It's pretty hard to punch yourself hard enough there to do much damage. Just a physical thing, your arm only bends so far. If you're having symptoms of a concussion like dizziness or vision disturbances or feel faint or the like, see a doc, but I'm guessing you're fine. When you're angry, punch a pillow.