I understand how you feel. Every times my husband and i have arguments he tell me to leave him alone or not to hold him and of course i do not listen to him and believe me it always make things worst. Try to talk to him when he is calm and do not do anything in haste.
You're the only one who can decide what your limit is. But personally I have a zero tolerance policy when it comes to any kind of domestic abuse. I literally don't care what his excuse is. If any guy hit, pushed, shoved, choked, slapped, or in any other way physically hurt me I would be out the door and would never come back. That's my policy. I don't allow disrespect. I was raised better than that.