I'd have dropped him the instant he touched me. Not all situations require that we draw a weapon. Had a similar situation happen years ago. An ex GF used to manage 7/11 stores the corp took back from franchisees. Usually crappie stores in crappie neighborhoods. One night I went to the store to learn someone had openly taken a couple of beers and candy bars but of course had gotten rid of them by the time the police arrived. The police banned him from the property, but he was coming back through the parking lot yelling being vulgar and threatening. Corp policy prohibited her and by association me from confronting him. Call the police and wait. He began to escalate throwing things at the window threatening her. At that time it was time to deal with him before he hurt someone or damaged property. Basically acted as if I was leaving and gave him time to approach me than told him the cops were coming so either back up his mouth or leave. He took a swing at me, I took a glancing blow unsure if my witness saw, I made sure they were watching and told him he hit like a female dog. He got a a decent shot in. His knee cap broke and his skull bounced off of a concrete car stop. He tried to get up but was unable too. When the police arrived they told me I could have soften him up a bit more before they got there and took him away. Apparently he was known for taking steroids and of course drinking. At court his family cried that I had cost him his football scholarship. The judge sympathized and gave him a year county to figure out who was responsible for that.
The point of all of this was as soon as he laid hands on you he gave you permission to end his inappropriate behavior. IMO make it end than and there. Allowing folks like these to get another chance encourages them, giving them time to build a resentment and plan a responce. Like the situation you are in now, he can come back with a gun. On the flip side you know his tactics, he is prone to push and shove. The next time he gives you a shove, knee cap him, its simple quick and effective even Godzilla will lay on the ground helpless with a broken knee. It works for feeble old little guys like me too wimpy to go toe to toe with big young buff bucks It will work for folks who have reach their normal growth potential as well. Be ready to draw & fire, I am a firm believer in not drawing unless I am going to shoot. So IMO do not draw your gun unless you feel you are in fear for injury or death. I believe a reasonable person would feel their life is in danger if someone who had threatened to kill them approached them in an aggressive manner and reached to draw something from concealment. You do not have to wait for him to complete the draw and start shooting. I wouldn't. Get ready, file a police report get a no contact order, make it clear that he has threatened your life, that you are in fear for your life, that you are afraid he will kill you, be prepared for a confrontation, make it happen on your terms. Remember you are a sane rational human being dealing with a drunken idiot. It is not that hard to what is the right term? "allow" him to make the wrong moves. You don't want to give the misimpression that you set him up. I am sure you want this to end, it is up to him on how it ends