I learned as a bouncer many years ago that the best defense is a good offense. Just before I was going to grab someone, I'd yell "Please don't cause any problems" loudly several times. That got everyones attention and was a proven defense when the law got there after I thumped him on the way out. Same could be said in your "follower" scenario. I'd only take it so long until I had to react, if a decent escape was impossible. Cowards have no guts to react to a forceful confrontation.