1) Track record & "stopping power" - must be a caliber with an excellent record in service.
2) Widely available ammo - must be a caliber you can find at any sporting goods store, and preferably not cost an arm, leg, or firstborn.
3) Controllability - must be able to perform follow-up shots in a timely manner.
Calibers I've decided meet this criteria: .45 ACP and .357 Mag.
Just personally, I wouldn't carry anything smaller than 9mm in an auto or .38 special in a wheelgun. I don't have confidence that anything smaller than those will do the job with a solid hit or two center-of-mass.
I have confidence that a modern JHP from either .45 ACP or .357 Mag center-of-mass will stop a fight; I have even more confidence that two of them will.