I think sometimes we over think the disparity of force issue.
Where it's commonly used against a shooter (who believes/states they were acting in self-defense) is the 240 lb. man who shoots the drunken 110 lb. female neighbor, or the 65 year old homeowner who shoots the 15 year old kid stealing the lawnmower.
It's not that a drunken woman or 15 year old boy can't kill you, it's that the perception of most people (LEO's and people on the jury alike) that we can avert injury to ourselves by retreating or using less lethal force in that particular situation.
(And please don't think that I am justifying the violent actions of the woman or boy in this question, just the fact that it's this type of scenario where the disparity of force issue is raised against us shooters).
It doesn't matter if your attacker is smaller than you, or fat, or skinny, or what ever. If your attacker is adult (or nearly so) and able bodied, then there is no disparity of force.
If the attacker is falling down drunk, in a wheelchair, an obvious juvenile, or the like, and now you've got a problem.