I'm a 1911 carrier and have 3", 4.1" and 5" 1911s at my "disposal."
I commonly carry the 4.1" as just like the gun better. My 3" is perfectly fine, of course and when carrying in a shoulder holster it does conceal better, but past that it really doesn't make that much of a difference and I'm a 5'3", 105 lbs woman.
I've tried carrying 5" before and while concealing wasn't too much of a problem the issue I ran into was that, because of my short torso, I couldn't clear the holster with the muzzle without my hand being in my armpit unless I carried small of the back.
So, I'd say no. There isn't a big difference in concealment from 3" to 3.5" or 4".