I try to keep concealed in public, but I stopped worrying about observant people catching a glimpse of my firearm many years ago. Most people, at least around here, don't seem to be overly concerned.
Last winter I was at a local shop getting new brakes on my truck. In the waiting room with me was a very nice lady, probably in her early-mid 70's. I got up to stretch my legs and she caught a glimpse of the XD-40 I was carrying and asked "is that a gun?" I fought the urge to say "no ma'am, I'm just happy to see you" and politely replied "yes it is". She started talking about how sad it is that mankind hasn't matured past the point where we feel the need to carry guns.... Just when I thought I had triggered an anti-gun speech, she continued to say that she supposed that even a most enlightened society would have its crazies and its wolves and that's why she wouldn't leave home without her .38! I was somewhat surprised, but extremely delighted by her statement. We had a wonderful conversation about guns, training, SA, politics and family until they pulled her car, with its new shocks around front and it was time for her to go.
No... I think I'd be more concerned about something peeking out of my nose than about an accidental exposure of something tucked in my waistband.