After a lot of bad situations, it seems like there's someone who says something like, "I KNEW something wasn't right!", but the majority talk themselves out of it (e.g. "Aw, gee, I'm just thinking it's odd because he's [pick one: wearing baggy pants/has piercings/looks like my girlfriend's ex/is a minority/etc.].

I believe it's better to stay aware (as you did) and accept your gut instincts. The old "Be polite, be professional, have a plan to kill everyone you meet."