I often leave mine at home. If I run into a situation where I need it while I'm on one of those trips... well that was my choice not to have it, and whatever I get is what I get.
When I'm hiking for enjoyment, it's all about lightening the load for me. On my usual long weekend trips, I try and pare down what I'm carrying as much as I can. That is the only way I can get where I want to go and still be able to say I enjoyed the trip at the end.
It's a gamble, as it always is when you choose not to carry, but as I said it's my choice. I carry most places that I can, but sometimes my desire for personal comfort and enjoyment are more important to me than the possibility that my gun could save my life on that particular trip. I have thought it through and weighed the options, and sometimes that's what I decide.