I wear 501's and have since high school. I also wear other button fly pants (BDU's and such) sometimes and this is not a problem for me. I learned before I turned 16 to start with the 2nd button down and leave the top one alone, then work back up when the business is over.... I guess it takes a little longer..... but for me it isn't frustration. But, this may change soon as I am getting older .
Yes, I agree that 501's are the best jeans on the planet.