Guess its just an individual preference. I got both.
For me, kydex seems to be like business attire and leather is for more casual dress. I tend to use kydex during work and high stress/strenous occasions. Leather gets used more during my socializing and/or relaxing activities.
Took me awhile to get used to wearing my kydex IWB C-TAC. I finally discovered that if I kept my drawers pulled up it would ride real nice between my inner undies and outer pants. It hardly ever touces my skin. I will occasionally put on an under shirt and it gets even more comfortable.