The lightest and smallest .357 Magnum I will carry is a 2 1/4" Ruger SP101 when I carry a .357 Magnum Revolver.
BTW: that SP101 is a BUG to a larger .357 Magnum, and not as a "primary". If you opt to carry one gun, and one caliber, for me it would be either a 3" or 4" .357 magnum or a 5" 1911, but then I very seldom carry one gun off duty.
Small .357 Magnums are not fun to shoot. If I can't shoot at least fifty rounds, qualification type course with one, I don't want to carry it. I can shoot fifty rounds through the SP101, not so the S&W 640, if both are loaded with my preferred .357 Magnum carry ammunition. You may be different, and be able to shoot the J-Frame for fifty rounds out to 25 Yards. I don't know.
Between the two calibers it would be a hard choice for me personally, but niether would be a "small" gun. I prefer the Gov't size in a 1911 and the 3" or 4" in a "primary" .357 Magnum revolver. I most often carry a 3" when I pack a .357 Magnum revolver, but carry two other revolvers as well, and one of them is another .357 Magnum.
I hope that helps to answer your question. If I was limited to only owning one gun, it would be a 4" .357 Magnum, but I'm not limited to such an option at this point in time, so it's not really a consideration for me right now. When I carry my 1911 I also carry a J-Frame.
Biker