A Mosin. :yup: M-44. Probably $100. Nice gun to learn on too.
Another vote for a Mosin Nagant.
Mosin Nagant. Pretty much the other end of the spectrum from the 10/22 as far as firepower and recoil, but its cheap and fun!
Mosin-Nagant (7.62X54r) or a Mauser of some sort (8mm Mauser AKA 8X57). Run the military surplus in either one, and the rounds will be pretty cheap relative to the power. Keep in mind with the surplus ammo, they will all have corrosive primers. That means that if you use it, you need to clean the gun immediately with something that will neutralize the salts in order to prevent rust. Windex or hot water will do just fine for that purpose, and the ammo is so cheap it's probably worth the extra couple minutes to account for the salty primers in your cleaning routine. Pretty much any Mosin you find for sale right now is pretty cheap, and the Yugoslavian 24/47 Mausers are going for low prices at the moment as well. The sites below will give you an idea what you can expect to pay for either type of rifle:
Well since you said "a little more power" I send my vote to the .22 magnum. Don't know if you intend to hunt but a scoped .22 magnum is a great little varmit rifle & kills better than it should. The .22 magnum has been around since the 1950's so there are plenty of good used scoped rifles in your price range and under!
I have never owned a .22 Hornet but I have friends that are head over heels in love with the little round. Always wanted one. Sounds like you have a fun quest ahead!
around here, I'll be paying 50 cents to $1 a round if I buy a .223.
I think I'll focus on .22 Magnum Henry lever-action or some bolt-action. They look like tons O' fun. :)