On Firefox, you can set the browser to erase your history when you close the browser. It's hard to do on IE because there are so many "History" files.
Dump IE, you'll never look back.
I also set Firefox to ask me every time a cookie is added. Then I have a choice to say no.
I keep IE just in case I need it to order something online (rare I can't do that on Firefox) but for browsing, Firefox is the way.