I really like Firefox. Once I tried it, it did feel like "everything seems a little bit off", but after using it for like a month or something, if felt completely normal. When I open up IE everything "feels a little off" now. It's just that they're both different. Actually, I seem to remember getting that same feeling once after IE had a large update.
Really, what "more" does Microsoft give you? I can't actually think of a single useful thing. And the pop-up blocking and "tabs" of Firefox are great.
If you want my advice, get Firefox and use it as your "secondary" browser. Just open it deliberately every now and then. After a month you'll get used to it, and then you can ask yourself if there's any reason for you to not make it your primary browser.
One of my brothers got hit with spyware. For no particular reason. He wasn't surfing for pron or anything (it was car stuff). Just random dumb luck. While we were cleaning up the computer, I pointed out that I was (mostly) immune to that "dumb luck" because of Firefox. All of my brothers tried using it (because cleaning up the computer was a pain in the ass), and today they all pretty much say "Why the hell was I ever using Internet Explorer?"