I've always been too paranoid to try replacing the batteries.
The way I see it, if you can't save your game then that makes it pretty much unplayable. You don't really have much to lose if the game stops working entirely if you messed up.
Pro-tip, something in the game boy pocket makes it drain the cartridge batteries if you leave the cartridge in while its off. So be careful out there!
Where did you hear about that?
I was playing my old copy of Crystalis, and to my dismay...it doesn't save anymore. It was fine a few weeks ago though.
Just to make sure you haven't forgotten, you need to hold in the Reset button when you turn the power off in order to make sure your save data will remain. This applies to even if you get a blank screen when trying to get the game to work.
Crystalis was re-released on the Game Boy Color, it might be less hassle to track down a copy of that.
Well I just bought Wario Land 1-4 so I may have to change the batteries for some of them.
Probably not. My 23-year-old copy of Zelda still hold saves (I still can't believe that game is older than I am...), and I'm always surprised to hear when a game's battery goes dead. Pokémon Gold/Silver/Crystal are an exception: they all contain a clock, so the battery goes dead after approximately six years.
Technically, Game Boy/Color games contain a CR2025 battery, but you can still replace them with a CR2032. It might even last a little longer, too.
I don't think Game Boy Advance games contain a battery, but they have a different type of screw and can't be opened with a 3.8mm GameBit so I don't know.
Something else which uses a CR2032 battery are computers, to power the clock when the computer is powered off. I had to replace mine a couple of years back. Some TV remotes also use the nickel-sized battery. I would be surprised if the GameCube and Wii didn't use it to power their clocks.