"They could support save states, but I have mixed feelings about that. On one hand, they're handy beyond description. On the other, they make it really easy to cheat by letting you try the same thing over and over until you get it perfect, then save again. That tends to take a lot of the enjoyment out of games I've played emulated. I personally find it a very difficult temptation to resist, unfortunately. I suppose they could make it menu driven, which would slow things down enough to make them less useful for cheating."
I think the ideal way to do it is make the save delete itself when you open it. Ideally what you need a save for is to stop playing a game and resuming it later. Then you can't abuse it but you don't have to dedicate hours to beating a game in one sitting. It's effectively just pausing the game to play it later.
Though I would recommend adding saves for high scores since some games didn't have that. Another example is that Excitebike originally didn't let you save your track designs and they later added a save for that in the GBA re-release. I think that sort of change is fine.