I think people can justify buying an iPhone more than they can justify a dedicated gaming system cause that's for children.
Plus the iphone is more convenient. You don't have to decide to take it with you, and if so, which games to grab. That is my issue with the DS and PSP, I never feel like lugging it around. So this gen, I've spent the vast majority of my handheld gaming time with Iphone games that are not as good as the DS and PSP games I own but never play.
The iPad is even more of a decision, in terms of bringing it with you for the day. That said, I don't think it'll beat out even the iPhone for portable gaming. Even though I would love the new version of Geometry Wars with it.