I agree Nintendo can make a great FPS. But like Kairon said: It is the objective of all game developers to make the best game possible, and although Nintendo is better than most it isn't like 3rd parties can't create games equally as great as Nintendo can.
But I really don't get your Smash Brothers argument...in fact Smash Bros. fits my argument better.
Nintendo set out to create a new fighting game, but instead of just making a 2D or 3D fighter with Nintendo characters. Or creating a new franchise like Rare did with Killer Instinct. Nintendo decided to create a new fighting game experience. It created an engine around King of the Hill and your damage level increasing instead of health decreasing. It added 4 players at a time, and simplified the buttons and combos down to make it approachable in a 4 player game. Then they added levels that played more like platforming games and items to attack enemies.
Basically Nintendo created the most original fighting game experience ever, by changing the formula.
Nintendo did the same thing to FPS with Metroid Prime series, turning a FPS into a FPA and changing the focus of the game. Nintendo do the same again with Pikmin creating an original and innovative Real Time Strategy game. So I look at Geist as a chance for Nintendo to create another great original experience. A game that can take FPS concept and imagination to a new level. A game that can rewrite the rules and break all the cliches of the genre.
I really don't want Nintendo just creating games that don't break the mold...even if it is the greatest game created in that design mold. I want them to reinvent genres, and set new standards for what we understand games to be. Then once they have done that, they can use that franchise to refine their original vision and perfect it.
I think Geist 2 would be a game capable of refining, and perfecting the series and Geist experience.