You are forgetting the millions of people who haven't upgraded yet, there is still room to bring them in.
A machine that tops PS4 wouldn't be that expensive, Nintendo has a lot more money, patents, and good will with their tech partners than I think you give them credit for. Also they wouldn't have to suffer massive losses, Wii U was sold at a loss but it turned profitable with the first game sold, they could afford to sell it at just enough loss to make it profitable at 2 games sold and considering that Nintendo has a high attach rate there is a chance they could do it. Also they have sole consoles for a loss before they do it all the time it isn't a new business model.
They could make a machine that is competitive and sell it for a reasonable $350 and people will buy it over the other two for many reasons. One the people who HAVE upgraded don't represent the entire gaming population, 2, there is a large number of them who are waiting to see what NX is and are holding off for that very reason, 3, people switch mid-generation ALL THE TIME, 4, Nintendo is not as fragile as you think they are they can afford a couple of years of heavy losses if it turns them profitable in the long term which would make more sense than going bancrupt pinching pennies. BTW, Nintendo is not as "conservative" as people claim they in fact are the biggest risk takers in the damn industry, they just don't always pay off.
Asking them to make the machine the entire gaming community is begging for would NOT be bad business sense it would sell like hotcakes just on the EFFORT and excitement alone it would garner.