The first character you choose in Octopath Traveler isn't stuck in your party for the entire game: you can switch them out after completing their fourth chapter. Since all chapter 4's have a recommended level of 45, there's no reason not to do their chapter 4 first, so you'll have full control over all four party slots for the seven other chapter 4's and likely the four Shrines for advanced jobs, late optional dungeons, late and post-game side stories, and the final dungeon and final boss. It's a weird restriction to put on players, for sure, but it's there for 1/2 or 2/3 of the game, not all of it.
I don't think any character in this game is useless. Alfyn is the only one that's more geared primarily towards boss fights, but the others are all useful in a lot of different situations. Primrose is the only character in the game who can use dark magic naturally and her buffs are important for boss fights. Tressa is the only character in the game who can use wind magic naturally and gives a lot of weakness coverage in general because she can also use spears and bows. With secondary jobs, you can make them even more useful. I never really had problems with rotating my party or having more of a support character locked into my party (I started out with Therion). The game is extremely flexible in letting you freely assign different roles to different characters.
When a boss powers up after recovering from Break status, you're not really supposed to just take their next attack. When they enter a more powerful state, that's your cue to Break them again before they can act (not restart the boss battle lol). If you have a Hunter with Leghold Trap (which you absolutely should for pretty much every boss fight), it's a trivial matter to Break them before it gets to that point (the Shield Points do increase, but the flip side is that you should know all or most of this weaknesses by then). After you Break them again, they usually return to a normal state. If the battle lasts long enough for them to power up again, you just Leghold Trap + Break them again.
That's probably why boss battles are taking you so long. Otherwise, 35-40 minutes makes no sense (especially if you're five levels over what's recommended). The majority of them are certainly not designed to take that long. I would say 35-40 minutes is more accurate for the entire dungeon exploration, boss fight, and post-boss fight cutscene cycle as a whole. Even then, I didn't always find it to be that long.
Getting to the final dungeon and the final boss isn't nearly as obtuse as you were led to believe. It really comes down to clearing all the chapter 4's, doing a couple of side story lines, and that's it. If you were doing all the side stories in the game anyway, you would not miss it. The side stories in question are also not "unmarked." The only maybe obtuse thing about the whole process is that the final side story needed to open the path to the final dungeon is outside of the towns, but again, it will still show up as an orange icon on your mini-map when you enter the right area (it's not unmarked). The game also does hint at the true ending fairly heavily through certain common elements showing up in a lot of chapter 4's and being left open-ended. Also, you do get credits after you clear either your first chapter 4 or your chosen character's chapter 4 (I'm not sure which it is exactly because I did my chosen character's chapter 4 first). You also get a nice illustration to finish the epilogue after each chapter 4 too.
To be honest, I'm not sure how you're playing the game to be running into a lot of these issues. Outside of the Shrines for the advanced jobs and the final boss, I thought the game was relatively tame in terms of difficulty and very fair overall. Once I learned to really make use of secondary jobs and the Break and Boost mechanics, I didn't have a problem going into areas or dungeons five levels underleveled and getting through just fine. You can starting ending boss fights comically fast when you really start experimenting with different job combinations and support skills too. In any case, it sucks that it's been such a drag for you so far.