All they would have to do is something like Paramount does. All of the live action shows and the 11 movies are canon (the Animated Series is not, most of the 11th movie takes place in a alternate timeline and thus doesn't screw up the real timeline). This means Star Trek novel writers, comic writers, game developers, etc. can do almost anything they want (obviously they need approval, Paramount would not allow something like sex stories) without messing up the canon.
Gene Roddenberry insisted that the novels and the animated series not be canon, and because of that there's all kinds of contradictory stuff in the old ones. In the last 10-15 years they've been pretty careful about what happens in the novels and the vast majority of stuff from that era fits in just fine with established canon, but it would be even more of a mess than Star Wars if some novels were canon and others weren't.
I personally consider Michael A. Martin and Andy Mangels' Enterprise relaunch novels, as well as Greg Cox's Eugenics Wars novels, to be canonical, even though they aren't officially part of it.
Just remember that none of the novels (or anything they contain), even the novelizations of the films, are canon. If it didn't happen in one of the live action TV series or movies, then it us no more canonical than fanscripts on the Internet. Some of the novels are pretty good and I hope that they start a new TV series (set in the real timeline, not this BS alternate timeline), this way they could make canon stuff like Bajor officially joining the Federation like they did in the DS9 re-launch novels. Roddenberry used to change his mind all the time on what was and wasn't canon. Things get more complex when you consider than some stuff has un-officially been decanonized (like the widely hated
Star Trek V: The Final Frontier, which has had most of its content contradicted by later movies/shows as Paramount tries to ignore the movie.
But like you said, Paramount decided to play it safe and just say that all novels are not canon. This prevents the mess that Lucasfilm has with Star Wars novels and the "Expanded Universe".