Wow, a topic someone like me can chew on... and maybe once the universe finally gets into alignment I'll be given a chance to expand on things. Anyway, first way back in the day of 64 Source I wrote an article about the topic of how to write good reviews, or an attempt to quantify my vision of it, and it dealt in some ways with this concept. I will tell you, in the 2 sites I was in the core of helping set up (including this one) there was serious discussion about ratings, scores, and how things would work. For a site wanting to score free games unfortunately I'd wager numeric scores are going to be necessary... the companies are out to get their word out and unfortunately for their marketing departments a consensus of 1,000 word + in-depth analytical reviews is harder to shove out there as an accolade as 5 major sites giving it a 9 or above. Sadly as well, whether right or wrong, the vast majority of people out there respond to the scores since it is all they have or can possibly comprehend. Somewhat revealing a bit of my angle to come on the site... for parents, in particular, a score is what drives them to buy the game for their kids.
To add even more complication and depth to this discussion though it is all difficult stuff, especially as you then introduce other factors, like distance. This is something on 64 Source we specifically (for a time) tried to address even with reviews 1 - 2 years after the fact of a same major game we'd reviews. How does an inspirational and incredible game of the moment given accolades out of the gate hold up over time? See something like Super Mario Brothers 3 (eclipsed but still an excellent example of the genre) against something like Goldeneye (loved the game originally but practically worthless beyond nostalgia now as it has been so thoroughly eclipsed). Some games, like movies, or TV, or books, or whatever age like fine wine... others you find have turned to Boones Farm or vinegar. The moment can make a big difference. So that introduces a new dimension. Is it fair to continue to hold something like Goldeyene (really, I loved the game, not bashing it, just it is a great example of a game great at its time more than all time) in as high esteem as other games with the same score that have aged more gracefully? Hell, perhaps games with lower scores that have aged better deserve more credit than GE just for their staying power. So even the power of MetaCritic and other score aggregation sites have only the power to rate games in terms of the here and now. This is especially an important reason to get solid VC and other reviews. Perhaps what was terrific then is well ported and just no longer relevant.
Another way to pee in an already tainted pool (and what my old Editorial really dealt with more) is context. I think moreso than numeric scores, lack of truth in score breakdowns (how many reviews do you see where a 8 game gets 4 sound but 10 graphics or what gave you... I swear 90% or more of reviews all breakdown scores will fall within +/- 1 point of the main score and themselves... is there truth in that?), or other bugaboos is context. Perhaps moreso than ever with the introduction of the Wii and the resurgence of "casual gaming" I really think, though it would make aggregation sites squirm, that game reviews almost need MORE scores.
I think in the old editorial it fell to series (if applicable), genre, and general audience... but perhaps now we need series (if applicable), genre, casual audience, and hardcore audience. Why? Because first if you're not a fan of the series or genre the score needs clarification. Is it transcendent (perhaps ala Smash Brothers for "fighting" as an example)? Is it a great example of the genre but unlikely to convert non-believers? Is it essential to someone who loves the series but still the same core game? Perhaps great for the series lover but a negative to someone outside. So context even in that sense DOES matter and one score for the game utterly fails to take that into account. But look at the Wii. Carnival Games? To the hardcore somewhere between tolerable and absolutely craptacular depending on your tastes. To casual gamers like my relatives (and based on sales, many many more people) who overlook the suspect graphics, dodgy controls, kiddie-pool shallowness... perhaps not the greatest thing since sliced bread but I will tell you that they play it on a regular basis to this day.
Blah, time to split but like I said, I hope to talk about it more with some ideas and once things roll may well put up a revised version of the old Editorial (that I sadly can't find, of course) that explores this problem. Thanks for a compelling discussion folks.
Topic: Making the Review Process Better (Read 26296 times)
