Post by: Halbred on January 03, 2009, 09:20:16 PM
I have a friend who often asks me for advice about video game purchases. He has both a PS3 and an Xbox 360. Despite my lack of owning the latter, it doesn't stop him from asking me "what I've heard" or "what I've read." And that's fine. I like being the go-to guy for gaming-related decisions. He often calls me at the store, game in hand, and says, "What do you think of Mirror's Edge?" I liked the demo, I tell him, but published reviews have been all over the map. Game Informer, EGM, and Play liked it, but Edge hated it, and even among the publications who gave it high marks, they were quick to point out its faults. I rattle off a list of review scores I've seen, thus illustrating the bipolarity of the press regarding Mirror's Edge. Then he asks me a question that renders all of those scores irrelevant: "But is it fun? Will I like it?"
Those are much tougher questions to answer. I daresay they are impossible to address in a standard number-based review format. Here's an example that hits close to home. Back when I first started here at NWR, my editors sent me Phantom Hourglass, a game I gave mediocre marks. The ensuing Talkback thread ballooned to eight pages of people yelling at me, proving beyond a doubt that enjoyment of ANY game, even one from a well-respected franchise, is completely subjective. Many people thoroughly enjoyed the very aspects of Phantom Hourglass that I criticized. And you know what? Nobody is wrong. I'm not wrong in giving the game a 7.5, and they're not wrong for giving it closer to a 10. The numbers don't mean anything.
Here's another example: Beyond Good & Evil is one of my favorite all-time games. Were I to review it today, with my NWR reviewer hat on, I would give it average scores in all counts. The graphics are kind of all over the place, the music is good in places, the control scheme is a little floaty, partner AI is questionable, and the game just can't decide what it wants to be. However, even though each criteria (Graphics, Sound, Control, Gameplay, Lastability, Final) would get average marks, only the final score, which here at NWR we disconnect from the others, gives any indication of how great the game is. BG&E would get 6's and 7's across the board, but probably a 9 from me at the bottom. BG&E has an emotional core that, until I played it, I had not experienced in a video game (except the MGS series). Now the inverse: I gave Phantom Hourglass fairly high marks in each individual category, but a lower final score.
Graphics, sound, control, none of this matters. What matters is whether the game is good or not. All you need is a final score. And even then, the final score is not definitive. I didn't like Phantom Hourglass, but many of my readers loved it. I love BG&E, but I know a lot of people who don't or wouldn't. I think Wario Land: Shake It! is one of the best games of 2008, but most publications barely gave it the time of day. It's all subjective, even the final score. It's just like my buddy asked me: "But is it fun? Would I like it?"
These are not questions I can put into numerical form. Whether you, the reader, will like a game or not is not a question I can answer. I can tell you that I, personally, thought the game was great (or sucked). Hell, there might be a guy out there who really liked Homie Rollerz. Me? I hated it, and I think you good people would similarly despise it, so I told you not to waste your time or money. And that's all I can really do. Scores don't mean anything. All I can do is tell you what my experience with any given game was. You might agree or disagree, but ultimately you have to make that call.
My feeling is that "scores" have stuck around to benefit aggregator sites, like MetaCritic and GameRankings. Review scores are collected at those sites and the resulting average affect things like stock prices, funding, and, ultimately, the consumer. It's a dirty business--one that leads to questionable situations like publishers allegedly bribing reviewers for higher numerical scores, or reviews being canned after giving a highly-advertised game a low score. Numerical values hold a lot of weight, more than I think they actually have.
If it were up to me, reviews would simply be narrations of one player's experience with any given game. Play and Edge, two of the more progressive gaming publications out there, have dramatically reduced the number of numbers in their reviews. One of them gives a final score, and the other has done away with scores entirely, leaving readers with a short "parting shot," basically the text that would go under NWR's "Final" score. Think about it--would that Phantom Hourglass review have been so lambasted were it lacking numerical scores? I really doubt it. You can argue about subjective scores, but it's tough to tell somebody that their opinion is wrong. I didn't like the game, simple as that, but you might. Numbers are infinitely more concrete than language.
The value of a game cannot be measured in terms of math, only experience and personal taste. What do you guys, as the readers, think of the standard review format? Do numbers help you? And if so, how?
Post by: NWR_Lindy on January 04, 2009, 01:34:49 AM
MetaCritic is neat as an at-a-glance site, but I think it's lunacy that companies actually sit there and tie business objectives to MetaCritic scores because, as Zach points out, review scores are completely arbitrary. One man's 6 is another man's 8, and while that's nice fodder for a discussion forum, basing your definition of success on those scores is simply not wise.
Post by: broodwars on January 04, 2009, 03:27:26 AM
Post by: Deguello on January 04, 2009, 04:16:08 AM
Quote
Think about it--would that Phantom Hourglass review have been so lambasted were it lacking numerical scores? I really doubt it.
I dunno, that whole part about Nintendo "changing the game for casual gamers" would have still drawn a lot of groans from the audience, being the tired canard it was in 2007.
The problem with releasing the hold of reviews on game quality stewardship is that you can then logically assume that anybody who enjoys X game, that makes it a 10 to them. It's a short leap from there to sales figures and next thing you know whatever game that sells the most is the one enjoyed the most and therefore the highest quality. I've been told that's bad... I think.
We, as reviewers, seem to have tricked ourselves into thinking we are necessary. We really aren't. After reviewing mountains of ****, I can say that my opinion of any of these games probably never influenced a single buying decision. Even when I review the occasional anticipated release, I can tell my review might as well have been posted on the inside of a church bathroom. And it's because of this, that I know every opinion I give is honest. I dread the numbers part, sure. But I do them anyway, without breaking out a slide rule and making sure I haven't given out too many 10s so I seem "credible."
The only real problem with numerical scores are the aggregator sites like you mentioned. Nothing encourages and incites 1984-ish groupthink than "Oh yeah, well Super Blood Racing got 89% on GameRankings. Take that, Wii Sports!" That and it's probably work theft, too.
And they already have something like what you want. User reviews on Amazon! That's where you'll find heartfelt customer reviews of the games they own and how they enjoy them. You'll find Mario Kart Wii has a 90%+ rating, besting Halo 3, and Wii Music, Wii Fit, and My Fitness Coach have higher scores than Call of Duty: World at War (which strangely has the same score as Wii Play.) Bedlam to be sure, customers enjoying their games against the wishes of professional reviewers.
Post by: Nick DiMola on January 04, 2009, 11:15:34 AM
In a perfect world, I'd prefer my reviews without a score. But, as Lindy pointed out earlier, aggregation sites like Meta Critic and Game Rankings have forced the entire industry into a 1-10 grading scale. So if you don't provide a score, they'll give you one, without consultation. Naturally this provides a major problem.
Kotaku made a valiant attempt at removing scores, but in my opinion failed. They provide what is essentially a list of pros and cons, but never really qualify on the whole whether the experience is worthwhile or not. I think it's important to imply a number or guide your reader towards the game's worth in your text. Whether or not a a score is provided at the bottom, reading through the text, readers should be able to make a decent guess how good that game actually is.
But again, lack of a number gets back to our role as reviewers, are we critics (ala Roger Ebert) or just recommenders (suggesting or dissuading a purchase)? In my mind, most people read a review to determine what action they are going to take with a game. Will they Buy it now, or wait for a price drop, or rent it, or just avoid it all together? I'm guessing most people simply internalize a score from 1-10 and make that call. As such I've always believed that blatantly pointing this out is the best course of action.
I know most staff disagrees with me on our purpose as reviewers, but I'd like to hear what you guys think. How do you consider reviews? Do you read them to gauge purchase-ability? Or do you read them for other reasons?
Post by: KDR_11k on January 04, 2009, 11:22:42 AM
Final scores are also useful to pick a review based on its view (e.g. when looking for positive or negative reviews to see different points of view) or quickly sorting a list of games for their probability of being something you'd like.
Of course there's no way for a single reviewer to account for different tastes and I don't think a reviewer can correctly assess how a game would go down with a group he's not part of. For a proper score you'd probably need different reviewers with different mindsets (competitive, "casual hardcore", new market, social, antisocial, ...) so people can select the evaluation from the one that matches their own mentality. Multiple reviewers with the same mindset are probably redundant and not useful though (I don't think you could get a good score for "casual gamers" even if you put the whole NWR staff on one game because all they can do is guess what such a person might think and they're likely going to be wrong).
I never put much weight into Amazon reviews since I can't know if the reviewer might be a plant or something and I don't think the reviewers are an unbiased sample either (i.e. people with certain oppinions are more likely to review so the results are not statistically useful). Plus some people even post "reviews" before the game is out, just ranking it according to the amount of hype they received...
Yeah, to me reviews are nothing but purchase recommendations.
Post by: NWR_Lindy on January 04, 2009, 01:17:16 PM
It's the same old "game critique" vs. "buyer recommendation" argument. They're really two different things. Art vs. product.
Post by: KDR_11k on January 04, 2009, 02:12:19 PM
Post by: NWR_Lindy on January 04, 2009, 02:32:04 PM
Post by: GoldenPhoenix on January 04, 2009, 06:40:31 PM
But how would I make the review process better? Well that is easy, ban Lindy from ever writing another review!
I'm j/k! Or am I? ;)
Post by: Kairon on January 04, 2009, 07:06:39 PM
Post by: NWR_Lindy on January 04, 2009, 07:51:30 PM
But how would I make the review process better? Well that is easy, ban Lindy from ever writing another review!
INSTA-BAN
Post by: Peachylala on January 04, 2009, 08:29:25 PM
- Either just do the reviews you guys have always done for the past decade, like the pros and cons, but get rid of the ratings for each aspect (ie: graphics, sound, etc.). A scale rating of 1 to 5 for a final score could suffice too.
- OR... have no rating element what-so-ever and just have the "end of review" blurb.
The second option could work for many of the Touch Generation games that Nintendo releases (like Wii Music). Personally, maybe you guys should use the rating system for the Virtual Console games. I love that system. ;D
Whatever floats your boat, I suppose.
Post by: shammack on January 04, 2009, 08:55:26 PM
I know most staff disagrees with me on our purpose as reviewers, but I'd like to hear what you guys think. How do you consider reviews? Do you read them to gauge purchase-ability? Or do you read them for other reasons?
Personally I read reviews to get information about how the game works and what the experience of playing it is like, so I can decide if it sounds like something I'm interested in trying for myself. If there are obvious problems like bugs, load times, or things that just don't work the way they're supposed to, I want to know about those too, but I pretty much ignore the reviewers' opinions.
Post by: Kairon on January 04, 2009, 10:15:13 PM
I know most staff disagrees with me on our purpose as reviewers, but I'd like to hear what you guys think. How do you consider reviews? Do you read them to gauge purchase-ability? Or do you read them for other reasons?
Personally I read reviews to get information about how the game works and what the experience of playing it is like, so I can decide if it sounds like something I'm interested in trying for myself. If there are obvious problems like bugs, load times, or things that just don't work the way they're supposed to, I want to know about those too, but I pretty much ignore the reviewers' opinions.
This. Exactly this. I don't depend on reviews to recommend games to me. I depend on reviews to provide me with the information I need to recommend a game to myself.
Whether that approach is common though I know not.
Post by: NWR_Lindy on January 04, 2009, 11:35:35 PM
Post by: Kairon on January 04, 2009, 11:44:02 PM
Hmm, most readers seem to really like our VC Recommendations rating system. Food for thought.
Then again, those VC games cost $5-10, not $30, $40, $50, or even $90.
Post by: oohhboy on January 05, 2009, 12:18:01 AM
Hmm, most readers seem to really like our VC Recommendations rating system. Food for thought.
Then again, those VC games cost $5-10, not $30, $40, $50, or even $90.
Second this. VC reviews work because they are so cheap. Such a system would remove half the reason for even having a review for full priced games.
Post by: TheYoungerPlumber on January 05, 2009, 01:57:47 AM
Post by: Shift Key on January 05, 2009, 03:34:29 AM
"art"? What art?
Oh no, its the Evan-signal. Now he's going to come in here swinging trashbags of rage about the old "games as art" debate!
find me a game hanging in the lourve. evan. go on!
Post by: Plugabugz on January 05, 2009, 04:01:05 AM
I reckon it can be done one of two ways:
1 - Three people group-review the game. I.E. NWR Review Podcast of Crimm, Yoshidious and a special guest reviewing Pokemon Dungeon again. The range of scores each present (with their own reasons) will help to even it out.
2 - Make no emotional/opinionated references to what you're reviewing and make only factual statements. Not seeing crimm getting wound up over pokemon dungeon will be a BLAST
Post by: KDR_11k on January 05, 2009, 04:44:47 AM
Post by: TheYoungerPlumber on January 05, 2009, 07:03:05 AM
http://www.askoxford.com/concise_oed/orexxview?view=uk
1 a formal assessment of something with the intention of instituting change if necessary. 2 a critical appraisal of a book, play, or other work. 3 a retrospective survey or report. 4 a ceremonial display and formal inspection of military or naval forces.
It's impossible to quantify "good graphics" or "good gameplay" in terms of purely "facts", with no opinion. Even if you start crunching polygonal numbers and number of music tracks, aesthetics for sound and visuals are always subjective. And even if you could, I don't think anyone would enjoy plodding through mathematicals proof of every bad aspect of a game. (Dissecting logically fallacies in game or control design can be fun in moderation, however.)
A review IS an opinion. But I can think of a simple solution for you: read multiple reviews. Regardless of entertainment medium, you should read multiple reviews unless you know your tastes align with a specific reviewer. I guess you could crack open a preview or FAQ if all you really care about are control mappings and game design, but even in those you'll find the author editorializes.
Post by: Flames_of_chaos on January 05, 2009, 11:52:08 AM
Post by: NWR_insanolord on January 05, 2009, 12:03:20 PM
Post by: Ian Sane on January 05, 2009, 12:40:40 PM
Regarding the letter grade system one of my favourite sites is VideoGameCritic.net. Everything is reviewed by one guy who has been gaming since the Atari days and has every system and is a serious game collector. Each game he gives a paragraph about it and a letter grade. I mostly like his site because it also covers a lot of retro games and when you're collecting for an old system it's a good reference to see what's worthwhile and what isn't (I suppose it would useful for the VC as well). But the guy has something a lot of reviewers don't: perspective. I remember years ago someone who worked for IGN's Playstation page was talking about how Final Fantasy VII got him into games. He was PAID to review Playstation games and his gaming experience didn't even cover the first two years of the console he reviews for? How often do you read a review where they're crapping on some game because it's 2D or it's a shmup? That reviewer doesn't remember when games like that were current. He classifies them as old and calls it a day. The guy I read has followed games forever so he isn't going to crap on a game with "old school" design because he still plays those games on his old consoles. At the same time he likes modern game design as well.
I think however the letter grade he uses is arbitrary. He could just as easly use numbers, I just like his format and feel he has credibility. One thing he also does is have a guest reviews RPGs because he doesn't care for the genre. I like that he will admit that. Not every genre is for everyone so if a reviewer clearly isn't really into a genre they should get someone else to look at it.
On that note, and I'm sure someone will crap on me for this, EGM's review crew is not a bad way to do it. Do they even still do that? Anyway, back when I used to read videogame mags they had four guys who each favoured certain genres more than others each give a score out of ten and I think, at least in theory, that's a great way to do it. You talk about some people liking Phantom Hourglass and others not. Well that's how you address that.
One thing you should never do though is just weigh a game's value as the sum of its parts. The score should never just be an indication on how well made the game is. Some games with no bugs, beautiful graphics and sound, and precise controls still are boring as hell while some slightly imperfect games are a blast to play. You have to think like a friend giving his opinion of a game and not some beta tester checking off a list.
Post by: NWR_insanolord on January 05, 2009, 01:30:13 PM
Post by: KDR_11k on January 05, 2009, 02:06:34 PM
I remember years ago someone who worked for IGN's Playstation page was talking about how Final Fantasy VII got him into games. He was PAID to review Playstation games and his gaming experience didn't even cover the first two years of the console he reviews for?
Why not? Hegives the perspective of someone who got into gaming with FF7 which I think a lot of people did. Would it be preposterous to recruit someone who started gaming with the Wii in order to review Wii games? Does being used to the idiosyncrasies of an older time really make you a better reviewer? From what I see those old preconceptions tend to get in the way whenever a paradigm shift occurs because the oldies remain locked into their old paradigm and judge everything by the old values (and complain that a Wii game isn't HD). Old gamers have their perspective but new gamers have a different perspective and without reviews from that perspective you're not going to get in touch with the new gamers.
I agree with your final point. Reviewers gave EDF scores in the 70s while still saying the game is an insane amount of fun. I think one review gave a 10/10 (or whatever their scale was) despite all the technical flaws and such because they really don't matter.
Post by: Halbred on January 05, 2009, 02:17:48 PM
Post by: Ian Sane on January 05, 2009, 02:55:11 PM
Quote
Why not? Hegives the perspective of someone who got into gaming with FF7 which I think a lot of people did. Would it be preposterous to recruit someone who started gaming with the Wii in order to review Wii games? Does being used to the idiosyncrasies of an older time really make you a better reviewer?
I think a reviewer should be very knowledgable of what he is reviewing. The first movie you ever saw was at one point the best movie you ever saw. Ignorance can have a significant effect on your opinion. Now videogames are old enough now that it's harder to have lived though all of it. With books and music and movies you CAN'T have lived that long. It's impossible to have that experience. But you should be well informed of what else is out there.
Is it not idiotic when reviewers praise Halo as the greatest FPS they've ever played? Even if you stick with just consoles Goldeneye and Perfect Dark were doing stuff that Halo "innovated" years before it came out. But ignorance of the past makes it out to be a better game than it is. I want an informed opinion. You have to have some knowledge of the subject matter to give that to me.
Regarding the Wii thing, well maybe if the audience of the site was strict the blue ocean group for which the Wii is their first real intro to videogames. In that case it would fit. But a videogame site like IGN? Totally inappropriate. The audience is experienced gamers and if your opinion has no perspective it isn't going to be of any use to them.
Post by: KDR_11k on January 05, 2009, 03:42:15 PM
Of course the audience for most review sites is experienced gamers, anyone else would get only worthless results from them so why would they bother going there?
Post by: GoldenPhoenix on January 05, 2009, 04:29:23 PM
Quote
The first movie you ever saw was at one point the best movie you ever saw.
Doesn't mean it was a good movie. Personally I think in some ways being an older Retro gamer can be a detriment in that you have built up biases from your nostalgia for the "classics" and really cannot connect all that well with a newer generation of gamers, most of which would think most Atari games are terrible and shallow.
Post by: Ian Sane on January 05, 2009, 04:36:55 PM
Quote
Careful, just because the features have been done before doesn't meant there's nothing good about a game. A game without new features can still deliver a combination that feels new or just plain better than what came before it. Also I think quite a few people never had an N64.
I didn't really word my example well. When Halo came out many reviewers were busting a nut over how it revolutionized console first person shooters and was so innovative when the only person who would ever think that is an ignorant one. And the N64 isn't some super obscure console like the Jaguar. A lot of people had it and any reviewer worth a sh!t should have come familiarity with it's biggest games.
Quote
Of course the audience for most review sites is experienced gamers, anyone else would get only worthless results from them so why would they bother going there?
I don't see your point. Is this bad? Should they change their format to accomodate a completely different audience? If non-gamers want their videogame review site then they can make it. Nothing stopping them in theory.
Post by: Halbred on January 05, 2009, 07:21:30 PM
You can argue Halo's overall merits, but I don't think there's much of a question that it gave the FPS genre a shot in the arm in a lot of ways.
I think the FPS genre is conversative ANYWAY. In my mind, there have only been three or four really groundbreaking FPS games: Wolfenstein 3D, Doom, Goldeneye, and Halo. But that's just me.
Post by: Nick DiMola on January 05, 2009, 08:53:27 PM
I see Ian's point overall, and I don't really have a problem with what he is saying, though, I wouldn't want every reviewers to be on the same page in terms of their historical knowledge and experience. Varying opinions give people the ability to identify with particular people to get their ideal review. Without people who are wet behind the ears, a number of younger gamers wouldn't be served particularly well, because us dinosaurs have very different expectations of games.
Post by: Halbred on January 05, 2009, 08:59:57 PM
And I think I'm just biased against the gamers who were raised on the PS2. :-) I'm that crotchety old gamer on the porch, yelling that kids today have it easy. Back in MY day, we didn't even HAVE save points! Some games didn't even have save SLOTS! You kids with your fancy respawns and online play...
My god...I've become Cranky Kong!
Post by: GoldenPhoenix on January 05, 2009, 09:09:04 PM
Yeah, I totally blanked on Half-Life. I'd argue with you about Tribes, though. The concept was there, but the execution was...questionable at times. Let's say this: Halo brought the FPS to the console gamepad in a way that hadn't been done *as well* before.
And I think I'm just biased against the gamers who were raised on the PS2. :-) I'm that crotchety old gamer on the porch, yelling that kids today have it easy. Back in MY day, we didn't even HAVE save points! Some games didn't even have save SLOTS! You kids with your fancy respawns and online play...
My god...I've become Cranky Kong!
Halo was archaic compared to PC FPS games when it came out. The game did nothing new or innovative beyond the rechargeable shield (Which may have been done before). Vehicles were done in Codename Eagle as well, the predecessor to the Battlefield series. Not to mention that Goldeneye had a vehicle based mission with the tank where you had full control over it. So Halo is nothing more than a cut and paste job from games before (though it was polished) and since console gamers at the time were not that experienced with the PC they ate it up. It is funny because if Halo had come out on the PC instead of the Xbox you probably would not hear of the game, and if you did it would be a minor title.
To bring this topic back on track, I think this is a good example of how flawed the experience argument is. What if a person is more experienced in FPS games on the PC? Should their opinion matter more than someone who really only has console FPS experience? What about other genres done on consoles and PC? How do you approach them?
Post by: NWR_Lindy on January 05, 2009, 11:41:10 PM
Halo was archaic compared to PC FPS games when it came out. The game did nothing new or innovative beyond the rechargeable shield (Which may have been done before). Vehicles were done in Codename Eagle as well, the predecessor to the Battlefield series. Not to mention that Goldeneye had a vehicle based mission with the tank where you had full control over it. So Halo is nothing more than a cut and paste job from games before (though it was polished) and since console gamers at the time were not that experienced with the PC they ate it up. It is funny because if Halo had come out on the PC instead of the Xbox you probably would not hear of the game, and if you did it would be a minor title.
This is getting ridiculous. I actually agree with GP on all of these points.
Post by: Kairon on January 06, 2009, 12:00:13 AM
Post by: NinGurl69 *huggles on January 06, 2009, 12:31:24 AM
Post by: Kairon on January 06, 2009, 12:59:16 AM
To bring this topic back on track, I think this is a good example of how flawed the experience argument is. What if a person is more experienced in FPS games on the PC? Should their opinion matter more than someone who really only has console FPS experience? What about other genres done on consoles and PC? How do you approach them?
I wouldn't want every reviewers to be on the same page in terms of their historical knowledge and experience. Varying opinions give people the ability to identify with particular people to get their ideal review. Without people who are wet behind the ears, a number of younger gamers wouldn't be served particularly well, because us dinosaurs have very different expectations of games.
I agree with these sentiments. Not to mention, there are TONS of differences even among people with plenty of experience. NWR alone should be proof of how longtime experienced gamers can have so many different takes on what comprises a quality game.
Post by: Halbred on January 06, 2009, 02:44:03 AM
Post by: GoldenPhoenix on January 06, 2009, 03:05:55 AM
Another game that was significant in regards to opening up the genre was Deus Ex as well (with a cool protagonist). That is one take on the genre that I hope to see more of, where you have a pretty wide open world, or at least ways of completing various tasks.
Though I am not a fan of the trend, I find it interesting now that it seems the FPS genre's direction is being dictated more and more by consoles. Though in some cases it turned out pretty good (Far Cry 2). So maybe in a few years the PC won't be the benchmark anymore for FPS because developers are focusing on consoles first and then porting it up.
Post by: Plugabugz on January 06, 2009, 03:24:53 AM
Post by: Stogi on January 06, 2009, 04:28:14 AM
Post by: KDR_11k on January 06, 2009, 05:13:37 AM
Ian, telling them to make their own publication is a dangerous thing for a manager of an existing publication. It's like a console maker saying "let them make their own console". The whole goal of a disruption is to grab markets that the competition will willingly surrender and then make them surrender more and more until there's nothing left. The manager of a publication should instead say "hey, there's a ton of people who aren't reading our publication, how can we get to them?".
Post by: GoldenPhoenix on January 06, 2009, 05:16:14 AM
Quote
Halo may have been a polished cut-n-paste job. So was WoW. Some genres just need polish.
That is true except it isn't like Halo was the only polished FPS game, especially compared to the PC. It would be like polishing Tomb Raider 1 (which they did) even though the genre has advanced beyond it.
Post by: Stogi on January 06, 2009, 05:28:58 AM
Regarding the letter grade system one of my favourite sites is VideoGameCritic.net.
He gave WiiMusic an F. No wonder you like this guy. ;)
Post by: KDR_11k on January 06, 2009, 05:56:59 AM
Post by: Kairon on January 06, 2009, 06:04:15 AM
unless you have personally reviewed both games, there is no congruency between scores. Thus, it is basically pointless to give out scores if you have multiple reviewers.
Of course, to have one person review everything is highly inpractical and serves little purpose for those who disagree with their stances on a basic level... would you like...say, Golden Phoenix, to review every single game for NWR Stogi?
Heck, and even then, a person might change their mind or mood or who knows what.
Take, for example, my movies collection. I have a whole range of movies of all different genres. Some of them I will readily admit are better than others. But the "best movies" aren't necessarily the ones I always pull out to watch. If I think a movie is worth four stars, but watch a two star movie way more often, does that make my personal evaluation of their respective qualities invalid? Or is quality of movie a separate issue from how much utility you can get from it?
Should a "quality rating" refer to end-user utility? To artistic value? To a lack of defects? Should it account for innovation? Uniqueness? A comparison to other efforts of its sort? Should it account for mood, taste, temperament, politics, social atmosphere, timeliness? Heck, should contributing factors like packaging or special features play a role? Say yes to more than one of these and already the equation for evaluating something becomes inherently complex and inherently unstable, especially over any extended period of time, even for just one person.
Post by: GoldenPhoenix on January 06, 2009, 06:12:14 AM
Quote
Golden Phoenix, to review every single game for NWR Stogi?
Yes!
Post by: KDR_11k on January 06, 2009, 06:50:04 AM
Post by: Stogi on January 06, 2009, 06:56:32 AM
Of course, to have one person review everything is highly inpractical and serves little purpose for those who disagree with their stances on a basic level... would you like...say, Golden Phoenix, to review every single game for NWR Stogi?
I may disagree with GP on a lot of issues, but not games. I understand what you mean and yet, you don't. Even if I disagreed with GP's reviews 100%, I still know where she stands. So if she says a game is horrible, I'll be sure to check it out.
Quote
Take, for example, my movies collection. I have a whole range of movies of all different genres. Some of them I will readily admit are better than others. But the "best movies" aren't necessarily the ones I always pull out to watch. If I think a movie is worth four stars, but watch a two star movie way more often, does that make my personal evaluation of their respective qualities invalid? Or is quality of movie a separate issue from how much utility you can get from it?
Uhhh, we're in agreement. Just cuz a movie might be of a better quality, it doesn't mean that it's your favorite. Your still being you, and thus congruent.
Quote
Should a "quality rating" refer to end-user utility? To artistic value? To a lack of defects? Should it account for innovation? Uniqueness? A comparison to other efforts of its sort? Should it account for mood, taste, temperament, politics, social atmosphere, timeliness? Heck, should contributing factors like packaging or special features play a role? Say yes to more than one of these and already the equation for evaluating something becomes inherently complex and inherently unstable, especially over any extended period of time, even for just one person.
Ok, I see where you went with this. And your proving my point. Giving a game or movie a number or letter is flawed from its conception. However, it is still less flawed when one person shares their opinion on everything, rather than many people sharing their opinions on somethings. No matter how much your opinion changes over the years, it's still ONE opinion.
QuoteGolden Phoenix, to review every single game for NWR Stogi?
Yes!
LOL
Post by: shammack on January 06, 2009, 07:54:37 AM
I still support the ice cream flavor-based scoring system.
Post by: KDR_11k on January 06, 2009, 08:53:02 AM
Post by: Kairon on January 06, 2009, 01:06:59 PM
First of all, calm down. I guess we didn't agree after all? Now, where do I begin?
Huh? Don't worry Stogi. I wasn't arguing with you. I was just elaborating on further points!... I can see how it's confusing because I added the first paragraph about Golden right before the post went up. Originally, my post started with "Heck, and even then" in which case it would be much clearer that I was agreeing with you and adding onto your statements.
Of course, to have one person review everything is highly inpractical and serves little purpose for those who disagree with their stances on a basic level... would you like...say, Golden Phoenix, to review every single game for NWR Stogi?
I may disagree with GP on a lot of issues, but not games. I understand what you mean and yet, you don't. Even if I disagreed with GP's reviews 100%, I still know where she stands. So if she says a game is horrible, I'll be sure to check it out.
Hmm.. yeah, I can see what you mean. That's how I am with movie critic Mick LaSalle: when he reviewed a movie I was able to tell if I'd like it or not independently of whether he liked it or not. Sometimes we agreed. Many times we didn't. But agreement never mattered, just my trust in him to write all I needed to form a preliminary judgement.
But this brings me back to my earlier post in this thread about deconstructionism. I find that I can relate better the better the reviewer manages to deconstruct the game dispassionately in their reviews. Even if I can translate a reviewer's opinion to my different tastes, I need to be able to trust them to have detailed all the aspects of a game appropriately that I can have specific reasons as to why the reviewer might have not enjoyed a game, but where I might, or vice verse.
If they don't do a good job of offering specific examples over which we may agree or disagree, I find that merely knowing how I differ from that person isn't very helpful in formulating my own stance.
Of course, I find this whole happy situation to be rare. I don't know if Mick LaSalle still writes reviews for the SF Chronicle.... all the reviews I read in Sacramento are from other people, and even in the SF Chronicle there were at least two other move reviewers. Like I said before, it's simply not practical to have one person do everything.
Post by: Peachylala on January 06, 2009, 01:20:46 PM
HOWEVER, if you guys do change the rating system for your reviews, please PLEASE keep the Pros and Cons. I love those, it gives un-needed sarcasm and I love that! ;)
Her Starfox Adventures review was great. Vote for GP everyone!QuoteGolden Phoenix, to review every single game for NWR Stogi?
Yes!
Post by: GoldenPhoenix on January 06, 2009, 01:28:38 PM
Hmm... from a previous comment made about the VC rating system, I see that it wouldn't work for normal game reviews (which is: Wii, WiiWare and DS games). Oh well, I like that system because it's right to the point.
HOWEVER, if you guys do change the rating system for your reviews, please PLEASE keep the Pros and Cons. I love those, it gives un-needed sarcasm and I love that! ;)Her Starfox Adventures review was great. Vote for GP everyone!QuoteGolden Phoenix, to review every single game for NWR Stogi?
Yes!or not
Ewww, I didn't like that review. I really should review some other things, I think the last game I ever reviewed was Zelda: Wind Waker.
Post by: KDR_11k on January 06, 2009, 02:01:20 PM
Post by: GoldenPhoenix on January 06, 2009, 02:15:13 PM
I'm not sure deconstructing a game is all that helpful. Sure, you need a description of what it's about but probably not every single mechanic involved (just the ones that affect the quality), a review is not a manual.
I actually agree with that. A review should answer points that people may have but have more focus on how well the pieces fit together. Also I like second opinion summaries as well.
Post by: Halbred on January 06, 2009, 03:05:14 PM
Post by: Kairon on January 06, 2009, 03:15:21 PM
Post by: Justin Nation on January 06, 2009, 03:18:28 PM
To add even more complication and depth to this discussion though it is all difficult stuff, especially as you then introduce other factors, like distance. This is something on 64 Source we specifically (for a time) tried to address even with reviews 1 - 2 years after the fact of a same major game we'd reviews. How does an inspirational and incredible game of the moment given accolades out of the gate hold up over time? See something like Super Mario Brothers 3 (eclipsed but still an excellent example of the genre) against something like Goldeneye (loved the game originally but practically worthless beyond nostalgia now as it has been so thoroughly eclipsed). Some games, like movies, or TV, or books, or whatever age like fine wine... others you find have turned to Boones Farm or vinegar. The moment can make a big difference. So that introduces a new dimension. Is it fair to continue to hold something like Goldeyene (really, I loved the game, not bashing it, just it is a great example of a game great at its time more than all time) in as high esteem as other games with the same score that have aged more gracefully? Hell, perhaps games with lower scores that have aged better deserve more credit than GE just for their staying power. So even the power of MetaCritic and other score aggregation sites have only the power to rate games in terms of the here and now. This is especially an important reason to get solid VC and other reviews. Perhaps what was terrific then is well ported and just no longer relevant.
Another way to pee in an already tainted pool (and what my old Editorial really dealt with more) is context. I think moreso than numeric scores, lack of truth in score breakdowns (how many reviews do you see where a 8 game gets 4 sound but 10 graphics or what gave you... I swear 90% or more of reviews all breakdown scores will fall within +/- 1 point of the main score and themselves... is there truth in that?), or other bugaboos is context. Perhaps moreso than ever with the introduction of the Wii and the resurgence of "casual gaming" I really think, though it would make aggregation sites squirm, that game reviews almost need MORE scores.
I think in the old editorial it fell to series (if applicable), genre, and general audience... but perhaps now we need series (if applicable), genre, casual audience, and hardcore audience. Why? Because first if you're not a fan of the series or genre the score needs clarification. Is it transcendent (perhaps ala Smash Brothers for "fighting" as an example)? Is it a great example of the genre but unlikely to convert non-believers? Is it essential to someone who loves the series but still the same core game? Perhaps great for the series lover but a negative to someone outside. So context even in that sense DOES matter and one score for the game utterly fails to take that into account. But look at the Wii. Carnival Games? To the hardcore somewhere between tolerable and absolutely craptacular depending on your tastes. To casual gamers like my relatives (and based on sales, many many more people) who overlook the suspect graphics, dodgy controls, kiddie-pool shallowness... perhaps not the greatest thing since sliced bread but I will tell you that they play it on a regular basis to this day.
Blah, time to split but like I said, I hope to talk about it more with some ideas and once things roll may well put up a revised version of the old Editorial (that I sadly can't find, of course) that explores this problem. Thanks for a compelling discussion folks.
Post by: KDR_11k on January 06, 2009, 04:13:49 PM
Post by: Justin Nation on January 06, 2009, 05:09:58 PM
So to go from being an average reviewer to a GREAT reviewer it is actually a pretty difficult road. First you (or really, ideally, the site itself) should establish a baseline "audience" the review is geared to in order to ensure that across reviews it is consistent. Then, perhaps, additional content could/should be added to the review to make it clear how you think "wearing another hat" would impact the review. Granted, this nuance would be lost on aggregators but since you established a baseline audience and are consistent towards it at least the primary view is worthwhile. For nuance the aggregator sucks anyway, so screw em.
Thoughts?
Post by: GoldenPhoenix on January 06, 2009, 05:17:09 PM
Precisely... an interesting phenomenon if you begin taking into account the genre or series. On the whole the newest Madden may truly be a great game on its own stand-alone merits. However, for people who have followed the series year after year maybe the purchase is less compelling overall since little was added from the previous year. The score should then somehow mean something different things to different people, almost being "qualified" up or down based on some other criteria. So maybe on a given game you'd get an 8 overall but to fans of the series you call a -1 for lack of differentiation. The question then would be what hat does the default review primarily reflect? Probably you'd need to be in the mind of the "mainstream gamer" which I would wager is neither casual or particularly hardcore. That is the "fair" score. Thing is, read people's positive and/or negative reviews of many games and you get a hodge podge. You get rabid fanboys of the series eating the game up despite its shortcomings (*cough* Final Fantasy games for the most part), you get jaded former fans bemoaning a shift they didn't like (see "Cel-da" criticisms for Wind Waker), or you get people who admit they aren't fans of a genre but reviewing a game anyway (perhaps Wii Music is a good example).
So to go from being an average reviewer to a GREAT reviewer it is actually a pretty difficult road. First you (or really, ideally, the site itself) should establish a baseline "audience" the review is geared to in order to ensure that across reviews it is consistent. Then, perhaps, additional content could/should be added to the review to make it clear how you think "wearing another hat" would impact the review. Granted, this nuance would be lost on aggregators but since you established a baseline audience and are consistent towards it at least the primary view is worthwhile. For nuance the aggregator sucks anyway, so screw em.
Thoughts?
It is nice to see relatively new posters actually use paragraphs and are articulate!
Post by: Justin Nation on January 06, 2009, 05:26:20 PM
It is nice to see relatively new posters actually use paragraphs and are articulate!
Uh, if you would review... in theory I may be one of the oldest possible posters there would be, just not recent. ;D
Post by: GoldenPhoenix on January 06, 2009, 05:26:59 PM
It is nice to see relatively new posters actually use paragraphs and are articulate!
Uh, if you would review... in theory I may be one of the oldest possible posters there would be, just not recent. ;D
I know that, but you don't post enough to be old. :P
Post by: Ian Sane on January 06, 2009, 05:45:52 PM
Quote
But look at the Wii. Carnival Games? To the hardcore somewhere between tolerable and absolutely craptacular depending on your tastes. To casual gamers like my relatives (and based on sales, many many more people) who overlook the suspect graphics, dodgy controls, I am a terrible poaster. -pool shallowness... perhaps not the greatest thing since sliced bread but I will tell you that they play it on a regular basis to this day.
With something like Carnival Games do people like it because it's a good game of its type and caters to its target audience perfectly or because they're just unaware of something better? A few years back I read an article by Roger Ebert about kids movies. As an adult they can be challenging for him to review but some kids movies are good enough that an adult can like them while some are so lazily made that you can tell the studio is intentionally half-assing it on the basis that kids won't notice how bad it is. I can't remember what movie he used as an example of good kids film (probably something by Pixar) but he shared a story about a conversation he had with someone in an elevator. The person was talking about taking his kids to one of the Scooby Doo movies. Ebert said that Scooby Doo was a very poor film and suggested the guy take his family to the good movie instead. He said they will likely enjoy that film more and probably will be disappointed by Scooby Doo. The guy blew him off.
I see a parallel between kids films and non-games. That's not necessarily an insult, it's just that both are examples of something that critics may have a hard time reviewing because they're targeted at an audience that will not have the knowledge and experience of the critic. Quality still matters. A kid doesn't have to see crap movies and a non-gamer doesn't have to play crap games. So I think a harsh review for something like Carnival Games is deserving. It's a poorly made product and even if you're the target demo there are well made non-games that you will enjoy as well as some well made games. You likely will enjoy these other options MORE you just don't realize it.
Ignorance isn't an excuse for poor taste and you don't deserve to be swindled by companies that know you don't know better. Now you shouldn't be a snob either but a critic should be able to point out that something sucks even if the target audience may end up liking it in ignorance.
Though the important thing is who reads reviews? It isn't the people who buy Carnival Games. When some really horrible film is number one at the box office all those people that saw it either didn't read any reviews or dismissed the reviews because "critics don't know anything". Reviews might as well be written for an audience that likes to make informed decisions as they're the only ones that will pay any attention to them.
Post by: Justin Nation on January 06, 2009, 06:18:32 PM
With something like Carnival Games do people like it because it's a good game of its type and caters to its target audience perfectly or because they're just unaware of something better? A few years back I read an article by Roger Ebert about kids movies. As an adult they can be challenging for him to review but some kids movies are good enough that an adult can like them while some are so lazily made that you can tell the studio is intentionally half-assing it on the basis that kids won't notice how bad it is. I can't remember what movie he used as an example of good kids film (probably something by Pixar) but he shared a story about a conversation he had with someone in an elevator. The person was talking about taking his kids to one of the Scooby Doo movies. Ebert said that Scooby Doo was a very poor film and suggested the guy take his family to the good movie instead. He said they will likely enjoy that film more and probably will be disappointed by Scooby Doo. The guy blew him off.
I see a parallel between kids films and non-games. That's not necessarily an insult, it's just that both are examples of something that critics may have a hard time reviewing because they're targeted at an audience that will not have the knowledge and experience of the critic. Quality still matters. A kid doesn't have to see crap movies and a non-gamer doesn't have to play crap games. So I think a harsh review for something like Carnival Games is deserving. It's a poorly made product and even if you're the target demo there are well made non-games that you will enjoy as well as some well made games. You likely will enjoy these other options MORE you just don't realize it.
Ignorance isn't an excuse for poor taste and you don't deserve to be swindled by companies that know you don't know better. Now you shouldn't be a snob either but a critic should be able to point out that something sucks even if the target audience may end up liking it in ignorance.
Though the important thing is who reads reviews? It isn't the people who buy Carnival Games. When some really horrible film is number one at the box office all those people that saw it either didn't read any reviews or dismissed the reviews because "critics don't know anything". Reviews might as well be written for an audience that likes to make informed decisions as they're the only ones that will pay any attention to them.
Hehe, I wouldn't defend Carnival Games. I'd still make sure a site score for it (assuming we're talking "average mainstream gamer") pretty bad. Despise the game. Shallow. Crap. Now, for a "casual gamer" I don't think Carnival Games suddenly becomes high art. I do think, however, that the criteria for depth and challenge in the game need to be reconsidered, and thus the score could be qualified, some, for the casual crowd who generally are less concerned with depth and challenge. Still, we are polishing a turd (the Mythbusters proved it could be done afterall)... so maybe with that it could earn an extra point to be fair. How much something would gain or lose would really depend. Well, and also your score could bring it down a point (or maybe even more) for Hardcore gamers, heck, maybe even give it the 1 it deserves for that crowd. You see though, the review would be smarter and stronger because it would speak to each type of gamer and give them advice rather than that type of gamer needing to try so hard to infer or read between the lines.
As for who reads reviews for something like Carnival Games I'd wager not too many but at the same time, given the outright scorn the game generally got from most sites who weren't reviewing for the casual crowd in the least, what incentive do they have to look? Of course we'll hate it, but that doesn't really help them because we're making no even passing attempt to see it in the light they would look at it in. We could still be wrong trying to qualify but at least with the attempt to connect with them we're doing a service that would make us unique... and that could drive hits, relevance, etc. I just see a major problem with reviews being the theory that they're somehow written for everyone when they couldn't possibly be. Ideally someone would at least try to address this, and the community could be better for it.
Post by: GoldenPhoenix on January 06, 2009, 06:34:07 PM
The hatred for Carnival Games is odd too because the game did NOT get bad reviews, it got average to slightly above average.
Post by: Halbred on January 06, 2009, 07:06:42 PM
And Ian is right--who reads reviews? My friend, who I discuss in the opening paragraph of the blog, does NOT. He goes to me, or the back of the box. He's not going to look up Far Cry 2 on IGN. I think a lot of people are like that. I'm not writing for those people, necessarily, I'm writing for people who actively seek out information on a game they're considering purchasing.*
*Which leads me to wonder why we bother reviewing games like Homie Rollerz.
And as gamers, we share a common language. When I read about a fellow gamer's experience with a game, in some vague way, I share it. And that narration lets me know whether I'll enjoy the game or not. A final score or a "parting shot" is a nice wrap-up, but you should be reading the review, not looking at the numbers.
Post by: GoldenPhoenix on January 06, 2009, 07:11:14 PM
See, but now we're all getting into MORE scoring territory. And we seem to be giving scores just for aggregators. Like Justin said, screw 'em. My review for Phantom Hourglass is but one of MANY on an aggregator, and I doubt my 7.5 shoved the average down to back-breaking levels. While I understand that aggregators exist, we as reviewers have no control over that, and we shouldn't write with aggregators in mind.
And Ian is right--who reads reviews? My friend, who I discuss in the opening paragraph of the blog, does NOT. He goes to me, or the back of the box. He's not going to look up Far Cry 2 on IGN. I think a lot of people are like that. I'm not writing for those people, necessarily, I'm writing for people who actively seek out information on a game they're considering purchasing.*
*Which leads me to wonder why we bother reviewing games like Homie Rollerz.
And as gamers, we share a common language. When I read about a fellow gamer's experience with a game, in some vague way, I share it. And that narration lets me know whether I'll enjoy the game or not. A final score or a "parting shot" is a nice wrap-up, but you should be reading the review, not looking at the numbers.
Well like I said it is silly to pigeon hole readers. I have no doubt I am not the only so called "core" gamer that enjoys more casual games as wel. So my best advice is to write to whoever you want and chances are you will catch someone's attention. BTW your review on PH was the best.
Post by: Ian Sane on January 06, 2009, 07:18:40 PM
Quote
still get lots of fun out of games like Carnival Games, Wii Music, Animal Crossing, and various other more casual focused games.
You are being so MEAN to Animal Crossing by putting it with that company. ;)
Post by: Kairon on January 06, 2009, 07:25:11 PM
It is nice to see relatively new posters actually use paragraphs and are articulate!
Wow, you managed to get on GP's good side!
Hehe, I wouldn't defend Carnival Games. I'd still make sure a site score for it (assuming we're talking "average mainstream gamer") pretty bad. Despise the game. Shallow. Crap.
Wow, you managed to get on her bad side too...
J/k
As someone who bought Carnival games wholly on GP's recommendation (and despite the game not catching my interest before really), I must say that it does not deserve the intense negativity it generates. It may not be high art, but it has a carnival micro game appeal... But let's not get into that again...
Post by: Kairon on January 06, 2009, 07:26:48 PM
Quotestill get lots of fun out of games like Carnival Games, Wii Music, Animal Crossing, and various other more casual focused games.
You are being so MEAN to Animal Crossing by putting it with that company. ;)
Yeah, GP. See? Even Ian agrees. Animal Crossing is a CORE game. &P
BTW your review on PH was the best.
...must...resist...arghing....AARRGGHHHH!!!
Post by: Halbred on January 06, 2009, 07:35:13 PM
Screw you, Kairon! Phantom Hourglass can suck it! ;-)
Post by: Justin Nation on January 06, 2009, 08:31:32 PM
Post by: NinGurl69 *huggles on January 06, 2009, 08:34:33 PM
Post by: GoldenPhoenix on January 06, 2009, 08:46:34 PM
Post by: Justin Nation on January 06, 2009, 09:39:02 PM
Post by: Kairon on January 06, 2009, 09:56:47 PM
Perhaps they're good if you want them to be true to the carny rigged tradition of games... :)
Does that mean the game should be lauded as a "simulation?" &P
Post by: Justin Nation on January 06, 2009, 09:58:51 PM
Post by: Nick DiMola on January 06, 2009, 10:26:52 PM
Ultimately, these experiences will serve as a guideline for someone reading your review. Regardless of your personal opinion of the game, readers will be able to take away tangible information about the game and how it actually plays and determine if those mechanics and concepts suit their preferences.
Of course, the reviewer also needs to avoid falling into the trap of becoming too biased, and downplaying positive aspects of the game simply because one aspect of the game truly bothered them. The final score, and any affected subsections of the game should only illustrate that frustration
Post by: NinGurl69 *huggles on January 06, 2009, 10:56:04 PM
nice thoughts
^ too bad that sort of stuff just doesn't happen with today's "media."
Post by: KDR_11k on January 07, 2009, 08:57:56 AM
Post by: Justin Nation on January 07, 2009, 10:44:13 AM
I think we had an argument like that before where I suggested allowing multiple scores with qualifiers for a review that are picked depending on what the reviewer wants to express but people kept saying we should have a fixed set of scores that all reviews should use instead of each review using its own set.
I think the people who disagreed with you are just limiting themselves unnecessarily and pigeonholing all reviews to only be able to count against one consistent set of potential qualifiers would ultimately weaken the point of having them at all. Sort of like how the Wii has suddenly legitimized the "casual audience" as distinct from mainstream and hardcore... where before it came along you wouldn't have had a need. Things change, times change, context changes.
A good example specific to only a handful of games but that would be an excellent qualifier if someone would offer it (mainly because people could truly find it beneficial): A qualifier on a review of something like Guitar Hero Aerosmith/Metallica/etc for fans of the band specifically. I rented it thinking I would love it because I love a lot of Aerosmith's music. Thing is, the mix of what they offered was mostly stuff that I wasn't stoked to play. So I could even see two qualifiers or more.
Fans of Aerosmith's entire catalog: + 2 (they actually did span a great deal of their catalog
Fans of Aerosmith's more current music: -1 (a lot of older stuff newer fans may not even recognize)
Guitar Hero fans neutral to Aerosmith: -2 (aside from the music itself it wasn't terribly compelling compared to the broader GH3)
Now, these qualifiers would need to be accompanied by a very clear and concise paragraph or two for the sake of really knowing why but I think it is a great example of qualifiers that could certainly be relevant at a game level. Just a matter of people being creative and possibly even... having some fun with it. Nah, reviewers are to be taken seriously, no room for that. ::)
Post by: NWR_Neal on January 10, 2009, 05:45:47 PM
I really like Justin's ideas of qualifiers because they account for whatever personal biases exist in reviews.
For example, if I reviewed GH:Aerosmith/Metallica, I would hold a bias against both bands because I'm not a huge fan of either band. However, if you were a big fan of either band, the games should be almost instapurchase.
Likewise with my Sonic Unleashed review (ugh, I don't know if I should even bring it up). For big fans of the hedgehog, that game would have a higher score.
I could go on with some more examples, but I'll just end it by saying I really like the idea of qualifiers.
Post by: Halbred on January 10, 2009, 07:40:17 PM
Post by: NWR_Lindy on January 10, 2009, 10:59:59 PM
ASIDE - We have bios here at NWR, but they're buried and never updated. Our future plans include bringing these bios more front-and-center.
Post by: NWR_Neal on January 11, 2009, 01:50:25 AM
Post by: GoldenPhoenix on January 11, 2009, 01:55:03 AM
Post by: Justin Nation on January 11, 2009, 07:28:15 AM
There is no perfect system. I just like the idea of the work being the responsibility of the reviewer. Go to some lengths to make your feelings plain. Take the time to try to put yourself into another perspective to evaluate the game in a different light, accounting for another person's legitimate perspective.
I think the Aerosmith or Sonic game reviews stand as great examples where the qualifiers help. If you give the game a bad review is it because of the game itself, the license, the lack of originality? Even for fans of the band the game could be terrible if the mechanics of the game are poor. Even fans of the music would be entitled to clearly know this rather than just flatly saying fans of the band will pick it up anyway. No, they'll pick it up anyway because nobody would address the shortcomings of the game in a way that fans of the band wouldn't disregard as someone just not liking the band. Again, I liked the game, I just think the situation it falls into illustrates the problem well.
Trust me, as verbose as I am (as you can see), there is easily a part of me that believes that the work a reviewer puts into their choices of words and turns of phrase should be properly appreciated. Writers put nuance into the reviews to be appreciated and people who short-cut to the score can miss a lot of quality points. Hell, the format of reviews for this very site, where the body of the review only is on the first page and numeric scores need to be clicked through to... it is something I collaborated on. Thing is, as a reviewer the job is to serve everyone, even the chronically lazy or just those in a hurry. So there is something to be said for taking the time to crafting consistent and clear condensed views. Having written in real print I can tell you a full review of a game in 250 words or less is a powerful exercise. I enjoy going the other direction (obviously) but perhaps the best test is to put equal time and effort into both. Qualifiers would just be the direct paragraph and score offset guided right to a legitimate audience, doing them a greater service than the "for the masses" review that addresses both everyone and no one.
Post by: NWR_insanolord on January 11, 2009, 11:00:21 AM
Post by: KDR_11k on January 11, 2009, 11:06:43 AM
Post by: Stogi on January 11, 2009, 01:39:08 PM
Post by: Kairon on January 11, 2009, 04:15:10 PM
Giving a game a numerical score is arbitrary enough, giving it a score based on how you think you might feel if you had someone else's opinions is crazy and getting close to Game Informer Paper Mario territory. Halbred's right, you just have to write well enough that someone reading the review could infer from the text if they'd like it more or less than the reviewer.
Yeah, with numerical scores as arbitrary as they already are, I'm not convinced that we should jump even deeper down the rabit whole with qualifiers.
This is an interesting topic, and I know that for some reason I feel uncomfortable with qualifiers like "add 1 point if X or take away 1 point if Y," but whether that's for a good reason or not, or for any reason at all, I still haven't figured out.
Post by: Justin Nation on January 12, 2009, 02:38:26 PM
Giving a game a numerical score is arbitrary enough, giving it a score based on how you think you might feel if you had someone else's opinions is crazy and getting close to Game Informer Paper Mario territory. Halbred's right, you just have to write well enough that someone reading the review could infer from the text if they'd like it more or less than the reviewer.
I think part of the reason for your hang-up with qualifiers is at the core of the first statement, that numeric scores are arbitrary. Not given thought perhaps but I would argue that a review without a numeric score on it would be much more arbitrary than one with it. I'd say either as a stand-alone could possibly qualify but if all you have is text for a review without any score whatsoever unless you write War and Peace you will fail to have any guarantee of your message getting through loud a clear. Just the use of the word "infer" gets you into highly treacherous territory. I suppose email is then to you a crystal clear mechanism for the exchange of thought and feeling? Couldn't you detect I was sarcastic or kidding or deadly serious or heartbroken just from a jumble of words I assembled with no other indicators for you to go on?
Perhaps my numeric scoring and qualifiers are then the emoticons on the written review. By simply putting down that arbitrary number you convey a great deal of hard information in a definitive way simple words could never hope to cleanly deliver. By that score you know what company the game keeps. By seeing the breakdown scores you know roughly what to expect in terms of sound, visuals, control, everything. Sure, you could waste a lot of words trying to get it across but words in that case are inefficient and, if anything, even less reliable a means of clearly communicating with everyone than even that arbitrary number that is often given more weight than it deserves.
The score is a necessary evil because print and true nuance do not go hand in hand. I could be a poor deliverer, you could be a poor receiver, we could have different societal or experiential perspectives that alter the meaning of what you or I would think are clear and concrete words. Inference lacking a personal connection, a great deal of previous exposure, or some other more concrete factor is doomed to losing the message.
Realize, the qualifier isn't necessarily just an excuse to tack on more points or scores or grades. It is an opportunity to clarify with a numeric changer complemented by text that explains the change. So rather than someone trying to infer my thoughts from my scoring or my scoring from my thoughts I could provide both and attempt to get something that more consistently approaches clarity.
Post by: Kairon on January 12, 2009, 03:06:19 PM
So rather than someone trying to infer my thoughts from my scoring or my scoring from my thoughts I could provide both and attempt to get something that more consistently approaches clarity.
Actually, I think this doesn't increase the accuracy (how true their understanding is) of someone viewing your review at all. It may, however, improve the precision (how consistent their understanding is) of their evaluation of your evaluation. Or not. *boggles*
I also agree that scores are necessary. But I also think there's a lot they don't leave said. I believe that the numerical system lacks the room for personal emotional evaluation, which is a very big reason why I believe werespond well to the rent-buy-avoid system (which clearly defines the expected satisfaction of a consumer's monetary investment).
I've said this before and I've said it again, I loved the SF Chronicles way of reviewing movies back when I read the paper:
-Guy jumping out of his seat
-guy clapping
-guy watching
-guy sleeping
-guy not there
That gives not only defined a clear multi-tiered structure for movies of different values, but it also gave clear and immediate emotional touchpoints. It worked even if you disagreed with a reviewer, because we're all familiar with being ready to fall asleep in a movie that someone else is watching with rapt attention. Finally, it gives a little wiggle room to the score too: the movie is roughly this good, roughly the same as another movie, it's up to you to decide which you like better. The truth is that consumers actually translate point scores to that vagueness as well, not caring how much something is over 80 in metacritic, just that it is over 80.
Post by: NWR_insanolord on January 12, 2009, 03:31:22 PM
Post by: Kairon on January 12, 2009, 04:04:22 PM
Using hypothetical scenarios to modify my review score, guessing what my emotional response would be if I were a different person, is significantly more arbitrary.
I think this is really what I have problems with in regards to qualifiers. Not only are you telling me whether I'll like the game and why, you're telling me how much I'd like it better if I was X, or if I was Y. It sort of assumes you can categorize and pidgeonhole and predict and commodotize readers. And it assumes that certain groups of people are homogonous and have the same tastes.
Just because I fall into category A doesn't mean I like X, or care less about Y, or care more about Z. And how DARE you make that assumption about me.
I'm not mad. &P I'm just prickling at the thought of how people like Nintendo owners, casual gamers, FEMALE gamers, and everyone who's ever been outside of the strictly defined "jhardcore gaming" group have been the subjects of prejudice for ages, and more and more as of late.
Post by: KDR_11k on January 12, 2009, 04:31:17 PM
2. I'm against a scale of buy-rent-avoid because it's impossible to shift (e.g. if I have a higher/lower purchase threshold) and doesn't have a "pricedrop to X" entry (which applies to nearly all games for me, another issue with low range scales since what I consider pricedrop territory would be retail for most and there's no differentiation in that range to tell my pricedrop games from my retail games). Also among the games that are worth buying there are still quality differences, some are more must-buy than others and a reader with a limited budget would not be able to determine a priority.
Generally there seems to be this advocating of less accurate or no scores but the quality range for games is there, it is perceptible and usually when someone asks for buying recommendations you will point out a few must-haves before the rest of the good stuff. If you just slap a 50 entry list of games you consider buyable in front of him he'll either be stumped or start with a random one (or the one with the brand he knows). You can sort a list for scores, you cannot sort a list for review impressions.
Post by: NinGurl69 *huggles on January 12, 2009, 04:37:52 PM
You should've first asked what types of games he likes.
Post by: KDR_11k on January 12, 2009, 04:54:17 PM
Post by: Justin Nation on January 13, 2009, 12:06:53 PM
To tell the truth I think what you would find at the end of all of this is that no matter what system you use or way you implement the scoring/lack of scoring/supplemented scoring much more rides on the quality of the reviewer than any other factor. Any of these systems, with the right reviewer, could work and be of benefit. How many people out there could write at the level to make it work regardless of format? OK, probably not too many. It is some hard stuff. I like to see the passion in here though of people asking the hard questions, thinking through their approach, and wanting it to be a good fit for the readers.
Post by: KDR_11k on January 13, 2009, 01:31:42 PM
Post by: Justin Nation on January 14, 2009, 09:43:27 AM
Post by: NWR_insanolord on January 14, 2009, 10:53:45 AM
Post by: Justin Nation on January 14, 2009, 07:27:25 PM
Technically, with things as they are, they could place something in the ball itself and you could begin to make things painfully precise... or take major steps towards it. Would that make the game better? No. But if accuracy was a concern they would do something about it.
Post by: Stogi on January 14, 2009, 08:52:36 PM
Post by: GoldenPhoenix on January 20, 2009, 08:42:05 PM
Post by: vudu on January 21, 2009, 02:12:04 PM
Actually I kind of like the IGN approach (along with others) where you have the main review along with another opinion.
I proposed NWR adopt this method years ago and was shot down (I can't remember by who). Maybe King Lindy will revisit the idea.
Post by: Halbred on January 21, 2009, 04:23:46 PM
Post by: vudu on January 21, 2009, 04:41:42 PM
Even if the "another opinion" section is added after the publication of the main review, it's still worthwhile provided it's a different take than on the original review. And honestly, it doesn't even have to be added to the official review. It might just be that the other reviewer adds his two cents in the talkback thread.
It just seems like it's something that's worth looking into. I've seen several reviews on the site comment about disagreeing with certain aspects of a review that they didn't write, but they didn't disagree enough to be worth writing another full-blown review. I think having an "another opinion" section would be a happy compromise.
Post by: NWR_Neal on January 21, 2009, 06:19:44 PM
That whole "ship to another staffer" process would still take a lot of time, especially since we're doing this volunteer and shipping takes time.
Something doesn't sit well with me adding content to the review after the fact. I'd much rather have some sort of Afterthoughts in the blog or something. Maybe even review our own review or something like that. I recall Jonny saying something in an RFN about how he wouldn't have given Twilight Princess a 10 now. Hell, in my short time period as a staffer, there's some games that I might second guess my score after some months. For example, Madden 09 really grew on me throughout the football year.
Post by: NWR_Lindy on January 22, 2009, 01:18:04 AM
However, I like the idea of going back and revisiting old reviews, and would like to see that incorporated into something we do in the future.
Post by: TheYoungerPlumber on January 22, 2009, 04:45:42 AM
Post by: Halbred on January 22, 2009, 04:37:30 PM
Post by: Stogi on January 22, 2009, 04:57:17 PM
Post by: Justin Nation on January 22, 2009, 08:44:00 PM
See? There's another problem with numbers. Even the same reviewer wouldn't give the same game the same score after the passage of time. So not only are reviews subjective as to the reviewer, they are subjective as to the time during which they are reviewed.
Ah, but to say that you presume that only the numeric score would change over time and not the body of your opinion... which would be entirely an untrue assumption. If you've warmed to the game it would be reflected in your words, same as your numeric scores. If you've become jaded you'd think it would also enter into their review, not just reflected in the score.
Post by: vudu on January 25, 2009, 06:58:55 PM
I argue: Why stop there? Since you're still playing the game six months later why not post a few paragraphs about your after-thoughts? Do you still agree with the score you gave it? Would you change anything about your review if you were writing it now instead of in August?
Post by: NWR_Lindy on January 25, 2009, 07:37:26 PM
Post by: NWR_Neal on January 25, 2009, 07:48:01 PM
Like Lindy said, we've got other priorities but I still want to see if I can go anywhere with it.
I read over my review before bumping it and I surprised myself by agreeing with my original score and text. Eventhough I enjoy the hell out the game, it's still an 8 in my book.
That and Mario Super Sluggers are fantastic games that just needed a few things added to make them better. The most grevious error Sluggers had was the lack of a season mode; Madden has a lot of control hiccups. I still love playing both games.
