Gaming Forums => General Gaming => Topic started by: that Baby guy on September 14, 2007, 09:09:14 PM
Title: How do you consider ratings?
Post by: that Baby guy on September 14, 2007, 09:09:14 PM
I was thinking a little bit about numerical ratings for video games. I can't seem to figure out how it works, honestly. There are so many different systems out there, as well as so many different games, and so many different methods used, that I think the industry needs some sort of shock to let people have a clearer understanding of what numerical or other ratings mean.
With movies, there's a similar problem, but since this industry has been around longer, people seem to have been able to earn an understanding. You have Roger and Ebert, with their thumbs, star ratings, and online percentage ratings. Roger and Ebert seem to actually be the least influential, IMO. With a two point system, as well as hundreds or thousands of questionable calls, they just aren't very trustworthy. Besides, one of them bad-mouthed games, too. The next, probably better system currently used is online rankings. Several sites like Yahoo, Fandango, and others gather up professional critic reviews, user reviews, and all the accompanied ratings to provide readers a look at what several think of the movie, as well as a consensus score in a few different categories. To me, this seems like a nice way to go, and the percentages assigned often fall in the same category as school grades do. The third is an older, more well-known system, which uses anywhere from one to four stars to rate a movie. A one-star movie is deemed the worst, while a four-star movie is considered the best. This system usually pairs the star-ranking with a mini-review or info-snippet, so movie-viewers can decide whether the movie and rating is appropriate to them.
Now, I have to ask how video games should be rated. What would be the most efficient way? What do you like? What do you dislike? What don't you understand?
For me, I hate that nearly every website and magazine out there uses a scale of ten or five, and very few have interpretations to what these numbers actually mean. To further muck these up, the reviewers themselves do not have a consensus on what numbers mean. Don't get me wrong. I'm not griping about NWR here. I find the entire industry is in this state. With so many different reviewers under each publication, so many different genres, and so many different sources out there, it seems impossible to learn much about any specific title by its rating. To add fuel to the fire, I've seen stories, heard rumors, and read speculation that sometimes, the reviewer is forced to alter their numbers by employer, without changing the content of the review.
I personally think that the best way to rate a video game might be to adopt something more similar to the star system. A general rating as well as an information quip could be useful. I second-guess that rating, though, because it seems to me that the industry is a little more competitive than what I suggest, and that people need to be able to learn more from the rating than even supplied there. Then again, I think that the star rating would really help to identify where a game stands in its respective genre, and perhaps provoke the interested party to take stock in the written review. It's hard to say, though. The important thing is that this system wouldn't work with numbers, really. It wouldn't fit in with what we see now, and it would take some getting used to. On a side note, I find that this is somewhat similar to NWR's pros and cons section, coincidentally, my favorite part of the reviews here.
I also thought that the percentage needs to be based on a more concrete and less abstract system when it comes to reviews. Most websites are afraid to give games numbers in failing ranges. Anywhere buy NWR, do you rarely see a four, and you never see a three. Perhaps if we considered ratings as more of a ranking. A 10 would place the game at 100%, which would mean that if you were to take every game released on current platforms (or comparative platforms, at least) that the game described would be in the very top percentile of all games. A fifty percent would mean the game is better than half the comparative games out there. A one percent would mean the game is worse than ninety-nine percent of everything that's been released comparably. This would hopefully encourage raters to use the lower spectrums more, and allow consumers a better idea at what is being looked at. The error in this system has to be a lack of subjectivity. If someone loves sports games or football in general, Madden might be in the top five percent, even if it isn't much of a great game. However, if our reviewer does not understand football rules, concept, strategy, or culture, Madden could wind up ranked below fifty percent. Since reviewers are typically just names to the consumer, such a discrepancy could wind up being very confusing, perhaps even more confusing than the way things are now. In essence, each individual's personal taste, as well as the genre of the game, could end up influencing the score, perhaps much more than the actual game's content could. I think that this type of rating would be most effective when paired with a list of scores that other games received by the reviewer. Given this, it could be a powerful concept, though it takes a considerably longer time to learn from this method of scoring.
Anyone else have their two cents?
Title: RE:How do you consider ratings?
Post by: GoldenPhoenix on September 14, 2007, 09:30:50 PM
Very interesting question, personally I like the 10 pt system, but I feel it should be more standarized. One thing I've really respected about Matt at IGN is that his numbers actually seem to mean something. Even if a game gets a 5 it means that the game may still be worth a shot, but don't expect anything more than average. While a 8 or above game means it is good to great. Now if you have a reviewer that sticks to that system (and also adopts a comparison scale like you suggested) I think it would work great. It is when reviewers don't have any standards and while one reviewer may give a game a 5 that could mean they thought the game was terrible, while another could give it a 7 and that is comparable to the 5 of another person.
Title: RE: How do you consider ratings?
Post by: King of Twitch on September 14, 2007, 10:01:07 PM
I also like the 10-point system, here's one way it could work
10: contains some revolutionary or extraordinary aspects that set it above the average game in recent memory. meticulous detail in every facet of the game. good length and maybe a few extra features; a game you want to show your grandchildren. 9: highly entertaining, well-rounded with a few extremely well-done elements 8: minor graphical flaws, maybe a little short, maybe not appealing to everyone, lacks a few features but still worth $50 7: spotty control and/or plain graphics, average overall. but if you like the genre you may find some things you like. not worth full price though 6: few redeeming qualities.. yet strangely may be spoken fondly of in 10 years. 5-0: Acclaim and Activision negative points: non-games
With those parameters you can easily look at the scale and tell which zone a game should lie. Even so, I prefer words words words with every review.
Title: RE: How do you consider ratings?
Post by: that Baby guy on September 14, 2007, 10:01:58 PM
Well, remember that most console titles now retail for $60, too.
Title: RE: How do you consider ratings?
Post by: King of Twitch on September 14, 2007, 10:07:12 PM
Automatic 7 for being over $50
Title: RE: How do you consider ratings?
Post by: KDR_11k on September 14, 2007, 11:04:23 PM
9-10: Great game, there's almost no unfun parts in the game. Many publications refuse to give a 10 (or 100%) on principle so IMO it's meaningless. 8: Good game. Mostly well executed though there may be a few minor unfun parts. 7: Average game (par). Sound concept and all but has a lot of unfun parts. Never pay full price for these! 6 and lower: The unfun is taking up most of the game time. Stay the hell away. 6 may be tolerable for a few minutes but anything below is lulz at best.
This is also consistent with the terminologsy I use (whenever I say good but not great I mean 8/10).
Title: RE:How do you consider ratings?
Post by: Kairon on September 15, 2007, 12:39:34 AM
1. a man in a chair jumping out of his seat clapping 2. a man in his chair clapping 3. a man in his chair watching attentively 4. a man in his chair sleeping 5. a chair with no man in sight
The reason I like that system is because it gives me an emotional base to touch on first, so that I can tell from a glance "this movie will wow me" or "this movie is average" or "this movie will put me to sleep." And it achieves this with infinitely more efficacy than any point or number system!
But aside from liking it for being more emotionally precise a rating system, it also remains vague enough that I can then read the review and get a sense of where the movie succeeds and fails. Reading comprehension is my best tool when tackling reviews, and the best reviewer won't be one whom I agree with, but the one whom I can always understand the viewpoint of. After all, we're all individuals, and I want to know if I will like the movie, not Mick LaSalle (who's awesome, by the way). A good reviewer can explain the experience such that I can easily say after reading a review that "I'll like this movie even though the reviewer didn't and rated it ____" or maybe even say "I won't like this movie even though it was rated _____."
Reviews like that are a joy to read, not because they break movies down into ridiculous areas like "visuals" or "sound" or "storyline" but because they preserve the holistic experience of entertainment, and can be fun just because an active reading of the review is almost like a conversation, the reviewer points out certain things and I get to decide whether I like them or not.
Title: RE: How do you consider ratings?
Post by: that Baby guy on September 15, 2007, 04:09:33 AM
So KDR and MJR, let me ask you this. What's the point of having a 1-5 when they all mean pretty much the same thing?
Title: RE:How do you consider ratings?
Post by: SixthAngel on September 15, 2007, 07:34:56 AM
A five point or five star system is the way to go. In a in a 10 point system anything 6 and below tends to be thought of as worthless. This isn't something that will change through effort of game sites, that is the way things are since middle school so it tends to carry on into these reviews. Look at MJR. He suggests a 10 point review system yet made 5 points equal to zero making the system back into a five point system.
5 points or stars differentiates the review from other concepts like grades and allows a far more reasonable approach to reviews.
Title: RE: How do you consider ratings?
Post by: King of Twitch on September 15, 2007, 08:38:07 AM
Yea it's more of a carryover from grade school and i was too tired to fill the rest in. 5-star would make sense but you couldn't give MK: DS 3.9 stars, it just wouldn't be the same.
Title: RE:How do you consider ratings?
Post by: IceCold on September 15, 2007, 08:48:53 AM
SixthAngel is completely right - while a 5 should be an average game which might be worth a shot, in the games industry that number is 7 instead. It's like in high school where the average of the class was always around 70% - that's carried over into people's mentality.
And that's why sites which try to right the problem by themselves, however honourable it may seem, are doomed to fail because the rest of the industry just won't change. And then the reviewed game has a lower score than it deserves when compared to other games.
Title: RE: How do you consider ratings?
Post by: NinGurl69 *huggles on September 15, 2007, 09:48:16 AM
It's either FIST or it's WEAK.
Title: RE:How do you consider ratings?
Post by: that Baby guy on September 15, 2007, 01:35:52 PM
Quote Originally posted by: IceCold SixthAngel is completely right - while a 5 should be an average game which might be worth a shot, in the games industry that number is 7 instead. It's like in high school where the average of the class was always around 70% - that's carried over into people's mentality.
And that's why sites which try to right the problem by themselves, however honourable it may seem, are doomed to fail because the rest of the industry just won't change. And then the reviewed game has a lower score than it deserves when compared to other games.
See, that's one reason why I think the star-type system is better. People don't write off a two-star movie like they do a 5/10 or a 2.5/5, even though each represents half of a full score.
The problem is that there isn't a right and wrong to making a video game. You can't say a game got 75% of itself correct. That's more of why I disagree with the high ratings some writers give out. And still, I must repeat that I hate it when the bottom half of a scale just seems like a placeholder. For some places, it seems like the scale is from 5-10, which, to me, is just a little sad.
Title: RE: How do you consider ratings?
Post by: UERD on September 15, 2007, 01:55:31 PM
I like Kairon's emotional-response idea. In any case, there should be solid benchmarks attached to each value. I know that not everything can be broken down objectively, but ratings based on comparable criteria can differ greatly between reviewers (or even the same reviewer over a period of time) based on things like changing perceptions and expectations, prior biases, etc.
Also, I find that reviews for niche games are kind of unhelpful. I really enjoyed Baten Kaitos Origins, for example, but that game got scores that were literally all over the board- and all those reviews hinged on the battle system, which was the 'niche' aspect of the game (more specifically, whether the reviewer liked it or not).
Title: RE:How do you consider ratings?
Post by: bustin98 on September 15, 2007, 07:39:18 PM
That niche market remark hits on what I think happens with Wii reviews.
I like the 10 point and I'd like to see the average score be a 5, but I still have a carry-over from school grading systems.
Another good point is why even have the lower numbers if they basically mean the same. In the same train of thought, why have a 10 because realistically no game is perfect, though a few games have elicited that response.
How about this idea: group the scores like:
1 : should be burned 2-4 : poor, almost unplayable 5-7 : average 8-9 : recommended 10 : near-perfect must buy
Where the grouped numbers mean about the same but I'd prefer a 9 over an 8, or a 7 over a 5. So it gives freedom for a reviewer to critically judge a game and at the same time give an idea of how they'd rank it amoung the other similarly graded games.
Title: RE:How do you consider ratings?
Post by: KDR_11k on September 15, 2007, 10:59:52 PM
Quote Originally posted by: thatguy So KDR and MJR, let me ask you this. What's the point of having a 1-5 when they all mean pretty much the same thing?
Dealing with expectations, it's automatically assumed that ~75% is par. Personally I'd use a par, above par, below par scale so the point of reference is no longer x/y but >0 and <0. That only creates confusion though as you can see in my user review of Wario Ware: Touched! In a later review I just wrote my thoughts down instead of using any consistent marks for scoring.
Reminds me, I should review something but I have nothing to review right now... Maybe I should get Tingle's Rosy Rupeeland?
Title: RE: How do you consider ratings?
Post by: Infernal Monkey on September 15, 2007, 11:44:55 PM
Games shouldn't be given a rating at all. It leads to everyone skipping the review text and just whinging about the little number at the end. But!
10 - Avoid At All Costs, Overrated Rubbish 9 - Klax Wave 8 - Number Eight 7 - Not Enough Skins 6 - Pizza Time 5 - Leftover Pizza Time 4 - Stupid Lisa Garbage Face 3 - Video Game 2 - Developed by Square-Enix 1 - Excellent for parties, recommended
Title: RE: How do you consider ratings?
Post by: couchmonkey on September 17, 2007, 06:12:52 AM
The only thing I know for sure is we don't need no 100-point scales. It just leads to confusion over whether Bald Space Marine 5 is really better than Rainbow Platformer 12 because it got a 9.7 instead of a 9.4.
Title: RE:How do you consider ratings?
Post by: that Baby guy on September 17, 2007, 06:44:14 AM
Quote Originally posted by: Infernal Monkey Games shouldn't be given a rating at all. It leads to everyone skipping the review text and just whinging about the little number at the end. But!
10 - Avoid At All Costs, Overrated Rubbish 9 - Klax Wave 8 - Number Eight 7 - Not Enough Skins 6 - Pizza Time 5 - Leftover Pizza Time 4 - Stupid Lisa Garbage Face 3 - Video Game 2 - Developed by Square-Enix 1 - Excellent for parties, recommended
Now, is six for TMNT games, or is it reserved for the little-known sequel to Burger Time?
Title: RE: How do you consider ratings?
Post by: Ian Sane on September 17, 2007, 07:13:07 AM
I find it easiest to understand number ratings if they work like school grades. So above 90% is exceptionally good, 80-89% is quite good, 70-79% is okay but there's a lot of room for improvement and once you start getting below that things start turning to crap. I don't know what schools the guys who say a 5 out of ten is average went to. It must have been one full of really stupid kids. 50 percent sucks.
Typically with games I'm following the titles I'm interested in anyway. I check over the Game Rankings and if some game I didn't expect is getting huge scores I'll check it out. I don't need to read a full review for games I'm not interested in who get crappy scores. And for games I really want to get I only need the "this doesn't suck" confirmation. It gets 8 or higher and I'm interested I don't need to read the review. I only read if then if it gets a bad score that I didn't expect to find out why.
The only problems I see with reviews is problems with the reviewers themselves. Some get wrapped up in hype and overrate. Some have a strong bias and underrate. Some are blatantly paid off to give a good review. But having a score out of 10 doesn't change that. If it was only a write up and we had to read every review that doesn't change IGN getting a bug up their butt about Mario Kart not having skins or overrating Star Fox Adventures.
One sight I visit frequently because it reviews older games is VideoGameCritic.net. Nearly everything is reviewed by one guy who collects games. Each review is about a paragraph and he assigns a letter grade like in school. It works really well. Not a bad format to copy. But I don't think all sites should use this method. The sheer number of titles on his site make the paragraph format ideal but a major site that only covers current games probably should provide more detail.
Title: RE: How do you consider ratings?
Post by: that Baby guy on September 17, 2007, 03:59:29 PM
I have to disagree, though. When you grade based on accuracy, sure, a school grade scale works, because you need to know well above fifty percent to be considered knowledgeable on the necessary material. Game's can't be detailed as accurate. There's no black or white, right or wrong. That's why the grade scale system doesn't work. The good is all lumped into 7-8 points, with too little space to say what's what, and still, not enough people reading the content when they don't know about the game. If they used a smaller system, I think more would at least skim the review, or if a full ten-point scale were used instead of the abysmal pseudo-ten-point scale, that at best uses the top half, things would be more clear.
Title: RE:How do you consider ratings?
Post by: Djunknown on September 17, 2007, 04:17:09 PM
Quote I don't know what schools the guys who say a 5 out of ten is average went to. It must have been one full of really stupid kids. 50 percent sucks.
I believe a magazine called Game Now or something like that (It was one of EGM's sister magazines) tried something like that. I was confused at first, I thought they were really, really tough on games. Still didn't make sense after a second read-through.
I'm a bit partial to the 10 point scale, with .5 intervals. If a game gets an 8.9, what's significantly wrong with it to merit a .1 difference? An 8.5 to me sounds like a great game, but either a minor artistic (re-used textures) or technical glitch (like long loading times) stops it from getting the coveted 9.0. A 9.0-9.5 are the games that's a thrill from begginning to end. Its a textbook example of its genre done right, but not perfectly.
For me, a 10 is when the developer maxes out the hardware. The art is fantastic. The frame rate is consistent no matter what. The localization (where available) is spot on. The voice acting (where available) is so good you want to listen to it again and again. Finally, its accessible. This title has the ability to gain new converts to the genre, or at a base level, detractors of the genre have to respect it. This number is rarely given out, and if its considered, he/she should run it by their peers and editors after an extensive process.
I think we'll never reach that catch-all scoring system, since we all have different standards and tastes, but as long as the reviewer sticks to their review guide, and can back up their reason for said score in a logical, reasonable manner. it shouldn't be too much of a problem.
Title: RE:How do you consider ratings?
Post by: vudu on September 18, 2007, 10:29:30 AM
IGN has both a number and a word rating (e.g. 9.5 & Incredible or 5.5 & Mediocre). Why not just dump the numerical score and simply go with the word? Good and Passable tell you more than a number does (which can be ambiguous and asinine at times). If a game is a good game, say so, rather than give it a 7.5 and force your audience to figure out for themselves whether a 7.5 is a good score or an average score.
Title: RE: How do you consider ratings?
Post by: Kairon on September 18, 2007, 10:45:50 AM
Yeah, like I previously stated, I believe far more in emotional benchmarks for reviews instead of numerical ones.
Title: RE: How do you consider ratings?
Post by: King of Twitch on September 18, 2007, 10:48:01 AM
You get emotional over everything though..
Title: RE: How do you consider ratings?
Post by: Kairon on September 18, 2007, 10:58:43 AM
That's why I believe in it! ^_^
Now, I know that Evan probably hates me for this, but I don't mean to say that we can't objectively look at these things and evaluate them. Merely that when recommending these things to other people, I believe in human emotional intelligence to be effective in describing the entertainment experience.
Title: RE:How do you consider ratings?
Post by: KDR_11k on September 19, 2007, 09:47:38 PM
Quote Originally posted by: vudu IGN has both a number and a word rating (e.g. 9.5 & Incredible or 5.5 & Mediocre). Why not just dump the numerical score and simply go with the word? Good and Passable tell you more than a number does (which can be ambiguous and asinine at times). If a game is a good game, say so, rather than give it a 7.5 and force your audience to figure out for themselves whether a 7.5 is a good score or an average score.
Words don't have a fixed scale, when I say great I mean 90+% but someone else may thing 95+% or 80%.
Title: RE:How do you consider ratings?
Post by: SixthAngel on September 20, 2007, 12:15:27 AM
Quote Originally posted by: KDR_11k
Quote Originally posted by: vudu IGN has both a number and a word rating (e.g. 9.5 & Incredible or 5.5 & Mediocre). Why not just dump the numerical score and simply go with the word? Good and Passable tell you more than a number does (which can be ambiguous and asinine at times). If a game is a good game, say so, rather than give it a 7.5 and force your audience to figure out for themselves whether a 7.5 is a good score or an average score.
Words don't have a fixed scale, when I say great I mean 90+% but someone else may thing 95+% or 80%.
One persons 95 could be another's 96 and one persons 892 could be another persons 899. You have to stop little differences from mattering in an overall review. Especially since the small increments are so individual that they really shouldn't exist and only cause bickering that "game x got .1 higher then game y so it is better." A five point star system or even a 5 word system tells the overall quality of the game. If a person wants to judge the smaller likes and dislikes that vary from person to person they can read the review and get an actual comparison to their own likes and dislikes.
Title: RE:How do you consider ratings?
Post by: vudu on September 20, 2007, 07:22:56 AM
Quote Originally posted by: KDR_11k Words don't have a fixed scale, when I say great I mean 90+% but someone else may thing 95+% or 80%.
That was my entire point.
It makes it so you're not looking at Metroid Prime scores and saying "they scored the first one a 95% but they scored the third one a 92% so the third one must not be as good as the first" and instead you say "both games are great, so if I enjoyed the first one chances are I'll enjoy the second one".
Title: RE: How do you consider ratings?
Post by: KDR_11k on September 20, 2007, 08:03:44 AM
It creates problems when you e.g. rate one game great and another excellent. To me excellent is above great but I've seen it the other way around in the labels on a rating system. Which word is better than which other word? What do the words mean for the play experience?
Title: RE:How do you consider ratings?
Post by: Kairon on September 20, 2007, 08:07:42 AM
Title: RE:How do you consider ratings?
Post by: vudu on September 20, 2007, 08:18:13 AM
Quote Originally posted by: KDR_11k It creates problems when you e.g. rate one game great and another excellent. To me excellent is above great but I've seen it the other way around in the labels on a rating system. Which word is better than which other word? What do the words mean for the play experience?
Title: RE: How do you consider ratings?
Post by: UERD on September 20, 2007, 08:38:05 AM
What if games had 'recommended for' ratings? For example, instead of saying '8/10' (or in addition to saying that), reviewers explicitly recommended the game to target audiences.
For example, 'I recommend game X for sci-fi first-person shooter fans who like a frenetic experience.' 'I recommend game Y for anyone who likes RPGs and adventure games with an extensive plot.' 'I recommend game Z even for people who didn't like the prior game in the series.'
Conversely, 'I don't recommend game A for people who want online deathmatch mode.' 'I don't recommend game B for people who like RPGs with turn-based battle systems.' 'I don't recommend game C for people who are graphics whores.
Title: RE:How do you consider ratings?
Post by: Freyr on October 24, 2007, 09:32:54 PM
I've been think hard about what could be a good rating system. I came up with a mixture of the star and letter grade system(-/+). with a modifier(feelings).
Here are the ranks.
One Star This game is beyond bad, to spend any time trying to comparing one star games, to give them a plus or minus will rot your brain. They are just bad, move along.
Two Star or These games have major flaws but there are some redeeming aspects. If your a fan of this genre, developer, series then give this a game a rent you might like it. No minus rating because if it's that close to a one star, it most likely is not worth saving.
Three Star or or These games have some minor flaws, but overall they are good game. This is were the bulk of games will go, and it's not a bad things. Unless you totally dislike this type of game you should give it a try. Plus/Minus is for the reviewer to show if they game is closer to a two or a four but just not quite making it. "It's barely a three or it's close to a four".
Four Star or This game's flaws are few, it is along the best games for it's system. This is a game that makes it worth buying the system it's on.
Five Star This game is one of the best for this generation, even maybe of all time. To try and rank this game against other five star game is a disservice to this game and other games ranked the same. It stands among the few that will be remembered and played, years into the future. There is no 4star plus or 5star minus/plus, because it would become a pointless pissing match.
The modifier. How did you personal feel when done playing the game. Need more possible answers but among them could be. Short but to the point Really enjoyed it. Angry about the game. Relief that it's over.
Title: RE: How do you consider ratings?
Post by: Kairon on October 24, 2007, 09:35:44 PM
Wow, I really like how you selectively used + and - ratings, making the extreme ends of the scale require a real absolute commitment, as well as suggesting that most games rank towards the middle in a sort of bell curve.
Title: RE:How do you consider ratings?
Post by: Freyr on October 24, 2007, 09:54:31 PM
Quote Originally posted by: Kairon Wow, I really like how you selectively used + and 1 ratings, making the extreme ends of the scale require a real absolute commitment, as well as suggesting that most games rank towards the middle in a sort of bell curve.
I think most rating system put people in the wrong frame of mind, my system most likely does too in a way I don't currently see. So give a shout if you see something that I can change.
A, B, C, D, F or 0%-100% systems are not great because most people went to school with a grading system using one or both of these ways to show how you did. Anything below C or 70% is considered terrible which on a percent scale leaves a huge part a barren wasteland from no use.
0.0 - 10.0 and 0-10 systems leaves a lot of room for nitpicking and most people change the score in their minds to a percent, which can leave a big part unused.
Edit Below: I'm playing Breath of Fire II on the Wii at the moment. It has many flaws but is still a good game. Even for it's time the user interface was a few generations behind compared to other RPGs, but does work. Translation compared to most snes RPGs, is flawed but par, if you fill in the blanks with your imagination it's quite good. Combat options is a little light but there is still some strategy, mainly because the monsters/boss pull no punches. If this game was released with a better interface from even around the time of chrono trigger, a new translation that isn't limited by storage, and give the characters combat options a little more verity, it could be a really good game. It's a game that wants to be better then it is. RankingModifier: Really enjoying it.
If it was released with the above comments it could be a aka a competent game, not everyone will care for it and it won't be a system seller but it will have following that will love it for what it is, a good game.
Title: RE:How do you consider ratings?
Post by: Shift Key on October 25, 2007, 02:27:43 AM
Quote Originally posted by: Professional 666 It's either FIST or it's WEAK.
The rating system was perfected by the Romans. And it resulted in a lot more people being thrown to the lions than occurs these days. Bring back the thumbs up/thumbs down system, I say.
Title: RE: How do you consider ratings?
Post by: vudu on October 25, 2007, 07:21:01 AM
It's the Star w/ Moon/Sun modifiers really just a slightly disguised letter grade? Even if you didn't mean it to be, readers would see a game as a B+. It doesn't solve the problem that anything below would never get used by reviewers or purchased by consumers.
Title: RE: How do you consider ratings?
Post by: Kairon on October 25, 2007, 07:27:40 AM
But Vudu, ANYTHING can be converted to a letter grade if you think hard enough. The trick is to DISGUISE it such that casual readers won't automatically do it in their head. The perceptual gap between a pictorial rating system and a number rating system achieves that, as does the use of only 5 main tent poles instead of 10 (because most casual readers won't/can't/don't do math).
Title: RE: How do you consider ratings?
Post by: vudu on October 25, 2007, 07:47:25 AM
True, but some hide it better than others. 5 stars with modifiers barely even counts as trying to hide it.
5 stars; 5 letters (A, B, C, D, F) Modifiers; +/-
It doesn't take a rocket scientist to translate 5 stars and a moon to an A-.
If you want to hide it better use words. And don't use 5 or 10, because those are easily translated; use 6 or 7. Also, words are a little more abstract and open to interpretation. Most people will infer that 4 stars is twice as good as 2 stars. But is "great" twice as good as "fair"?
Title: RE: How do you consider ratings?
Post by: Kairon on October 25, 2007, 08:02:03 AM
What do restuarants use? 4 stars? And do movies and TV ratings also use 4 stars?
Title: RE:How do you consider ratings?
Post by: vudu on October 25, 2007, 08:34:57 AM
Yes, yes and yes. But I think the whole point of this discussion (and Evan's editorial) is that a review system such as this isn't appropriate for games. A meal lasts an hour and while the cost is sometimes high, you would have had to eat anyway. A movie lasts a couple hours and doesn't cost very much. Television programs are short and generally don't cost you anything extra to watch any one particular program.
Games on the other hand are fairly expensive, especially for kids (who tend to be a major consumer of the product), generally last anywhere from 10 to 50 hours, and require a great deal of commitment on the part of the consumer because they're not passive in nature. They require you to put forth some effort to enjoy. I'd compare reviewing games more akin to reviewing books. Which makes it my turn to ask you a question; what do book reviews use?
Title: RE: How do you consider ratings?
Post by: that Baby guy on October 25, 2007, 10:04:25 AM
No, the whole point is that a 4 star system would work far better for games than the current system.
I don't know what books use for ratings, but if five or six people go to a movie, it's about the price of a game, and more than five or six people play games. The idea is that movie reviews and ratings want you to choose what you like, and don't do the choosing for you. A 2-star movie can be worthwhile to people who like that type of movie, whereas a game receiving about 50% is usually not worth anyone's time from most reviewers, and reviews won't even be read. The idea behind movie ratings is that the lowest rating means it really isn't worth it for anyone, and everything above that caters to an audience somewhere, and you have to decide if that audience is you.
With game ratings, it seems like a game is for everyone, for everyone with money to burn, or for no one, with no real distinguishing categories beyond that that scores can add. That's how I feel about things, at least.
Title: RE:How do you consider ratings?
Post by: Freyr on October 25, 2007, 11:22:40 AM
The uses of moon/sun in my example was just because I was limited by the message board in how I could nicely show the grading system. The -/+ lower/upper modifier could be displayed nicer in real use. Also there is no A- or B+ in the system I talked about before . Here is the system displayed as a rough chart.
Title: RE: How do you consider ratings?
Post by: that Baby guy on October 25, 2007, 11:41:07 AM
If you were to drop the lines, I think that might work alright. I don't like the lines in there, because there's too many ranges with that, IMO. The three star meaning average is nice, I like that. It makes a normal bell curve, in a way.
Title: RE:How do you consider ratings?
Post by: Freyr on October 25, 2007, 01:06:34 PM
Quote Originally posted by: thatguy If you were to drop the lines, I think that might work alright. I don't like the lines in there, because there's too many ranges with that, IMO. The three star meaning average is nice, I like that. It makes a normal bell curve, in a way.
Title: RE: How do you consider ratings?
Post by: KDR_11k on October 25, 2007, 10:02:29 PM
For some reason that rating looks like "rated Major General"
Title: RE: How do you consider ratings?
Post by: oohhboy on October 25, 2007, 10:31:06 PM
You guys do realize that Evan's editorial was well meaning, but fundamentally flawed. Regardless of much he would like to romanize that he is reviewing games for a higher purpose, at the end of the day, a review and it's attached score is a purchase recommendation.
Disguising or encoding the score is pointless and only serves to provide unnecessary confusion. It is all very pretty Freyr, but all that data can be expressed more effectively as a 7 bit number (128). It is a a lot more concise, direct. People my not like how a score should be a point or five higher or lower here and there, but they clearly missed the point. They should have asked themselves whether they were willing to buy the game at full price based on that review.
The score is a useful summing up that can be used at a multiplier against the going price of a game to see whether you might think it is worth your hard earn dollars. When you walk up to that counter to buy that game, you made an assessment as to whether the price your were paying is worth the utility (FUN) gained out of the product.
For those people with bell curve hard-ons, just because a curve does not sit on the magical number 5 make it any less invalid. It may very well be that 75% of games out there are not worth the time of day to play. Scaling it just so it sits center is just window dressing. Every system that some one has come up with in this thread is just more of the same window dressing.
Title: RE: How do you consider ratings?
Post by: UltimatePartyBear on October 26, 2007, 05:58:02 AM
It's impossible to come up with a rating system that people can't assign their own biases to. Maybe it would be a little more difficult if you made up your own terms, like "I rate this game treskelopf out of voondentor," but you'd have to define those terms in order to be able to communicate your meaning, and it is those definitions that people will map to other systems they already know. It would be hard enough for you not to do it for them just to write the definitions.
As for trying to come up with something that means something emotionally, that's just going to cause confusion. If people are free to assign their own ideas to the goofy pictures you use for ratings, they're not likely to assign the same meanings you do, and then you have failed to communicate at all.
Title: RE:How do you consider ratings?
Post by: Freyr on October 26, 2007, 06:40:38 AM
Title: RE:How do you consider ratings?
Post by: Kairon on October 26, 2007, 07:42:11 AM
Quote Originally posted by: PartyBear As for trying to come up with something that means something emotionally, that's just going to cause confusion. If people are free to assign their own ideas to the goofy pictures you use for ratings, they're not likely to assign the same meanings you do, and then you have failed to communicate at all.
If they're looking at the end score, you're already not communicating. The text of the review is for communication. The review score itself, whether it be pictorial, numerical, or emotionally-based, is just a quick-and-dirty touchpoint for those who aren't particular enough to read the actual review anyways.
Oohhboy has it right: review scores are nothing more than a recommendation. And you don't recommend something by saying it got 87% out of 100%. You recommend something with abstract, human-centered, EMOTIONAL terms like "better than average," "excellent," or "meh." I really believe that appealing to, and using, our emotional intelligence will allow game review recommendations to be far more meaningful, and perhaps even convey more information than a dry, draconian, dispirited number system.
Title: RE: How do you consider ratings?
Post by: oohhboy on October 26, 2007, 11:21:52 AM
You are some what mistaken Kairon. It is not only the score that is the recommendation, it is also the body of the review as I mentioned earlier. The score is meant to be a dry counter point to he emotion of the review. Hence it is pointless trying to assign emotional value to the number.
Title: RE: How do you consider ratings?
Post by: IceCold on October 26, 2007, 11:26:38 AM
I like your star plus/minus ratings, Freyr, but the as for the "meter" rating, it would be great for relative scores. However, like oohboy said, a game that is past the centre of the bell curve might still warrant a "not recommended".
Title: RE: How do you consider ratings?
Post by: ThePerm on October 26, 2007, 01:17:51 PM
nice ratings meter! a bell curve makes sense
Title: RE:How do you consider ratings?
Post by: Freyr on October 26, 2007, 05:20:42 PM
Title: RE: How do you consider ratings?
Post by: oohhboy on October 26, 2007, 05:34:06 PM
You guys know what a bell curve is? It is a form of mathematical representation of raw data in to something that humans can easily interpret.
NOT A SCALE.
If I was to plot all the scores from NWR on to a leaf and stem graph, it would form a crude bell curve. I would bet real money that the crest of that curve will not sit in the center.
Title: RE: How do you consider ratings?
Post by: Kairon on October 26, 2007, 05:57:37 PM
So are we now arguing where to put the crest of the curve?
Title: RE: How do you consider ratings?
Post by: KDR_11k on October 27, 2007, 02:23:59 AM
No, he's arguing that it's pointless to complain about that. Some people complain that reviews don't average at 5.0. I've posted my views on the matter in the talkback thread about Evan's editorial.
Title: RE: How do you consider ratings?
Post by: Kairon on October 31, 2007, 08:06:23 AM
Oohhboy in the talkback thread to Svevan's NWR editorial on the subject just gave me an awesome idea!
Quote Originally posted by: oohhboy Rubbish. For as long art has existed, we have applied ratings and scales to it. For things like classical art, there is no limit to that scale. An item can be brought for a million dollars or a hundred million. Is that not an arbitrary value tied to a scale we call money?
... why don't we rate games... using MONEY!
Why isn't the review score a monetary amount that the reviewer says would be a "fair price" for the game? That way we wouldn't have 9.0's or 8.0's... we'd have games that are great for $30 but cost $50, and games that people should and would pay $60 for (Zelda titles for example) but only cost $40 (Zack & Wiki)!
...bad idea? ... yeah, probably... but it's a new one at least!
Title: RE: How do you consider ratings?
Post by: vudu on October 31, 2007, 08:21:41 AM
No cross posting.
Also, money's not an absolute value. If I have more money than you do I may be willing to part with more money for a game I enjoy just as much as you. We're not communists.
Can I get a Hell Yeah?
Title: RE: How do you consider ratings?
Post by: IceCold on October 31, 2007, 11:20:19 AM
Also, price is determined through supply and demand, not through critical merit. Popular games like Madden inherently would have a higher price than, say, Okami, even though they may not be as good.