Okay, maybe I'm ranting a bit here, and maybe I'm being overly sensitive to the strong reaction to our score. But it got me thinking: What are we looking for in our games as professional game critics and gamers? Are we losing our sense of what makes games fun in our chase to be seen as thoughtful and progressive? Or are we so intent on our industry being taken as "seriously" as other entertainment mediums that games that don't push obvious envelopes (other than graphics) can't be lauded as excellent?
I don't think anyone's going to stop you from lauding a game as excellent. What I object to is this continued reliance on numerical scores, as if they mean something. There are so many subjective factors that go into a score that it's pure fantasy to think a single number actually means anything. You're like a drug addict who can't figure out why people complain about your habit.
Instead of scores, focus on concise, direct opinion pieces. Words are infinitely more flexible than numbers, and with them you can express a game's greatness just right. Then you won't ever have to rationalize statements like this:
Is the game [we gave a 10 out of 10] perfect? Not by a mile.If a game isn't perfect, don't give it a score that makes people think it is. Don't give it a score at all.
I actually have to say that I like score based reviews systems as long as the reviews have a non-biased critique method. (A standard rule of excellence should be developed and applied across the board for examining all games against.) Add summarized pros and cons and it's even better for today’s fast paced multi-tasking lifestyles. =)
One point the editor did touch on though that I thought was interesting was the public’s perception of what games should rank in the top percentile.
It reminds me of how much marketing is effecting end user perception about what classifies as a great game... so much so that gamers are programmed to defend their well known brands against lesser known titles.
I don't buy the editors rant against the seriousness and intellectualism of the industry though... that just seems like reactionism in his own head. I really don't understand why some people seem to be so quick to defend mindless glitzy gameplay against 'the ominous threat of creative experimentation'. It's not like mindless games with awesome graphics are the minority...
I would prefer scoring systems that are 1-5, along the lines of movies. Most reviewed games fit between 6 and 10 anyways, plus, a 1-5 scoring system makes a top score (5) less of a "perfect" score and more of a "incredible" score.
So what's the angle on grading students? Seems a useful shorthand to assign grades, though the "10 is perfect" confusion critique obviously would still stand. Was Goldeneye a perfect game when released? Arguably. Fight Night? I don't believe the answer is no grades, but reasonable ones.
Now to expect shorthand to precisely convey the innermost thoughts of a reviewer is obviously luidcrous, but I'd rather know Clayfighter is a 2 (Blue Suede Goo, yo! +1) and Civ IV a 9 when I'm deciding which to rent.
The quick answer on students is that, at least in my field, are expected to reach certain (mostly) concrete goals. Solve this equation, plot that graph, etc. What is the analagous set of (mostly) concrete goals that a game must achieve?
The real trouble comes when you ask several teachers to grade the same student assignment. Unless, as is the case with a teacher I know very well, there is a very detailed point breakdown created for each problem ahead of time, it's going to be subjective. Right this minute I'm looking at a set of papers where each problem has X number of points with no breakdown of how they're to be distributed. I'm positive another instructor would choose to weight the mistakes differently.
What we hope happens is that given sufficient data (graded assignments) that two instructors would come to about the same letter grade in the end.
Upper level classes are harder, where a typical question might be "prove a theorem" or "compute the numerical solution to the differential equation using such-and-such methods". In the former, a student might prove a theorem based on a false assumption (without realizing it). Does that make it all right, all wrong, or somewhere in between? On the latter, one can imagine solving the wrong problem, having a bug in the code, using the right method on the right problem but with an inappropriate step size, and so on. How to measure? That's an individual instructor's call.
That's my quick response. Back to, appropriately enough, grading. Will try to post more later if I can untangle the analogy in my head enough.
to jvm: Ha, glad you're not in the humanities. ;^)
From the article:
"Is the game perfect? Not by a mile. Would we change the score if we could do it again? No."
It occurred to me that this is single game I've played on an Xbox 360 (was on display at the mall a while back). It's far FAR from a 10. Quite boring, replayability for all but a very specialized subset of gamers seems low, and I left thinking the 360 is still a ways away from being a toddler. Reminds me of the posts Matt's been chunking in here about the lack of options for PSP and, well, the 360.
Similarly, when I bought my PS2 there wasn't much worth buying. SSX was a good choice, but Tekken 3 or whatever else I bagged went back pretty quickly. It may have been one of the best available, but that didn't help its grade.
The 360 is, at best, apparently going through a growing stage. This doesn't mean it's time for grade inflation, mind you (nor is it time to toss grades out, as Matt has suggested). It means if a 6.8 or a even a 9.7 is appropriate, well, it'd danged well better be used.
Not real impressed with phrases like, "[readers think a game with a 10] had better be all things to all people while slicing, dicing, and julienne-ing the f#&k out of everything" either, I'm afraid. Call me an elitist subjectifier. I'm down widit.
You know, Roger Ebert doesn't like his own star rating system (which boils down to a 1-9 rating scale). He admits that Basic Instinct 2 may deserve a score better than the 2 1/2 stars he gave it, but the necessity of rating movies relative to each other on a numerical scale sometimes results in strange scores due to the necessity for scores to relate quantitatively to each other. People don't always know how much they like something, and the degree to which you like a set of things doesn't always follow a transitive, A > B > C ordering. I think it’s obvious this also applies to games.
Readers see a 10 as being the same thing as four stars from Ebert: the best score a game can achieve means the reviewer liked it without qualification. It does not imply "perfection," but it does mean that the reviewer saw something there other than "nothing wrong with it." In the words of immortal words of Hobbes (the tiger), good is more than the absense of bad.
By that measuring, game that innovate nowhere deserve average scores even if they are technically accomplished, because while they may have no bad points, they also have no good ones.
I automatically append "...to someone who enjoys that genre" to any score I read. Fight Night 3 may be the most technically accomplished, well playing BOXING game ever made but that doesn't mean I'd rather play it than even an average vertical shooter. A system that allows reviewers to rate the out-of-genre appeal of a title would be nice. Of course, games that mix genres could pose a difficult beast to judge.