Gaming Game Scores - How To Fix Game Reviews
Should video game reviews include some kind of overall score? Are they getting abused? Are they useless? Are people not bothering to read the actual review because the final score pretends to provide an adequate evaluation of the game all by itself?

The Point Problem
This topic has been brought up a lot lately with the deluge of game review sites tripping over themselves in an effort to hand out perfect 10's as quickly as possible to the most hyped releases over the last couple of months (Bioshock and Halo 3, I'm talking about you...)

They perhaps understand the dirty little secret of game reviews: Players too often do not use reviews to educate themselves as consumers, but rather to confirm their own (often pre-existing) opinion of a game. In the case of the most heavily-hyped games, most players' opinion of the games are set long before they ever play the game. Chalk it up to successful marketing - so long as the game is of decent quality and has the shiny, it doesn't even have to live up to the hype. By the time the realization sets in that the game really isn't on the level of the Second Coming, the game has already gone through several iterations of its sales half-life, and players are already following the Pied Piper Hype of the Next Big Thing. The reviews that have delayed in order to tell the true and complete story are pretty much worthless by then.

That could be a problem with or without scores, admittedly. But the little "10's" that get doled out like foil-wrapped chocolates on release day for the most heavily hyped games are just the shiny little lures to bring in eyeballs onto advertisement-laden websites. Eyeballs that belong to poor schmucks like me who are stuck at work on release day for a hotly anticipated title. We sit at our desks, counting the minutes to quitting time to go home and enjoy the hours of gaming bliss that has been promised to us for the last 90 days in a multi-million dollar marketing blitz. So we hit the websites to find affirmation that our patience will not be in vain, and we immediately screen out anything below a 90%.

Or something.

Should We Lose Scores Altogether?
GameSetWatch has a feature by Nayan Ramachandran of HDRL on game scores in reviews ... and why they are misleading, uninformative, and arbitrary. And why it is - for all practical purposes - impossible to assign a numerical value to a game's quality with anything relating to objectivity.

A quote on the challenge of meeting expectations in RPGs and general perception of value: "Role playing games, on the other hand, are notorious for lengthy play time, often ranging between 40 and 70 hours worth of gameplay for the same price as a copy of a 8 to 12 hour action game. Should the game's value be judged on how many hours it lasts, or on how much it costs relative to the total play time? Should a game without multiplayer be judged poorly because of its lack of the function, or should every game be judged purely on its own merits?"

What Scores Would Be Meaningful?
Could we live without scores? Without some measurable bottom-line of comparison? I don't know... it seems unnatural to fight against them. We crave those bottom-line evaluations. But maybe it really is the granularity and the pretense at objectivity that is really the problem, not the scoring itself.

Reviewing a game is always going to be a subjective experience. Trying to feign objectivity really comes down to lying to the reader. I know that some sites and magazines have attempted to add together weighted values in all categories for a final score. This is fundamentally wrong-headed, as we have all experienced games that were much more (or much less) than the sum of the parts. And many categories don't get represented. And the "cap" enforced by the system prevents one or two really phenomenally stand-out features from properly ... well, standing out.


Quality, Not Quantity
The industry has painted itself into a corner by rating quantity of features as highly as quality. If two games are comparable in every category, but one game features both multiplayer and single-player game, but the other only offers single-player, it's natural that a reviewer would feel compelled to give the game with "more" a higher score.

They shouldn't.

It's great if a game offers both. The fact that a game may offer both experiences may be meaningful to me, as a consumer. There have been games I passed on because they did not offer multiplayer. I prefer playing RTS games single-player, but I will be unlikely to buy the game if I can't take it online and "prove my skills" against other players, or play it with friends.

But that's not a qualitative difference. That's like making the decision between a sports simulation and a fantasy RPG in my mind. I like fantasy RPGs. I play very few sports titles, regardless of their quality. A game should receive a score based upon how well it executes against its stated or implied purpose.

Now, I would expect a reviewer to perhaps grant some leniency towards a title which attempts a far greater scope. For example, a massively multiplayer RPG having a few problems with polygons passing through each other during battles shouldn't be dinged as hard as a straightforward FPS with the same issues. They should be weighted based upon their impact on the game as a whole.

Lower the Granularity, and Lose The Grading Curve
And the whole concept of trying to compare apples to oranges on some kind of "grading curve" with games is also ludicrous. Just because one game earns top marks should not in any way preclude any other game from reaching such lofty levels if it is inferior or less ambitious in any way. Lower levels of granularity (like the well-understood, well-used four-star or five-star rating system used across multiple industries) removes this temptation.

On the flip side, a higher encourages direct comparison. After all, how do you decide whether a game should get an 87% or an 88%? Too often, this means comparing a game to every other 87 or 88 scoring game --- and the reviewer may not even agree with the score given to the preceding games.

Naturally, the standard of excellence will rise (and possibly even fall) with the change in technology and the gaming culture. Ultima IV and Doom 1 were clearly pinnacles of excellence when they were first released, but if released today they'd not quite cut the mustard for a five-out-of-five. That's okay and to be expected.

Maybe it's because I'm not a professional reviewer, but I have a very difficult time ranking my own favorite games in any kind of order. While it can be an amusing exercise, it's not something I could claim to be definitive in my own mind, let alone for anyone I'm recommending (or dis-recommending) the game to. I personally feel that the difference between an "8" and a "9" is hard to grasp as a consumer or reviewer, let alone the difference between an "8.3" and an "8.6".

Now, I admit that you can get to the point where it might feel too limited, like the Roger Ebert's "thumbs up / thumbs down" approach (and I admitted to chaffing a little under GameTunnel's 3-score system, but that was as much the labels as being limited to three levels of recommendation). But I think once you get beyond four or five levels of recommendation (or not), its just more confusing than useful.

He Shoots, He Scores...?
In the end, I think people like to see the concrete bottom line. While I personally wouldn't mind seeing review scores go away, score-less reviews will only appeal to a minority. But I think we're best served by sites that give a much less granular score, that score each game based upon their own merits rather than feature comparisons with other titles, and leave the details in the text. Let people read to find out if that three-star game is for them or not.


(Vaguely) related drivel:
* Fallout Over the Fallout 3 Trailer
* Indie RPG News Roundup, October 25th
* Game Reviews - What Are They Good For?

Discussion in the Forum, If You Feel Bold...

Labels: Biz

- posted by The Rampant Coyote @ 5:17 AM (Permalink)