Here’s Why We Should Stop Using Review Scores

Late last year, Jim Sterling made an argument in favor of review scores that I think is worth refuting. As Sterling points out, lots of people like review scores, and the first few major outlets who eliminate their scoring systems will probably see a decrease in their readership, at least temporarily. Given those facts, he argues, why rock the boat?

The thing is, MetaCritic scores are widely (if mysteriosuly) respected, and sometimes used to determine game developers’ compensation. As such, Sterling misunderstands the problem, and also underestimates his own culpability in it. His main assertion is that review scores are not inherently evil, even if they can be put to evil uses. This observation is both unassailably true and face-palmingly beside the point.

22230_screenshots_2011-09-08_00007

Look. Movies get scores. TV shows get scores. But no one takes those numbers half as seriously as we all seem to take game review scores, specifically because there is a widespread cultural understanding that movies and TV shows can’t be boiled down to some set of objective evaluative criteria. By approaching videogame reviews with such reverence, we’re reinforcing the idea that videogames should be approached primarily as technology, as software—that a new Call of Duty is more like a new release of Photoshop than a new Bond movie.

Now, there’s nothing wrong with valuing games as technology, nor is there anything wrong with wanting a review to cover a game’s more objective, more technical aspects—does it run well and render nicely?—and yes, it does make a certain kind of sense to rate those qualities numerically, or with letter grades, or whatever. But the exercise becomes silly once we’re assigning quantitative values to “story” or “fun factor,” or for that matter, to aesthetics as well as graphics.

219890_screenshots_2013-02-10_00003

More to the point, it’s heartless and nutty to address game developers (who make pieces of software that run on computers, yes, but who also make narratives, experiences, and other squishy things) and say, “Hey, this clock you made? It doesn’t function properly,” and feel that those developers therefore should, objectively, be paid less for their labor. Hey, you can’t argue with the numbers!

Paying someone less for their work because that work didn’t sell is the kind of coldly logical capitalism that can be hard for mere flesh-and-blood mortals to swallow, ethically and empathetically speaking—and so wanting to take the quality of their work into account, regardless of raw sales, is a commendably humane impulse, and a potential counter-wight to the often unaccountable cruelties of the market.

But believing that MetaCritic provides an objective measure of quality—wantning the numbers the do your work for you, even though no one really even knows how MetaCritic arrives at those particular numbers—takes us right back into the realm of hardhearted pragmatism, with the additional drawback that (unlike pure, balance sheet-driven, Adam Smith-flavored exploitation) it isn’t even logically defensible or strategically sound. In short, it’s bad, and also stupid.

620_screenshots_2011-05-01_00001

Sterling says “The argument that we should abolish review scores hinges on the belief that all reviewers secretly loathe them, and would jump at the chance to be free of their numerical shackles.”

Eh, not really. The argument is more that, whether you like using review scores or not, you’re doing harm by using them. By attaching a score to your review, you’re knowingly feeding the snarling, shambling, bonus-garnishing behemoth that MetaCritic has become. None of the misconduct outlined in that Kotaku piece is reviewers’ fault, of course, but it’s within their power to stop it from happening. If game reviewers would simply stop affixing scores to reviews, then they’d starve the beast, and that would be that.

Sterling’s own metaphor, over-the-top though it may be, is actually the best way to explain this problem. “If somebody stabs,” he asks, “do we punish the knife, or the psycho holding it?” Here review scores are a knife, and the developers denied their bonuses (for example) are the stabbed, and those who denied them their bonuses (again, just for example) are the madmen doing the stabbing.

But it would follow that reviewers using reviews scores are the ones handing the madmen the knives. A stabbing isn’t the knife’s fault, sure—but if you hand someone a knife, knowing full well that they intend to stab someone with it, then the stabbing is at least partially your fault, isn’t it?

So really, the argument against review scores is: put away the stupid knives, Jim. We know they’re fun to play with, but it’s just not worth anybody getting hurt.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s