What’s the Score?
January 25, 2012 22 Comments
I’m currently writing an FAQ for this site, and one of my answers ran so long I figured I would just break it off into its own editorial. The most common question I get is, well, actually it’s “Are you really a girl?” The answer is yes. At least it was the last time I checked. Hang on, I need to consult a diagram.
I guess that’s what it looks like. Yep, still a girl.
Now then, the second most common question I get is “why aren’t there any review scores here?” I get this one daily and it comes in a wide range of variations. “How come there’s no score?” “These aren’t really reviews. Reviews have scores.” “I can’t tell how good a game is because you didn’t give it a score!” I even get the occasional developer asking me to make an exception and assign their game a score. I usually respond with “fine, it’s a 2 out of 10” regardless of the game’s quality because that number is every bit as meaningless to me as an 8 would be.
When I play a video game, my brain does a lot of things. It thinks about how the graphics look, how the game controls, whether the overall execution is good, and most importantly, whether or not I’m having fun. The one thing it doesn’t do is spit out some arbitrary number that is the sum of all those thoughts. Simply put, I think review scores are total bullshit. I don’t think any game can be broken down into a simple number. Many sites try to do this and the results are usually baffling to me. A reviewer can spend pages upon pages ripping a game a new asshole and then close the review by giving a game a 4 out of 5, or a 9 out of 10, or seven thumbs up, or 11 gold stars.
I think people tend to skip through reviews on professional sites and go straight to the final score. I know this is probably true because I’ve been guilty of it from time to time. But the result is writers all become interchangeable and devoid of any real personality. The only way my site can grow in readership is if my writing is good enough to leave an impression on the readers. If they’re skipping what I write and going straight to a meaningless number, I can’t do that. When you read one of my reviews, you actually get to learn about the game and maybe decide for yourself whether you want to play it. If you care about my opinion on a game, it’s usually not too hard to decipher how I felt about it. A number would actually help nobody, because it doesn’t explain how I felt about a game, or why I felt the way I did.
And really, aren’t scores just flame baiting? Affirmation that your favorite game is exactly as good as you think it is? The way gamers act about scores, you would think they were handed down unto the people on stone tablets from Mount Sinai. It’s just a fucking number, people. When Uncharted 3 was reviewed by IGN, it immediately resulted in two warring factions of dweebs taking turns shouting “I told you so!” or “What a bunch of bullshit!” at each other for days. Mind you, the fucking game hadn’t even come out yet. Not one person involved in this desecration of the human species had played it themselves. Yet within seconds of the review going live, before anyone could have possibly had the time to actually read the damn thing, the fighting was on. The Sony fanboys were rubbing it in the faces of Microsoft fanboys, who were decrying it as IGN’s official jump-the-shark moment. Not one person involved actually knew anything about the game or how the writer came to that conclusion, and they probably never will.
World peace? Hell, we’re ready to shed blood over pre-release game reviews.
Sure, fanboyism played a part in that, but that situation wouldn’t have happened if IGN had the balls to shit-can the whole fucking rating system and just let people figure out for themselves whether the writer actually liked the game or not. So where are the benefits? Consumers become less likely to know if there are aspects of the game that cater to their tastes or not. Developers are less likely to learn what could be improved about their game. I suppose it might in some way benefit me. I could get listed on Metacritic or get the arbitrary number posted on a developer’s website. You know, assuming it’s a good number.
By the way, this isn’t exclusive to the world of video games. Just read any reviews of TV shows on IGN. Even if an episode is a stinker that the writer clearly didn’t like at all, it usually still gets some kind of highish-sounding number. Here’s an example. Read the review carefully. It sure as hell to me sounds like Ramsey Isler thought the episode completely sucked. Yet he gave it a 5.5 out of a possible 10. I actually read through it trying to figure out where those 5.5 points came from. Going off his writing, I couldn’t find them anywhere. Maybe one or two points tops, but 5.5 points came out of someone who wrote that? What the fuck?
Doesn’t that make him kind of lose his credibility a bit? To spend all that time writing about something only to then throw out a seemingly random number is kind of silly. But he did it, and so do lots of critics. Well I don’t really want to do that. At best, a score would leave most people with a vague understanding of how I felt about a game. I could give something a 10 out of 10, but maybe the reasons I got that number are things you wouldn’t like about the game. Or maybe a game I give a 1 out of 10 to had aspects I hated that you normally love. If you look at a number, you don’t get that information. So in a nutshell, that’s why I don’t have scores.
Oh, and because I shamelessly ripoff Yahtzee.
Indie Gamer Chick 
The only reason I ever look at scores is if I want to have a rough idea of the game’s quality without spoilers (note: sometimes I find the specifics of a game’s mechanic a spoiler, Vagrant Story is a good example of this for me), which reaffirms your point about reviews with scores not being read. You should totally put a trilby on that little birdy on the top of the site one day
I barely even play XBLIG games, but I rather enjoy reading your reviews of them. You offer a lot of design insight instead of just being another data point for Metacritic.
Although I’m also guilty of trying to get a quick idea of a game via review score, I’ve found I get much more milage out of a “bottom line” type of paragraph that’s able to wrap up general observations on the game’s design and quality in various areas. If I’m curious about something in it, then I’ll start actually reading through the review, skimming to the parts I’m interested in.
Sadly, it doesn’t work so well for some reviews on IGN, Gamespy, et. al that might seriously criticize one aspect of the game on page one, then on page two decide to wrap up with “Overall, not bad!” And then the game gets a 7.
I used to believe in scores, or at least percentages. I think it was because of my background. Nintendo Magazine System was my game purchasing handbook throughout early-mid 90s – my formative game playing years – and it always gave a percentage score, plus scores for different aspects of each game.
It didn’t trouble me that reviewers like yourself and the Gear Fish gents don’t award them, but I preferred to give scores in my own reviews. Then as I wrote more, I came to the realisation that deciding on a score was more trouble than it was worth.
It was easy giving pointless stool bucket House of Cockroach 20% or the excellent ZP2KX: Zombies & Pterodactyls! 90%, but most games fall somewhere in the middle. They have good points and bad points, and trying to decide whether one flawed-but-generally-ok game was fractionally better than another in order to award them appropriate scores was just frustrating.
When I realised I couldn’t decide which to give a higher score to between a fun but technically rubbish game, and a better developed but less enjoyable game, I suddenly saw the light.
So yes, I think not only are scores not a great asset to the reviewer, they’re not even that useful to the reader. In fact, I’ve found that part of the art of reading a review site is working out how much your tastes overlap. There’s one site where I typically loathe all their 5/5s, but enjoy their 3/5s. Why? Because while they would use a phrase like ‘unrelentingly bleak’ as a criticism, I’d see it as high praise.
While I think (and elaborate elsewhere in this comment stream) that review scores are worthless, rankings are not. While it can be impossible to compare apples and oranges, you might still like oranges better. It IS meaningful that Kairi has a top ten. It is especially meaningful she is willing to reorder and change that list. While it can mean certain death for the game that falls out of the top ten, it does mean something to me that I can look at that list as stating: “These are my favorites, the ones I’m still playing or still relish my time having played.”
That speaks to me whereas : “Graphics 8.5, Audio 9.0, NextGen 8.5, Overall 8.5 GREAT (not an average)” says nothing at all
Oh, and Kairi is definitely a girl. I punched her on the boob and it didn’t fall off. Under lab conditions and everything.
I’m going to need a detailed report on your findings….for science….
Tune in to the next episode of Cops.
NOOOO!!!
Boobs are precious!
Awww…now that the boob has been punched I’ll have to scrap building the “Boob Defense Force 2012” game and start working on “Boob Avenger” instead.
In the world of games (and even other media) scores should go away. They are worthless and pointless. I have had this argument many times. I might point out a game I am really having fun with and some people, while I am talking, go and look up the metacritic score and say “Oh, but it’s only a 65… not getting it”.
What a stupid way to decide on things.
Even today, if I choose to read a gaming rag review (meaning the usual idiots at IGN, GameSpot, etc.) I have learned to tune out the score, the headline, and the talking points (the quick hit leads that try to sell you the game in bullet point form). I read the review looking for any insights into the REVIEWER. I want to know, what kind of games does this person like that this compares to. Maybe I HATE those games. Maybe I LOVE games that you are saying suck. Maybe I will read the game uses a control scheme I HATE or a save system I LOVE even though you hate it.
I read the review for NHL12 from one of these rags. The teaser link calls it “An expertly tailored experience”. The opening paragraph ends with: “Combining all of this with NHL 12’s core gameplay — which is excellent — gives gamers plenty of reasons to drop their hard-earned cash.” Then the next paragraph launches into problems, of which I will merely sample what is a longer list of nitpicking: “There are little touches that make the game feel inauthentic. Your computer-controlled opponents pretty much never ice the puck, go offside, or make other mistakes that are fairly common in any given game.” “Penalties are too rare, but when called, the refs have a single-minded obsession with interference.”
The author loved things I don’t give a crap about. Awesome graphics with all new animations, the actual players sitting on the bench. Fans in the stands. I personally know I don’t give a rats ass about this. I want to feel like I am playing/coaching a real hockey game. Even though the review score is high, I could tell from these comments I would hate this game.
Somehow “These are all minor gripes, but these problems (and others) can compound and detract from NHL 12 feeling as pinpoint authentic as it could have been.” and a “Great” rating belonged in the same review. I have no clue why. I wonder if the reviewer was bought off by EA (my opinion alone and not necessarily the opinion of anyone else) because I just don’t see these statements as compatible.
But that’s the point isn’t it? Scores are WORTHLESS. The careful prose of the reviewer tells me if I want to play a game. I’ve purchased a few games based on this site’s recommendation and my reading of the words. So far, it’s been a reasonable investment for time and money spent ($2 and a few afternoons).
Well, IGN and GameSpot are particularly troublesome because they receive all their revenue from advertising. Who pays the most advertising dollars? Big publishers like EA. Who will withdraw their spending if their games receive bad reviews? The very same.
I sympathise with your experience of people dismissing games. I find it frustrating recommending games to friends. I can explain not only why I like a game, but exactly why a particular friend will like it too, and they’ll refuse to play it because it’s got mediocre reviews. I just gave a detailed recommendation tailored to you! And you know I have similar taste to yours! Grrrr!
It has been many years since I wrote a game review for IGN (see http://pc.ign.com/articles/164/164897p1.html) and I found the ratings/scoring part to be very difficult. Not sure if I fall into the “usual idiot” crowd at IGN or not. 😛 As a reviewer I had no choice but to provide a score since that was their format, but I did try to provide a detailed review as well.
With separate ratings for presentation, graphics, sound, gameplay, and lasting appeal and space for comments regarding each one, I think it’s a good format. Just a simple percentage score without any information on how it was rated wouldn’t be useful, I agree.
I suppose if you’re providing a score at all there’s always the possibility that people won’t bother to read the review, though, as Kairi mentioned above.
Unfortunately I feel that review scores are a necessary evil to get your reviews retweeted and noticed. Sometimes it’s not enough to just have a good write-up. Honestly I couldn’t give two shits about the reader that scans straight to the bottom and reads only the review score. If the score weren’t there, would they bother to read the rest of the article in the first place? I lean towards ‘No’ as the answer.
I’d rather attract readers who want the whole package in terms of a review. A good summary describing what works and what doesn’t and a review score if they’re still on the fence about buying something.
I hope to prove you wrong.
Incidentally, I rather enjoy Giantbomb’s take on it:
A single review score.
3 stars is “meh, if you like the genre/setting/style/characters, you have a decent chance of enjoying the game”
Above that, it’s “good” and “highly recommended”, and below it’s “problematic” and “this game has a deep and abiding hatred for your mental and physical well-being”
If [proverbial] you are going to have a rating system, that’s a good model to follow.
And if not, no big deal.
I like review numbers as a jumping off point. If a game gets a 2 out of 10, I’ll pass. If a game gets 7+, then I will read the full context of the review and decide from that if I want to give it a chance. I like some sort of scores… something here that might work well would be a good / bad / ugly or similar system.
You are all Pavlovian trained to look for your scores, you salivate at the thought of some ridiculous number that was created by by the big game sites in order to skew numbers and sell ads. If you really believe games are art then reviews do not have scores. I have never seen an art show review with a score. That Mona Lisa… 9 outta 10!
It’s the same with breaking out he scores, Graphics 10, Sound 9, Packaging 10, Colors 10, Gameplay 2: Final Score 8.2! Go Buy it! And game publishers see, I sold your crap to the masses so buy more ads!
These are great reviews, indie reviews for indie games.
Good god yes! When I initially started writing reviews for games, I was very deliberate in my decision to NOT using some ridiculous scoring system, for many of the reasons you listed here. It just doesn’t do ANYBODY any favors having a system like that in place. When somebody reads a review, they’re reading it to see whether or not it sounds like something they should give a shit about, but instead they scroll down and see “8/10” and immediately think, “Hmmm… seems decent, but not great. I’ll wait on it.”
… I actually know somebody who has done that on multiple occasions. Doesn’t read a DAMN thing and just looks at the score, as if it’s psychically imparting some kind of irrefutable super-logic straight to his brain, so he has NO need whatsoever to actually read anything about the game. It’s infuriating! You can’t see the pros and cons of a game by looking at a score, therefore you have no idea if you and the reviewer even share the same opinions!
It’s especially bad when the overall score is actually an average of all the sub-scores. Generally a score of ‘8’ is considered “Great.” But if the sound and visuals were atrocious, and there was little to no replay value, the score would get dragged through the mud. My overall experience may have still been GREAT (presentation be damned), but thanks to that score averaging bullshit, that’s not what readers will get out of it.
Anyways, I love that you don’t use a scoring system. Keep NOT doing that shit. Hopefully the more we decry that busted system of reviewing games, eventually it’ll be changed. It’s not much, but sites like this are a start…
I have my gaming blog too and from running Google Analytics on it I know ppl spend an average of 1:30 minutes on it. I know many of you guys have your own websites too and I’d be curious to know if you have similar stats, anyway the point is: ppl on the internet are always in a hurry, that’s the real problem.
Put a score or not, most of your readers will just skim through your reviews at best. How to engage readers, that’s the real trouble. As for myself, I started trying to write very concise and short reviews, three paragraphs maximum, using bold text here and there to emphasize key terms in them, so even the most distract reader can get a glimpse of it.
Of course there could be other ways to keep ppl reading: writing short paragraphs, using a simple style, being funny and witty as Kairi here, employing an inverse pyramid writing style… also, putting the score at the end, maybe it should be put at the start, ’cause putting it at the end means the reader goes to it, sees it and then he’s ready to close the website already.
So to wrap it up, scores are a problem because often ppl just looks at them, so the real challenge IMHO is to make ppl actually read what you wrote.
I often ponder this little issue. My own writing style often leads my previews / reviews to be in the ballpark for 1400-1800 words, which I hear is a little larger than many people can deal with, apparently.
At first I thought maybe I should change my writing style to make my write-ups shorter and easier to digest, but then it just takes me back to one of the things I hate about scoring systems, which is that it DUMBS THINGS DOWN.
I absolutely HATE the idea that people are so completely ridiculous and all over the place that they can’t sit down and READ an entire fucking review for like 5-10 minutes. Seriously, think about what that says about people: They can’t just sit down and read for 5-10 minutes, it’s TOO MUCH for them.
I do NOT accept that. Maybe it’ll damage my potential readership, but I simply do not care. I have things to say, and I will say them in exactly the way my mind forms them. Unless these people are illiterate, they can sit down and read for a few minutes.
Just like with review scores, it’s only ever gonna change if people actually start fighting it. I refuse to blunt my writing, and I don’t think anybody else should either. Big name sites like IGN have no choice but to stick to the industry standards, but we smaller folk, the “indie” writers, we can write however we damn well please, and I think we should stay true to whatever our natural style of writing may be.
I think if your writing is GOOD, people will read it. If you write about something they’re interested in, people will read it. That’s how you get people to stick around for your write-ups.
If you shorten your original writing to the bare essentials, all you’re really doing is making it easier for people to skim over your work.
I write about a game until I have nothing else to say. It might be 1,000 words. It might be 300. All that matters to me is that my thoughts on the game are properly conveyed in their entirety.
Well, writing short posts isn’t the only way to try to keep your readers reading, as I pointed out there are other ways too. I’d just like to add that keeping it short doesn’t necessarily mean dumbing it down, personally I just try to cut all the redundancy and get straight to the point, hopefully also being entertaining meanwhile. To make an example, if I want to review Gears of War 3, do I really need to spend half page recounting all the plot of the saga, knowing that my readers have probably read about it on countless other websites already, and if they haven’t, then probably they’re not interested at all in it?
Anyway my point was that “score yes/score no” is IMHO a false problem, the real problem is: how do I make ppl read what I write? Taking the score out often means that instead of just reading the number ppl just read the final comment or the final paragraph.
And even if we like to think that it’s just dumb ppl that skim here and there, fact is that the internet itself, with its multi-tabbed browsing in multi-tasking environments, encourages “hit and run” reading behaviours.
Wait, what’s with the hate on Ramsey Isler’s review of that South Park episode? Overall, it was a negative review, but he mentioned plenty of things that he either thought were “decent” or that he actually liked and laughed at, and for that the show got at least some points. You don’t have to read it that hard to get that.
Besides, could you IMAGINE if a South Park episode got a 1 or 2 on IGN? People already freaked the hell out over a 5.5. IGNers would’ve lost their damn minds on a lower score.