Let us start with an amusing anecdote then…
Once upon a time, in the land where I live, there was a print magazine covering video games. Its name was 64’ler. People who’re familiar with the semi-famous German magazine 64’er may be excused to think that it was the Turkish version of the same magazine. It wasn’t. It wasn’t half as professionally done and though the articles in it were about the same kind of home computers the Turkish version was exclusively about games. Really… About the most similar thing between 64’er and 64’ler was the logo.
Of course, just like any magazine reviewing games, 64’ler also had a scoring method. They favored the percentage system in which the game would receive a score between 0 and 100. Plus a little percentage sign in front of that number. What that meant was anyone’s guess. We, the readers had easily figured out that the higher the number is, the better the game is… But what does it actually mean when a game scores 80%? Does this mean the game is better than 80% of all the games in the market? Or does it mean that 80% of the game is good and the remaining 20% is bad? One wonders, of course, in case the latter is correct, exactly which 20% of the game in question actually sucks…
When Cinemaware released their latest game “It Came From The Desert”, the reviewers of 64’ler got really excited; and rightfully so… The game was really good for its time. So good in fact that the staff working at 64’ler decided to give the game a perfect score of 100%. The reviewer also explained how this new game from Cinemaware could achieve such a feat. According to him there was nothing bad about it. It simply was perfect. It was the first and the only totally perfect game they have seen. It was both better than 100% of all the games in the market and 100% of it was really good.
But then something ridiculous happened. A few issues later a game called “Wings” was released and that game was also good. In fact the people at 64’ler thought “Wings” was better than “It Came From The Desert”. 64’ler was obviously filled with Cinemaware fans at the time. But back then it really was hard for anyone who owned an Amiga not to be a Cinemaware fan.
For 64’ler, “Wings” was so good that its review score was 101%.
This made sense for them. They have really thought “It Came From The Desert” was perfect. Their belief in its perfection and brilliance was so strong that they have seen fit to write 100% underneath the article which reviewed it. But now that they had seen a game which was obviously better than “It Came From The Desert”, the only logical solution was to give that game 101% percent. The game was so good that it was better than 101% of all the games in the market. I guess that means it was pro-actively better. It already was better than 1% of the games yet to be released… Or that its theoretical levels and missions which do not exist in our space-time continuum are good too.
We did not care… “Wings” was just THAT good. It was awesome. And no scale could judge awesomeness… At least not properly.
Percentage of the Germans
This case does not demonstrate the ridiculousness of the percentage system which is so popular among game critics today. But it’s ridiculous enough to get things started. Of course, 64’ler was a Turkish magazine and Turks are not exactly known for their superior planning skills. So in the hands of people who can plan and make up rules really well, this percentage system should work… Right?
It’s a loaded question of course. It would be the basest form of demagogy if I don’t make it clear right now that I believe the percentage system doesn’t work. I know… It’s not a huge revelation nor does it require anyone to be a rocket scientist to figure out. Still, for the few people who may not be familiar with the subject matter, let us summarize how things work:
A critic reviews the game and assigns it a score between 0 and 100. Maybe 1 and 100, I don’t know. Because if it’s really between 0 and 100, then 64’ler’s “Wings” review actually makes some sense. Anyway… The problem is the way this scale’s been used. The distribution of scores is not linear. Instead it’s like this:
If a game is good it gets something between 90 and 100. You can buy this game without thinking much. In most cases you won’t be disappointed.
If a game is average it gets something between 80 and 90. These games are merely not bad… You can buy them if you are a fan of the genre or if you are really interested in that specific game for some freak reason, but you last dime should better be spent on something else.
If a game gets something between 70 and 80, then it’s “meh”. These games are not really good. Under normal circumstances you should not buy them, but maybe you just like that movie and now you want to play the game, or maybe you’re a fan of the company who made the game, or maybe you’re a fan of the franchise.
That’s pretty much how it works. Oh… What about games with a below-70 score. Well, they’re all crap. They’re so bad that you should not even breathe the air which just so unluckily happens to be in the same room as a below 70-game.
So yeah… This is pretty much how it doesn’t work…
Germans tried fix this in a magazine called PC Player. Not the British magazine. The German one…
The legendary German print magazine PC Player which was published between 1993 and 2001 was intent on fixing the percentage system. Their usage of this familiar system was so different from their competitors that they actually warned the reader every month not to compare their scores with other magazines.
Their first rule was simple and logical enough. The percentage scale gives us a potential score between 1 and 100. So let’s use the whole scale for a change. Let’s stop giving 80 to average games. Instead let us give 50 to that game. Anything above 50 is a good game, for anything with a score above 50 is above average. Anything below 50 is a bad game. But the magnitude of the goodness and badness depends on its proximity to 50. 50 was never a clean cut line. So obviously there wasn’t much difference between 49 and 51. They were both average games. More or less…
But this wasn’t all.
The PC Player system declared beforehand that no game shall ever receive a score of 100, for no game is perfect. Regardless of how good the game is, 100 was just not to be given as a review score.
On top of that, during a calendar year, only a few games should get something between 90 and 100. One or two games… Maybe three… But that was it. These games were considered to be masterpieces; games which would be fun not only for the fans of the genre but also for those who are not fans of that particular type of game. They also received the PC Player Platinum Award. You’d know these games are really, really good.
Then there was the PC Player Gold Award. Every month, only a few games were able to get that, for games with a score between 80 and 90 earned a Gold Award. You also instantly knew that these games are really, really good.
For reference, games with a Platinum Award were top three games of that year and Gold Award games were runner-ups.
Done? No. These are Germans we’re talking about.
Every review had a chart displaying the scores of similar games together with the score of the game in question, so that the reader can quickly compare it to other games in the same genre. Yes, Gran Turismo is a good racing simulator, but is it better than Forza? Are there better games? Is this game my best choice?
All these questions were answered. But see, this represented a problem. Meditate on the problem below:
“Bushido Blade 2” was a really good fighting game back when it was released in 1998. But how does it hold up to “Street Fighter 4” today? Would you rather buy “Street Fighter 4” or hunt for a PSone and a vintage CD of “Bushido Blade 2”?
PC Player solved this problem by extracting 5 points from a game’s score for every year after it was released. So let’s say both “Street Fighter 4” and “Bushido Blade 2” received a flat 90 when they were reviewed. “Bushido Blade 2”s score today would be a mere 20. At least according to PC Player…
But Does It Work?
Fooled you… You all know we’re focusing on the wrong problem here. The problem is not the percentage scale, or our ridiculous stars or letter marks…
The percentage scale DOES work, for the reasons I have stated above. All this info is known to both critics and gamers. Regardless of how “wrong” the language is on a technical level, when both parties can communicate errors in grammar are really irrelevant.
The scores are not the problem. They are a consequence. We get the scores… We all do.
What doesn’t work is the review itself. Not the score which comes after.
David Jaffe looks at all these AAA titles getting 90+ scores and wonders how we’re able to produce so many awesome games?
Are all these games really awesome?
Consider how the movie industry can produce a 90+ hit only every now and then. Check Rotten Tomatoes if you’re not sure about this.
Either our games are really, amazingly good… Or there is something wrong with the reviewing process our critics use.
But before we can even begin to determine what’s wrong with our reviews we should define our subject. We should ask ourselves a simple question.
WTF is a review?