Reviewers vs. Users » Forum Post by Draginol

It happens with user reviews and fanboyism as well; reviews that admit there are far more cons than pros, yet because it's part of a beloved franchise it somehow gets a 7 or 8. Huh? 8? Didn't you just spend three paragraphs telling me how much it sucked?

Well, a lot of "final" point scores tend to be an average of scores for a few categories that a game is judged by. IGN and Gametrailers at least clearly let people see this. A reviewer might bash the "story" into the ground over 3 paragraphs because it's worth talking about, but if everything else about the game is great, he doesn't really have much to write about those things to fill 3 more paragraphs with how great they are. So even if it seems like a review is overly negative, in many cases it's just a detailed description of the one aspect that's not up to par, with no complaints about the others. So the average score might still come out high, even though a review sounds negative.

The games that are universally bad in every imaginable aspect usually reflect this in their scores.

Successfully updated karma reason!

I think a lot of it is sample size and self-selection. There aren't nearly as many reviewers as users, so the averages are statistically questionable. At the same time, users are more likely to review games that they enjoyed (or really hated, although my unresearched guess is that negative reviews are much less common than positive reviews), hence the ratings are higher.

Reviewers also have different ranges than users. Reviewers almost never give above a 9.5/10 and almost never give below a 6.5/10. Users have fewer qualms giving 10/10 or 0/10.

Successfully updated karma reason!

Philocthetes

Join Date 10/2003

+153

Quoting DKL,

reply 45

Regarding user reviews objectivity, was reading one site recently and saw some 0/10 reviews for E:WoM based on accusations that Brad and his team have right wing political views. Now THATS a way to judge the game

Re "objectivity," reviews are all about finding someone with common subjective tastes to help you find stuff you like and avoid boring or annoying stuff. The only role "objectivity" might play is in hard-numbers stuff like speaker wattage or video card minimums, and that is always secondary to an interest in something like home sound systems or graphics-heavy software.

Re bashing Elemental because Brad's a 'right-winger,' that's just silly. Stardock has more than a few longtime games forum posters who are various sorts of non-mainstream/right-winger. We might get little or no affection from Draginol or Frogboy, but some of us have various forms of 'stature' in 'the community.' More importantly, I'm close to certain that Brad actually respects the idea of a loyal opposition, unlike too many folks currently claiming to be 'true patriots' or whatever. And I'm not just sweet-talking because I want the Elemental (and hopefully GC3) code to demote currency to a secondary resource in the basic game mechanics.

Successfully updated karma reason!

@OP

As much as people like to claim their view is the majority, there is no way to find out what the player consensus is. So you can't really compare the two.

But if im being less pedantic about the question id say a reviewer opinion is one from someone that wont play the game outside of their job. (If they do play the game outside of their job then technically their opinion is a player opinion.) and thats why there is a difference.

Successfully updated karma reason!

It will be interesting when the big *cough* professional reviewers check out Elemental's expansion (the sold separately one) because by that time the game will be so changed they won't know most of it was patched in over time. Who knows? Maybe we'll see an expansion of the year award like GC2 TotA.

Successfully updated karma reason!

From one quick look at f.e. GameSpots reviews and its users, I cannot trust the users as their "reviews" are 5 sentences long.

A review must mention pros'n'cons and tell you what works and what doesn't.

About professional reviews....I'm looking for a gamereviewersite to become my "home" for gamereviews but it's hard to decide. PCGamer is out of the question because of their "75% from us means 50% from other magazines" <--- Yes they've said that! They're inflating scores and affecting metacritic!

Another thing which I believe is that reviewers MUST mention what types of games they like and what they don't like. In SuperPlay many years ago the reviewers were listed with their preferred gametypes and what they didn't like.

In one number, they reviewed DOOM for Playstation (this was back in 1995.) One of the reviewers, a guy called Martin-san (san is a honorary title used in ancient Japan. He loves JRPGs like Final Fantasy and such) reviewed it and part of the review went like this: "Zzzzzz.....shoot monsters, get blue skullkey, enter blue door, fight more monsters, collect weapons and ammo, get in trap and fight Barons of Hell and Cacodemons very close to you. Die 3 times until you survive. I really hate games like DOOM."

He gave it 42%(!)

The other guy who reviewed DOOM likes shooters and gave it a high score. Don't recall the score but it was what a FPS fan at the time would have given DOOM. Like atleast 85%.

I wished there were more reviews like that since it shows a a review from two different perspectives. One that loves the genre and another guy who dislikes it.

Kinda like if I had to review Baldurs Gate or IceWind Dale. Without help I wouldn't have gotten anywhere and would probably give it like 50% or something.

Successfully updated karma reason!

^^ eh? PC Gamer are hands down the best PC reviewing magazine.

Successfully updated karma reason!

Q:Why does blockbuster titles get high ratings?
A: As long as there's a single rating that has to cover gameplay, graphics, interface, sound, then a game with good graphics and sound, and at least passable design and gameplay otherwise - which the big titles nearly invariably has - is going to have a high score.

Q: Why does reviewer and user scores differ?
A1: Many users will pad stats if they like/dislike a game or something about it, often for irrelevant reasons. (All 0 and many 10 scores.).
A2: Personal preference. Not everyone likes the same game. Some reviewers try to have objective scores to rate a game by to minimize this, but that can often lead to ratings that does not reflect the game. Some reviewers are up front about their likes and dislikes; liking a genre is as dangerous to a score (Especially if it fails to live up to its hype but is still a good game) as disliking it.
A3: Longevity, updates, patches are seldom reflected in reviews, but very often so in user scores.

As some others here I've become less concerned with the actual score and more with the 'meat' of the matter... but it is still important, because if a game has a low score I might not look at it in the first place.

I've come to like GameTrailers' reviews, because they give a decent summary of the strong and weak points of the game, including game footage. Not all reviews are good, but there's been some good ones. (The review of Alpha Protocol, for instance, slammed its combat system but also made it clear WHY - and that reason was one I do not care about, so I knew I could ignore that part of the score.).

Successfully updated karma reason!

Khardis does indirectly raise the good point that one big reason why user review scores will always be lower is because of the 0s. A reviewer who hates a game won't give it a 0. He will give it a 50 or maybe even a 30 if he thinks it's truly awful. But they will almost always give it some points. A user on the other hand will happily give it a 0 if they think it's a bad game, or sometimes just because they hate one feature. This will cause user scores to be significantly lower on average.

Now it's also true that users are more likely to give a 10 then a reviewer. But it's not as big a deal if a user gives a 10 when a reviewer gives a 92, as it is if a user gives a 0 and a reviewer gives a 50.

Successfully updated karma reason!

Quoting FadedC,

reply 59

Khardis does indirectly raise the good point that one big reason why user review scores will always be lower is because of the 0s. A reviewer who hates a game won't give it a 0. He will give it a 50 or maybe even a 30 if he thinks it's truly awful. But they will almost always give it some points. A user on the other hand will happily give it a 0 if they think it's a bad game, or sometimes just because they hate one feature. This will cause user scores to be significantly lower on average.

Now it's also true that users are more likely to give a 10 then a reviewer. But it's not as big a deal if a user gives a 10 when a reviewer gives a 92, as it is if a user gives a 0 and a reviewer gives a 50.

I don't know why all these 0's or 1 stars are such a big deal. You have a polar group doing the opposite thing at the same time, spamming perfect scores wherever they go because they like a game, love a company, or just love the idea and think that one day, some time in the horizon a poor game will become good so they give it 10s now.

Successfully updated karma reason!

Quoting Nesrie,

reply 60

I don't know why all these 0's or 1 stars are such a big deal. You have a polar group doing the opposite thing at the same time, spamming perfect scores wherever they go because they like a game, love a company, or just love the idea and think that one day, some time in the horizon a poor game will become good so they give it 10s now.

They are a big deal because they are much further away from the average score then the 10s. If most people give the game a 7, but 10 people give it a 0 and 10 people give it a 10, then the 0s have a much bigger influence on the average then the 10s do.

In order for the 10s and 0s to have equal impact, the average score would have to be a 5. And because we interpret a 5 to mean awful, very few games get that.

Successfully updated karma reason!

Quoting FadedC,

reply 61

Quoting Nesrie,
reply 60

I don't know why all these 0's or 1 stars are such a big deal. You have a polar group doing the opposite thing at the same time, spamming perfect scores wherever they go because they like a game, love a company, or just love the idea and think that one day, some time in the horizon a poor game will become good so they give it 10s now.

They are a big deal because they are much further away from the average score then the 10s. If most people give the game a 7, but 10 people give it a 0 and 10 people give it a 10, then the 0s have a much bigger influence on the average then the 10s do.

In order for the 10s and 0s to have equal impact, the average score would have to be a 5. And because we interpret a 5 to mean awful, very few games get that.

No, I don't think that's the case. If a game is hitting around 5-6 because its less than average, and you have a bunch of fans throw in 10s and then you have people throwing in 0's... its almost the same. You also have to keep in mind for the sites that have stars, you can't actually give zero stars but you can give max stars.

Successfully updated karma reason!

Quoting Nesrie,

reply 62

No, I don't think that's the case. If a game is hitting around 5-6 because its less than average, and you have a bunch of fans throw in 10s and then you have people throwing in 0's... its almost the same. You also have to keep in mind for the sites that have stars, you can't actually give zero stars but you can give max stars.

Well note that I specifically said that a game getting a 5 was the only case when the scores would be balanced out. Very few games average 5 though, which is why on average the 0s will impact much more then the 10s. A 5 is generally viewed as an F.

Good point about the stars though. 2.5-3 stars is about average, so star systems will tend to be less effected by extreme scores.

Successfully updated karma reason!

Quoting FadedC,

reply 63

Quoting Nesrie,
reply 62

No, I don't think that's the case. If a game is hitting around 5-6 because its less than average, and you have a bunch of fans throw in 10s and then you have people throwing in 0's... its almost the same. You also have to keep in mind for the sites that have stars, you can't actually give zero stars but you can give max stars.

Well note that I specifically said that a game getting a 5 was the only case when the scores would be balanced out. Very few games average 5 though, which is why on average the 0s will impact much more then the 10s. A 5 is generally viewed as an F.

Good point about the stars though. 2.5-3 stars is about average, so star systems will tend to be less effected by extreme scores.

I mean it's not that I don't share your concern. I just think that the people that rate games artificially high are of equal concern as the ones that rate them very low. For some reason fanbois tend to get a pass when haters don't, and really, we could do without either of them in the score board. While I rate games I like fairly high, I don't typically give them max points or stars or whatever and often point out negatives for even games I adore. And I think the average for games should be in the 5-6-7 range with the higher scores reserved for extraordinarily good or bad games.

Successfully updated karma reason!

Quoting Nesrie,

reply 64

I mean it's not that I don't share your concern. I just think that the people that rate games artificially high are of equal concern as the ones that rate them very low. For some reason fanbois tend to get a pass when haters don't, and really, we could do without either of them in the score board. While I rate games I like fairly high, I don't typically give them max points or stars or whatever and often point out negatives for even games I adore. And I think the average for games should be in the 5-6-7 range with the higher scores reserved for extraordinarily good or bad games.

The part about sharing my concern may be where you misinterpret what I say. I'm not expressing a concern or making any kind of judgement about people who rate high or low. I'm just saying that statistically the overall effect of the artificial scores is to lower the average user score. I agree with you that it would be better if the average score was a 5-6, but that is not currently the case.

I think most people tend to view games scores as simililar to academic grades, where anything less then a 65/100 is a F.

Successfully updated karma reason!

Quoting FadedC,

reply 65

Quoting Nesrie,
reply 64

I mean it's not that I don't share your concern. I just think that the people that rate games artificially high are of equal concern as the ones that rate them very low. For some reason fanbois tend to get a pass when haters don't, and really, we could do without either of them in the score board. While I rate games I like fairly high, I don't typically give them max points or stars or whatever and often point out negatives for even games I adore. And I think the average for games should be in the 5-6-7 range with the higher scores reserved for extraordinarily good or bad games.

The part about sharing my concern may be where you misinterpret what I say. I'm not expressing a concern or making any kind of judgement about people who rate high or low. I'm just saying that statistically the overall effect of the artificial scores is to lower the average user score. I agree with you that it would be better if the average score was a 5-6, but that is not currently the case.

I think most people tend to view games scores as simililar to academic grades, where anything less then a 65/100 is a F.

Well I originally said I didn't think the 1 star or bottom scores were such a big deal because there are inflated high scores that counter them. You then said it was a big deal which I took to mean you were making a judgment because you said it was a big deal. In any event, I made an assumption which appears to have been incorrect.

That last bit is certainly true which is unfortunate. I mean a "C" score to me for a game means its fine and some people will love it but maybe it's not so mainstream that gazillions will line up at midnight to get it.

Successfully updated karma reason!

Quoting Nesrie,

reply 66

Well I originally said I didn't think the 1 star or bottom scores were such a big deal because there are inflated high scores that counter them. You then said it was a big deal which I took to mean you were making a judgment because you said it was a big deal. In any event, I made an assumption which appears to have been incorrect.

That last bit is certainly true which is unfortunate. I mean a "C" score to me for a game means its fine and some people will love it but maybe it's not so mainstream that gazillions will line up at midnight to get it.

Ah no I just meant that it was a big deal in the sense that it had the overall effect of lowering the average score. Kind of a big deal from a statistical perspective, but maybe not from a real world one!

For the last bit, that's also why scores can be really inconsistant. There is no one definition for what a particular score means, particularly among users. A score which means average to you might mean awful to someone else.

Successfully updated karma reason!

Quoting FadedC,

reply 67

Ah no I just meant that it was a big deal in the sense that it had the overall effect of lowering the average score. Kind of a big deal from a statistical perspective, but maybe not from a real world one!

For the last bit, that's also why scores can be really inconsistant. There is no one definition for what a particular score means, particularly among users. A score which means average to you might mean awful to someone else.

True on both account. I know metacritic tried to level things out a bit in comparing scores, but yeah we as gamers don't really have a way to have consistent meaning between the scores at this point, user scores or pros.

Successfully updated karma reason!

Tridus

Join Date 03/2008

+110

Quoting FadedC,

reply 61

They are a big deal because they are much further away from the average score then the 10s. If most people give the game a 7, but 10 people give it a 0 and 10 people give it a 10, then the 0s have a much bigger influence on the average then the 10s do.

In order for the 10s and 0s to have equal impact, the average score would have to be a 5. And because we interpret a 5 to mean awful, very few games get that.

They're not a big deal. The problem isn't with the users, it's with reviewers. With the rare exception, an AAA game will get a review between 7-10. Always. The other numbers might as well not exist. For something like Fallout: NV, 7 is considered a failing score by Bethesda.

Users on the other hand actually will give out a 0 if the game is unplayably broken (like say Final Fantasy 14). The review sites won't. And honestly, that game doesn't deserve better then a 2.

Successfully updated karma reason!

Quoting Tridus,

reply 69

They're not a big deal. The problem isn't with the users, it's with reviewers. With the rare exception, an AAA game will get a review between 7-10. Always. The other numbers might as well not exist. For something like Fallout: NV, 7 is considered a failing score by Bethesda.

hehe it might be worthwhile to go back and reread the discussion between me and Nesrie regarding what I meant by "a big deal".

Successfully updated karma reason!

Quoting Tridus,

reply 69

Quoting FadedC,
reply 61

They are a big deal because they are much further away from the average score then the 10s. If most people give the game a 7, but 10 people give it a 0 and 10 people give it a 10, then the 0s have a much bigger influence on the average then the 10s do.

In order for the 10s and 0s to have equal impact, the average score would have to be a 5. And because we interpret a 5 to mean awful, very few games get that.

They're not a big deal. The problem isn't with the users, it's with reviewers. With the rare exception, an AAA game will get a review between 7-10. Always. The other numbers might as well not exist. For something like Fallout: NV, 7 is considered a failing score by Bethesda.

Users on the other hand actually will give out a 0 if the game is unplayably broken (like say Final Fantasy 14). The review sites won't. And honestly, that game doesn't deserve better then a 2.

Yeah I think we talked about this earlier maybe about how you read a review that pretty much says the game is a piece of garbage, the reviewer can't believed it was released it in this state and then give it something like a 7 which completely baffles me because, to me, 7 is an acceptable score.

Successfully updated karma reason!

Quoting FadedC,

reply 65

I think most people tend to view games scores as simililar to academic grades, where anything less then a 65/100 is a F.

I never have. I also can't, since our academic grades are G, VG and MVG and it's being changed in less then a year to have 13 steps.

I wonder if it was american reviewers who started that standard....and now I know it is because of their academic gradesystem....

95-100 A+

90-94 A

85-89 B+

80-84 B

75-79 C+

70-74 C

65-69 D+

60-64 D

55-59 E+

50-54 E

45-49 F+

40-44 F

Correct?

Myself I see it like this:

10 - A masterpiece that got everything right

9 - A truly great game with almost nothing to complain about.

Will edit tomorrow

Successfully updated karma reason!

Simple answer is because most commercial game's journalists aren't really gamers at all, they are , well journalists

I see a trend where even a poor shallow game will get a very high score as long as it's relatively bug free, polished and accessible. In fact I think to be a games reviewer these days, being a "n00b" is some sort of prerequisite.... and lets not get into the big publishers rumored payouts for a good score.

I very rarely find myself agreeing with any critics these days or there scores, except maybe Yahtzee, but he doesnt give scores and even he hates RTS/TBS and fighting (which is my favorite genre's). So I generally don't trust what he says either.

In the end try a demo or rent.

Too Human was a game that was slaughtered in the press, but me and my friends love it. It's not polished or user friendly, but man once you get a hold of the combat system it's a hell of a lot of fun.

Because the general panning I suspect we will never see the sequels and that makes me a sad panda.

As for scores I generally don't like them because people tend to just focus on that. No game is ever perfect so I don't think any are entitled to a 10.

Successfully updated karma reason!

Talking about reviews and perhaps how to manage it a bit better - try looking at www.Boardgamegeek.com that seems to manage things quite well, I wonder what a computer game version of that might look like.

Successfully updated karma reason!

Quoting RFHolloway,

reply 74

Talking about reviews and perhaps how to manage it a bit better - try looking at www.Boardgamegeek.com that seems to manage things quite well, I wonder what a computer game version of that might look like.

I like that website, but it's a mess to wade through.

Successfully updated karma reason!