The Problem with Online Ratings

by Sinan Aral
published in MIT Sloan Management Review, 2014
Ref SMR55224
 

HThe Problem with Online RatingsWe are inundated with other people’s opinions, observes the author of this thought-provoking article, but is our faith in online ratings misplaced? And is the heart of the problem our herd instincts?

We browse books on Amazon and see other readers’ reviews; we compare hotels based on user ratings, and we look at ‘thumbs up’ and ‘thumbs down’ icons when on YouTube. For the most part, consumers view these online ratings as trustworthy – but are they really?

The author describes an experiment that he and two colleagues conducted on a news website. On the site, users could rate news articles and comments by voting them up or down. The researchers randomly manipulated these scores and the results were striking. The positive manipulations created a positive social influence bias that persisted over five months and ultimately increased the final ratings by 25%. Negatively manipulated scores, meanwhile, were offset by a correction effect that neutralized the manipulation: although viewers of negatively manipulated comments were more likely to vote negatively (evidence of negative herding), they were even more likely to positively 'correct' what they saw as an undeserved negative score.

This social-influence bias, explains the author, snowballs into disproportionately high scores, creating a tendency toward positive ratings bubbles. A positive vote didn’t just affect the mean of the ratings distribution; it pushed the upper tail of the distribution out as well, meaning a single positive vote at the beginning could propel comments to ratings stardom. 

Purchase the full article now

About the author

Sinan Aral is the David Austin Professor of Management and an associate Professor of Information Technology and Marketing at the MIT Sloan School of Management, Massachusetts, US.

Back to list of featured articles

Share this page:

A resource for business

Journal montage
 
We distribute over 8,000 articles published in a wide range of leading management journals and newsletters. 
  • Effective Executive
  • IESE Insight
  • International Commerce Review: ECR Journal
  • Ivey Business Journal
  • MIT Sloan Management Review

Find out more