Reviews, reviewed: the great ratings hoax

December 12, 2018

It’s 102 minutes of nothing interesting happening AT ALL. What happens in this movie? Is there anything interesting happening at all that anybody can point out to? I don’t think so. It’s just 102 minutes of random boring people talking about random boring stuff. There is nothing to be well acted or well done because there is nothing happening during the whole movie. I think it would be way more interesting to just film random conversations between random people in bars.

There is absolutely zero chemistry between the two main actors. (…) I’ve seen better simulated “attraction” or “love” in porn dialogues. No, forget that, I can detect more love and attraction when two butterflies dance together or I see rabbits mating.

This is an excerpt from a review of Casablanca, a timeless classic with the marvelous Humphrey Bogart and Ingrid Bergman, widely recognized as one of the best movies of all time.

Of course, there’s no pleasing everyone. In the digital sphere, you’ll often bump into finely (and not-so-finely) crafted examples of consumer fury. People asking what’s so great about the Great Wall of China, complaining beaches are too sandy, or, my personal favorite, carping about the Grand Canyon: “Nature is crap.”

While not every negative review is as amusing or articulate as this one, there is something about them that we find appealing. We’re hardwired to pay extra attention to negative bits of information and weigh them disproportionately against what would otherwise be a sea of glowing reviews.

Bad is worse than good is great

“In general, and apart from a few carefully crafted exceptions, negative information receives more processing and contributes more strongly to the final impression that does positive information.”

That is the conclusion at which Roy F. Baumeister, Ellen Braslavsky, Kathleen D. Vohs and Catrin FInkenauer arrived after conducting exhaustive research published in the Review of General Psychology.

From an evolutionary standpoint, paying closer attention to the potential for bad outcomes makes perfect sense. “We believe that throughout our evolutionary history, organisms that were better attuned to bad things would have been more likely to survive threats, and, consequently, would have increased probability of passing along their genes. Survival requires urgent attention to possible bad outcomes, but it is less urgent with regard to good ones.”

More specific analysis of feedback suggests that bad comments have a stronger effect on people’s perception of their own performance than good ones. People are more concerned about avoiding bad than about maximizing good, which points to the greater motivational power of negative feedback.


Two decades in the making

Online reviews first started appearing in 1999, with three major websites: RateItAll, Deja, and Epinions, all of which have been acquired or closed off in recent years. We may never even have heard of them, but their legacy clearly lives on. Today online reviews are so common, and platforms such as Amazon and review fora like TripAdvisor and Yelp have grown so much in popularity, that the phenomenon is extensively studied by market research companies and academics alike. We’re more aware than ever of how much they influence our purchasing habits.

According to data from BrightLocal’s 2018 Local Consumer Review Survey, 86% of consumers read reviews for local businesses, and positive reviews make 68% of consumers more likely to use them. It also shows that 91% of 18 to 34 years olds trust online reviews as much as personal recommendations.

But should they?

Can you really trust online reviews?

On November 1, 2017, six months after first listing The Shed at Dulwich online, the journalist Oobah Butler finally made it onto TripAdvisor’s list of top rated restaurants in London. The only problem? His restaurant didn’t exist.

In an article for VICE, Butler explains how he managed to fake his way onto the most coveted position of TripAdvisor with just a £10 burner phone, reviews planted by his friends, a ludicrous menu based on moods — perhaps too silly not to be believable? — and equal parts nonsense and mystique.

“A restaurant that never existed,” he said, ”The highest ranked in one of the world’s biggest cities, on perhaps the internet’s most-trusted reviews site”.

Butler became fixated on the fact that positive reviews he was writing for restaurants at £10 a piece played a role in determining restaurants’ ultimate fate — even though he’d never eaten at a single one of them. Wherever there’s money to be made, there’s someone rigging the game, and that’s especially true with online reviews — there’s quite a bit of evidence that fake reviews are endemic in the industry.

They are openly traded on the web (just a quick search through will get you dozens of freelancers offering to write “awesome reviews”), but there’s all sorts of shenanigans going on. Besides paying strangers to pose as customers, brands also plant employee reviews, give freebies or even full refunds in exchange for positive feedback (or so the ads strongly imply), as well as trash competitors by posting negative reviews on their website.

But even true, honest-to-god reviews can be a bit sketchy. They’re highly subjective, circumstantial, and sometimes even emotional. What works for you — or even what doesn’t — won’t necessarily have the same impact on others. Moreover, when Langhe, Fernbach, and Lichtenstein studied whether online consumer ratings are good indicators of a product quality, based on objective tests from consumer reports, they found very low correlations. While it’s barely a surprise that human beings don’t necessarily excel at being objective, the question remains. Perhaps our trust in online reviews really is misguided.

Especially when we look at the individuals who write them.


The people behind the reviews

According to a recent study by MIT Sloan Professor Duncan Simester and Eric T. Anderson’s study, very few customers write reviews. In fact, only 1.5% of the brand’s customers write them. Which means those 15 people out of a 1000 customers are influencing the behavior of the other 985.

When they took a closer look, they realized reviewers had a similar profile: “Reviewers are more likely to be married, have higher household incomes, and are more likely to hold graduate degrees. They also purchase almost four times as many items, they have been customers for longer, they return more items, and they purchase more items at a discount,” Duncan says. He further explains that this small portion of reviewers is not representative of the other customers.

And it’s not just a question of demographics. Their motivations also come into play. Although they can be genuinely trying to help other people make decisions, or even help the company by giving them feedback and advice on future product development, their feedback can also be motivated by economic incentives; by feeling the need to spell out every single letter in f-r-u-s-t-r-a-t-i-o-n, in reference to Christmas gifts that didn’t make it in time to be lavished on our teary-eyed mothers; by the need to belong in certain online reviewer communities or to build credibility and authority.

Whatever is driving people to write them, reviews are at best imprecise but moderately helpful indicators of a product’s performance. At worst, they are downright deceitful.

How to spot a fake review

It may sound easy, but as it turns out, we’re about as good at being objective about a product’s qualities as we are at telling the fakes from the real ones. A team of researchers in Cornell hired freelance writers to create fake positive reviews, which they then intermixed with 400 real ones from TripAdvisor, and compared how humans fared against their in-house algorithm. Granted, their “human sample” was not very large, as it was comprised of just three volunteer undergraduate students, but they couldn’t tell them apart more than a coin toss would.

Despite that, there are a few things we can look out for. In their study, Duncan and Anderson explored some linguistic cues associated with deceptive reviews. “Perhaps the strongest cue associated with deception is the number of words: deceptive messages tend to be longer.”

Exhibit A: Example of a review exhibiting linguistic characteristics associated with deception

I have been shopping at here since I was very young. I recently bought gloves for my wife that she loves. More recently I bought the same gloves for myself and I can honestly say, “I am totally disappointed”! I will be returning the gloves. My gloves ARE NOT WATER PROOF !!!! They are not the same the same gloves!!! Too bad.

But there are other red flags, he explains. “They are also more likely to contain details unrelated to the product (“I also remember when everything was made in America”) and these details often mention the reviewer’s family (“My dad used to take me when we were young to the original store down the hill”). Other indicators of deception include the use of shorter words and multiple exclamation points.”

Users writing fake reviews will also most likely have profiles with single reviews, and they tend to use personal pronouns such as “I” and “me” more often, which is probably an attempt to “enhance the credibility of their reviews by emphasizing their own presence in it.”

It’s not that we should disregard electronic word of mouth altogether, and roll our eyes every time someone buys a multi-color motion sensor toilet light because reviewers said it changed their lives (I mean, whatever works). We should, however, take online reviews with a grain of salt.

Beaches aren’t too sandy, nature isn’t crap, and Casablanca is an absolute delight. But don’t take my word for it.

The post Reviews, reviewed: the great ratings hoax appeared first on Unbabel.

About the Author

Profile Photo of Content Team
Content Team

Unbabel’s Content Team is responsible for showcasing Unbabel’s continuous growth and incredible pool of in-house experts. It delivers Unbabel’s unique brand across channels and produces accessible, compelling content on translation, localization, language, tech, CS, marketing, and more.