GaudiLab/Shutterstock.com

The science of why crowdsourced reviews can be so misleading.

Where can you devour an elk burger smothered in melted cheese and onion rings? Which diner has the greasiest loaded potato skins?

It’s no secret that many of us turn to crowdsourced sites to gather information and weigh options. Quantcast estimates that Yelp’s traffic—combining desktop and mobile visitors—is in the realm of 5,270,925 unique monthly users. We’re trying to find the experience that gels with our specific cravings, however niche. But are we actually learning anything from this “collective intelligence?”

Turns out, we’re suckers. In a study published in Science, researchers found that reviewers are easily manipulated by “social influence bias,” a feedback loop in which positive reviews beget more positive reviews. Another study, this one by Harvard Business School professor Michael Luca, corroborated these findings, concluding that restaurant reviews are influenced by ones preceding them—sometimes bumping up Yelp-style evaluations by as much as half a star.

This mushrooming doesn’t occur with negative write-ups, though—vitriolic reviews tend to be tempered by other, more neutral or enthusiastic ratings. The Science authors write, “Whereas positive social influence accumulates, creating a tendency toward ratings bubbles, negative social influence is neutralized by crowd correction.” They call for more thoughtful analysis of the factors at play in our decision making, so that we’re not just a horde of Internet-weary sheeple:

Future research that further explores the mechanisms driving individual and aggregate ratings—especially in in vivo social environments—will be essential to our ability to interpret collective judgment accurately and to avoid social influence bias in collective intelligence.

An Italian newspaper recently pointed out how easy it is to manipulate crowdsourced rankings on TripAdvisor. (GongTo/Shutterstock.com)

The Italian newspaper Italia a Tavola recently proved how necessary that enhanced insight is. Staffers scammed the ratings system by creating a profile for a fake restaurant in Moniga del Garda then posting 10 glowing ratings (under different usernames) over the course of a month, Jezebel reported. Within weeks, La Scaletta had the highest TripAdvisor ratings in town—despite the fact that it didn’t exist.

Why were people so quick to take the reviews at face value? “Stories that come from other people [are viewed as] much more believable than information from companies, because our working assumption is that [individuals] don't have an ulterior motive,” says Sarah G. Moore, assistant professor of marketing at the University of Alberta. Additionally, given a lack of identifying information, we assume that reviewers are similar to ourselves, found a study published in the Journal of Marketing Research. This presumption of similarity makes reviews seem relevant and trustworthy to readers, Moore adds.

Moore also says that vivid storytelling draws readers in. “Individual anecdotes of delicious desserts or snails in salads are very memorable and concrete—like urban legends compared to reading the telephone book,” she says. Even if a reader suspects that a review may not be entirely honest, “we're not vigilant about that,” Moore notes. “We're caught up in the story, and trying to figure out if we too would like or hate this restaurant if we went there.”

​Though TripAdvisor fired back with an assertion that it has procedures in place to detect and weed out duplicitous posts, Jezebel notes that Italian authorities slapped the site with a €500,000 fine last year for failing to implement sufficient safeguards against trolling or misleading content.

How do you avoid getting duped? Christopher Chabris, an associate psychology professor at Union College, explained to the Boston Globe:

The case where the wisdom-of-the-crowd effects work well is where each person brings their own observation and knowledge, however imperfect and idiosyncratic. When you don’t have that independence and everyone sees the history of other people’s opinions, you can get big biases in the outcome.

There are plenty of real-world ramifications of this digital ranking system, from a black market for soliciting paid positive reviews to court cases evaluating the legality of potentially defamatory—and perhaps fraudulent—reviews published under a pseudonym. Some proprietors have even accused crowdsourced review sites of burying positive evaluations and shaking down business owners to buy ads in exchange for promoting the glowing reviews, CBC News reports.

Misleading reviews aren’t as egregious as shady or coercive business practices—but try remembering that when you arrive somewhere hungry for a pizza, only to discover that the restaurant doesn’t serve up anything at all.  

Top image: GaudiLab / Shutterstock.com.

UPDATE: This post has been updated to include quotes from Sarah G. Moore of the University of Alberta.

About the Author

Most Popular

  1. Coronavirus

    The Post-Pandemic Urban Future Is Already Here

    The coronavirus crisis stands to dramatically reshape cities around the world. But the biggest revolutions in urban space may have begun before the pandemic.

  2. Perspective

    Coronavirus Reveals Transit’s True Mission

    Now more than ever, public transportation is not just about ridership. Buses, trains, and subways make urban civilization possible.

  3. A pedestrian wearing a protective face mask walks past a boarded up building in San Francisco, California, U.S., on Tuesday, March 24, 2020. Governors from coast to coast Friday told Americans not to leave home except for dire circumstances and ordered nonessential business to shut their doors.
    Equity

    The Geography of Coronavirus

    What do we know so far about the types of places that are more susceptible to the spread of Covid-19? In the U.S., density is just the beginning of the story.

  4. Coronavirus

    The Coronavirus Class Divide in Cities

    Places like New York, Miami and Las Vegas have a higher share of the workforce in jobs with close proximity to others, putting them at greater Covid-19 risk.

  5. photo: South Korean soldiers attempt to disinfect the sidewalks of Seoul's Gagnam district in response to the spread of COVID-19.
    Coronavirus

    Pandemics Are Also an Urban Planning Problem

    Will COVID-19 change how cities are designed? Michele Acuto of the Connected Cities Lab talks about density, urbanization and pandemic preparation.  

×