Bgreenlee/Flickr

Small differences in ratings can really help (or hurt) restaurants.

Over the past half-decade, Yelp has become the king of the crowd-sourced-reviews hill, with some 27 million reviews and an average of 70 million unique visitors per month. But despite its prominence, its effects have been difficult for researchers to pin down. How much do Yelp ratings affect a restaurant's business?

Anecdotal evidence -- the great tailor whose positive Yelp reviews led to the business's expansion -- is common, but sussing out a more widespread effect is hard. It's a chicken-or-the-egg problem. Did a restaurant fail because its reviews were bad or because its food was bad, garnering negative reviews? Did another succeed because of its winning food and service, or because positive reviews brought customers to its doors? How can you ever know?

In a new paper in The Economic Journal, Michael Anderson and Jeremy Magruder of Berkeley demonstrate a tool for cracking that nut wide open. They find that the difference of a half-star in a Yelp rating can make a huge difference for the rate at which a restaurant is booked up with reservations on a given night. How do they know that they are seeing the effect of the rating and not the quality of the restaurant itself? They use a statistical tool known as "regression discontinuity."

Here's how it works. When you look at a Yelp page for a restaurant, you will see the average of its reviews in a number of stars at the top -- e.g. 3.0, 3.5, 4.0, 4.5, and 5.0. But of course those discrete rankings represent a much smoother distribution. In other words, when Yelp takes the average of its rankings, it gets numbers that are not 3.0 or 3.5, but that fall in between. Thus two restaurants can have very, very similar averages, say 3.24 and 3.26, but be rounded into different half-star categories, 3.0 and 3.5.

This feature of the data gives the researchers the material they need to be sure that the effect they're seeing comes not from the quality of the restaurant but the bright stars of the Yelp review.

Their study of nearly 400 restaurants in San Francisco during the late-summer/early-fall of 2010, found a large difference in reservation availability for restaurants whose average rounded up versus those whose average rounded down. For example, here is what the distribution of 7 pm availability looks like, between restaurants rounding to 3.5 stars and those rounding up to four:

yelpreviews7pm.png

As you can see, there is a big drop in availability for restaurants right around the point at which reviews shift from rounding down to rounding up, not a gradual decline as you would expect if consumers were responding only to the change in quality. (The researchers note that the drop actually observes a few tenths of a point before the threshold, but they explain this simply: "Average ratings drift over time," they write. "A restaurant currently just below the threshold is thus likely to have been above [it] in the preceding months.") Overall, they see effects of varying sizes for different reservation times and different ratings thresholds. The seven o'clock time slot sees the most significant effects: "Here," they describe, "moving from 3 to 3.5 stars is associated with being 21 percentage points more likely to have sold out all 7:00 PM tables and moving from 3.5 to 4 stars makes restaurants an additional 19 percentage points more likely to have sold out all tables."

The researchers make some back-of-the-envelope calculations to estimate the potential effect slight changes in Yelp reviews could have on profits, which the gauge to be an additional $816 per week for a bump up the review metrics. That level of specificity is necessarily a bit thin, but the fact remains: Very slight changes in Yelp reviews can result in big swings for the restaurateurs below.

Top image: Customers wait in line for a sandwich at the highly reviewed Bakesale Betty in Oakland, California. (bgreenlee/Flickr)

This post originally appeared on The Atlantic.

About the Author

Most Popular

  1. a photo of yellow vest protesters in Paris, France.
    Equity

    To Understand American Political Anger, Look to ‘Peripheral France’

    French geographer Christophe Guilluy has a controversial diagnosis of working-class resentment in the age of Trump, Brexit, and the Yellow Vests.

  2. A rendering of a co-living building in San Jose.
    Life

    The Largest Co-Living Building in the World Is Coming to San Jose

    The startup Starcity plans to build an 800-unit, 18-story “dorm for adults” to help affordably house Silicon Valley’s booming workforce.

  3. A photo of a street barrier in New Delhi
    Equity

    What’s Behind New Delhi’s Gated Communities?

    India’s capital city is full of private residential “colonies” protected by locked gates. But many claim the barriers don't stop crime and cause traffic chaos.

  4. Transportation

    Why Are Little Kids in Japan So Independent?

    In Japan, small children take the subway and run errands alone, no parent in sight. The reason why has more to do with social trust than self-reliance.

  5. a photo of a striking Uber/Lyft driver
    Transportation

    Uber and Lyft Really Don’t Want California to Pass This Worker Rights Bill

    As California considers a gig-work bill to make ride-hailing drivers employees eligible for benefits and bargaining rights, Uber and Lyft ask for compromise.

×