Incentivized online reviews inflate product ratings, sales, even when disclosed

New research based on thousands of reviews posted on Amazon shows that when people received free products in exchange for reviews, their ratings were significantly inflated — and product sales were stronger — even though reviewers disclosed that they received the product for free.

These incentivized reviews, where reviewers receive discounted or free products, are common on many online retailers. Amazon no longer allows compensated reviews, but incentivized reviews continue to proliferate on sites like Walmart, Yelp, and Overstock. And influencers on Instagram and YouTube routinely hype products in paid reviews. The new study suggests that these disclosures do not protect customers from systematically inflated reviews and that new policies are necessary.

The study, performed by University of Florida and University of South Carolina researchers, comes as the Federal Trade Commission is investigating the problem of incentivized reviews. The FTC recently implemented guidance that reviewers must disclose any incentives they receive. But that guidance doesn’t go far enough, according to the study’s authors.

“Disclosure is not effective to protect consumers,” said Jinhong Xie, a professor in UF’s Warrington College of Business, who, with her co-authors, shared their findings with the FTC. The article by Xie, UF’s Woochoel Shin and Sungisk Park of the University of South Carolina will be published soon in the journal Management Science.

Their study was based on every review for more than 5,000 products across broad categories like electronics, home and kitchen, toys, and appliances. Their data come from both before and after Amazon shut down their original review system in 2016. This provided a natural experiment for the researchers to test the effect of these compensated reviews on the products.

Until October of 2016, businesses selling on Amazon worked directly with reviewers to offer free products in exchange for a review. Many reviewers were prolific, writing hundreds of reviews. Some were semi-professional, and they protected their future earnings by posting overwhelmingly positive reviews so businesses would continue working with them.

“We see that for exactly the same product, incentivized reviews are systematically more positive than organic reviews,” said Park, a professor of business at South Carolina. Incentivized reviewers rated products nearly 0.5 stars higher on average on a five-star scale. In discussions he had with reviewers, Park found some who believed they were posting their honest opinions, despite receiving free samples and consistently posting five-star ratings.

“This disclaimer does not discipline the reviewers at all,” Park said. “I’m saying I received a free sample, but I’m still giving a five-star rating.”

After Amazon shuttered the program, they began purging existing incentivized reviews. This gradual removal of the reviews allows the researchers to see how product ratings and sales changed when products lost the advantage of inflated reviews.

“The removal of disclosed incentivized reviews led to a significant sales reduction,” Park said. Customer satisfaction also increased. “Once Amazon deleted these disclosed reviews, then consumers became happier. There was a smaller number of one-star reviews after the policy change.”

While Amazon ended their business-to-reviewer incentive program, they still offered Vine reviews. Through Vine, Amazon plays middleman between reviewers and businesses looking for reviews of new products. Since businesses no longer select their own reviewers, people have less incentive to offer glowing praise for middling products. The Vine program is still in practice.

The researchers discovered that these marketplace-led review programs provide enough independence to encourage honest reviews from participants, ultimately protecting consumers.

“If you’re Amazon, you’re not interested in inflating the rating of one specific seller. Amazon has an incentive to keep the marketplace fair, but they also want to provide information to consumers. Reviewers, knowing that the party that hired them does not want to inflate the rating, offer fairer reviews,” said Shin, a UF professor of business. “We showed there is no rating inflation on Vine reviews.”

Other retailers, like Home Depot, also provide this kind of independence by matching reviewers to products instead of allowing sellers to select their own reviewers. But reviews in exchange for free products, or straightforward paid reviews, are very common on social media sites like YouTube and Instagram.

“Our proposal is to change the system by requiring more independence in reviews,” said Xie. “It’s hard to change the individual behavior of reviewers, consumers, or sellers. But you can change the system to make it fairer for everyone.”