Amazon’s review-for-free-stuff policy wrecks its reviews

These days, you can access a massive library of independent product reviews from sites like Consumer Reports and The Sweethome, and you don’t even have to get off the couch. But who wants to bother with all that when you have Amazon (AMZN) product reviews right next to the “add to cart” button?

This is the power Amazon product reviews wield today. That power has swelled as more people now shop on Amazon using mobile devices than desktop computers, where it’s harder to jump from Amazon’s app to a browser and back if you want more than just the Amazon reviews.

Unfortunately, the whole review system is corked, thanks to an Amazon policy that permits its vendors to trade free or discounted products for reviews.

Amazon forbids fake and paid reviews, but the company has a massive loophole—a “sole exception” in its policies for free or discounted product. If vendors provide free products in exchange for reviews, they have to tell people getting the freebies they welcome positive and negative feedback. For their part, reviewers just have to add a disclaimer in the review text itself.

These disclaimers have become so common—one in five reviews, according to review aggregator and analyzer ReviewMeta—many consumers may accept them without question.

As if diluting more objective reviews isn’t bad enough, Amazon accommodates vendors even further by factoring these paid reviews in the actual one-to-five star rating system, although an Amazon spokesperson noted verified purchases—which now must be at a typical price—and helpful reviews were given more scoring weight last year.

Even at a lesser weight, counting these incentivized reviews can skews scores, making a big difference in a product’s overall rating. According to ReviewMeta’s data, incentivized reviews are 0.38 stars more positive than their more-objective counterparts. That might not seem like much, but when the average overall review is 4.4, it can boost products into the very highest percentiles on the curve. That makes them come up at the top of the Amazon search results.

This isn’t just a problem for vendors who give away products in exchange for reviews—it’s a problem for consumers. ReviewMeta pointed to a black umbrella that got 121 reviews with an average score of 4.7 stars. Only four reviews came from people who didn’t get a free umbrella. They averaged an abysmal 2.7 rating.

Amazon’s dominance in the marketplace has lit a fire under vendors to thread this loophole. Last year, survey data of 2,000 users from e-commerce company Bloomreach showed that 44% of people bypass the web and search for products directly on Amazon.