Yelp’s ChatGPT deal turns attention to how it displays reviews

Contractors have long had mixed feelings about online review platforms and pay-per-lead marketplaces. But as those players strike deals with AI companies, the stakes of understanding what those deals mean are climbing. 

Yelp sits at the center of it.  

What’s happening: In late July, Axios reported that Yelp struck a partnership with ChatGPT to feed the AI tool its trove of local business information, including reviews and photos, to surface in responses and recommendations. 

• The move is part of a broader trend in home services, as Angi has integrated with Google’s Gemini, Thumbtack with Anthropic’s Claude, and both with ChatGPT.

Why it matters: Consumers are increasingly turning to AI for local service queries. With ChatGPT poised to lean more on Yelp as a source, a contractor’s Yelp presence could carry more weight than before. 

• According to a July study of over 28 million AI responses to local business queries, Yelp is cited 3.4 times more than the next closest local discovery platform, excluding Google. 

Yes, but: Yelp doesn’t display every review a business receives. Its software sorts reviews as recommended or not recommended — the former appear on a company’s Yelp page, while the latter are effectively hidden and not factored into its star rating. 

What they’re saying: “My take on Yelp is that it is a completely unserious review platform for home service contractors at this point in time,” Tucker Yarbrough, CFO of Oklahoma-based Yarbrough & Sons, tells Homepros. 

• “It might be a year before we see [a customer] again, or we might never see them again. And so the problem that I [have is] the way in which they are trying to incentivize the review game,” he says.
• “Most of us aren’t scrolling on our phone leaving a review for everything that we touch. But that’s the idea that Yelp has, which is, ‘We’re only going to show the reviews of our power users,’” he adds.
• “If ChatGPT is going to start using this as the means by which they gauge a company… for lack of a better term, they’re only showing the reviews of people who need to go touch grass.”

As of publication, Yarbrough & Sons has 98 Yelp reviews, of which 11 are recommended and visible, with a 3.6-star rating. On Google, the company has a 4.9-star rating across 1,307 reviews.

How it works: In a statement to Homepros, a Yelp spokesperson says its software uses “hundreds of signals” to determine whether reviews are recommended, but that the company doesn’t publicly disclose “every signal” to prevent bad actors from gaming the system. 

• Those signals include solicited reviews, which “tend to be biased compared to reviews that consumers are motivated to write on their own,” per the spokesperson, as well as suspected conflicts of interest and reviews that appear to have been written by AI.
• Reviews can bounce between recommended and not recommended at any time, “as the automated software learns more about the reviewer and the business,” the spokesperson adds. 
• The company says advertising on Yelp has no influence on whether a business’s reviews are recommended. 

Zoom in: Even reviews that can be traced to real, verified customers — including those that weren’t solicited or don’t involve a conflict of interest — aren’t guaranteed to be recommended. 

• “[Yelp] is going to claim they aren’t ‘trusted sources.’ Me, being the contractor… can’t I prove to you that they’re a trusted source by saying, ‘Here’s the email address of the customer I serviced, and it’s tied to the Yelp account?’” says Joseph Moeller of Phoenix-based Desert Diamond Air.
• A former Yelp employee tells Homepros that several factors, including reviewers not being “that well known to the platform,” can influence a non-recommendation, “which is like Yelp’s little loophole way of saying, ‘We’re not saying they’re not real. We’re just saying their review isn’t credible.’”
• According to Yelp’s website, “Potentially unreliable reviews include those written by less active users that [our recommendation software] doesn’t know enough about to recommend their opinion to our community.”

The other side: Josh Campbell, founder and president of Dallas-based Rescue Air and Plumbing, is a self-proclaimed “Yelp super-user.”

• “Their whole platform is based on giving good, authentic, real answers for somebody to find the right service,” he says.
• “I think it speaks more to what is really happening in the world than Google does. I think that’s probably why OpenAI says, ‘Hey, this makes sense. We’re getting closer to the truth out here.’”
• “I think [Yelp] believes in those systems so deeply that they don’t really care if anybody likes it or dislikes it… They believe that it’s authentic and real and it serves the Yelp user at the highest level.”

Of note: According to Yelp, 76 percent of all reviews on its platform were recommended as of June 30.

• Of all reviews on the Yelp pages of Yarbrough’s, Moeller’s, and Campbell’s businesses, recommended reviews account for 11 percent, 27 percent, and 45 percent, respectively. 

Zoom out: ChatGPT currently generates local service responses using a mix of information from Google, Yelp, and other platforms; however, it remains unclear how much additional weight it will give Yelp following the new partnership.

• “How Yelp content is surfaced in ChatGPT is up to OpenAI,” a Yelp spokesperson tells Homepros.

OpenAI, the maker of ChatGPT, didn’t respond to several requests for comment before publication.

What we’re watching: As part of the partnership, consumers will soon be able to use Yelp’s “Request a Quote” feature directly within ChatGPT, though an official launch date has yet to be announced.