Who is this research for? Customer experience executives, health care administrators, reputation managers, AI governance teams, and professionals deciding how generative AI should be used in customer communications.
Top Answer
Companies using generative AI to respond to customer reviews may face a difficult tradeoff between efficiency and transparency. Research in a health care setting suggests consumers often cannot tell whether a response was written by a human or Generative AI (GenAI). But when they are told AI generated the response, they tend to be less satisfied with it, which may reduce their willingness to consider the provider. That doesn’t mean businesses should hide AI use. The findings instead point toward careful disclosure, human oversight, and strong privacy safeguards.
Executive Summary
GenAI can make responding to online reviews faster and less labor-intensive. But does knowing that AI wrote a response change how customers judge the company behind it? New research from Dr. Daniel Villanova (Department of Marketing, Sam M. Walton College of Business) and Dr. Javad Mousavi (Iowa State University) examines that question in health care, an industry where online reviews can influence provider choice and customer communications can involve sensitive information.
The researchers first analyzed 16,393 real-world health care reviews and provider responses posted before widely available tools such as ChatGPT entered public use. They asked ChatGPT-3.5 to respond to the same reviews and compared its responses with those written by service providers. The analysis found systematic differences in the language, including GenAI responses that were more emotional and less analytic. Yet consumers generally could not distinguish AI-generated responses from human-generated ones.
That changed when AI's involvement was disclosed. Across controlled experiments, consumers reported lower satisfaction with responses when they knew GenAI had generated them. That lower satisfaction was associated with a reduced likelihood of considering the health care provider in the future. Importantly, the pattern appeared for both positive and negative reviews, suggesting that simply using AI for difficult complaints is not the only concern.
For health care organizations, there is another concern: privacy. In the researchers' analysis, ChatGPT-3.5 sometimes repeated potentially sensitive medical information from reviews and sometimes appeared to promise actions that might not actually be underway. The authors caution that these behaviors can create privacy, legal, ethical, and trust risks.
For leaders, the implication is that organizations must decide where GenAI belongs in customer communication, what requires human review, how AI use should be disclosed, and what safeguards prevent an efficient response from becoming a customer trust or privacy problem.
Expert Insights: What should health care leaders know about AI-generated customer responses?
How should health care companies balance transparency about AI with the risk of negative customer reactions?
Dr. Daniel Villanova notes: “We found consumers have less negative reactions when the firm discloses its own GenAI use compared to when the review platform adds a disclosure tag to the response. With platforms and regulatory bodies both interested in transparency, health care companies will need to seriously consider how they use GenAI in the response workflows – writing responses or merely helping hone human-generated responses. Non-disclosure might be a non-starter.”
→ Takeaway: Health care companies should consider disclosing GenAI use themselves and carefully define AI’s role in customer response workflows rather than waiting for platforms or regulators to make that use visible.
Where should health care companies draw the line between AI assistance and human involvement in customer communications?
Dr. Daniel Villanova adds: “Customers writing negative reviews want to be taken care of. Customers writing positive reviews appreciate gratitude. Shopping this completely out to GenAI, especially in the sensitive and care-focused context of health care, could be squandering a chance for heroic service recovery or for cementing a customer’s positive experience. Plus, our results show that when a consumer sees these GenAI disclosures on responses to reviews, even if the consumer didn’t write a review themselves, it leads to negative reactions. It would seem the cost savings would need to be immense to overcome this risk, but that of course depends on the market in which the company operates, the company’s cost structure, and how consumer feelings about GenAI evolve over time.”
→ Takeaway: Use AI to support customer communications without automatically replacing human engagement, especially when a response offers an opportunity to recover trust or reinforce a positive experience.
What safeguards should health care leaders put in place before AI responds to reviews containing sensitive customer information?
Dr. Daniel Villanova explains: “The development of AI agents enables having a reviewer agent or agents help provide some control over what responses ship. This could reduce specific response content risks, but two key issues remain. First, the negative consumer response risk is not particularly dependent on the content of the response, just the fact that GenAI was used. Second, low failure rates for leaking sensitive information applied to an extremely large number of responses could still be a very large number of HIPAA violations. Healthcare might be a context where having a human in the loop is critical.”
→ Takeaway: Human oversight may remain critical in health care because even a low AI failure rate can create significant privacy risk when applied across a large volume of customer responses.
Link to the Original Research
Published in Journal of Public Policy & Marketing (2026)
Frequently Asked Questions
Can consumers tell when AI wrote a company’s response to an online review?
In this research, consumers generally could not distinguish GenAI-generated responses from human-generated responses when they were not told the source. The researchers found linguistic differences between responses produced by ChatGPT-3.5 and service employees, but those differences did not make the source obvious to consumers. That matters because customers may assume a natural-sounding response represents direct human engagement even when AI produced it. For companies, the finding makes transparency and disclosure an important governance question rather than assuming customers will recognize AI-generated communications themselves.
Does disclosing that a response was written by AI affect customers?
The research suggests it can. Participants were less satisfied with a health care provider's response when they were told it had been generated by AI. Lower satisfaction with the response was then associated with a lower likelihood of considering the provider in the future. The researchers found this pattern across both positive and negative review contexts. The evidence therefore suggests that disclosure itself can influence how consumers interpret an otherwise similar response, although the studies focused specifically on health care and should not automatically be generalized to every industry.
Is it better for a company to disclose its own use of AI?
The findings suggest proactive disclosure may matter particularly for consumers who consider themselves knowledgeable about generative AI. Among these consumers, a platform-initiated disclosure could produce a more negative reaction than a firm-initiated disclosure. The researchers suggest knowledgeable consumers may interpret outside disclosure as a signal that the company was less transparent about its AI use. That does not establish one universally optimal disclosure strategy, but it gives businesses a reason to think carefully about whether customers should learn about their AI practices directly from the company or from someone else.
What are the risks of using generative AI to respond to health care reviews?
The research identifies risks beyond consumer acceptance. In its analysis, ChatGPT-3.5 sometimes repeated potentially sensitive medical information contained in reviews, which could create privacy concerns in a health care setting. The system also sometimes generated responses suggesting that the organization was taking actions that might not actually have been underway. These examples highlight why health care organizations should not evaluate GenAI solely by how natural or empathetic its writing sounds. Privacy safeguards, accurate claims, appropriate prompting, data governance, and human oversight may all be important when sensitive customer information is involved.
Should businesses stop using AI to respond to customer reviews?
The research does not establish that businesses should stop using AI for customer communications. Instead, it highlights tradeoffs that leaders should consider. GenAI may reduce the time and labor required to respond to reviews, but disclosure can affect customer evaluations, while inappropriate responses can create privacy or credibility risks. The study was also conducted in health care and used ChatGPT-3.5, so the authors caution against assuming the results apply identically to newer models or every industry. The managerial challenge is determining where AI adds value while maintaining appropriate transparency, safeguards, and human oversight.
