Simple words in online reviews reveal hidden patterns in patient experience

Can simple words like "and" or "then" in online reviews help health care providers learn about their patients' experiences? 

New research shows they can, with a new analysis showing that the most common theme in negative reviews of health care facilities involved communication and administrative issues, while complimentary reviews focused on reassuring and gratifying interactions with clinicians and staff. This was uncovered by an analysis of the specific words and terms used in online reviews, which was then used to find correlations with positive or negative reviews. The work was done by a study team at the Perelman School of Medicine at the University of Pennsylvania who published their findings in JAMA Network Open. 

Because so many are guided in their choices by online reviews of many different industries and products, the researchers view online reviews as an untapped resource for discovering true patient sentiments. 

The current ways in which health care facilities measure the patient experience and then take action often takes a lot of time, creating a system that lags between the actual encounter and improvements. If the real-time data from online reviews could be incorporated into tools that are easily viewed by health systems and their providers, they could make almost immediate changes for more positive experiences." 

Anish Agarwal, MD, MPH, senior author, deputy director of the Center for Insights to Outcomes and assistant professor of Emergency Medicine

The researchers analyzed data from more than a million Yelp reviews spanning seven years (2017 through 2023) of almost 139,000 health care facilities across the United States. Using a data analysis tool called the Differential Language Analysis ToolKit (DLATK), they specifically looked at 1- and 2-star (negative) reviews, as well as 4- and 5-star (positive) reviews. 

The negative 

Roughly 46 percent of all reviews were 1- or 2-stars. The word "not" somewhere in the text correlated most strongly to the negative reviews. Payment issues and poor treatment were most strongly tied to negative reviews, almost equally. 

Communication was a very common theme among the words the researchers determined to be correlated to negative reviews. "Told," "Said," "Call," and "Asked" were among the top words associated with bad reviews. Previous work by Agarwal and his colleagues had also highlighted "told" as the most common term in negative online hospital reviews. 

Phone conversations (and the hold times associated with them) were common negative themes, along with feelings of unfair payment experiences and perception of poor medical treatment, although the latter was not as common as the researchers believed it would be, Agarwal said. 

The positive 

More than 50 percent of the analyzed reviews were 4- or 5-stars, with "and" as the top term associated with those reviews by a good margin, followed by "great." 

Positive interactions with clinicians and staff emphasizing kindness was a top theme in the positive reviews. "Friendly," "helpful," and "kind," were all top terms. According to the analysis, the ability of staff to relieve patients' anxiety was also a top theme uncovered, resulting in "nervous" and "scared" being terms that were actually associated with positive reviews. 

Pushing toward the positive 

Health care reviews took a significant dip amid the COVID pandemic, and the sentiments have yet to rebound. However, the research team believes that their work points to real opportunities to improve care. 

"We believe these are systemic issues rather than isolated complaints," said first author Neil Sehgal, ME, an associate fellow in the Penn Leonard Davis Institute for Health Economics. "We're interested in understanding how specific facility characteristics, such as size, ownership, or patient volume, might shape patient sentiment and communication challenges." 

Agarwal is particularly interested in understanding more about the health care facilities who consistently score well. 

"I'd love to see where and which hospitals consistently knock this out of the park and, despite the last five to ten years, continue to provide high-quality and highly well-regarded care," Agarwal said. "If we spend more time learning from those doing it well, we can all move forward."