Study reveals how online misinformation 'ticks' and suggests new strategy to stop it

Parenting communities on Facebook were subject to a powerful misinformation campaign early in the COVID-19 pandemic that pulled them closer to extreme communities and their misinformation, according to a new study published by researchers at the George Washington University.

Previous research has shown that social media feeds the spread of misinformation. However, how this happens was unclear, leaving social media platforms struggling under the deluge of new material that is posted everyday. Researchers at GW set out to better understand how the Facebook machinery helps misinformation thrive and spread through the platform's network of online communities.

By studying social media at an unprecedented scale, we have uncovered why mainstream communities such as parents have become flooded with misinformation during the pandemic, and where it comes from,. Our study reveals the machinery of how online misinformation 'ticks' and suggests a completely new strategy for stopping it, one that could ultimately help public health efforts to control the spread of COVID-19."

Neil Johnson, professor of physics, GW

Johnson and a team of GW researchers, including professor Yonatan Lupu and researchers Lucia Illari, Rhys Leahy, Richard Sear and Nico Restrepo, began by looking at Facebook communities, totaling nearly 100 million users, that became entangled in the online health debate through the end of 2020. Starting with one community, the researchers looked to find a second one that was strongly entangled with the original, and so on, to better understand how they interacted with each other.

The researchers discovered that mainstream parenting communities were exposed to misinformation from two different sources within Facebook. First, during 2020, alternative health communities, which generally focus on positive messaging about a healthy immune system, acted as a key conduit between mainstream parenting communities and pre-Covid conspiracy theory communities that promote misinformation about topics such as climate change, fluoride, chemtrails and 5G. This strengthened the bond between these communities and allowed misinformation to flow more freely. Second, a core of tightly bonded, yet largely under-the-radar, anti-vaccination communities, which were found adjacent to the mainstream parenting communities, was able to continually supply COVID-19 and vaccine misinformation to the parenting communities. What's more, neither the alternative health communities nor the anti-vaccination communities were particularly large groups by Facebook's standards, meaning they could still be flying under the radar of platform moderators.

"Our results call into question any moderation approaches that focus on the largest and hence seemingly most visible communities, as opposed to the smaller ones that are better embedded," Johnson said. "Clearly, combatting online conspiracy theories and misinformation cannot be achieved without considering these multi-community sources and conduits."

Facebook has previously tried to combat misinformation through the use of information banners at the top of Facebook communities to provide official health guidance and advice. According to the researchers, these banners failed to stop the mainstreaming of conspiracy theories and misinformation because they targeted a limited inner core of extreme communities while most of the parenting communities and other conspiracy theory communities live outside of that core.

Johnson and his team note that similar behaviors will arise on any social media platform with built-in community features. The team hopes to address these other platforms in future work.