Video game offers new insights into autistic perception

People with autism often come at life from a different angle-seeing details others miss, focusing on patterns rather than faces, experiencing heightened or dimmed physical sensations. Although researchers have tracked some of the many ways the neurodevelopmental disorder alters perception of the world, the precise brain mechanisms involved remain obscure, says Boston University researcher Benjamin Scott. One reason: research often excludes people with more severe symptoms.

Scott and his colleagues have created a more inclusive approach, a video game that measures how people with autism understand and interpret information-and that can be played by those across the disorder's spectrum of symptom severity, including by people who are nonverbal. The researchers say it could spur fresh insights into how the autistic brain works and help refine the development and testing of new drugs and therapies.

Around 1 in 31 children in America have autism, according to the nonprofit Autism Speaks, with symptoms that affect communication and other social skills and that include restricted and repetitive behaviors. At the moment, scientists studying these issues often rely on subjective assessments from family members-particularly of people with significant communication issues-rather than input from the patients themselves. The game changes that.

"We found we can get these kids to engage and tell us, in their own language, how they're perceiving the world," says Scott, a BU College of Arts & Sciences assistant professor of psychological and brain sciences.

During a trial, he and his colleagues used the game to pinpoint a potential cause of some perceptual impairments: a phenomenon called "noisy evidence integration," which relates to how the brain handles and computes a flood of sensory information. The findings were published in Science Advances.

According to BU autism researcher Helen Tager-Flusberg, the ability to pull a broader range of kids into research is also exciting for families, who often want their children to participate in studies, but have frequently found them shut out because they can't complete research tasks.

"Understanding more about the kinds of difficulties autistic people have is always helpful," says Tager-Flusberg, director of the BU Center for Autism Research Excellence and a CAS professor emerita of psychological and brain sciences. She was one of Scott's coauthors on the paper, along with cognition and decision expert Joseph McGuire, a CAS associate professor of psychological and brain sciences.

"Importantly, the game was successful across the full range of the autism spectrum, including those falling in the profound autism category or with known genetic syndromes," she says. "We also demonstrated that we could successfully collect a considerable amount of both survey and experimental data remotely, which allowed us to include autistic people with quite varied backgrounds from all over the US."

How people with autism manage sensory input

The game is a hunt for gems. Players choose a character, then follow a treasure map to grab as many jewels as possible. As they work their way through the game, flashes appear on the edge of the screen-if one side sparkles more, it's hiding gems.

While the players are adding to their horde, researchers are tracking how well they spot the flashes, which vary in number, duration, and timing.

"We can collect some very detailed information about how you're making decisions, perceiving the world, and learning to play the game," says Scott. The researchers named the game Geode (for Gathering Evidence to Optimize DEcisions); it was programmed by Quan Do (CAMED'25)-now a postdoctoral scientist at Howard Hughes Medical Institute-with graphics help from an outside designer.

There are no instructions-players just jump in and get started, learning the gameplay and rules as they go. That's an important factor in leveling the playing field for people across the autism spectrum, says Scott, and is what allows children with severe communication-related symptoms to participate.

As part of the study, the researchers had more than 300 adolescents, all aged between 11 and 17, play the game. Just over 200 players had autism, covering the breadth of the disorder's spectrum; the remainder were neurotypical siblings.

Although most players-including those with profound autism symptoms-quickly grasped the game's rules and adjusted as it became harder, the learning curve differed significantly between those with and without autism. As the researchers adjusted game elements-like the timed light pulses-they saw that autistic people found it harder to utilize all the sensory evidence being thrown their way. Overall, the teens with autism took longer to learn how to play and had reduced accuracy compared to people without the neurodevelopmental disorder.

"The deficit seems to be in a component of perception called perceptual integration, where you are accumulating information over time to form a complex idea about the world," says Scott.

In everyday life, that complex idea might be something as seemingly routine as figuring out what someone is saying, pulling in the information from individual words, then sentences, and compiling those with visual cues from the speaker to understand their message and meaning. For someone with autism, each piece of information may come with an extra piece of distortion or noise; the more information, the more accumulated feedback.

"One of the longstanding ideas in autism is that there's a change in the excitatory-inhibitory balance of neural circuits and that change can lead to altered neural activity that can produce noisy perception," says Scott.

The game is not designed to be a tool for diagnosis-it can tell you how people see the world and learn, he says, but not pinpoint autism versus another neuropsychiatric disorder. He speculates it could one day be refined to figure out specific traits or deficits from one disorder to another, or even understand symptom severity in a particular condition, but that's not on the team's agenda yet. For now, Scott sees a greater potential for its use in clinical trials.

"Clinical trials are underway for a variety of different drugs across autism and related neurodevelopmental disorders," he says. "And this game could be an outcome measure."

A child could play the game at the start of a trial to get a baseline measure of their skills, then again after the intervention, "to see whether it alters the way they play this game."

Comparing behavior and brain circuitry between animals and humans

The game's no-instructions approach has its roots in Scott's research on how animals make decisions. In his Laboratory of Comparative Cognition, Scott has analyzed how songbirds learn and add new skills and how rats choose between speed and accuracy on a task-and it means he's used to study participants who can't easily follow instructions.

"We work with nonverbal subjects all of the time, so we've developed tricks to use reinforcement learning and nonverbal feedback to get you to come in very naively and learn the complex mechanics of a game," says Scott.

As well as helping include people with more severe communication issues, that foundation brings a potential benefit for scientists looking to better align studies of human and animal behavior. Most major breakthroughs in brain science start in animals, and having a tool that can help researchers compare findings-for example on the underlying genetic and brain circuitry changes in autism-to humans could be a major benefit.

"We can more cleanly compare their behavior and understand the underlying circuitry," says Scott. Because animals and humans can play virtually identical games, "we can directly measure whether autism risk genes affect humans and animals in the same way."

For someone whose science has been focused on lab models and animals, Scott says working with human patients was enriching in multiple ways, describing them as "the fourth collaborator on the project." Listening to the challenges that families face opened up potential future research questions, like, why is it that some autistic people experience skill regression?

"The science is great, and I hope we make progress," says Scott, "but really, we're inspired by the kids themselves and the families that are trying to make it all work."

Postdoctoral associate Sucheta Chakravarty (now an assistant professor of radiology at Wake Forest University), Yutong Li (GRS'22,'26), and Vanessa Torres-Lacarra (GRS'26) also contributed to the study. This research was funded with support from the Simons Foundation.