Research shows how the brain merges faces and voices to read emotions

Communicating effectively with others involves recognizing emotions from faces and voices. How does the brain process and integrate emotional information from visuals and sounds that others display? New in JNeurosci, Olivier Collignon, from the Catholic University of Louvain, Stefania Mattioni, currently at Ghent University, and colleagues mapped responses in people's brains to faces and voices expressing different emotions. 

When participants perceived faces or voices separately, the emotions that were expressed triggered a wide range of brain activity in areas that respond to faces and voices. Mattioni elaborates, "Importantly, these areas were not limited to their preferred sense: Several face-selective regions also carried information about emotions heard in voices, while voice-selective regions carried information about emotions seen in faces. In some regions, the brain even represented the same emotion similarly whether it was seen or heard." When study participants observed faces and voices together, this combination reshaped how the brain processed and evaluated emotions compared to faces or voices alone. 

According to the researchers, this work suggests that the brain systems that process faces and voices interact and, in some areas, represent emotions in a similar way. This observation may help people quickly build an understanding of how a person is feeling by looking at their facial expressions and hearing the tone of their voice. The findings from this study could inform future research on whether these mechanisms are impaired in individuals who struggle to integrate social cues.