Perceiving the emotion in communication by persons with combined visual and auditory impairment

Sensory impairments are common among the elderly population. Difficulties in
communication are one of the major problems these people face. Not being able
to communicate properly could ultimately lead to a reduced quality of life.
Although the actual words (lexical content) in speech is essential, to truly
understand a speaker*s intentions one must additionally gauge his/her emotions.
Emotion in communication is carried by cues in both the auditory and visual
domains. While it is known the two modalities interact, for communication in
general and emotion in communication perception (ECP) in particular, the
underlying mechanisms remain unknown. Even more so, there is no scientific
knowledge on what effects visual and hearing impairments have on ECP, while we
anecdotally hear complaints from this patient population that they have
difficulties with ECP, it can be assumed that there are complex interactions
between hearing impairment (HI) and visual impairment (VI) that may prevent
current rehabilitation programs to be effective to the full degree. To be able
to develop more effective treatments, it is essential to first understand how
HI and VI affect multimodal emotion perception during communication.

The main objective of the study is to understand the individual and combined
effects of VI and HI on ECP. Secondary objectives are to create the proper
tools to study these effects, understand the interactive effects of the
auditory and visual modality on ECP, and to unravel the neural mechanisms
involved in ECP.

The study is a non-invasive observational study, with both within- and
between-subject comparisons. During screening visual and auditory functioning
will be measured. In all experiments the participants will attend to recordings
(audio and/or video) that convey emotion and asked to identify the conveyed
emotion. In addition, questionnaires on auditory and visual function,
personality, and emotional processing will be filled in.

All methods proposed are non-invasive. Data collection will take 3 to 8 hours,
depending on the experiment, which will be divided into shorter testing
sessions of 2 to 4 hours. The sessions will include multiple breaks, which
participants will be encouraged to take as often as needed. By distributing
testing over multiple sessions conducted on multiple days as needed, as well as
frequent breaks, will minimize fatigue for the participants and help maintain
alertness for better data quality. Neither risks nor benefits are associated
with participation.