Behavioral Manifestations of Listening Effort

Listening Effort has been described as a reflection of the cognitive resources necessary for speech understanding. Measures can broadly be classified into Subjective, Physiological and Behavioral measures of Listening Effort (LE). Subjective measures of LE involve the listener rating or answering a questionnaire on the presented auditory stimuli. Some examples of Subjective measures are the multidimensional speech, spatial and qualities (SSQ) scale and Listening effort rating scale. However, these measures suffer from individual bias, for example, what one person might find takes more effort, another might not.

Physiological measures establish relationships between changes in the central and/or autonomic nervous system activity and LE. Effect of LE on central nervous system activity include methods such as using Functional Magnetic Resonance Imaging (fMRI) to compare processing of noise vocoded versus clear sentence stimuli and using Electroencephalography (EEG) to monitor memory/cognitive load. Some methods of measuring effects on Autonomic Nervous system are through Pupillometry and Skin Conductance. Research has found that fluctuating pupil size may be an indicator of mental task load and/or attention, stress and memory. Furthermore, Pupillometric methods have been used to correlate changing measures like pupil size and listening tasks. There are some drawbacks of using pupillometry to measure LE- Absolute pupil size and change in pupil size for presented speech change according to the age of the listener. For instance, Older adults have been found to have smaller absolute pupil size and show lesser change in pupil size when moving from difficult to easier listening conditions. These drawbacks can be accounted for by normalizing the pupil size.

Behavioral measures take into account that the there is a decline in cognitive functions over prolonged mental effort used an auditory dual task paradigm where the primary task was word recognition and the secondary tasks were memory recall and Visual Response times. Apart from the Dual tasks, participants were also asked to assign a subjective rating based on SSQ. Findings revealed 1) that there is empirical evidence linking repeated instances of effortful listening and subsequent cognitive failure and 2) self- report measures did not show the same changes as the behavioral measures. It has been suggested that this may indicate that subjective (self-report) measures and behavioral measures assess separate aspects of fatigue. It may also be due to subjective measures having individual biases.

As mentioned above, there are numerous studies of subjective, objective and behavioral measures of LE, yet, there are no studies of the actual behavior employed by listeners to reduce listening effort. Further, researchers do not understand how these behaviors influence the ability of the listener to monitor and maintain conversational speech. Listeners must rely on both peripheral and central auditory mechanisms to process speech. As the auditory signal increases in complexity, likewise there is an increase in the auditory processing required to understand speech. The effort that the auditory system must put forth to process complex signals is even greater if the signal degraded or imbedded in competing signals such as background noise. This requires increased exertion and mental effort on the part of the listener, and results in a reduction in ability to maintain focus and an increase in fatigue. These issues are further amplified in the presence of a peripheral hearing disorder as listeners do not have the mechanisms required to properly process even simple and unchallenged auditory signals. Research has demonstrated decreasing word recognition accuracy with decreasing Signal to Noise Ratio (SNR) and demonstrated an increase in listening effort when speech was presented in increasingly negative SNRs. Numerous studies have reported on the increase in listening effort required for individuals with hearing loss.

The purpose of this investigation is quantify listening effort based subjective, objective and behavioral measures at different Signal to Noise Ratios (SNR).