Noise is present in most work environments, including emissions from machines and devices, irrelevant speech from colleagues, and traffic noise. Although it is generally accepted that noise below the permissible exposure limits does not pose a considerable risk for auditory effects like hearing impairments. Yet, noise can have a direct adverse effect on cognitive performance (non-auditory effects like workload or stress). Under certain circumstances, the observable performance for a task carried out in silence compared to noisy surroundings may not differ. One possible explanation for this phenomenon needs further investigation: individuals may invest additional cognitive resources to overcome the distraction from irrelevant auditory stimulation. Recent developments in measurements of psychophysiological correlates and analysis methods of load-related parameters can shed light on this complex interaction. These objective measurements complement subjective self-report of perceived effort by quantifying unnoticed noise-related cognitive workload. In this review, literature databases were searched for peer-reviewed journal articles that deal with an at least partially irrelevant “auditory stimulation” during an ongoing “cognitive task” that is accompanied by “psychophysiological correlates” to quantify the “momentary workload.” The spectrum of assessed types of “auditory stimulations” extended from speech stimuli (varying intelligibility), oddball sounds (repeating short tone sequences), and auditory stressors (white noise, task-irrelevant real-life sounds). The type of “auditory stimulation” was related (speech stimuli) or unrelated (oddball, auditory stressor) to the type of primary “cognitive task.” The types of “cognitive tasks” include speech-related tasks, fundamental psychological assessment tasks, and real-world/simulated tasks. The “psychophysiological correlates” include pupillometry and eye-tracking, recordings of brain activity (hemodynamic, potentials), cardiovascular markers, skin conductance, endocrinological markers, and behavioral markers. The prevention of negative effects on health by unexpected stressful soundscapes during mental work starts with the continuous estimation of cognitive workload triggered by auditory noise. This review gives a comprehensive overview of methods that were tested for their sensitivity as markers of workload in various auditory settings during cognitive processing.
Keywords: Cognition, measure, noise, task, workload
|How to cite this article:|
Grenzebach J, Romanus E. Quantifying the Effect of Noise on Cognitive Processes: A Review of Psychophysiological Correlates of Workload. Noise Health 2022;24:199-214
| Introduction|| |
Noise is the proportion of the perceived soundscape that is irrelevant to the listener’s goals., For that reason, noise in the context of occupational safety and health, public health, and environmental protection is often defined as unwanted sound that can cause disturbance, annoyance, impairment, or damage (see DIN 1320). Sound consists of superimposed acoustic waves stemming from several sound sources. Only some of these superimposed acoustic waves might be relevant to the listeners’ goals at a time. However, the listener might not be aware that a particular momentarily irrelevant sound source will become relevant (like signaling danger). If the goals of the listener change, auditory information judged to be noise, may become a relevant signal. This observation highlights the importance of permanent auditory vigilance. Accordingly, the auditory perceptual system has certain peculiarities unseen in other modalities., The human visual system allows averting the gaze from a visual object or closing the eyelids. In contrast, auditory sensory information cannot be suppressed in such a manner. Instead, attentional resources need to be directed to another sound stream to implicitly ignore an unwanted auditory stream. Still, the irrelevant sensory information is processed to a varying degree, even if it is intended to be suppressed. Moreover, the probability that the suppressed stream dominates conscious attention is still elevated. Therefore, the information filtered out as noise can be best described by its unfavorable effect on cognition by distraction. The auditory cognition constantly monitors and segregates the auditory information. This filtration process comes at a particular cognitive workload.,,
Mental work and the underlying processes are particularly prone to additional cognitive workload. The preservation of the momentary cognitive state is a prerequisite for cognitive activities. The processing of information can be disrupted by mind wandering (internally induced) and unexpected sensory input (externally induced). The externally induced change in the cognitive state can be caused by occupational noises. Noise, even below the permissible sound pressure levels and at a short exposure duration, lowers concentration, often triggers stress, and may require additional workload to maintain the current work performance. Many studies find evidence that the individual perception in a subjective rating of noise sounds is negative or detrimental to the perceived psychological wellbeing.,,,
At work, the employee behaves goal-oriented: consequently, available cognitive resources are focused on work tasks. Thus, the prevention of auditory disturbances during this focused process plays a crucial role in the design of the workplace. Work tasks can be fulfilled at a lower cognitive workload if the workplace is designed ergonomically. One aspect of acoustic ergonomics is the soundscape: noise is among the most frequent and annoying factors in many workplaces. For instance, machine noise from close-by production areas, conversations of colleagues (density of office space), traffic sound from an open window (density of urban space), and the neighbors’ children playing in the corridor (home office) − the sources seem endless and are partly interwoven with the core job duties. For example, in a pedagogic setting, a teacher in a classroom needs to focus attention on a selected pupil’s voice, even in the presence of irrelevant speech. This scenario underlines how job characteristics may co-occur with certain stressful acoustic settings. The mental processing of information is becoming the dominant element of many professions. In digitalizing societies, mental work is gradually superseding more and more tasks. Regulators have identified noise at work as a health issue and invested time in the evaluation, prevention, and protective measures. Effects of noise on the employee are classified as either auditory or non-auditory. Auditory effects of noise relate to noise-induced hearing impairment; non-auditory effects include, among others, annoyance, stress, and effects on work and task performance. The scope of this review is non-auditory effects of noise.
If the noise is consciously perceived as disturbing, the impact of noise adversely affecting task performance can be prevented by adapting the workspace: for example, reducing noise emissions at the source by selecting low-noise work equipment, reshaping the workspace by taking room acoustics measures tailored to the work activity, organizational measures leading to spatial or temporal separation from the noise source to reduce noise exposure and, as a last resort, wearing hearing protection. According to European occupational health and safety regulations, employers shall assess risks to the safety and health of employees at workplaces and take measures to avoid or reduce the risks. In terms of noise, that means employers bear the responsibility of evaluating and minimizing the risk of the worker related to activities exposed to noise. If necessary, the employer shall measure the level of noise that workers are exposed to and take action to eliminate the risks arising from exposure to the noise source (see Directive 2003/10/EC of the European Parliament and of the European Council). Nevertheless, noise below the permissible exposure limits was also found to subconsciously affect cognition, which might chronically accumulate into negative health effects. In other studies, where no effect can be measured on the performance and the subjectively experienced workload,, researchers assumed that the cognitive system activates a compensatory strategy that invests additional workload in overcoming noise-related demands.
To expand the scientific knowledge as a prerequisite for improving occupational health and safety regulations, especially with regard to non-auditory noise effects, it would be helpful to have an unambiguous marker that can objectively track the actually invested workload. Until now, the effects of noise on a primary cognitive task have been evaluated mostly with retro- and introspective diagnostic instruments and task performance comparisons., Besides detrimental effects on task performance, non-auditory effects also comprise the noise-induced workload. The self-reported individual perception through questionnaires is a powerful tool that allows cheap and fast insights into the experienced cognitive workload. For example, the NASA, N. (1986). Task load index (tlx) v. 1.0 manual. NASA, NASA-Ames Research Center Moffett Field is an instrument that allows ratings of the perceived workload during a just finished task. However, limitations of a subjective rating scale are not only the inconsistencies in individual judgments, depending on emotional states, memory capacity, and willingness to cooperate. Also, the current workload cannot be monitored online during the processed task itself. In contrast, objective measurements that correlate with (neuro-) physiological markers that represent activities in the cognitive system have the potential to track noise-related workload online. In this way, the unchanged task performance (and subjective reports) between a silence and noise condition can be explained better.
The advances in the required sensor technology (e.g., miniaturization, price reduction, mobility), mobile applications, and development of software for complex mathematical analysis pipelines (e.g., regression, classification, networks) drive the assessment of cognitive workload in objectively measurable units. In the digital world, the quantification of human resources allows for better workforce planning and prevention of work-related diseases. The scope of the review is to identify measurement methods, including sensitive parameters, and describe the acoustics settings they are most commonly used in. In this fashion, scientists can combine the appropriate measure in their respective occupational investigations for sensitivity tests.
A broad spectrum of measuring techniques has been found to be sensitive to the momentary cognitive workload in different instruments such as pupillometry, electroencephalogram (EEG), functional near-infrared spectroscopy (fNIRS), and electrocardiogram (ECG). However, those objective measurement methods are often used in a controlled laboratory setting, sometimes within an experimental paradigm that contextualizes the findings and limits generalizability. The attributability of specific proportions of cognitive workload (task- or noise-related) is a major research area that needs further investigation. In this review, we offer a starting point on which measurement technique is applicable and in which combination it might be administered to cover realistic work settings. That is, the employee is usually engaged in a primary cognitive task covering already some of the available cognitive resources (task-related workload). If during this ongoing cognitive process, noise is present, the listener may have to invest more effort to maintain performance on the primary cognitive task. This review has a particular focus on the objective measurement methods that allow the quantification of noise-related workload in the presence of an already demanding primary cognitive task. Several reviews deal with the impact of short- and long-term exposure to noise on chronic health.,, However, we examine here publications with the acute, non-auditory effects of noise on cognitive workload and the respective measurability.
Since the amount of mental work is increasing in the work-life of many employees, the facilitation of safe work environments is fundamental for future acoustic ergonomics. The prevention of noise effects helps improve the quality of work, the personal experience of work, and, subsequently, psychological health (technical rules for the ordinance on workplaces). In this paper, we review the literature of the last 10 years on the quantification of noise-related workload during an ongoing mental task. Along the following four dimensions that establish the search string, we investigated which measurement methods are being used and showed effects on which specific parameters. We aim to describe qualitatively in a comprehensive review of empirical findings regarding the objective measure of workload (1. measurement dimension) in adverse auditory conditions (2. acoustic dimension) during an ongoing cognitive process (3. cognitive task dimension), which generates mental workload (4. workload dimension).
| Method|| |
The research question can be answered from many viewpoints (e.g., speech-perception effort, general noise stress, auditory split attention task). We adopted a reduced form of the PRISMA framework to synthesize the evidence. Due to the heterogeneity in the screened approaches chosen in the publications examined, not all items of the PRISMA framework could be addressed. Our methodological approach consisted of developing a search string, the reproducible search requests in representational literature databases, a technical and human filtration process, and the classification of the findings within the prescribed four dimensions. Ultimately, the connection within and between the dimensions can be observed in publication-based categories, allowing statements in which the combination of auditory stimulation, cognitive task, and measurement methods effects can be expected. In this way, conclusions upon the clusters of evidence can be made regarding which measurement methods should be used in which setting.
Search string design
The search string was developed in a discussion by two researchers based on expert knowledge and refined by strategies that help prevent bias. The design was supported by learnings from bibliographic methods, such as formulated by Page et al. In addition, the risk of bias was minimized by carefully selecting terms with neutral meanings concerning the expected effect.
Since we were interested in publications satisfying all four dimensions (acoustic, cognitive task, workload, measurement), we had to specify search terms for each dimension. As independent variables, the acoustic environment and the cognitive task are defined. The workload and the measurement dimensions contain dependent variables. As shown in [Table 1], the dimensions of cognitive task and measurement were split into two. In this hierarchical order, the search engine needed to satisfy all dimensions with at least one included search term each. The search terms animal*, child* should not appear in the final selected publication. The search engines usually committed themselves only to the title, abstract, and the (author selected) keywords, but that differed inconsistently between databases. The full text is not regarded in this primary step.
Seven literature databases (Web of Science, Scopus, EBSCO, ILO, JSTOR, PubMed, SAGE) were queried for the search string in German and English language on March 8, 2021 between 11 am and 1 pm. The timespan of considered publications extended from 2010 to 2021. Depending on the syntactic commands the literature database required (e.g., AND, OR, NOT), the above search terms were transformed and translated in the fitting format to appease the search field logic. For example, in the “web of science” the exact search string looked like this in a shortened form (“…” symbolize the remaining search terms found in [Table 1]):
TS=((acoustic* OR “background speech” OR …) AND (cognit* OR mental) AND (arith* OR attend* OR attention* OR …) AND (allocat* OR arous* OR …) AND (assess* OR activit* OR correlat* OR evaluat* OR …) AND (*adrenaline* OR biofeedback …)) NOT TS=(animal* OR child*)
The filtration process consisted of the removal of the duplicates (technical and human assistance), the screening of the titles, the screening of the abstracts, and finally, the full-text screening. The topical inclusion criteria of the screening process were based on the cardinal dimensions that define the search string (acoustic, cognitive task, workload, measurement). Exclusions could happen along all stages of the screening steps (title/abstract/full-text). Only peer-reviewed journal articles were considered that had been published in a journal with an impact factor available on Scopus. The general quality of the article was rated to meet “good scientific practice” in terms of depth of description and study design.
From eligible publications, relevant information was extracted in an abbreviated form to allow for an overview and evidence synthesis. That included information on…
- the sample (sample size, gender distribution, age, occupation, exclusion criteria, exclusion count, location, count of experimental groups, experimental setting, session duration)
- the auditory stimulation (number of acoustic conditions, parameter modulated, description of sound type, duration, and position relative to the cognitive task)
- the cognitive task (number of cognitive tasks, name of the task, the tested cognitive system, test name [if applicable], difficulty levels per task, performance parameter)
- the moderator variables (general moderators [e.g., personality], subjective rating, name of the subjective effort scale, subscales)
- the measurement methods (number of measurement methods, name of the method, company and product name, measurement type, analyzed parameter, time of measurement relative to the cognitive task, invasiveness
- the results (summary thereof, effect size)
- the discussion (summary thereof)
The whole process was executed by a researcher and, after that, discussed critically in a group of five researchers of the same institute and three external senior scientists. Finally, the resulting data extraction table was further analyzed and aggregated to reveal fundamental insights into the presented evidence. The data extraction table can be found in Appendix 1.
Based on the results identified during the extraction process, three dominating categories could be identified within the publications. Often the auditory stimuli used in the publications can be conflated within a single category because the underlying design is similar. Different task-relevant auditory stimuli were applied in combination with noise in the acoustic dimension: many of the speech-related studies used various corpora of text, sentences, and words that are eventually masked by noise or irrelevant speech. Evidence-based on speech stimuli was condensed into a single category in the deployed information reduction process (C1–C3, see results). The three categories were set up to establish a model of the evidence, allowing conclusions within the categories along the four analyzed dimensions. In this way, we could overcome a fragmented insight and get the whole picture of the heterogeneous publication sample.
| Results|| |
During the search in the selected databases, 5479 publications were accumulated. Two thousand and three hundred fifteen publications were identified as duplicates and removed (42%). The titles of the remaining 3164 publications were screened for the thematic focus, which was not satisfied for 2279 publications (72%). The thematic focus of a publication had to fulfill the four dimensions that were introduced above. During the next screening of the abstracts of 885 publications, 730 were, again, thematically unfit and removed (82%). At this stage, the journal required a journal impact factor released in the Scopus network. In the last stage of the filtration process, the full text of 155 publications was screened, and the eligibility was rated. Based on a thematic focus, 84 publications were eligible for data extraction and consequently screened for the included evidence and experimental details. Seventy one publications were removed after full-text screening (46%), see [Figure 1]. The data extraction table can be accessed in Appendix 1.
|Figure 1 Filtration process. Remaining publications after each step of filtration.|
Click here to view
The analysis of the bibliographic properties of the selected 84 publications revealed a dominance of the search term “listening” in the title of the publication. Further, a growing number of publications were published in the considered time slot from 2010 (1 publication) to 2020 (11 publications). The most frequent journals where the publications were published in were “Ear and Hearing” (9 publications, 11%), “Hearing research” (5 publications, 6%), and “Trends in hearing” (4 publications, 5%). The literature databases Scopus (49 publications) and Web of Science (21 publications) contributed the most publications finally considered.
All publications consisted of intervention studies that controlled different experimental conditions on a selected sample of study participants. The size of the samples varied around M = 34 participants, SD = 18 with, if stated, around M = 18 females, SD = 18. In the following analysis, the statements are based solely on information from the publications that provided them. Publications without information stated are not included in the conclusions. The participant sample was in most cases stemming from the Netherlands (17 publications, 20%), USA (9 publications, 11%), Germany (6 publications, 71%), China (4 publications, 5%), or France (4 publications, 5%). Most of the studies were situated in a laboratory (67 publications, 80%) or were executed in a simulation environment (14 publications, 17%), and in-field studies summed to 3 publications (4%).
The cardinal dimension in this review is the acoustic dimension. We are interested in how the cognitive workload is modulated in the presence of task-irrelevant sound. Therefore, studies were considered that include both partially and completely task-irrelevant sounds. In this fashion, it is possible to use evidence from a wider set of publications that may eventually shed light on this complex interaction. Most studies had two separate acoustic conditions (28 publications, 33%), 3 (27 publications, 32%), 1 (16 publications, 19%) acoustic condition(s), 4 (9 publications, 11%), 5 (2 publications, 2%), 17 (1 publication, 1%), and 30 (1 publication, 1%). In 42 publications (50%), a silent condition was introduced as a baseline condition. That means, besides the 16 publications with a single acoustic condition (19%), another 26 publications presented studies without a silent condition (31%). The silent condition was defined by the absence of the otherwise present auditory stimuli. This condition might differ from a more controlled silent condition, as it is available in acoustic research laboratories only. The sound pressure level of the non-silent acoustic conditions in the studies varies between 35 and 80 dB(A).
Categories identified within the acoustic dimension
Within the acoustic dimension, three broad categories could be identified. Each publication would be allocated to one of the three categories based on the used auditory stimulus and experimental paradigm: “listening effort” (C1), “auditory stressor” (C2), and “oddball” (C3). These three categories describe which auditory stimulation and experimental paradigm were included in the specific publication. A publication in the “listening effort” category (C1) contains to-be-remembered masked speech, see publications.,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, The “auditory stressor” category (C2) contained a stimulus independent of the cognitive task and contained various auditory materials, see publications.,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, The “oddball” category (C3) contained a task-independent stimulus that followed the oddball experiment design (repetitive short sounds with a break in the pattern), see publications.,,,,,,,,, The subsequent analysis of evidence is performed within the categories C1 to C3, see [Figure 2].
|Figure 2 “The fibers of the chord”: schematic of the topical review process.|
Click here to view
In the following section, each category (C1–C2) is described along with the four dimensions: the acoustic dimension, the cognitive task dimension, the workload dimension, and the measurement dimension. In this way, it is possible to grasp which measurement method (and parameter) was combined with which acoustic settings and which cognitive tasks. The following table gives an overview of the three categories and the four dimensions, [Table 1]. The cells of the table are intended to help researchers to select the appropriate combination for their own research question, as it has been used in the literature. A detailed description is eventually given in the following paragraphs.
Category 1 (C1) “Listening Effort”
In the “listening effort” research, the workload stems from the challenges in the intelligibility of the acoustic speech signals. Listening effort emerges when mental resources are deliberately allocated to overcome acoustic obstacles while listening to speech.. Again, in an occupational setting, a teacher in the classroom must withstand a specific workload when listening to a voice of a selected pupil (pupil A, male pitch, location x) that is masked by the voice of another pupil (pupil B, female pitch, location y). In this situation, the auditory stimulus is partially “irrelevant” (pupil B) and can be labeled as noise. Therefore, the listener must exert selective attention to direct attentional resources onto the target voice (pupil A). The relevant signal (pupil A) is necessarily processed to extract the targeted linguistic information. In this scenario, the auditory system can filter the interfering speech based on spatial cues (location) and voice cues (male/female pitch), among others. In this vein, the perception of speech is regarded as a cognitive task that is dependent on working memory capacity and linguistic capabilities., Therefore, the processing intensity varies by the external, acoustic factors (e.g., speech degradation by noise) and internal cognitive factors (e.g., intelligence).
Examples for the listening effort category are publications with the titles: “Cognitive processing workload during listening is reduced more by decreasing voice similarity than by increasing spatial separation between target and masker speech,” “Listening effort during sentence processing is increased for non-native listeners: A pupillometry study,” and “Impact of stimulus-related factors and hearing impairment on listening effort as indicated by pupil dilation.”
C1 “Listening Effort”: acoustic dimension
The category “listening effort,” C1, is a so-called mono-modal category because the experiment is limited to one modality. The speech perception in the listening effort category represents a cognitive task, and the masker sound represents the noise in the acoustic dimension. In both dimensions, the acoustic dimension, and cognitive task dimension are taking place in the auditory modality. However, the following two identified categories (i.e., auditory stressor, C2, and oddball, C3) might rely on two modalities at the same time, for example, the visual, for the cognitive task, and the auditory modality, for the noise. For that reason, both categories are bi-modal since the noise is partially independent of the cognitive task. Furthermore, both categories cover different aspects of occupational settings, like speech-related or text-processing tasks. If a listening effort study dealt with hearing impaired and non-native speaker samples, only the evidence from the normal hearing and native speaker sample was considered in the following analysis.
In the listening effort category, C1, the acoustic conditions (from 1 to 30 possible conditions) systematically manipulated the signal-to-noise ratio (SNR) to influence the intelligibility of the presented target speech. By changing the target and masker voice differences (e.g., gender, spatial, mono/binaural, single/multi-talker), the SNR was adopted. In some studies, the masker’s voice was replaced by a non-speech masker (white noise, stationary noise, pink noise, fluctuating noise), speech-shaped noise, and noise-vocoded speech. In other cases, the masker was replaced by multi-talker babble (e.g., four-talker, two-talker, five-talker, 10-talker, 30-talker) noise or a recording from a complex soundscape (e.g., café, open-plan office, swimming pool). The masker sound usually starts before the target speech and ends shortly after. During the retention and recall phase, no noise sound is presented in almost all cases.
C1 “Listening Effort”: cognitive task dimension
The task for the participant in the listening effort paradigm is often to repeat (recall and verbalize) the whole or parts of the sentence, keyword(s), and single words from the just presented target speech stimulus material. Different cognitive functions are engaged in such a speech reception threshold task (e.g., memory, attention, sensory, linguistics processes). The response could be verbal, by keyboard/computer interface (multiple-choice), or visual closeness tests. The cognitive task is to perceive the target speech in a masked stimulus, to memorize the relevant part of the stimulus, and finally to recall and respond. Consequently, both the noise and the cognitive task are hearing-based only without activating other modalities like vision. The auditory stimulus is partly relevant and partly irrelevant.
C1 “Listening Effort”: workload dimension
The induced cognitive workload (e.g., processing workload, listening effort, stress, demand, mental resource allocation) is estimated by task performance (correctly perceived target speech) and is doubled by objective measurements like pupillometry (mean/maximal peak amplitude), EEG, ECG, EDG, endocrinological makers (salivary cortisol), fNIRS, and fMRI (measurement dimension), see below in the measurement dimension. In addition, some studies investigated moderator variables like cognitive abilities, memory span tests, working memory capacity, need for recovery, reading span score, tiredness, fatigue (fatigue visual analog scale), perceived subjective workload (e.g., NASA TLX), listening effort, and linguistic closure abilities.
C1 “Listening Effort”: measurement dimension
The cognitive load is often investigated between conditions with modulated intelligibility and quantified by one measure. Always, the measure is accompanied by a behavioral parameter (performance).
The studies showed that the pupil dilation (amplitude, slope, latency of amplitude) was modulated with the listening effort: the lower the intelligibility, the higher the amplitude of the dilation of the pupil.[45–56,58–61,64–66, 69, 71, 74, 75, 77, 79-81] Often such effects were measured in a fixed retention period (“offline”) following the stimulus presentation and, as well, during the presentation period itself (online). No such effect was found in one pupillometry study. EEG-based publications reported time- and frequency-based effects for late positive potential components, P3, N1, N2, theta and alpha band power, N1, N2, and P3, and power analysis.
Endocrinological markers in the saliva (cortisol, CgA) did not reveal any effects. Cardiovascular activity studies found effects for the pre-ejection-period reactivity, mixed results for the heart-rate variability,, but no effect for high-frequency heart-rate variability. The effects on skin conductance (EDG, EDA) were somewhat mixed.,, Hemodynamic studies reported in regard to our research question unclear effects for the HbO2 concentration in the prefrontal cortex, measured by fNIRS,, and various activations in other brain regions, measured by fMRI.
Category 2 (C2) “Auditory Stressor”
Attentional resources are competed for by incoming sensory information because they are limited. Attention refers, in this view, to the depth of processing: auditory stimuli are processed only until the cognitive system can decide with a certain probability that the auditory stimulus is relevant. Thus, irrelevant stimuli are less processed than relevant stimuli for which more memory capacity is reserved, informational streams are prioritized, and conscious perception and transformation of stimuli are more likely.
An “auditory stressor” is a stimulus that is irrelevant to the current primary cognitive task. Still, the auditory stressor cannot be blatantly ignored but is processed to a certain extent. In this category, all content of the auditory sensory information is distractive and includes no information necessary for handling the primary cognitive task (or signal a threat). In this case, the participant needs to focus available mental resources only on the cognitive task.
Examples of the auditory stressor category are publications with titles such as: “Quantifying the effects of external factors on individual performance,” “Effects of low-intensity noise from aircraft or from the neighborhood on cognitive learning and electrophysiological stress responses,” and “Increased anxiety induced by listening to unpleasant music during stress exposure is associated with reduced blood pressure and ACTH responses in healthy men.”
C2 “Auditory Stressor”: acoustic dimension
Studies with an auditory stimulus categorized as an “auditory stressor” often contained a silence condition. In the remaining conditions, low-noise, silent white noise, light/pleasant music, nature sounds, and safe sound conditions are contrasted against high-noise, industrial noise, rock/unpleasant music, ambient noise, and threat sound conditions. Like the oddball stimulus (see below C3), the auditory stressor was presented during different stages of the primary task. However, the oddball stimulus is somewhat artificial compared to the real-life sounds used in the auditory stressor studies. Often the auditory stressor is presented for a longer time (e.g., up to around 15 min), during which several cognitive tasks must be executed.
C2 “Auditory Stressor”: cognitive task dimension
The cognitive tasks that played a role in these publications reach from the text-processing task (listening, reading, processing, writing), Stroop task, mental arithmetic task, proactive interference, response inhibition, flanker task, logics, recall/shifting/updating memory/attention, problem-solving, (simulated) car driving, n-/2-back task, delayed response task, gambling, sustained attention to response task (SART), a complex control task, stress test, deliberated eye-movement task, simulated surgery task, and object tracking task. Such tasks were used to test various cognitive systems (attention, memory, concentration, among others) in applied settings found at work. Sometimes, different difficulty levels (e.g., easy, medium, arduous task) of the cognitive tasks were applied and factored eventually with the acoustic conditions (e.g., silent, loud 1, loud 2). As a result, the cognitive task’s performance parameters varied strongly (e.g., hit rate, accuracy, reaction time).
C2 “Auditory Stressor”: workload dimension
The workload was varied with different levels of difficulty in the cognitive tasks. The load necessary to shield a cognitive process against noise potentially interacts with the mental workload stemming from the primary task. However, the cognitive tasks evaluated here are usually not in the auditory modality but are dominated by another modality (e.g., visual). That means that the cognitive processes in this category might block the auditory stimulus as a whole, and not just partially, as it was in the listening effort category, C1.
C2 “Auditory Stressor”: measurement dimension
In the auditory stressor category, the heterogeneity of the applied measurement technologies was larger than in the listening effort and oddball categories. Different measurement methods were applied: behavioral, cardiovascular, [86, 87, 90–92, 94, 99, 102–105, 107, 108, 112–114] skin conductance,,,,, endocrinology,, EEG,,,,,,, fNIRS,,,,, pupillometry,,, fMRI,,, and eye-tracking.,,,,
In all studies, the presence of noise affected at least one parameter of each measurement method. Several different movement types of a manually controlled computer mouse for behavioral measurements, blood pressure,, heart-rate variability (HRV),,,,,, heart rate, RR-intervals for cardiovascular activity, fluctuations in skin conductance,,,,, ACTH for endocrinology, N400, prefrontal cortex N100, feedback-related negativity, P300, steady-state visual potentials for EEG, O2Hb in fNIRS,,,, pupil dilation,,, respiration rate,, frequency ratio of HR, the power of the frequency bands,, default mode network in fMRI and frontal and parietal areas,, the gaze shifts in the eye-tracking,,, and urinary cortisol.
No effects were found in publications for the cortisol level, norepinephrine response, inter-beat-intervals,,, and eye movements.
Category 3 (C3) “Oddball”
The oddball paradigm is a well-established auditory paradigm to study the fundamentals of auditory processing. An oddball stimulus consists of a repeating sequence of, usually, two types of sounds: frequent “standard sounds” that are similar or equal and infrequent or unpredictable (or “odds”) “deviant sounds” that differ from the standard sounds perceptibly. Besides the insights into the fundamental structure of the auditory system, the paradigm allows for studying a process called attention capture. This way, it is possible to measure subjects’ susceptibility to distractions from an irrelevant stimulus, like the deviating oddball sound. The oddball paradigm can be realized as an active and passive response format. Either a deliberate response by the participant is required (active oddball, like, verbal response, button-press, among others) or an automatic or reflexive response from (passive oddball: e.g., EEG, pupil dilation) is recorded during the passive perception (non-responding) of the oddball stimulus (difference between standards and deviants) − or mixtures of both is applied.
Therefore, the passive oddball paradigm can function as a secondary task, which is combinable with various cognitive tasks. The attentional processes driving the oddball deviant detection occupy the remaining cognitive resource that is available besides the workload of the primary task. In this scenario, the oddball sounds are irrelevant to the primary task and can be staged at different phases of ongoing cognitive processes.
Examples for the oddball category are publications with the titles like: “Auditory task irrelevance: a basis for inattentional deafness,” “Disruption in neural phase synchrony is related to identification of inattentional deafness in real‐world setting,” and “Event-related potentials associated with auditory attention capture in younger and older adults.”
C3 “Oddball”: acoustic dimension
The considered oddball does not consist of several different conditions but a single one. However, the oddball paradigm can be used easily in real-life settings like an aircraft cabin or movie screening as a dual-task. The oddball sounds can be either single frequency sounds or sounds from the environment like a barking dog or a ringing bell, among others. Most considered publications do not require an active response. The time point the oddball paradigm is applied varies between the studies depending on the primary cognitive task.
C3 “Oddball”: cognitive task dimension
The oddball paradigm was combined with the following primary cognitive tasks: n-back task, verb-generation-task, aircraft piloting, movie watching, mental imagery tasks, speech-in-noise task, visuomotor tasks, and steering tasks. The primary task produced a cognitive workload that was eventually accompanied by the oddball task’s cognitive workload. The “oddball task” is to detect the deviant sound, which requires the auditory system to form a prediction about incoming sensory information.
C3 “Oddball”: workload dimension
Both the primary and secondary tasks evoke a workload that usually was not analyzed separately. Instead, the difference between the standard and deviant patterns was monitored. In this way, as a secondary task, the oddball task covers the supposed “remaining” cognitive workload. Once no more cognitive load capacity is available, the performance in the deviant detection of the oddball task decreases. The investigated cognitive system referred to dual-/multi-tasking, (visual, auditory, and cross-modal) attention, shielding, goal-directed behavior, (short-term) working memory, executive functions, decision making, perceptual load, general attentional resources. In addition, some studies applied moderator variables to investigate the subjectively perceived effort (NASA TLX, Stanford sleepiness scale).
C3 “Oddball”: measurement dimension
The oddball paradigm is a classic EEG experiment.,,,,,,, One study considered was Magnetoencephalography-based (MEG). In the EEG experiments, analysis can be based on a time-domain analysis (event-related potential, ERP) or frequency-domain analysis (power spectrum). For the ERPs, amplitudes in defined time-slots relative to the auditory event can be investigated: mismatch negativity (MMN), deviant-related negativity (DRN), late positive potential (LPP), P3a, P300, N1, N2, and early/late/frontal P3.
In the majority of the studies, effects can be detected for at least one of the investigated ERPs even in the absence of an active response,,,,,,,,, as well as for the MEG in the auditory steady-state response. Similarly, the power analysis reveals effects in the phase reset, alpha, beta, theta band, power ratios, and the general “brain state.” ,,,
| Discussion|| |
The estimation of the current cognitive workload in humans is complex. However, measuring cognitive workload stemming from two cognitive processes (one task- and one noise-related process) that are potentially partially interconnected (modality general) and partially separate (modality specific) is an even more arduous endeavor. Psycho- and neurophysiological correlates of cognitive workload, like ERPs, variations in cardiovascular activity, and dilations of the pupil, have received attention from research communities for different purposes. Objective measures are often used for understanding fundamental psychological functions. As well, the measurements are capable of predicting the performance in cognitive tasks and estimating the current workload in various situations. In this vein, correlates reveal to some extent the involvement of a particular cognitive process when confronted with a certain type of task. Nowadays, it is possible to record and monitor changes in the cognitive state with a sensor unobtrusively. In that case, the worker can be immersed in a task without distractions from subjective effort reports. Thus, the well-established subjective report of cognitive workload can be partially replaced and extended with a myriad of data from objective measurement methods introduced above. The recorded parameters represent specific aspects of the activity of the mind (e.g., neuronal activity, hemodynamic blood flow, muscle contractions). Additionally, data from body sensors have a high granularity (sampling rate) and can be monitored during the task processing in real-time.
However, the question that we raised is whether such measurement methods can be used as a marker for cognitive workload in situations with and without noise. The reviewed results indicate that the described measurement methods can detect noise-related effects. In the following, we want to structure the challenges formulated in the literature to help develop measurement systems for occupational safety evaluations (i.e., combinations of sensors). We propose occupational scenarios in which the findings can be further studied in-depth. Finally, an integrated measurement system of several sensors relies on the selection of suitable sensory channels (e.g., EEG and PD) that have demonstrated sensitivity to some extent and benefit from each other when combined. A limitation of this review is the absence of a quantitative sensitivity comparison between objective measures used. The sensitivity should be investigated in future studies with an experimental setting that is identical to all tested objective measures, which was not the case for the publications reported.
The measurement problem of cognitive workload becomes eminent in the limitation: cognitive workload can only be measured relative to a specific cognitive test or mental task. The design of the cognitive task determines how the cognitive system is tested. A cognitive task might be designed to affect the targeted cognitive system (and relevant subsystems) but is inherently limited in terms of generalizability. Vice versa, the capability of the cognitive system to adapt to an infinite number of ever-changing demands of the environment excludes the possibility of a single encompassing test to grasp the involved cognitive resources exclusively. When applying objective workload correlates, the desire is to circumvent this task-specificity and estimate workload universally (like “50% of the memory capacity is remaining”). The effects of noise on cognitive workload differ contextually and depend on the momentary intentions of the listener. Noise might evoke less cognitive workload if no cognitive task is being processed. This highlights the missing generalizability of almost all classical laboratory psychological paradigms. Instead, simulated work environments might offer the required complexity along with the controllability of an experiment (e.g., virtual reality).
Workload measurements depend on the individual listener’s capacity for the task, the resilience to noise in general, and the sensitivity of the measurement method. Therefore, the outcome of the total workload from combined noise- and task-related processes is an interaction of several factors. Ultimately, we summarized three occupational settings that are oriented on the categories formed based on the literature. In the three scenarios, those objective measures are discussed that report the most frequently significant results in each category. The scenarios provide guidance for designing experiments in three occupational settings and lay the foundation for sensitivity comparisons. First, if studying workers’ mental workload who must make phone calls in an open-plan office, the measurement methods from “speech-related scenarios” are a recommended starting point, see occupational setting 1. Second, if a worker must read and understand a technical manual close to a noisy production facility, experiments can be designed best along with findings from the occupational setting 2: “distractive sounds.” Third, if control room workers are investigated who monitor continuously over hours relevant and irrelevant auditory and visual information, partially informative sound scenarios can be helpful, see occupational setting 3. As a most promising approach, we propose a multidimensional approach with a set of sensors that integrates data from two or more sensory channels to overcome weaknesses that measurement techniques might have stand-alone, see prospects.
Occupational setting 1: speech-related scenario (listening effort)
The listening effort category is characterized by a stimulus that is in part relevant (“target”) and in part irrelevant (“masker”). The field of listening effort research deals with challenges in the intelligibility of speech signals. Acoustic speech signals are masked by sound (e.g., background noise, competing talkers). The consensus among researchers is that listening effort is the energy needed to meet the demands stemming from understanding a distorted speech signal. Notably, the theory emphasizes “motivation” (willingness to understand the target speech), “fatigue” (extinction over time of involved resources), and the localization of cognitive resources (experienced and invested listening effort).
The disturbance, that is, the masker noise or speech, is a piece of auditory information that needs to be split from the target speech. For instance, the similarity between masker sound and the target speech substantially influences the difficulty of attending the target speech. The cognitive system strained in a speech reception task is primarily auditory (e.g., speech reception test). If a universal attentional resource is involved that constitutes the cognitive workload, it would not be split among modalities (as it is necessary for the bimodal auditory stressor category). Notably, the noise in this category is not presented during the retention phase but only in the perception phase, followed by the processing phases (memorizing the understood speech before retention). Consequently, the research area “listening effort” is interested in the momentary capacity to process speech. As a limiting factor, cognitive abilities played a role.
The measurement dimension in listening effort publications is dominated by pupillometry, the dilation of the pupil in the eye. The effects of varying speech intelligibility are reflected in the maximal peak and the latency of the maximal peak during the task. On a smaller magnitude, EEG studies report effects in ERPs and frequency-based parameters for listening effort. Both pupil dilation and EEG are fast-responding markers of cognitive workload that correlate with activations in the central nervous system representative of cognitive mechanisms involved. Findings in EEG-based studies double the characteristic brain activity involved in pupil dilation, which is mainly controlled by phasic brainstem activations. The momentary pupil diameter results from the balance between the sympathetic and parasympathetic nervous systems. The dilation of the pupil is triggered by activity in the locus coeruleus. Therefore, the fluctuations in pupil dilation are often interpreted as a proxy for cognitive load. The reaction of the measured pupil dilation effects differs for external (e.g., physiology of the eyelid) and internal factors (e.g., linguistic complexity interacts with knowledge). A limitation that should be considered when designing studies in the field.
Occupational setting 2: distractive sound scenario (auditory stressor)
The auditory stressor stimulus is irrelevant to the cognitive task. Various real-life disturbing sounds from the environment have been used in the analyzed publications. Therefore, the auditory stressor stimuli can be combined with numerous psychological tests, paradigms, and cognitive tasks and allow a bimodal setting. The exposure duration in this category is much longer than in both the oddball and the listening effort categories. In this category, the cognitive task is to focus attention on the primary task and suppress the attention capture of the secondary task (noise). The auditory information is completely irrelevant and withholds no information relevant to the listener.
On the measurement dimension, the auditory stressor category was dominated by cardiovascular correlates of workload (ECG; HRV, HR) and skin conductance (EDG; skin conductance fluctuations). In sum, EEG, fNIRS, and eye-tracking played a minor role in the analyzed literature. Naturally, the auditory stressor category contains a wide variety of sounds. The focus of such studies is typically directed towards external stressors and cognitive workload to trigger arousal, that is, the stimulation of the ascending reticular activating system. It can be observed in the terminology of such publications that workload is used ambiguously as the effector and, at the same time, describes the outcome in the cognitive system (e.g., a high load condition evokes high load in the cognitive system). In this field of study, individual resilience might play less of a role, and emotion or mood is sometimes considered a moderator.
Occupational setting 3: partially informative sound scenario (oddball)
The oddball experiments usually did not contain a silence condition. Instead, the cognitive resources are occupied with the pattern building induced by the auditory oddball stimulus. That relates mainly to the auditory attentional resources that are remaining to detect the deviant sound. Various experimental psychological paradigms and ecologically valid tasks were combined with the oddball stimulus to test multi-tasking and attentional resources. Clearly, the oddball paradigm is a classic EEG paradigm and is therefore dominated by monitoring EEG responses to standards and deviant sounds. On the one hand, event-related potentials were investigated (MMN, DRN, LPP, among others), and, on the other hand, markers in the power spectrum showed effects (phase reset, alpha, beta theta power band).
In this way, the attentional bottleneck can be studied during, for example, a dual-task situation where the auditory sensory information (secondary task) is irrelevant to the primary task. For example, if all attentional resources are bound to the primary task, a phenomenon coined “inattentional deafness” occurs. It describes the momentary deafness to a sound due to exhaustion of attentional resources and selective attention for another competing primary process. The oddball task allows applications outside the laboratory in the real dynamic world as part of an online monitoring system to measure the momentary cognitive workload in, for example, a dual-task setting.
| Conclusion|| |
In this review, we can show that in various occupational settings, different applied measuring methods will be able to capture aspects of the capacity utilization in the cognitive system due to auditory load combined with a mental task. The resilience towards auditory disturbances, like noise, may represent an independent resource that is coupled to general resources and differently developed among individuals. The individual factor in the workload measurement was not sufficiently addressed in many considered publications. It means that the measurement technique is not individually calibrated to the capacity of an individual in a certain task (trait). Instead, the authors of the publications are usually interested in the functional relations (inter-individual overlap). The publications reporting the use of the listening effort paradigm are dominantly investigated with pupillometry and EEG, capturing neurophysiological correlations of cognitive workload. Similarly, the oddball publications dominantly report using EEG to detect quantitative effects. Both settings include a (partially) irrelevant auditory stimulus, and effects are measured in activities of the central nervous system. The domination of the neurophysiological markers in these two auditory attention settings hints at two aspects: First, the effect stemming from the auditory task is often not “stressful” enough to modulate the peripheral correlates like heart rate or change the skin conductance but is a rather subtle effect that can be measured on the neuronal level. Second, the timescale of the effects is relatively short and, therefore, needs to be measured in a low latency correlate more directly connected to the neuronal underpinnings, for example, EEG and PD. The latency of peripheral correlates cannot resolute short-time scale disturbances, for example, EEG and EDG. As a third option, both obstructions may coexist and interact. In contrast, if the auditory stressor includes a completely irrelevant auditory stimulus, dominantly peripheral correlates like ECG and EDG have been applied that aim on the body activation (e.g., stress). Such publications are not primarily focused on the fine-grained attention resources and attentional transitions but broader scale body responses, like arousal and stress. Cognitive workload can induce stress and general body activation as indicated by ECG and EDG responses. Arousal is regarded as a far-reaching activation of the body to facilitate task demands rather than providing resources in a focused fashion for one specific subsystem. Instead, processing capacity during listening to a two-speaker scenario must be moved adaptively from one voice to the other. Monitoring different aspects of mental load in multimodal measurement systems might uncover effects that are indistinguishable in single parameters (e.g., due to ceiling and floor effects).
From the perspective of the considered publications, pupillometry, EEG, EDG, and ECG should be combined in an integrated measurement system to cover responses from the three theoretical escalation levels of perceptual attention, cognitive workload, and whole-body stress. EEG and pupillometry can capture fast neuronal responses in response to subtle attentional responses to noise. At the same time, these correlates might not differentiate persisting noise-induced stress that effects the cardiovascular system or skin responses. Especially regarding field studies in an occupational setting, the identified measurement methods shall be further assessed along with usability aspects: reliability (contextual stability), mobility (sensor), and intrusiveness (during the measurement). The integration of the signals delivered from several channels (i.e., multiple sensor channels/techniques in one integrated measurement system) would also allow filling momentary gaps of signals that are temporarily uninformative (i.e., missing data) or coarse on a certain parameter at all times (i.e., resolution). Common ways to integrate signals (and remove artifacts) from many sensor channels may comprise machine learning approaches and other established classification methods.,,
In our view, a multidimensional approach that covers the identified sensor channels holds the potential to predict task- and noise-related cognitive workload, see [Figure 3]. The momentary cognitive workload might be estimated best by gathering information on (1) “subjectively” perceived workload (self-report, e.g., for data labeling in machine learning), (2) task performance (how “well” is the task performed, e.g., hit rate, latency, lapses/errors), and (3) “objectively” measured workload from several sources ([neuro]-physiological correlates of workload).
|Figure 3 The proposed multidimensional cognitive workload estimation approach with desired sensor channels.|
Click here to view
Ideally, the measurement may consider parameters to allow statements considering the individual case: capability regarding the cognitive task, resilience regarding the noise disturbance (i.e., susceptibility), motivation regarding the fulfillment of the task, and the overcoming of obstacles like noise, which includes the expected reward and experienced/invested effort, and mood.
Thanks to Georg Brockt, Fabian Heisterkamp, Hannah Rolf, and Andreas Wojtysiak for helpful comments on the manuscript draft.
VI. Appendix 1
Data extraction table: “Appendix1.PDF”.
Financial support and sponsorship
Nil.Conflicts of interest
There are no conflicts of interest.
| References|| |
Fink D. A new definition of noise: noise is unwanted and/or harmful sound. Noise is the new ‘secondhand smoke’. In Proceedings of Meetings on Acoustics 178ASA. Acoustical Society of America; 2019.
Kahneman D, Beatty J. Pupil diameter and load on memory. Science 1966;154:1583-5.
Patterson RD. Auditory warning sounds in the work environment. Philos Trans R Soc London B Biol Sci 1990;327:485-92.
Bregman AS. Asking the “what for” question in auditory perception. Perceptual Organization. Routledge; 2017 99-118.
Warren RM. Auditory Perception: A new New Synthesis. Vol. 109. Elsevier; 2013.
Shinn-Cunningham BG. Object-based auditory and visual attention. Trends Cogn Sci 2008;12:182-6.
Snyder JS, Gregg MK, Weintraub DM et al.
Attention, awareness, and the perception of auditory scenes. Front Psychol 2012;3:15.
Basner MD, Babisch W, Davis A et al.
Auditory and non-auditory effects of noise on health. Lancet 2014;383:1325-32.
Van Dijk F, Souman A, De Vries F. Non-auditory effects of noise in industry. Int Arch Occup Environ Health 1987;59:133-45.
Stansfeld SA, Matheson MP. Noise pollution: non-auditory effects on health. Br Med Bull 2003;68:243-57.
Gaillard AW. Concentration, stress and performance. In Performance Under Stress. CRC Press; 2018 75-92.
Braboszcz C, Delorme A. Lost in thoughts: neural markers of low alertness during mind wandering. Neuroimage 2011;54:3040-47.
Banbury SP, Berry DC. Office noise and employee concentration: Identifying identifying causes of disruption and potential improvements. Ergonomics 2005;48:25-37.
Sander ELJ, Marques C, Birt JR et al.
Open-plan office noise is stressful: multimodal stress detection in a simulated work environment. J Manage Organ 27; 2021 1021–37.
Schlittmeier SJ, Hellbrück J. Background music as noise abatement in open‐plan offices: A a laboratory study on performance effects and subjective preferences. Appl Cogn Psychol 2009;23:684-97.
Schlittmeier SJ, Feil A, Liebl A, Hellbr Ck JR. The impact of road traffic noise on cognitive performance in attention-based tasks depends on noise level even within moderate-level ranges. Noise Health 2015;17:148.
] [Full text]
Fujiwara D, Lawton RN. Noise and subjective wellbeing. Handbook on Wellbeing, Happiness and the Environment. Edward Elgar Publishing; 2020.
Young MS, Brookhuis KA, Wickens CD, Hancock PA. State of science: mental workload in ergonomics. Ergonomics 2015;58:1-17.
Lundquist P, Holmberg K, Landstrom U. Annoyance and effects on work from environmental noise at school. Noise Health 2000;2:39.
] [Full text]
Litchfield P, Cooper C, Hancock C, Watt P. Work and Wellbeing wellbeing in the 21st Century †. Int J Environ Res Public Health 2016;13:1065.
Howard AE. The Changing Nature of Work. Jossey-Bass; 1995.
Quick JCE, Tetrick LE. Handbook of Occupational Health Psychology. American Psychological Association; 2011.
Basner M, Babisch W, Davis A et al.
Auditory and non-auditory effects of noise on health. Lancet 2014;383:1325-32.
European P. and U. The Council of the European, Directive 2003/10/EC of the European Parliament and of the Council of 6 February 2003 on the minimum health and safety requirements regarding the exposure of workers to the risks arising from physical agents (noise). Off J Eur Union 2003;42:38-44.
van den Bosch F KA , Welch D, Andringa TC. The evolution of soundscape appraisal through enactive cognition. Front Psychol 2018; 9: 1129.
Bengtsson J, Waye KP, Kjellberg A. Evaluations of effects due to low-frequency noise in a low demanding work situation. J Sound Vibrat 2004;278:83-99.
Liebl A, Haller J, Jödicke B, Baumgartner H, Schlittmeier S, Hellbrück J. Combined effects of acoustic and visual distraction on cognitive performance and well-being. Appl Ergon 2012;43:424-34.
Brocolini L, Parizet E, Chevret P. Effect of masking noise on cognitive performance and annoyance in open plan offices. Appl Acoust 2016, 114: 44–55.
Popp RF, Maier S, Rothe S et al.
Impact of overnight traffic noise on sleep quality, sleepiness, and vigilant attention in long-haul truck drivers: results of a pilot study. Noise Health 2015;17:387.
] [Full text]
Hart SG. NASA task Task load Load index Index (TLX). 1986.
Yannakakis GN, Martínez HP. Ratings are overrated! Front ICT 2015;2:13.
Tsouti V, Boutopoulos C, Zergioti I, Chatzandroulis S. Capacitive microsystems for biological sensing. Biosens Bioelectron 2011;27:1-11.
Sanches P, Hook K, Vaara E et al.
Mind the body! Designing a mobile stress management application encouraging personal reflection. Proceedings of the 8th ACM Conference on Designing Interactive Systems; 2010.
Radüntz T, Fürstenau N, Mühlhausen T, Meffert B. Indexing mental workload during simulated air traffic control tasks by means of dual frequency head maps. Front Physiol 2020;11:300.
Beatty J. Pupillometric Measurement of Cognitive Workload. California Univ Los Angeles Dept of Psychology; 1977.
Borghini G, Astolfi L, Vecchiato G, Mattia D, Babiloni F. Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness. Neurosci Biobehav Rev 2014;44:58-75.
Herff C, Heger D, Fortmann O, Hennrich J, Putze F, Schultz T. Mental workload during n-back task—quantified in the prefrontal cortex using fNIRS. Front Hum Neurosci 2014;7:935.
Dussault C, Jouanin JC, Philippe M, Guezennec CY. EEG and ECG changes during simulator operation reflect mental workload and vigilance. Aviat Space Environ Med 2005;76:344-51.
Pirrera S, De Valck E, Cluydts R. Nocturnal road traffic noise: A a review on its assessment and consequences on sleep and health. Environ Int 2010;36:492-8.
Stansfeld S, Haines M, Brown B. Noise and health in the urban environment. Rev Environ Health 2000;15:43-82.
Hegewald J, Schubert M, Freiberg A et al.
Traffic noise and mental health: a systematic review and meta-analysis. Int J Environ Res Public Health 2020;17:6175.
7, A.A., Technische Regel für Arbeitsstätten-Lärm. Gemeinsames Ministerialblatt 2018;24:456-69.
Page MJ, McKenzie JE, Bossuyt PM et al.
The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Int J Surg 2021;88:105906.
Grames EM, Stillman AN, Tingley MW, Elphick CS. An automated approach to identifying search terms for systematic reviews using keyword co‐occurrence networks. Methods Ecol Evol 2019;10:1645-54.
Zekveld AA, Rudner M, Kramer SE, Lyzenga J, Rönnberg J. Cognitive processing load during listening is reduced more by decreasing voice similarity than by increasing spatial separation between target and masker speech. Front Neurosci 2014;8:88.
Gwyneth AL, Gavin MB. Autonomic nervous system correlates of speech categorization revealed through pupillometry. Front Neurosci 2020;13:1418.
Borghini G, Hazan V. Listening effort during sentence processing is increased for non-native listeners: A pupillometry study. Front Neurosci 2018;12:152.
Ohlenforst B, Zekveld AA, Lunner T et al.
Impact of stimulus-related factors and hearing impairment on listening effort as indicated by pupil dilation. Hear Res 2017;351:68-79.
Koelewijn T, Zekveld AA, Lunner T, Kramer SE. The effect of reward on listening effort as reflected by the pupil dilation response. Hear Res 2018;367:106-112.
Koelewijn T, Versfeld NJ, Kramer SE. Effects of attention on the speech reception threshold and pupil response of people with impaired and normal hearing. Hear Res 2017;354:56-63.
Koelewijn T, Shinn-Cunningham BG, Zekveld AA, Kramer SE. The pupil response is sensitive to divided attention during speech processing. Hear Res 2014;312:114-20.
Koelewijn T, de Kluiver H, Shinn-Cunningham BG, Zekveld AA, Kramer SE. The pupil response reveals increased listening effort when it is difficult to focus attention. Hear Res 2015;323:81-90.
Zekveld AA, Kramer SE. Cognitive processing load across a wide range of listening conditions: Insights insights from pupillometry. Psychophysiology 2014;51:277-84.
McGarrigle R, Rakusen L, Mattys S. Effortful listening under the microscope: Examining examining relations between pupillometric and subjective markers of effort and tiredness from listening. Psychophysiology 2021;58:e13703.
McGarrigle R, Dawes P, Stewart AJ, Kuchinsky SE, Munro KJ. Pupillometry reveals changes in physiological arousal during a sustained listening task. Psychophysiology 2017;54:193-203.
Koo M, Jeon J, Moon H et al.
Effects of noise and serial position on free recall of spoken words and pupil dilation during encoding in normal-hearing adults. Brain Sci 2021;11:277.
Bertoli S, Bodmer D. Novel sounds as a psychophysiological measure of listening effort in older listeners with and without hearing loss. Clin Neurophysiol 2014;125:1030-41.
Zekveld AA, Kramer SE, Festen JM. Cognitive load during speech perception in noise: The the influence of age, hearing loss, and cognition on the pupil response. Ear Hear 2011;32:498-510.
Zekveld AA, Kramer SE, Festen JM. Pupil response as an indication of effortful listening: the influence of sentence intelligibility. Ear Hear 2010;31:480-90.
Winn MB, Edwards JR, Litovsky RY. The impact of auditory spectral resolution on listening effort revealed by pupil dilation. Ear Hear 2015;36:e153-e165.
Wang Y, Naylor G, Kramer SE et al.
Relations Between Self-Reported Daily-Life Fatigue, Hearing Status, and Pupil Dilation During a Speech Perception in Noise Task. Ear Hear 2018;39:573-82.
Plain B, Richter M, Zekveld AA, Lunner T, Bhuiyan T, Kramer SE. Investigating the influences of task demand and reward on cardiac pre-ejection period reactivity during a speech-in-noise task. Ear Hear 2021;42:718-31.
Mackersie CL, Macphee IX, Heldt EW. Effects of hearing loss on heart rate variability and skin conductance measured during sentence recognition in noise. Ear Hear 2015;36:145-54.
Kramer SE, Teunissen CE, Zekveld AA. Cortisol, chromogranin A, and pupillary responses evoked by speech recognition tasks in normally hearing and hard-of-hearing listeners: A pilot study. Ear Hear 2016;37:126S-35S.
Koelewijn T, Zekveld AA, Festen JM, Kramer SE. Pupil dilation uncovers extra listening effort in the presence of a single-talker masker. Ear Hear 2012;33:291-300.
Alhanbali S, Dawes P, Millman RE, Munro KJ. Measures of listening effort are multidimensional. Ear Hear 2019;40:1084-97.
Hjortkjær J, Märcher-Rørsted J, Fuglsang SA, Dau T. Cortical oscillations and entrainment in speech processing during working memory load. Eur J Neurosci 2020;51:1279-89.
Andéol G, Suied C, Scannella S, Dehais F. The spatial release of cognitive load in cocktail party is determined by the relative levels of the talkers. J Assoc Res Otolaryngol 2017;18:457-64.
Zhang Y, Lehmann A, Deroche M. Disentangling listening effort and memory load beyond behavioural evidence: Pupillary response to listening effort during a concurrent memory task. PLoS One 2021;16:e0233251.
Finke M, Büchner A, Ruigendijk E, Meyer M, Sandmann P. On the relationship between auditory cognition and speech intelligibility in cochlear implant users: An ERP study. Neuropsychologia 2016;87:169-81.
Winn MB, Moore AN. Pupillometry reveals that context benefit in speech perception can be disrupted by later-occurring sounds, especially in listeners with cochlear implants. Trends Hear 2018;22:2331216518808962.
Rowland SC, Hartley DEH, Wiggins IM. Listening in naturalistic scenes: what can functional near-infrared spectroscopy and intersubject correlation analysis tell us about the underlying brain activity? Trends in Hearing, 2018;22:2331216518804116.
Kelly M, McMahon C, Boisvert I et al.
Objective assessment of listening effort: coregistration of pupillometry and EEG. Trends Hear 2017;21:2331216517706396.
Kadem M, Herrmann B, Rodd JM, Johnsrude IS. Pupil dilation is sensitive to semantic ambiguity and acoustic degradation. Trends Hear 2020;24:2331216520964068.
Catherine MM, Boisvert I, de Lissa P et al.
Monitoring alpha oscillations and pupil dilation across the performance-intensity function. Front Psychol 2016;7:745.
Francis AL, MacPherson MK, Chandrasekaran B, Alvar AM et al.
Autonomic nervous system responses during perception of masked speech may reflect constructs other than subjective listening effort. Front Psychol 2016;7:263.
Zekveld AA, Festen JM, Kramer SE. Task difficulty differentially affects two measures of processing load: The the pupil response during sentence processing and delayed cued recall of the sentences. J Speech Lang Hear Res 2013;56:1156-65.
Seeman S, Sims R. Comparison of psychophysiological and dual-task measures of listening effort. J Speech Lang Hear Res 2015;58:1781-92.
Lau MK, Hicks C, Kroll T, Zupancic S. Effect of auditory task type on physiological and subjective measures of listening effort in individuals with normal hearing. J Speech Lang Hear Res 2019;62:1549-60.
Alhanbali S, Munro KJ, Dawes P, Carolan PJ, Millman RE. Dimensions of self-reported listening effort and fatigue on a digits-in-noise task, and association with baseline pupil size and performance accuracy. Int J Audiol 2021;60:762-72.
Winn M, Edwards JR. The impact of spectral resolution on listening effort revealed by pupil dilation. J Acoust Soc Am 2013;134:4233.
Mackersie CL, Cones H. Subjective and psychophysiological indexes of listening effort in a competing-talker task. J Am Acad Audiol 2011;22:113-22.
Wisniewski MG, Thompson ER, Iyer N, Estepp JR, Goder-Reiser MN, Sullivan SC. Frontal midline θ power as an index of listening effort. NeuroReport, 2015;26:94-9.
Manan HA, Yusoff AN, Franz EA et al.
The effects of background noise on brain activity using speech stimuli on healthy young adults. Neurol Psychiatry Brain Res 2013;19:207-15.
Carneiro D, Novais P. Quantifying the effects of external factors on individual performance. Fut Generat Comp Syst 2017;66:171-86.
Trimmel M, Atzlsdorfer J, Tupy N, Trimmel K. Effects of low intensity noise from aircraft or from neighbourhood on cognitive learning and electrophysiological stress responses. Int J Hyg Environ Health 2012;215:547-54.
Jezova D, Hlavacova N, Makatsori A, Duncko R, Loder I, Hinghofer-Szalkay H. Increased anxiety induced by listening to unpleasant music during stress exposure is associated with reduced blood pressure and ACTH responses in healthy men. Neuroendocrinology 2013;98:14450.
Du M, Jiang J, Li Z, Man D. The effects of background music on neural responses during reading comprehension. Sci Rep 2020;10:18651.
Jahncke H, Hygge S, Halin N, Green AM, Dimberg K. Open-plan office noise: Cognitive performance and restoration. J Environ Psychol 2011;31:373-82.
Sauer J, Nickel P, Wastell D. Designing automation for complex work environments under different levels of stress. Appl Ergon 2013;44:119-27.
Horrey WJ, Lesch MF, Garabet A, Simmons L, Maikala R. Distraction and task engagement: How how interesting and boring information impact driving performance and subjective and physiological responses. Appl Ergon 2017;58:342-48.
Abbasi AM, Motamedzade M, Aliabadi M, Golmohammadi R, Tapak L. Combined effects of noise and air temperature on human neurophysiological responses in a simulated indoor environment. Appl Ergon 2020;88:103189.
Sinanaj I, Montandon ML, Rodriguez C et al.
Neural underpinnings of background acoustic noise in normal aging and mild cognitive impairment. Neuroscience 2015;310:410-21.
Mandrick K, Peysakhovich V, Rémy F, Lepron E, Causse M. Neural and psychophysiological correlates of human performance under stress and high mental workload. Biol Psychol 2016;121:62-73.
Qu J, Rizak JD, Zhao L, Li M, Ma Y. Effects of visual working memory on brain information processing of irrelevant auditory stimuli. PLoS ONE, 2014;9:e89989.
Stella B, Linda G, Monicque ML. Acute stress modulates feedback processing in men and women: differential effects on the feedback-related negativity and theta and beta power. PLoS One 2014;9:e95690.
Shavit-Cohen K, Golumbic EZ. The dynamics of attention shifts among concurrent speech in a naturalistic multi-speaker virtual environment. Front Hum Neurosci 2019;13:386.
Sabri M, Humphries C, Verber M et al.
Neural effects of cognitive control load on auditory selective attention. Neuropsychologia 2014;61:269-79.
Parent M, Peysakhovich V, Mandrick K, Tremblay S, Causse M. The diagnosticity of psychophysiological signatures: Can we disentangle mental workload from acute stress with ECG and fNIRS? Int J Psychophysiol 2019;146:139-47.
Gabbard R, Fendley M, Dar IA, Warren R, Kashou NH. Utilizing functional near-infrared spectroscopy for prediction of cognitive workload in noisy work environments. Neurophotonics 2017;4:041406.
Smucny J, Rojas DC, Eichman LC, Tregellas JR. Neuronal effects of auditory distraction on visual attention. Brain Cogn 2013;81:263-70.
Yun SH, Park SJ, Sim CS et al.
The responses of subjective feeling, task performance ability, cortisol and HRV for the various types of floor impact sound: A pilot study. Ann Occup Environ Med 2017;29:13.
van der Zwaag F MD , Dijksterhuis C, de Waard D et al.
The influence of music on mood and performance while driving. Ergonomics 2012;55:12-22.
Sauer J, Kao CS, Wastell D, Nickel P. Explicit control of adaptive automation under different levels of environmental stress. Ergonomics 2011;54:755-66.
Cottrell ND, Barton BK. The impact of artificial vehicle sounds for pedestrians on driver stress. Ergonomics 2012;55:1476-86.
Sabine H, Andreas K. Quantifying intermodal distraction by emotion during math performance: an electrophysiological approach. Front Psychol 2019;10:439.
Huiying W, Sze NN, Zeng Q, Hu S. Effect of music listening on physiological condition, mental workload, and driving performance with consideration of driver temperament. Int J Environ Res Public Health 2019;16:2766.
Alvarsson JJ, Wiens S, Nilsson ME. Stress recovery during exposure to nature sound and environmental noise. Int J Environ Res Public Health 2010;7:1036-46.
Johansson R, Holmqvist K, Mossberg F, Lindgren M. Eye movements and reading comprehension while listening to preferred and non-preferred study music. Psychol Music 2012;40:339-56.
Hodgetts HM, Vachon F, Tremblay S. Background sound impairs interruption recovery in dynamic task situations: Procedural procedural conflict? Appl Cogn Psychol 2014;28:10-21.
DiGirolamo GJ, Patel N, Blaukopf CL. Arousal facilitates involuntary eye movements. Exp Brain Res 2016;234:1967-76.
Liu X, Iwanaga K, Koda S. Circulatory and central nervous system responses to different types of mental stress. Ind Health 2011;49:265-73.
Cvijanović N, Kechichian P, Janse K, Kohlrausch A. Effects of noise on arousal in a speech communication setting. Speech Commun 2017;88:127-36.
Waterland P, Khan FS, Ismaili E, Cheruvu C. Environmental noise as an operative stressor during simulated laparoscopic surgery. Surg Laparosc Endosc Percutan Tech 2016;26:133-6.
Crameri L, Hettiarachchi I, Hanoun S. Feasibility study of skin conductance response for quantifying individual dynamic resilience. IEEE Transactions on Systems, Man, and Cybernetics: Systems 2020 -October:1764-71.
Wolfgang E, Kattner F, Klippenstein E, Kreis M, Marquis-Favre C. Short-term noise annoyance and electrodermal response as a function of sound-pressure level, cognitive task load, and noise sensitivity. Noise Health 2020;22:46-55.
Gao J, Liu S, Feng Q et al.
Quantitative evaluations of the effects of noise on mental workloads based on pupil dilation during laparoscopic surgery. Am Surg 2018;84:1951-56.
Callan DE, Gateau T, Durantin G, Gonthier N, Dehais F. Disruption in neural phase synchrony is related to identification of inattentional deafness in real-world setting. Hum Brain Map 2018;39:2596-608.
Morrison C, Kamal F, Campbell K, Taler V. The influence of different types of auditory change on processes associated with the switching of attention in younger and older adults. Neurobiol Aging 2020;96:197-204.
Morrison C, Kamal F, Campbell K, Taler V. Event-related potentials associated with auditory attention capture in younger and older adults. Neurobiol Aging 2019;77:20-5.
Dehais F, Duprès A, Blum S et al.
Monitoring pilot’s mental workload using erps and spectral power with a six-dry-electrode eeg system in real flight conditions. Sensors (Basel) 2019;19:1324.
Friedrich EVC, Scherer R, Sonnleitner K, Neuper C. Impact of auditory distraction on user performance in a brain-computer interface driven by different mental tasks. Clin Neurophysiol 2011;122:2003-9.
Scheer M, Bulthoff HH, Chuang LL. Auditory task irrelevance: a basis for inattentional deafness. Hum Fact 2018;60:428-40.
Scheer M, Bülthoff HH, Chuang LL. Steering demands diminish the early-P3, late-P3 and RON components of the event-related potential of task-irrelevant environmental sounds. Front Hum Neurosci 2016;10:73.
Stephan G, Arnau S, Karthaus M, Reiser JE, Wascher E. Age-related differences in pro-active driving behavior revealed by EEG measures. Front Hum Neurosci 2018;12:321.
Yokota Y, Naruse Y. Phase coherence of auditory steady-state response reflects the amount of cognitive workload in a modified N-back task. Neurosci Res 2015;100:39-45.
van der Heiden RMA, Janssen CP, Donker SF, Kenemans JL. The influence of cognitive load on susceptibility to audio. Acta Psychol (Amst) 2020;205:103058.
Pichora-Fuller MK, Kramer SE, Eckert MA et al.
Hearing impairment and cognitive energy: the framework for understanding effortful listening (FUEL). Ear Hear 2016;37:5S-27S.
Poeppel D. The neurobiology of language. In Marcus G, Freeman J, eds. The Future of the Brain: Essays by the World’s Leading Neuroscientists. Princeton, NJ: Princeton University Press, 2015;139-48.
Poeppel D, Idsardi WJ, Van Wassenhove V. Speech perception at the interface of neurobiology and linguistics. Philos Trans R Soc B: Biol Sci 2008;363:1071-86.
Francis AL, MacPherson MK, Chandrasekaran B, Alvar AM.Autonomic nervous system responses during perception of masked speech may reflect constructs other than subjective listening effort. Front Psychol 2016;7:263.
Stephen CR, Douglas EHH, Ian MW. Listening in naturalistic scenes: what can functional near-infrared spectroscopy and intersubject correlation analysis tell us about the underlying brain activity? Trends Hear 2018;22:2331216518804116.
Baddeley A. Working memory. Science 1992;255:556-9.
Milham MP, Banich MT, Barad V. Competition for priority in processing increases prefrontal cortex’s involvement in top-down control: an event-related fMRI study of the stroop task. Cogn Brain Res 2003;17:212-22.
Attneave F. Some informational aspects of visual perception. Psychol Rev 1954;61:183.
Dalton P, Hughes RW. Auditory attentional capture: implicit and explicit approaches. Psychol Res 2014;78:313-20.
Robinson C, Ahmar N, Sloutsky V. Evidence for auditory dominance in a passive oddball task. In Proceedings of the Annual Meeting of the Cognitive Science Society
Ghani U, Signal N, Niazi IK, Taylor D. ERP based measures of cognitive workload: A review. Neurosci Biobehav Rev 2020;118:18-26.
Hughes AM, Hancock GM, Marlow SL, Stowers K, Salas E. Cardiac measures of cognitive workload: a meta-analysis. Hum Fact 2019;61:393-414.
Marshall SP. The index of cognitive activity: Measuring measuring cognitive workload. In Proceedings of the IEEE 7th conference on Human Factors and Power Plants. IEEE; 2002.
Wickens CD. Multiple resources and mental workload. Hum Fact 2008;50:449-55.
Krol LR, Freytag S, Fleck M, Gramann K, Zander TO et al.
A task-independent workload classifier for neuroadaptive technology: Preliminary data. 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2016:003171–003174.
Gevins A, Smith ME. Neurophysiological measures of cognitive workload during human-computer interaction. Theor Issues Ergon Sci 2003;4:113-31.
Wöstmann M, Erb J, Kreitewolf J, Obleser J. Personality captures dissociations of subjective versus objective hearing in noise. R Soc Open Sci 2021;8:210881.
Peelle JE. Listening effort: How how the cognitive consequences of acoustic challenge are reflected in brain and behavior. Ear Hear 2018;39:204-14.
Rudner M. Cognitive spare capacity as an index of listening effort. Ear Hear, 2016;37:69S-76S.
Raveh D, Lavie N. Load-induced inattentional deafness. Attention Percep Psychophys 2015;77:483-92.
Radüntz T, Mühlhausen T, Freyer M et al.
Cardiovascular biomarkers’ inherent timescales in mental workload assessment during simulated air traffic control tasks. Appl Psychophysiol Biofeedback 2021;46:43-59.
Radüntz T. The effect of planning, strategy learning, and working memory capacity on mental workload. Scientific Reports, 2020;10(1): p. 1-10.
BAuA, Friedrich-Henkel-Weg 1-25, 44149 Dortmund
Source of Support: None, Conflict of Interest: None
[Figure 1], [Figure 2], [Figure 3]