Highly mobile computing devices promise to improve quality of life, productivity, and performance. Increased situation awareness and reduced mental workload are two potential means by which this can be accomplished. However, it is difficult to measure these concepts in the ‘wild’. We employed an ultra-portable battery operated and wireless functional near infrared spectroscopy (fNIRS) to non-invasively measure hemodynamic changes in the brain’s prefrontal cortex. Measurements were taken during navigation of a college campus with either a hand-held display, or an augmented (...) reality wearable display. Hemodynamic measures were also paired with secondary tasks of visual perception and auditory working memory to provide behavioral assessment of situation awareness and mental workload. Navigating with an augmented reality wearable display produced the least workload during both secondary tasks. The hemodynamics associated with errors were also different between the two devices. Errors with an augmented reality wearable display were associated with increased prefrontal activity and the opposite was observed for the hand-held display. This suggests that the cognitive mechanisms underlying errors between the two devices differ. These findings show fNIRS is a valuable tool for assessing new technology in ecologically valid settings and that head-mounted displays offer benefits with regards to mental workload while navigating, and potentially superior situation awareness with improved display design. (shrink)

There is growing interest for implementing tools to monitor cognitive performance in naturalistic work and everyday life settings. The emerging field of research, known as neuroergonomics, promotes the use of wearable and portable brain monitoring sensors such as functional near infrared spectroscopy (fNIRS) to investigate cortical activity in a variety of human tasks out of the laboratory. The objective of this study was to implement an on-line passive fNIRS-based brain computer interface to discriminate two levels of working memory load (...) during highly ecological aircraft piloting tasks. Twenty eight recruited pilots were equally split into two groups (flight simulator vs. real aircraft). In both cases, identical approaches and experimental stimuli were used (serial memorization task, consisting in repeating series of pre-recorded air traffic control instructions, easy vs. hard). The results show pilots in the real flight condition committed more errors and had higher anterior prefrontal cortex activation than pilots in the simulator, when completing cognitively demanding tasks. Nevertheless, evaluation of single trial working memory load classification showed high accuracy (>76%) across both experimental conditions. The contributions here are two-fold. First, we demonstrate the feasibility of passively monitoring cognitive load in a realistic and complex situation (live piloting of an aircraft). In addition, the differences in performance and brain activity between the two experimental conditions underscore the need for ecologically-valid investigations. (shrink)

It is currently assumed that exposure to an artificial blue-enriched light enhances human alertness and task performance, but recent research has suggested that behavioural effects are influenced by the basal state of arousal. Here we tested whether the effect of blue-enriched lighting on vigilance performance depends on participants’ arousal level. Twenty-four participants completed four sessions (blue-enriched vs. dim light x low vs. high arousal) at 10 pm on four consecutive days, following a repeated-measures design. Participants’ arousal was manipulated parametrically through (...) the execution of a cycling task at two intensities (low vs. moderate), and was checked by monitoring their heart rate. On each session, distal and proximal skin temperatures were recorded as a neuroergonomic index of vigilance, while participants performed a 20-min psychomotor vigilance task (PVT) under either blue-enriched light or dim light conditions. The Positive and Negative Affect Schedule (PANAS), Karolinska Sleepiness Scale (KSS) and Borg Rating of Perceived Exertion Scale (RPE) were used to measure subjective psychological state. The results showed that the exercise-induced manipulation of arousal produced robust alerting effects in most measures, while the lighting manipulation only attenuated subjective sleepiness and enhanced positive affect, but it did not influence behaviour or physiology. Acute exposure to a blue-enriched light was practically ineffective when the arousal level was over baseline. The present research favoured the use of acute physical exercise over acute exposure to blue-enriched lighting in order to boost humans’ alertness when necessary, as in work settings where maintaining optimal levels of attention is difficult (shift work, night-work, vigilance tasks) and necessary to prevent human error and accidents. (shrink)

Human mental workload is arguably the most invoked multidimensional construct in Human Factors and Ergonomics, getting momentum also in Neuroscience and Neuroergonomics. Uncertainties exist in its characterization, motivating the design and development of computational models, thus recently and actively receiving support from the discipline of Computer Science. However, its role in human performance prediction is assured. This work is aimed at providing a synthesis of the current state of the art in human mental workload assessment through considerations, definitions, measurement (...) techniques as well as applications, Findings suggest that, despite an increasing number of associated research works, a single, reliable and generally applicable framework for mental workload research does not yet appear fully established. One reason for this gap is the existence of a wide swath of operational definitions, built upon different theoretical assumptions which are rarely examined collectively. A second reason is that the three main classes of measures, which are self-report, task performance, and physiological indices, have been used in isolation or in pairs, but more rarely in conjunction all together. Multiple definitions complement each another and we propose a novel inclusive definition of mental workload to support the next generation of empirical-based research. Similarly, by comprehensively employing physiological, task-performance, and self-report measures, more robust assessments of mental workload can be achieved. (shrink)

Recent advances in neuroimaging technologies have rendered multimodal analysis of operators’ cognitive processes in complex task settings and environments increasingly more practical. In this exploratory study, we utilized optical brain imaging and mobile eye tracking technologies to investigate the behavioral and neurophysiological differences among expert and novice operators while they operated a human-machine interface in normal and adverse conditions. In congruence with related work, we observed that experts tended to have lower prefrontal oxygenation and exhibit gaze patterns that are better (...) aligned with the optimal task sequence with shorter fixation durations as compared to novices. These trends reached statistical significance only in the adverse condition where the operators were prompted with an unexpected error message. Comparisons between hemodynamic and gaze measures before and after the error message indicated that experts’ neurophysiological response to the error involved a systematic increase in bilateral dorsolateral prefrontal cortex (dlPFC) activity accompanied with an increase in fixation durations, which suggests a shift in their attentional state, possibly from routine process execution to problem detection and resolution. The novices’ response was not as strong as that of experts, including a slight increase only in the left dlPFC with a decreasing trend in fixation durations, which is indicative of visual search behavior for possible cues to make sense of the unanticipated situation. A linear discriminant analysis model capitalizing on the covariance structure among hemodynamic and eye movement measures could distinguish experts from novices with 91% accuracy. Despite the small sample size, the performance of the linear discriminant analysis combining eye fixation and dorsolateral oxygenation measures before and after an unexpected event suggests that multimodal approaches may be fruitful for distinguishing novice and expert performance in similar neuroergonomic applications in the field. (shrink)

Over the last decade, the efforts towards unraveling the complex interplay between the brain, body, and environment have set a promising line of research that utilizes neuroscience to study human performance in natural work contexts such as aviation. Thus, a new discipline called neuroergonomics is holding the promise of studying the neural mechanisms underlying human performance in pursuit of both theoretical and practical insights. In this work, we utilized a neuroergonomic approach by combining insights from ecological psychology and embodied (...) cognition to study flight expertise. Specifically, we focused on the Mirror Neuron system as a key correlate for understanding the interaction between an individual and the environment, suggesting that it can be used to index changes in the coupling of perception-action associated with skill development.In this study, we measured the EEG mu suppression as a proxy of the Mirror Neuron system in experts (pilots) and novices while performing a distance estimation task in a landing scenario. To survey the specificity of this measure, we considered central, parietal and occipital electrode pools and analyzed alpha (8-13 Hz) and beta (18-25 Hz) rhythm bands. We hypothesized that in experts vs. novices, specific neural sensorimotor brain activity would underpin the connection between perception and action in an in-flight context. Preliminary results indicate that alpha and beta suppression was area-specific irrespective of groups, present in the central electrodes placed over the motor areas. Group analysis revealed that specifically alpha mu rhythm, but not beta, was significantly more suppressed in pilots vs. novices. Complementing these findings we found a trend in which the strength of mu suppression increased with the sense of presence experienced by the pilots. Such sensorimotor activation is in line with the idea that for a pilot, a distance judgment is intimately associated with the function of landing. This reflects the ability to use optical invariants to see the world in terms of the capabilities of the aircraft (e.g., reachability and glide angle).These preliminary findings support the role of embodied simulation mechanisms in visual perception and add important insights into a practical understanding of flight expertise, suggesting sensorimotor mechanisms as potential neuro-markers. (shrink)

Recent advances in neuroimaging technologies have rendered multimodal analysis of operators’ cognitive processes in complex task settings and environments increasingly more practical. In this exploratory study, we utilized optical brain imaging and mobile eye tracking technologies to investigate the behavioral and neurophysiological differences among expert and novice operators while they operated a human-machine interface in normal and adverse conditions. In congruence with related work, we observed that experts tended to have lower prefrontal oxygenation and exhibit gaze patterns that are better (...) aligned with the optimal task sequence with shorter fixation durations as compared to novices. These trends reached statistical significance only in the adverse condition where the operators were prompted with an unexpected error message. Comparisons between hemodynamic and gaze measures before and after the error message indicated that experts’ neurophysiological response to the error involved a systematic increase in bilateral dorsolateral prefrontal cortex (dlPFC) activity accompanied with an increase in fixation durations, which suggests a shift in their attentional state, possibly from routine process execution to problem detection and resolution. The novices’ response was not as strong as that of experts, including a slight increase only in the left dlPFC with a decreasing trend in fixation durations, which is indicative of visual search behavior for possible cues to make sense of the unanticipated situation. A linear discriminant analysis model capitalizing on the covariance structure among hemodynamic and eye movement measures could distinguish experts from novices with 91% accuracy. Despite the small sample size, the performance of the linear discriminant analysis combining eye fixation and dorsolateral oxygenation measures before and after an unexpected event suggests that multimodal approaches may be fruitful for distinguishing novice and expert performance in similar neuroergonomic applications in the field. (shrink)

Conventional wisdom suggests mid-task rest as a potential approach to relieve the time-on-task (TOT) effect while accumulating evidence indicated that acute exercise might also effectively restore mental fatigue. However, few studies have explored the neural mechanism underlying these different break types, and the results were scattered. This study provided one of the first looks at how different types of fatigue-recovery break exerted influence on the cognitive processes by evaluating the corresponding behavioral improvement and neural response (EEG power spectral) in a (...) sustained attention task. Specifically, 19 participants performed three sessions of psychomotor vigilance tasks (PVT), with one session including a continuous 30-min PVT while the other two sessions additionally inserted a 15-min mid-task cycling and rest break, respectively. For behavioral performance, both types of break could restore objective vigilance transiently, while subjective feeling was only maintained after mid-task rest. Moreover, divergent patterns of EEG change were observed during post-break improvement. In detail, relative theta decreased and delta increased immediately after mid-task exercise, while decreased delta was found near the end of the rest-inserted task. Meanwhile, theta and delta could serve as neurological indicators to predict the reaction time change for exercise and rest intervention, respectively. In sum, our findings provided novel evidence to demonstrate divergent neural patterns following the mid-task exercise and rest intervention to counter TOT effects, which might lead to new insights into the nascent field of neuroergonomics for mental fatigue restoration. (shrink)

Recent advances in neuroscience have paved the way to innovative applications that cognitively augment and enhance humans in a variety of contexts. This paper aims at providing a snapshot of the current state of the art and a motivated forecast of the most likely developments in the next two decades. Firstly, we survey the main neuroscience technologies for both observing and influencing brain activity, which are necessary ingredients for human cognitive augmentation. We also compare and contrast such technologies, as their (...) individual characteristics (e.g., spatio-temporal resolution, invasiveness, portability, energy requirements and cost) influence their current and future role in human cognitive augmentation. Secondly, we chart the state of the art on neurotechnologies for human cognitive augmentation, keeping an eye both on the applications that already exist and those that are emerging or are likely to emerge in the next two decades. Particularly, we consider applications in the areas of communication, cognitive enhancement, memory, attention monitoring/enhancement, situation awareness and complex problem solving, and we look at what fraction of the population might benefit from such technologies and at the demands they impose in terms of user training. Thirdly, we briefly review the ethical issues associated with current neuroscience technologies. These are important because they may differentially influence both present and future research on (and adoption of) neurotechnologies for human cognitive augmentation: an inferior technology with no significant ethical issues may thrive while a superior technology causing widespread ethical concerns may end up being outlawed. Finally, based on the lessons learned in our analysis, using past trends and considering other related forecasts, we attempt to forecast the most likely future developments of neuroscience technology for human cognitive augmentation and provide informed recommendations for promising future research and exploitation avenues. (shrink)

Car driving is considered a very complex activity, consisting of different concomitant tasks and subtasks, thus it is crucial to understand the impact of different factors, such as road complexity, traffic, dashboard devices, and external events on the driver’s behavior and performance. For this reason, in particular situations the cognitive demand experienced by the driver could be very high, inducing an excessive experienced mental workload and consequently an increasing of error commission probability. In this regard, it has been demonstrated that (...) human error is the main cause of the 57 % of road accidents and a contributing factor in most of them. In this study, 20 young subjects were involved in a real driving experiment, performed under different traffic conditions (rush hour and not) and along different road types (main and secondary streets). Moreover, during the driving tasks different specific events, in particular a pedestrian crossing the road and a car entering the traffic flow just ahead of the experimental subject, were acted. A Workload Index based on the Electroencephalographic (EEG), i.e. brain activity, of the drivers has been employed to investigate the impact of the different factors on the driver’s workload. Eye-Tracking technology and subjective measures were also employed in order to have a comprehensive overview of the driver’s perceived workload and to investigate the different insights obtainable from the employed methodologies. The employment of such EEG-based Workload index confirmed the significant impact of both traffic and road types on the drivers’ behavior (increasing their workload), with the advantage of being under real settings. Also, it allowed to highlight the increased workload related to external events while driving, in particular with a significant effect during those situations when the traffic was low. Finally, the comparison between methodologies revealed the higher sensitivity of neurophysiological measures with respect to Eye-Tracking and subjective ones. In conclusion, such an EEG-based Workload index would allow to assess objectively the mental workload experienced by the driver, standing out as a powerful tool for research aimed to investigate drivers’ behavior and providing additional and complementary insights with respect to traditional methodologies employed within road safety research. (shrink)