Neuroimaging-guided Adaptive Training in Flight Simulators

• ¹ School of Biomedical Engineering, Science and Health Systems, Drexel University, United States
• ² Lockheed Martin (United States), Advanced Technology Laboratories, United States
• ³ University of Pennsylvania, Department of Family and Community Health, United States
• ⁴ Children's Hospital of Philadelphia, Division of General Pediatrics, United States

Many high-stakes professions dealing with life-or-death situations such as doctor, airplane pilot, and military mission commander require large amounts of time, effort, and money during training. This training is usually offered at specialized schools or facilities; however, the methods used often rely on outdated standards of expertise assessment. Because only performance is measured, and not the mental workload and effort involved, it is possible for people to leave undertrained and not yet ready, or overtrained, having wasted valuable time and resources. Therefore, it is necessary to develop a training protocol utilizing objective neuroimaging measures of workload to improve the efficiency of training. In this study, our results point toward the benefits of incorporating neuroimaging-based direct brain measures over a standard training protocol. Mental workload is commonly defined as the proportion of resources used as a function of maximum brain power. In other words, although there is an average level of max and optimal workload, each person’s limits vary, and may be different depending on the task at hand. There are four main ways to measure workload; through primary task performance, secondary task performance, subjective measures, and neurophysiological recordings. By combining these measures with an emphasis on neuroimaging, a graph of optimal workload can be plotted. In previous studies, it has been found that the highest rate of learning occurs at the peak of an inverted U-shaped plot of workload, where too little engagement associates with boredom, and too high relates to mental overload. As such, neuroimaging can be used to determine this level for individuals. In this study, we set out to investigate whether prefrontal cortex activity-related mental workload measures can be used to improve the efficiency of training. For monitoring the prefrontal cortex activity, a wearable and portable neuroimaging technique, functional near-infrared spectroscopy (fNIRS), was used (Ayaz et al., 2013). We used a flight simulator, Prepar3D, with three custom developed scenarios that include realistic piloting tasks from our earlier studies (Ayaz et al., 2012, Choe et al., 2016). These tasks included: situational awareness, which presented a video recording from the cockpit of a plane and required subjects to recall various aspects of the flight from dashboard gauges; a landing task, in which subjects controlled a plane in flight and were required to land it safely; and the ring task, where subjects flew a plane through rings that moved side to side. Each task had up to 9 different difficulty levels, and over the course of four sessions spread out over two weeks, subjects would advance or drop through the levels depending on their performance. (Mark et al., 2018). Twelve subjects volunteered for the study and were randomly assigned to one of two conditions: a control group, who progressed according to just their performance, and the neuro group, who progressed based on a combination of performance and neuroimaging-based workload (Fig 1). The experiment was designed such that two established measures of effective training could be elucidated from the results: transfer and retention. Transfer is defined as how well skills trained in one specific task can be utilized in a functionally similar but different and unique task. This was modeled by making subjects perform each task at the maximum difficulty level at the end of session four, which could not be reached through advancing normally in the given time. Retention is defined as the decay (or lack thereof) of a trained skill over time. We measured this using reference tasks at the minimum difficulty at the start of each session. This allowed for a pure measure of skill and a baseline to compare each of the four sessions equally. Our preliminary results indicate that, in the landing task, the neuro group advanced to higher levels of training than the control group in the same amount of time (F3,178=8.07 p<0.002) (Fig 2). Over the course of the four sessions, the control group showed a steady decrease in fNIRS-measured workload, whereas the neuro group demonstrated more consistent levels of neural oxygenation (F3,129=3.17 p<0.05), which is in line with the inverted U-shape theory of optimum workload (Fig 3). Neither group showed a change in retention performance, but the neuro group showed a significant decrease in mental workload over time, which is what we predicted from the personalized training, as workload decreases as skill increases (F3,48=3.73 p<0.04). In addition, while both groups had comparable performance during the transfer task, the neuro group had significantly lower mental workload (F1,10=7.06 p<0.025), again demonstrating the more efficient acquisition of skill as compared to the control group (Fig 4). In conclusion, here we utilized a novel method of integrating neuroimaging procedures into the training of complex flight tasks. Our results suggest that a neuroimaging-informed training results in higher efficiency as compared to the control group using standard training metrics based solely on performance measures. Neuro group subjects progressed to higher levels in the same amount of time as control subjects, showed consistent levels of workload in line with the neuro efficiency hypothesis, and required less mental effort during the transfer tasks. Our next steps are to investigate this with other training domains and paradigms to determine if these results apply to a more generalized context of skill acquisition and learning. Figure 1. Adaptation procedures for both control and neuro groups. “L” and “H” represent low and high workload, respectively Figure 2. Average level reached per session for landing task (F3,178=8.07 p<0.002) Figure 3. Workload correlates during training show decrease in control group and consistency in neuro group for landing task (Sessions: F3,129=3.17 p<0.05) Figure 4. Workload correlates in transfer task show lower oxygenation and thus workload in neuro group (F1,10=7.06 p<0.025)