Assessment of Mental Stress among Undergraduate Students Using Novel Fusion Method on EEG and fNIRS Features

• ¹ American University of Sharjah, United Arab Emirates

Background: Stress is a major contributing factor to chronical disorders and productivity loss. Students spend most of their time studying, often burdened with high workloads and time pressure, a factor that contributes to increase the level of stress and decrease their academic performance. Stress activates the hypothalamus-pituitary-adrenocortical axis and the sympathetic nervous system leading to the release of stress hormone (cortisol) in the adrenal cortex. The continuous release of cortisol leads to several chronical diseases including anxiety, depression, stroke and even addiction. Therefore, an accurate stress detection method at its early stage is very important to clinical intervention, diseases prevention as well as for academic performance. While different neuroimaging modalities have been proposed to detect mental stress, each modality experiences certain limitations. Neuroimaging tools such as functional magnetic resonance imaging (fMRI) and positron emission tomography (PET) have good spatial resolution, yet are limited in terms of temporal resolution and susceptibility to motion artifacts. They also require the test subjects to remain at fixed position during scanning. Electroencephalography (EEG) constitutes a possible alternative neuroimaging technique that does not possess the same limitations. The EEG has temporal resolution in the order of a few milliseconds, which makes it suitable for measuring cortical changes during workplace activities. However, the EEG is generally considered as having poor spatial resolution (compare to excellent fMRI modality) and being highly prone to motion artifacts. Functional near-infrared spectroscopy (fNIRS) has allowed human cortical activity to be measured during unconstrained movements, the temporal resolution is sub-second, and the spatial resolution is on order of 1 cm2 at best. In order to overcome these constraints, it is advisable to combine the EEG with fNIRS modality to provide a complementary nature and improve the detection rate of mental stress. Goal: In our previous studies, individual EEG and fNIRS have shown their potential to quantify three levels of stress (Al-shargie et al., 2016b; Al-shargie et al., 2018). In this study, we propose a novel fusion approach to discover the association between the temporal features of EEG and the spatial features of fNIRS under stress by using Joint Spare Canonical Correlation (JSCCA) analysis method. Methods: Twenty-five undergraduate students performed a mental arithmetic task under two different conditions; neutral-control (5 min) and stress (5 mi) conditions (the stressors were based on time pressure and negative feedback on performance (Al-Shargie et al., 2016a)). The brain activities measured simultaneously at the prefrontal cortex (PFC) using EEG (BrainMaster 24E system; 7-electrodes) and fNIRS (OT-R40, Hitachi Medical Corp, Japan; 23 channels) techniques. EEG and fNIRS signals were preprocessed to remove the noise and artefacts, then decomposed into a set of components (EEG features extracted at different frequency bands; Theta: 4-8 Hz, Alpha: 8-13 Hz, and Beta: 14-30 Hz) in the temporal domain and (fNIRS oxygenated hemoglobin (O2Hb) at the frequency interval of 0.001- 0.1 Hz) in the spatial domain, respectively. Then, JSCCA with generalized fused lasso penalty was used to jointly estimate multiple pairs of canonical vectors with both shared (EEG+fNIRS) and modality-specific (EEG or fNIRS) patterns (Fang et al., 2016). The tuning parameters of JSCCA model were estimated using a cross validation method. The performance evaluation of the proposed method in detecting stress was then performed using support vector machine (SVM) with radial basis function. The classification accuracy, sensitivity, specificity, area under the receiver operating characteristic curve (AROC), positive prediction value (PPV) and negative prediction value (NPV) were calculated for individual modality and after fusion. Two-sample t-test was used to check the significant of the proposed method. Results and Conclusion: The results of EEG and fNIRS showed that, brain activities significantly decreased under stress compared to control condition, among all students p<0.002±0.001. The results of JSCCA method outperformed individual EEG and fNIRS in all the classification performance metrics with p<0.001 as shown in Table 1 as well as outperformed our previous original version of canonical correlation analysis method (Al-Shargie et al., 2017). The statistical analysis of JSCCA fusion confirmed the localization of mental stress to the right ventrolateral PFC. The classification performance showed that fusion of O2Hb+Alpha outperformed fusion of O2Hb+Theta and fusion of O2Hb+Beta in all the metrics. The results strongly recommend JSCCA fusion for future multimodal EEG and fNIRS studies.