Brain activity in the action observation network is lateralized during action execution, with greater activation in the contralateral hemisphere to the side of the body used to perform the task. However, it is unknown whether the AON is also lateralized when watching another person perform an action. In this study, we use fNIRS to measure brain activity over the left and right cortex while participants completed actions with their left and right hands and watched an actor complete action with (...) their left and right hands. We show that while activation is lateralized when the participants themselves are moving, brain lateralization is not affected by the side of the body when the participant is observing another person’s action. In addition, we demonstrate that individual differences in hand preference and dexterity between the right and left hands are related to brain lateralization patterns. (shrink)

Brain-computer interface multi-modal fusion has the potential to generate multiple commands in a highly reliable manner by alleviating the drawbacks associated with single modality. In the present work, a hybrid EEG-fNIRS BCI system—achieved through a fusion of concurrently recorded electroencephalography and functional near-infrared spectroscopy signals—is used to overcome the limitations of uni-modality and to achieve higher tasks classification. Although the hybrid approach enhances the performance of the system, the improvements are still modest due to the lack of availability of (...) computational approaches to fuse the two modalities. To overcome this, a novel approach is proposed using Multi-resolution singular value decomposition to achieve system- and feature-based fusion. The two approaches based up different features set are compared using the KNN and Tree classifiers. The results obtained through multiple datasets show that the proposed approach can effectively fuse both modalities with improvement in the classification accuracy. (shrink)

Over the last few decades, neuroimaging techniques have transformed our understanding of the brain and the effect of neurological conditions on brain function. More recently, light-based modalities such as functional near-infrared spectroscopy have gained popularity as tools to study brain function at the bedside. A recent application is to assess residual awareness in patients with disorders of consciousness, as some patients retain awareness albeit lacking all behavioural response to commands. Functional near-infrared spectroscopy can play a vital role in identifying these (...) patients by assessing command-driven brain activity. The goal of this review is to summarise the studies reported on this topic, to discuss the technical and ethical challenges of working with patients with disorders of consciousness, and to outline promising future directions in this field. (shrink)

This abstract presents a novel brain-computer interface (BCI) framework to control a prosthetic leg, for the rehabilitation of patients suffering from locomotive disorders, using functional near-infrared spectroscopy (fNIRS). fNIRS signals corresponding to walking intention and rest are used to initiate and stop the gait cycle and a nonlinear proportional derivative computed torque controller (PD-CTC) with gravity compensation is used to control torques of hip and knee joints for minimization of position error. The brain signals of walking intention and (...) rest tasks are acquired from primary motor cortex in the left hemisphere for nine subjects. After acquiring brain signals, in order to remove motion artifacts and physiological noises, the performance of six different filters i.e. Kalman, Wiener, Gaussian, hemodynamic response filter (hrf), Band-pass and finite impulse response, is evaluated. Afterwards, six different features are extracted from oxygenated hemoglobin signals and their different combinations were used for classification. The classification performance of five different classifiers i.e. artificial neural network, quadratic discriminant analysis, linear discriminant analysis (LDA), Naïve Bayes and support vector machine (SVM) is used. The classification accuracies obtained from SVM using hrf filter are significantly higher (p < 0.005) than the other combinations of classifier and filters. These accuracies are 77.5%, 72.5%, 68.3%, 74.2%, 73.3%, 80.8%, 65%, 76.7%, and 86.7% for all nine subjects. The control commands generated using classifier initiate and stop the gait cycle of the prosthetic leg whose knee and hip torques are controlled using PD-CTC to minimize position error. The proposed scheme can be effectively used for neurofeedback training and rehabilitation of lower limb amputation of paralyzed patients. (shrink)

Electroencephalography and functional near-infrared spectroscopy have potentially complementary characteristics that reflect the electrical and hemodynamic characteristics of neural responses, so EEG-fNIRS-based hybrid brain-computer interface is the research hotspots in recent years. However, current studies lack a comprehensive systematic approach to properly fuse EEG and fNIRS data and exploit their complementary potential, which is critical for improving BCI performance. To address this issue, this study proposes a novel multimodal fusion framework based on multi-level progressive learning with multi-domain features. The (...) framework consists of a multi-domain feature extraction process for EEG and fNIRS, a feature selection process based on atomic search optimization, and a multi-domain feature fusion process based on multi-level progressive machine learning. The proposed method was validated on EEG-fNIRS-based motor imagery and mental arithmetic tasks involving 29 subjects, and the experimental results show that multi-domain features provide better classification performance than single-domain features, and multi-modality provides better classification performance than single-modality. Furthermore, the experimental results and comparison with other methods demonstrated the effectiveness and superiority of the proposed method in EEG and fNIRS information fusion, it can achieve an average classification accuracy of 96.74% in the MI task and 98.42% in the MA task. Our proposed method may provide a general framework for future fusion processing of multimodal brain signals based on EEG-fNIRS. (shrink)

As automobile manufacturers have begun to design, engineer, and test autonomous driving systems of the future, brain imaging with functional near-infrared spectroscopy can provide unique insights about cognitive processes associated with evolving levels of autonomy implemented in the automobile. Modern fNIRS devices provide a portable, relatively affordable, and robust form of functional neuroimaging that allows researchers to investigate brain function in real-world environments. The trend toward “naturalistic neuroscience” is evident in the growing number of studies that leverage the methodological (...) flexibility of fNIRS, and in doing so, significantly expand the scope of cognitive function that is accessible to observation via functional brain imaging. While more than a decade’s worth of study in this field of fNIRS driving research has led to many interesting findings, the number of studies applying fNIRS during autonomous modes of operation is limited. To support future research that directly addresses this lack in autonomous driving research with fNIRS, we argue that a cogent distillation of the methods used to date will help facilitate and streamline this research of tomorrow. To that end, here we provide a methodological review of the existing fNIRS driving research, with the overarching goal of highlighting the current diversity in methodological approaches. We argue that standardization of these approaches will facilitate greater overlap of methods by researchers from all disciplines, which will, in-turn, allow for meta-analysis of future results. We conclude by providing recommendations for advancing the use of such fNIRS technology in furthering understanding the adoption of safe autonomous vehicle technology. (shrink)

Human gait is a complex activity that requires high coordination between the central nervous system, the limb, and the musculoskeletal system. More research is needed to understand the latter coordination's complexity in designing better and more effective rehabilitation strategies for gait disorders. Electroencephalogram and functional near-infrared spectroscopy are among the most used technologies for monitoring brain activities due to portability, non-invasiveness, and relatively low cost compared to others. Fusing EEG and fNIRS is a well-known and established methodology proven to (...) enhance brain–computer interface performance in terms of classification accuracy, number of control commands, and response time. Although there has been significant research exploring hybrid BCI involving both EEG and fNIRS for different types of tasks and human activities, human gait remains still underinvestigated. In this article, we aim to shed light on the recent development in the analysis of human gait using a hybrid EEG-fNIRS-based BCI system. The current review has followed guidelines of preferred reporting items for systematic reviews and meta-Analyses during the data collection and selection phase. In this review, we put a particular focus on the commonly used signal processing and machine learning algorithms, as well as survey the potential applications of gait analysis. We distill some of the critical findings of this survey as follows. First, hardware specifications and experimental paradigms should be carefully considered because of their direct impact on the quality of gait assessment. Second, since both modalities, EEG and fNIRS, are sensitive to motion artifacts, instrumental, and physiological noises, there is a quest for more robust and sophisticated signal processing algorithms. Third, hybrid temporal and spatial features, obtained by virtue of fusing EEG and fNIRS and associated with cortical activation, can help better identify the correlation between brain activation and gait. In conclusion, hBCI system is not yet much explored for the lower limb due to its complexity compared to the higher limb. Existing BCI systems for gait monitoring tend to only focus on one modality. We foresee a vast potential in adopting hBCI in gait analysis. Imminent technical breakthroughs are expected using hybrid EEG-fNIRS-based BCI for gait to control assistive devices and Monitor neuro-plasticity in neuro-rehabilitation. However, although those hybrid systems perform well in a controlled experimental environment when it comes to adopting them as a certified medical device in real-life clinical applications, there is still a long way to go. (shrink)

The aim of this study was to explain previous inconsistent results regarding the effects of positive affect on creative cognition based on the motivational dimensional model of affect theory and provide the underlying neural correlates of the effects of different approach-motivation intensities of positive affect on creative processes (creative idea generation and creative idea evaluation) using the functional near-infrared spectroscopy (fNIRS) technique. Sixty participants were randomly assigned to three groups (high-approach-motivated positive affect (HAM), low-approach-motivated positive affect (LAM) and affectively (...) neutral state (NS)) to complete corresponding tasks in creative idea generation and creative idea evaluation stages. In the creative idea generation stage, the results indicated that the LAM group achieved a higher performance in flexibility and originality than the NS group. However, compared with the NS group, the LAM group exhibited a lower sensitivity in the creative idea evaluation stage. The fNIRS technique provides an ideal approach with high ecological validity for exploring the issue of affect and creativity. In current study, fNIRS results showed that the HAM group exhibited significantly more activation in the bilateral frontal lobe than the LAM group and NS group in the creative idea generation stage. This result could reflect individuals attempting to overcome a narrowed attentional scope state through the activation of bilateral frontal cortical resources in the creative process. In the creative idea evaluation stage, compared with the NS group, the LAM group showed significant deactivation in the right dorsolateral prefrontal cortex, which is a brain area associated with executive, controlled processing. Moreover, the approach-motivation intensity altered the connectivity between the left prefrontal cortex and left superior temporal gyrus in the creative idea generation stage. Thus, the influences of positive affect on creative cognitive processes (both creative performance and hemodynamic responses) were modulated by the approach-motivation intensity in the creative idea generation and creative idea evaluation stages. (shrink)

Multi-modal integration, which combines multiple neurophysiological signals, is gaining more attention for its potential to supplement single modality’s drawbacks and yield reliable results by extracting complementary features. In particular, integration of electroencephalography and functional near-infrared spectroscopy is cost-effective and portable, and therefore is a fascinating approach to brain-computer interface. However, outcomes from the integration of these two modalities have yielded only modest improvement in BCI performance because of the lack of approaches to integrate the two different features. In addition, mismatch (...) of recording locations may hinder further improvement. In this literature review, we surveyed studies of the integration of EEG/fNIRS in BCI thoroughly and discussed its current limitations. We also suggested future directions for efficient and successful multi-modal integration of EEG/fNIRS in BCI systems. (shrink)

This study explored the differentiated neural correlates of mental rotation in preschoolers with high and low working memory capacity using functional near-infrared spectroscopy. Altogether 38 Chinese preschoolers completed the Working Memory Capacity test, the Mental Rotation, and its Control tasks. They were divided into High-WMC and Low-WMC groups based on the WMC scores. The behavioral and fNIRS results indicated that: there were no significant differences in MR task performance between the High-WMC and Low-WMC group ; the Low-WMC group activated (...) BA6, BA8, BA 9, and BA 44, whereas the High-WMC group activated BA8, BA10 and BA 44 during mental rotation; significant differences were found in the activation of BA44 and BA9 between the High-WMC and Low-WMC groups during mental rotation; and the High-WMC and Low-WMC groups differed significantly in the activation of BA 9 and BA10 during the control tasks, indicating that both areas might be responsible for the group differences in working memory. (shrink)

The aging of attentional and emotional functions has been extensively studied but relatively independently. Therefore, the relationships between aging and the interactions of attentional and emotional processes remain elusive. This study aimed to determine how age affected the interactions between attentional and emotional processes during adulthood. One-hundred forty adults aged 18–79 performed the emotional variant of the Attention Network Test, which probed alerting, orienting, and executive control in the presence and absence of threatening faces. During this task, contexts with varying (...) levels of task preparatory processes were created to modulate the effect of threatening faces on attention, and functional near-infrared spectroscopy (fNIRS) was used to examine the neural underpinnings of the behavioural effects. The behavioural results showed that aging was associated with a significant decline in alerting efficiency, and there was a statistical trend for age-related deficits in executive control. Despite these age differences, age did not significantly moderate the interactions among attentional networks or between attention and emotion. Additionally, the fNIRS results showed that decreased frontal cortex functioning might underlie the age-related decline in executive control. Therefore, while aging has varying effects on different attentional networks, the interactions of attentional and emotional processes remain relatively unaffected by age. (shrink)

Neuroimaging technologies have improved our understanding of deception and also exhibit their potential in revealing the origins of its neural mechanism. In this study, a quantitative power analysis method that uses the Welch power spectrum estimation of functional near-infrared spectroscopy signals was proposed to examine the brain activation difference between the spontaneous deceptive behavior and controlled behavior. The power value produced by the model was applied to quantify the activity energy of brain regions, which can serve as a neuromarker for (...) deception detection. Interestingly, the power analysis results generated from the Welch spectrum estimation method demonstrated that the spontaneous deceptive behavior elicited significantly higher power than that from the controlled behavior in the prefrontal cortex. Meanwhile, the power findings also showed significant difference between the spontaneous deceptive behavior and controlled behavior, indicating that the reward system was only involved in the deception. The proposed power analysis method for processing fNIRS data provides us an additional insight to understand the cognitive mechanism of deception. (shrink)

Hyperscanning studies using functional Near-Infrared Spectroscopy have been performed to understand the neural mechanisms underlying human-human interactions. In this study, we propose a novel methodological approach that is developed for fNIRS multi-brain analysis. Our method uses support vector regression to predict one brain activity time series using another as the predictor. We applied the proposed methodology to explore the teacher-student interaction, which plays a critical role in the formal learning process. In an illustrative application, we collected fNIRS data (...) of the teacher and preschoolers’ dyads performing an interaction task. The teacher explained to the child how to add two numbers in the context of a game. The Prefrontal cortex and temporal-parietal junction of both teacher and student were recorded. A multivariate regression model was built for each channel in each dyad, with the student’s signal as the response variable and the teacher’s ones as the predictors. We compared the predictions of SVR with the conventional ordinary least square predictor. The results predicted by the SVR model were statistically significantly correlated with the actual test data at least one channel-pair for all dyads. Overall, 29/90 channel-pairs across the five dyads presented significant signal predictions withthe SVR approach. The conventional OLS resulted in only 4 out of 90 valid predictions. These results demonstrated that the SVR could be used to perform channel-wise predictions across individuals, and the teachers’ cortical activity can be used to predict the student brain hemodynamic response. (shrink)

Although the neural systems that underlie spoken language are well-known, how they adapt to evolving social cues during natural conversations remains an unanswered question. In this work we investigate the neural correlates of face-to-face conversations between two individuals using functional near infrared spectroscopy and acoustical analyses of concurrent audio recordings. Nineteen pairs of healthy adults engaged in live discussions on two controversial topics where their opinions were either in agreement or disagreement. Participants were matched according to their a priori opinions (...) on these topics as assessed by questionnaire. Acoustic measures of the recorded speech including the fundamental frequency range, median fundamental frequency, syllable rate, and acoustic energy were elevated during disagreement relative to agreement. Consistent with both the a priori opinion ratings and the acoustic findings, neural activity associated with long-range functional networks, rather than the canonical language areas, was also differentiated by the two conditions. Specifically, the frontoparietal system including bilateral dorsolateral prefrontal cortex, left supramarginal gyrus, angular gyrus, and superior temporal gyrus showed increased activity while talking during disagreement. In contrast, talking during agreement was characterized by increased activity in a social and attention network including right supramarginal gyrus, bilateral frontal eye-fields, and left frontopolar regions. Further, these social and visual attention networks were more synchronous across brains during agreement than disagreement. Rather than localized modulation of the canonical language system, these findings are most consistent with a model of distributed and adaptive language-related processes including cross-brain neural coupling that serves dynamic verbal exchanges. (shrink)