Functional Near-Infrared Spectroscopy Indicates That Asymmetric Right Hemispheric Activation in Mental Rotation of a Jigsaw Puzzle Decreases With Task Difficulty
Murat Can Mutlu
Sinem Burcu Erdoğan
Ozan Cem Öztürk
Reşit Canbeyli
Hale Saybaşιlι- 1Institute of Biomedical Engineering, Boğaziçi University, Istanbul, Turkey
- 2Department of Medical Engineering, Acιbadem Mehmet Ali Aydιnlar University, Istanbul, Turkey
- 3School of Sport Exercise and Health Sciences, Loughborough University, Leicestershire, United Kingdom
- 4Department of Psychology, Boğaziçi University, Istanbul, Turkey
Mental rotation (MR) is a cognitive skill whose neural dynamics are still a matter of debate as previous neuroimaging studies have produced controversial results. In order to investigate the underlying neurophysiology of MR, hemodynamic responses from the prefrontal cortex of 14 healthy subjects were recorded with functional near-infrared spectroscopy (fNIRS) during a novel MR task that had three categorical difficulty levels. Hemodynamic activity strength (HAS) parameter, which reflects the ratio of brain activation during the task to the baseline activation level, was used to assess the prefrontal cortex activation localization and strength. Behavioral data indicated that the MR requiring conditions are more difficult than the condition that did not require MR. The right dorsolateral prefrontal cortex (DLPFC) was found to be active in all conditions and to be the dominant region in the easiest task while more complex tasks showed widespread bilateral prefrontal activation. A significant increase in left DLPFC activation was observed with increasing task difficulty. Significantly higher right DLPFC activation was observed when the incongruent trials were contrasted against the congruent trials, which implied the possibility of a robust error or conflict-monitoring process during the incongruent trials. Our results showed that the right DLPFC is a core region for the processing of MR tasks regardless of the task complexity and that the left DLPFC is involved to a greater extent with increasing task complexity, which is consistent with the previous neuroimaging literature.
Introduction
In recent years, the number of studies conducted with functional near-infrared spectroscopy (fNIRS) has increased due to several advantages it has over other functional imaging techniques such as electroencephalography (EEG), functional magnetic resonance imaging (fMRI), and positron emission tomography (PET). fNIRS technology offers a more cost-effective and user-friendly alternative to fMRI and PET in measuring cerebral energy metabolism besides having the major advantage of being field-deployable. Furthermore, its portability allows recordings of the functional cerebral hemodynamics induced by cortical activation in daily life environments. Numerous studies have demonstrated the reliability of fNIRS in measuring cortical hemodynamic responses during various lateralized cognitive tasks involving language (Herff et al., 2013; Jasinska and Petitto, 2013; Sugiura et al., 2015; Fu et al., 2016), visual/visuospatial (Herff et al., 2013; Baker et al., 2018; Curtin et al., 2019) and mathematic processing (Herff et al., 2013; Artemenko et al., 2018). However, studies that have investigated visuospatial processing with fNIRS is limited in number, particularly for mental rotation (MR) tasks, the decoding of which might have crucial importance in the design of fNIRS based brain-computer interfaces.
Mental rotation is a spatial cognitive ability that involves mentally rotating two or three-dimensional objects. Since the development of the first mental rotation test by Shepard and Metzler (1971), mental rotation skill has been widely investigated by numerous research groups employing different behavioral and neuroimaging methods. While initial studies depended mostly on behavioral responses and usually measured reaction time as a metric of cognitive workload and difficulty (Shepard and Metzler, 1971; Cooper and Shepard, 1975), more recent studies have employed neuroimaging methods such as fMRI (Jordan et al., 2001; Halari et al., 2006; Seydell-Greenwald et al., 2017), fNIRS (Shimoda et al., 2008) and PET (Harris et al., 2000) to understand the neurophysiological processes underlying MR skill. In these studies, hemispheric dominance in mental rotation skill has been a major scope of the investigation, but the results still remain to be a matter of debate. While a majority of the studies claimed a right-hemispheric superiority in mental rotation with different imaging modalities (Ratcliff, 1979; Papanicolaou et al., 1987; Deutsch et al., 1988), others found a left-hemispheric dominance (Mehta and Newcombe, 1991; Alivisatos and Petrides, 1997) or a bilateral activation (Cohen et al., 1996; Vingerhoets et al., 2002; Hugdahl et al., 2006; Shimoda et al., 2008). While various factors may contribute to the discrepancy in these results, variations in task complexity and the resultant cognitive effort appear to be the first and foremost issue to be addressed. Mehta and Newcombe (1991) found that left hemispheric lesions affected the mental rotation of 3D-shapes, while Ratcliff (1979) and Ditunno and Mann (1990) found that right hemispheric lesions impaired the mental rotation of 2D-shapes. It is claimed that left-hemispheric activity increases with the complexity or difficulty of the task possibly because of two factors: applied rotation strategy (piecemeal vs. holistic) and involvement of other cognitive elements such as verbalization (Corballis, 1997). Corballis (1997) claimed that simpler shapes are being rotated in more holistic fashion, while more complex shapes are being rotated in more piecemeal fashion during which the subjects focus on certain features or attributes of the shapes. For the latter factor, increased task complexity requires various additional cognitive resources to be utilized during the task.
Since findings regarding the hemodynamic activation patterns observed during mental rotation tasks are not consistent in terms of lateralization, the present study aimed to assess whether MR elicits a lateralized neural response in the prefrontal cortex regions and, if so, where the hemodynamic activity is localized. Another aim was to investigate the effect of task difficulty on prefrontal cortex activation. Overall, the study was designed to show that fNIRS is reliably capable of capturing the cerebral hemodynamics during mental rotation tasks, and thus can be used in decoding of cognitive abilities. For these purposes, a novel mental rotation task based on jigsaw puzzle piece-template pairs with various categorical difficulty was designed, and fNIRS data were collected during execution of these tasks in a daily life environment rather than inside an MRI or PET apparatus. Several fNIRS studies have derived spatial and temporal features from hemodynamic responses during mental rotation tasks for fNIRS-Brain Computer Interface (BCI) applications (Herff et al., 2013; Banville et al., 2017; Qureshi et al., 2017). However, to the best of our knowledge, no fNIRS study has examined the spatial and temporal features of the cerebral hemodynamics of mental rotation tasks of varying difficulty at a group level in detail to reveal the underlying physiological mechanisms in the prefrontal cortex.
Materials and Methods
Participants
Fifteen healthy volunteer subjects, ages 21–35, participated in the study. One subject was excluded from analysis due to poor signal to noise ratio in fNIRS recordings. The remaining subjects were eight females and six males (11 right-handed and 3 left-handed). All subjects were informed before the experiment and gave written consent. The study was approved by the local ethical committee, Human Research Ethical Committee of Boğaziçi University.
fNIRS Data Acquisition
A NIRSport system (NIRx Medical Technologies, LLC, Berlin, Germany) with a set-up of 22 channels consisting of 8 light sources (emitting near-infrared light at 760 and 850 nm) and 7 detectors covering the frontal cortex was used to collect hemodynamic data (Figure 1A). The sampling rate of the signal was 7.8125 Hz and the source-detector distance was 3 cm. Probe sensitivity profile was computed with the Atlas Viewer toolbox (Aasted et al., 2015) of Homer2 Software (Huppert et al., 2009). AtlasViewer provides wavelength-specific sensitivity maps of photon propagation for each. In Figure 1A, the optical sensitivity profile of the probe is mapped on a standard brain template with a 10/5 global EEG electrode system. The sensitivity of the probe for detecting brain hemodynamics was calculated with the method explained in Aasted et al. (2015) and presented as a temperature plot ranging from 0 (red) to -2 (blue) in log10 units. The channel locations are coregistered onto a standard brain in MNI space using the NIRS_SPM toolbox in Figure 1B (Singh et al., 2005; Jang et al., 2009; Ye et al., 2009). The percentage of Brodmann areas covered by each channel was computed with the spatial registration toolbox of NIRS_SPM according to the Rorden’s brain atlas (Rorden and Brett, 2001) and are presented in Table 1.
Figure 1. (A) Probe configuration and photon sensitivity profile. Each red dot corresponds to a light source while each blue dot represents a detector. Channels are formed between adjacent source-detector pairs and represented with yellow lines and labeled accordingly. The sensitivity of the probe to detecting brain hemodynamics is presented as a heat map in log10 units. (B) Channel locations projected onto a standard brain template in MNI space.
Table 1. The MNI coordinates and the percentage of Brodmann areas covered by each projected channel.
Experimental Design
The study was carried out in a silent and dimly lit room. The temperature of the room was controlled with an air conditioning system and set approximately at 23°C. Subjects sat in front of a computer screen with a 50 cm eye-to-screen distance and all experimental cues were presented on a 24-inch computer monitor. After the acquisition of demographic data, the experimental protocol was explained in detail to each subject. During the experiment, subjects had to decide whether a puzzle piece shown on the screen could fit the gap in the middle of a puzzle template, shown after the puzzle piece, by pressing either the “S” or “K” key on the keyboard, indicating a “Yes” (“Congruent”) or “No” (“Incongruent”) answer respectively. There were three conditions: (1) Perfect Match (PM) condition in which the piece can fit into the template without requiring a mental rotation, (2) Match (M) condition in which the piece would fit into the template only if the subject mentally rotated the piece, and (3) a Non-Match (NM) condition in which the piece could not fit into the template (Figure 2). Each subject made decisions for 24 trials consisting of 6 PM, 6 M, and 12 NM conditions presented in random order. All of the Perfect Match and Match trials formed a “congruent” condition and required a “Yes” answer, while Non-Match trials formed an “incongruent” condition requiring a “No” answer. Each experiment began with a baseline fNIRS recording of 30 s, after which the subject pressed the “Space” key on the keyboard to start the experiment. At the beginning of each trial, a plus sign (“+”) was presented for 500 ms to briefly alert the subject to the onset of a new trial. Then, a puzzle piece was presented for 500 ms, followed by a white noise masking image presentation for 750 ms to eliminate the after-image effect of the puzzle piece. Subsequently, the template was shown for 3 s during which the subject had to give a “Yes” or “No” answer by pressing “S” or “K” on the keyboard, respectively. Afterward, a white noise masking was shown for another 750 ms. All of the visual cues and images were rendered at the center of the screen in order to eliminate a possible visual field effect. The total duration of a trial was 5.5 s and the inter-trial interval was set randomly for each trial, ranging from 9 to 12 s, in order to prevent subjects’ anticipatory hemodynamic responses (Figure 3). The experiment was designed with PsychoPy 3.0.6 (Peirce et al., 2019). Reaction times and accuracy rates for each trial were collected as behavioral data with PsychoPy software. fNIRS data were collected continuously during the experiment.
Figure 2. Examples of perfect match, match, and non-match trials.
Figure 3. Experimental procedure. A 30 s long-baseline recording was followed by a 0.5 s long fixation point shown in the center of the computer screen. After turning off the fixation point, subjects saw a randomly chosen jigsaw puzzle piece, followed by 0.75 s long white noise masking image. Subsequently, the puzzle template was shown on the screen for 3 s during which the subjects had to give a “Yes” (“Matched”) or “No” (“Unmatched”) answer. Another 0.75 s long white noise masking image was followed by an inter-stimulus interval randomly chosen from the range of 9–12 s.
Data Preprocessing and Feature Selection
Data preprocessing was performed with a combination of nirsLAB (a free analysis software that comes with the NIRSport system), Homer2 and custom-made scripts written in MATLAB R2017B. Raw intensity data were first visually examined in nirsLAB. Then, the coefficients of variation (CV) method (Hocke et al., 2018; Pfeifer et al., 2018) was employed to eliminate the channels with poor signal quality. The threshold for CV was set to 7.5% as used by Hocke et al. (2018). A preprocessing pipeline based on HOMER2 functions was constructed. The raw intensity data were converted into optical density (OD) data with hmrIntensity2OD function. Data segments that included motion artifacts were identified by using hmrMotionArtifact.m function with the following parameters: tMotion = 0.5, tMask = 1, STDEVthresh = 10, and AMPthresh = 1. PCA filtering was applied for motion artifact removal by using hmrMotionCorrectPCA.m with the parameter nSV set to 0.8. Motion corrected OD data were filtered with a third-order Butterworth bandpass filter with high and low cut-off frequencies of 0.01 and 0.2 Hz, respectively. Lastly, oxygenated hemoglobin (HBO) and deoxygenated hemoglobin (HBR) concentrations were calculated from filtered optical density data by using hmrOD2Conc.m function. Only HBO data were used in the analysis as it was shown that HBO is a more reliable indicator of cortical activation (Watanabe et al., 2002; Dravida et al., 2017).
Data Analysis
Behavioral Data Analysis
Accuracy was calculated by dividing the number of successfully answered trials by the total number of trials for each subject. In the reaction time dataset, for the trials during which the subject could not give a valid answer within the 3 s response period, reaction time was set at 3 s, the maximum. Due to the unbalanced number of trials among conditions, we calculated the average reaction time and accuracy per condition for each subject. Therefore 14 × 3 matrices (number of subjects × conditions) were created for reaction time and accuracy separately and were subjected to one-way repeated measures ANOVA.
fNIRS Data Analysis
For each trial, 14 s long block segments were created which spanned a 2 s pre-stimulus baseline interval and a 12 s long duration after the onset of stimulus presentation. Each segment was detrended and classified as a Perfect Match, Match or Non-Match condition trial. We defined a new parameter, called Hemodynamic Activity Strength (HAS), for hemodynamic time series as in the following: First, the time point corresponding to the maximum concentration value between 4.5 and 12 s period after the onset of a trial is determined. Then, a mean peak value (MPV) is calculated within a window, which spans 1.5 s before and after that peak. The mean value of the 2 s pre-stimulus baseline window is calculated (MB) and subtracted from the MPV. Lastly, this value is divided by the standard deviation of the pre-stimulus baseline window.
Hemispheric lateralization analysis
At the single subject level, whether a channel was deemed significant was tested for each condition as follows: (1) Single-trial HAS parameter of each trial (n = 6 for PM and M conditions; n = 12 for NM condition) was computed separately for each condition and channel with the abovementioned method. (2) One sample student t-tests were performed on the vector of HAS parameters belonging to each condition and channel separately. (3) Channels having statistical significance (p < 0.05) were considered as active channels.
Hemispheric Lateralization (HL) parameter was calculated for each subject and condition with the following formula:
where NRC denotes the number of active channels in the right hemisphere; NLC denotes the number of active channels in the left hemisphere; NC is the total number of channels in a hemisphere (i.e., NC equals to 10); and HLsub,con denotes the hemispheric difference parameter for a subject and condition. With this formula, the hemispheric lateralization parameter is normalized between -1 and 1, where positive values denote right hemispheric lateralization and the negative values denote left hemispheric lateralization.
To assess whether there is a statistically significant hemispheric lateralization in a condition, 14 × 1 vectors of hemispheric lateralization parameters belonging to each condition were subjected to a one-sample student t-test. Furthermore, to assess whether there is a difference between conditions in terms of lateralization, hemispheric lateralization vectors were subjected to one-way ANOVA with the condition as the main effect.
Hemodynamic activity analysis
To localize the hemodynamic activity, group level HAS parameter was computed as follows: (1) For each channel and subject, the trial time series of each condition were block averaged, resulting in a single time-series data for each condition and channel in a subject (see section fNIRS Data Analysis). (2) A HAS parameter was calculated on this block-averaged time series with the abovementioned method. (3) Lastly, 14 × 3 matrices (subject × condition) were generated with group level HAS parameters for each channel.
To localize the statistically significant hemodynamic activity at the group level, one sample student t-test was performed on group level HAS parameter separately for each channel and condition. To assess contrasts among different conditions, a channel based one-way repeated measure ANOVA was performed with the condition as the main effect. Post hoc t-tests with Bonferroni correction were performed for pairwise comparisons for the following contrasts: Match > Perfect Match, Non-Match > Perfect Match and Non-Match > Match. Thresholded t-statistics depicting significant activation were mapped onto a standard head template for each condition and contrast. Including channels as a main factor in ANOVA analyses should be avoided in fNIRS studies because of the systematic bias caused by variations in optical properties of different brain regions, as suggested in the works of Yanagisawa et al. (2010) and Kujach et al. (2018). Thus channel was not included as a main factor in ANOVA. All statistical analyses were conducted with IBM SPSS Statistics 2015 and MATLAB R2017B.
Results
Behavioral Data
Reaction time and accuracy data are summarized in Table 2.
Table 2. Summary of the behavioral data in mean ± standard error.
Reaction Time
We found a significant difference among conditions in terms of reaction time [F(2, 39) = 4.931, p = 0.015]. Bonferroni corrected post hoc analysis of paired t-test results showed that reaction times for the Match trials were significantly longer than those for the Perfect Match trials (p = 0.02), while no significant difference was found between the Non-Match and the Match (p = 0.272) or between the Non-Match and the Perfect Match (p = 0.99) conditions, even though Non-Match required slightly longer reaction times than Perfect Match condition.
Accuracy
One way repeated measure ANOVA results did not show any significant difference among conditions for accuracy data [F(2, 39) = 0.236, p = 0.791].
Reaction Time vs. Accuracy
We calculated Pearson’s correlation between accuracy and reaction time for each condition. A negative correlation between the two behavioral metrics was found for each condition but only Match (p = 0.012, R = −0.648) and Non-Match (p = 0.026, R = −0.591) conditions surpassed a statistically significant threshold while Perfect Match condition did not (p = 0.165, R = −0.393).
fNIRS Data
Hemispheric Lateralization Analysis
Hemispheric lateralization data are listed in Table 3. One sample student t-test results showed that Perfect Match condition had a statistically significant right hemispheric lateralization (HL = 0.071 ± 0.029, p = 0.027), while Match (HL = 0.029 ± 0.049, p = 0.566) and Non-Match (HL = 0.036 ± 0.064, p = 0.588) conditions did not present statistical significance even though they show a right hemispheric lateralization. Besides, 57% of the total number of active channels from all subjects were located on the right hemisphere in Perfect Match condition, while only 39% of active channels were located on the left hemisphere. One way repeated measure ANOVA revealed that there was no significant difference among conditions in terms of hemispheric lateralization [F(2, 39) = 0.22, p = 0.806].
Table 3. Hemispheric Lateralization (HL) parameter of each subject.
Hemodynamic Activity Analysis
The p-values obtained from one-sample t-tests for each channel showed that significant hemodynamic changes occurred in 8 out of 22 channels for Perfect Match condition; 18 channels for Match condition; and 19 channels for Non-Match condition (p < 0.05). Figure 4 depicts the thresholded t-statistics parameter for each condition on a head model. In Perfect Match condition, neural activity was localized mainly in the right hemisphere (2 left, 5 right, 1 medial) while a bilateral neural activity was observed during Match (9 left, 8 right and 1 medial) and Non-Match (9 left, 8 right and 2 medial) conditions (Table 4).
Figure 4. Hemodynamic activity strength parameter map for (A) Perfect Match; (B) Match; and (C) Non Match conditions. Channels depicting statistically significant activation (p < 0.05) were mapped onto a standard head model with their thresholded t-statistics score for each condition.
Table 4. Significantly active channels and corresponding brain regions for each condition at the group level.
One-way repeated measure ANOVA revealed significant hemodynamic activity differences among conditions in two channels: Channel 9 [F(2, 26) = 6.500, p = 0.024] and Channel 14 [F(2, 24) = 4.059, p = 0.039]. These channels correspond to the right Brodmann Area (BA) 9, 10, 46, and left BA 10, 46, respectively. However, the majority of the signal collected by these channels come from right and left DLPFC, respectively (Table 1). Post hoc t-tests with Bonferroni correction showed that HAS of Match condition was stronger than Perfect Match condition only in the left DLPFC (p = 0.014); HAS of Non-Match condition was stronger than Match condition only in the right DLPFC (p = 0.005); and HAS of Non-Match condition was stronger than Perfect Match condition in only right DLPFC (p = 0.004) (Figure 5).
Figure 5. Hemodynamic activity strength parameter map depicting the contrasts for (A) Match > Perfect Match; (B) Non Match > Perfect Match; and (C) Non Match > Match conditions. Channels depicting statistically significant contrast (p > 0.05) were mapped onto a standard head model with their thresholded t-statistics score for each condition pair.
Discussion
The aim of this study was to investigate the neural activity lateralization in the prefrontal cortex in response to mental rotation tasks with varying complexity and to demonstrate the reliability of fNIRS for capturing the neurophysiological mechanisms underlying mental rotation. For this purpose, a novel mental rotation task based on mentally fitting a jigsaw puzzle piece into a template was designed. Our results demonstrated a right hemispheric activation in all mental rotation tasks while recruitment of the left hemisphere with increasing task complexity.
Behavioral Data
Assuming task difficulty is accompanied by longer reaction times as demonstrated by Shepard and Metzler (1971), the reaction time data revealed task difficulty as Match > Non-Match > Perfect Match. Perfect Match condition requires no mental rotation of the puzzle piece, thus it shows similarity to simple delayed match-to-sample task (DMTS). However, for the Match and Non-Match conditions, subjects needed to recruit further cognitive processes such as mental rotation, inner speech, and verbalization to make a judgment. Therefore, it can be assumed that in perfect match trials subjects only maintain the relevant information, but in the other two conditions, they had to manipulate it as well. The maintaining and manipulating processes will be discussed later in this report.
High accuracy rates (>%94 ± 1.5) observed in all three conditions indicated that all subjects actively participated in the tasks (Table 2). Even though no statistically significant difference was found in terms of the accuracy values among the three conditions, the Match condition had the lowest accuracy rate (%94.1 ± 2.8) suggesting that it was the most cognitively demanding task. On the other hand, negative correlations between reaction time and accuracy for all three conditions indicated that increased reaction time was associated with increased task difficulty (Shepard and Metzler, 1971; Cooper and Shepard, 1975).
fNIRS Data
When the hemodynamic activity localization was analyzed at the group level for three conditions in terms of hemispheric lateralization parameter, a right hemisphere dominant activation was found for the Perfect Match condition whereas bilateral activations were found for Non-Match and Match conditions (Figure 4). Right DLPFC was found to be activated in all three conditions which was in accordance with many studies reporting right hemispheric activity during mental rotation tasks (Ditunno and Mann, 1990; Corballis, 1997; Harris et al., 2000; Hugdahl et al., 2006). Subjects had to keep the shape of the puzzle piece in their mind until the template is shown and manipulate it where necessary in order to arrive at a judgment. Therefore, visual working memory is essential for all three conditions. It is shown that right DLPFC is involved in visual working memory processes (Leung et al., 2002; Slotnick and Moo, 2006; Daniel et al., 2016) which was supported by our findings. As mentioned above, perfect match trials have a similarity with the simplest form of DMTS tasks. Daniel et al. (2016) described this type of DMTSs as having three phases: cue, delay, and response. In this type of DMTSs, subjects need only to maintain the relevant information in their visual working memory without further manipulation. Thus, activating only the right DLPFC might provide adequate neural sources for solving the perfect match trials. On the other hand, in match and non-match conditions, subjects had to manipulate the information stored in visual working memory in order to arrive at the correct answer. D’Esposito et al. (1999) demonstrated that mental tasks requiring the manipulation of the information stored in working memory are more difficult than tasks requiring only maintaining the relevant information. It is likely that the recruitment of left-hemispheric resources during Match and Non-Match conditions was because of the additional mental strategies (manipulations) employed as the task becomes more difficult. Feedback from subjects revealed that they tended to employ complex strategies during Match and Non-Match conditions such as focusing on local attributes (i.e., number and position of the protrusions) of the shapes rather than global attributes, and/or verbalization and inner speech. Multiple studies have also shown left hemispheric dominance in the processing of local features of complex shapes while right hemispheric dominance in processing global features regardless of the stimuli type (Christie et al., 2012; Brederoo et al., 2017). Language related brain activities are mostly localized in the left hemisphere (Frost et al., 1999) and inner speech is known to be regulated by the left hemisphere (Scholkmann et al., 2013). Another study by Hugdahl et al. (2006) suggested that verbalization might be one of the strategies used by subjects during mental rotation tasks and may explain the left-hemispheric activity. Significantly active channels found in the left hemisphere for Match and Non-Match conditions cover the BA 8, 9, 45, and 46 which are known to be involved in language processing. Hence, mental strategies deployed by subjects during Match and Non-Match tasks might explain the bilateral activation found in the prefrontal regions (Figure 4).
The location of the significant activation found for Match > Perfect Match contrast corresponds to the left DLPFC (Figure 5). As mentioned above, subjects used additional mental strategies (i.e., verbalization/inner speech) in Match condition compared to the Perfect Match condition due to the increased complexity of the task. It was shown that left DLPFC is involved in the processing of various linguistic tasks (Hugdahl et al., 1999; Abrahams et al., 2003). Hugdahl et al. (2006) also claimed that verbalization strategy might be the reason for left-hemispheric activity during the processing of mental rotation tasks. Therefore, we assume that stronger left DLPFC activity in Match condition compared to Perfect Match condition in our experimental design might be due to the use of inner speech and verbalization of local attributes of the stimuli in order to complete the Match trials. For the contrasts depicting Non-Match > Match and Non-Match > Perfect Match, the location of significant activation covered DLPFC of the right hemisphere (Figure 5). A stronger activation localized in the left hemisphere would be expected for Non-Match > Perfect Match and Match > Non-Match contrasts due to the increasing task complexity as revealed by behavioral metrics. However, a dominant right hemispheric activity observed in Non-Match compared to the other two conditions leads us to interpret the results from another perspective. In the Non-Match condition, subjects were exposed to incongruent stimuli and they were expected to correctly reject the unmatching piece-template pair. It is possible that this correct rejection mechanism may require strong conflict monitoring. Chevrier et al. (2007) used event-related fMRI to investigate the neural response for successful and unsuccessful inhibition of incongruent trials during the Stroop test and found that the right DLPFC is one of the active regions. A meta-analysis of Go/No-Go tasks carried out by Simmonds et al. (2008) also demonstrated that the right DLPFC is one of the brain regions found to be in association with successful inhibitions of No-Go trials. In our task, subjects had to recognize that the piece could not fit into the template in the Non-Match condition in contrast to the Perfect Match and Match conditions, thus replying with a “No” response in order to successfully reject the trial.
Top-Down vs. Bottom-Up Modulation
In a meta-analysis review, Tomasino and Gremese (2016) explored the impact of the employed strategy (i.e., top-down modulation) and stimulus type (i.e., bottom-up modulation) on brain activation during mental rotation tasks. They included studies where the specific motor or visual imagery strategy instructions were given at the onset of the experiment. Their findings indicated that the mental rotation network included activations in the inferior frontal and middle frontal gyrus bilaterally, regardless of the stimulus type and type of strategy followed.
Our results indicated that the right DLPFC activation was common for all trial types. Right DLPFC activation is associated with visual working memory processes and may not necessarily be influenced by the type of strategy. As previous literature suggests (Corballis, 1997; Hugdahl et al., 2006), subjects may also employ secondary cognitive processes during mental rotation tasks such as local feature processing and inner speech, which are known to be regulated by left DLPFC resources. We propose that the bilateral activations observed during Match and Non-Match conditions in our experimental design are likely to be related to the use of secondary cognitive processes. It should be noted that Tomasino and Gremese (2016) excluded the studies in which the subjects were not instructed with a specific mental rotation strategy. In our study, we did not give any specific instructions to our subjects on whether to follow motor or visual imagery strategies at the onset of the experiment. Nonetheless, regarding the top-down regulation consensus reported in Tomasino and Gremese (2016), we may speculate that the bilateral DLPFC activation observed during the match and non-match conditions might be due to the usage of motor imagery-based strategy in addition to the abovementioned secondary cognitive processes.
Regarding the frontal cortex regions for mental rotation of non-bodily objects, Tomasino and Gremese (2016) reported activation in inferior frontal gyrus bilaterally and middle frontal gyrus. Our design included only non-bodily stimuli. Bilaterally activated frontal regions observed during Match and Non-Match conditions are in accordance with the findings of Tomassino et al. (2015). Our findings indicate that the left hemispheric activation increases with task difficulty for non-bodily stimuli.
Recommendation for Future Work
In our protocol, the Match condition was the most cognitively demanding condition as supported by both behavioral and hemodynamic metrics (Table 2 and Figure 4). Post-experimental feedback from subjects suggested that their decision-making strategy involved counting the number of protrusions and recesses and/or a mental rotation strategy involving the dynamic spatial transformation of the puzzle piece. Seepanomwan et al. (2015) stated that the task difficulty increases with the number of distinctive features of stimuli. There are two distinctive features in our stimulus design: (i) the number of local features (i.e., protrusions and recessions), and (ii) their orientation. During the non-match conditions, checking either of the distinctive features would be sufficient to arrive at a decision. Subjects may prefer either counting the local features or rotating the puzzle piece. The Match conditions were more challenging than the Non-Match conditions because the subjects had to check both distinctive features, as checking only the number of local features would not be sufficient to make a correct decision (e.g., both the piece and the template may have two protrusions and recesses, yet the protrusions and recesses might be on the consecutive edges for the piece, while they might be located on the opposite edges for the template. Therefore, they had to check the second distinctive feature as well.). However, we should note that more challenging protocols involving match and non-match conditions can also be designed where it will be necessary to consider a mental rotation in all non-match trials (Seepanomwan et al., 2015). In future designs, different strategies might be employed to make the non-match condition more cognitively demanding, such as using unusual rotation angles (e.g., 15, 145, 20, 220 degrees) instead of multiples of 90 degrees, designing the puzzle piece and template with asymmetrical distinctive features.
Removal of instrumental, systemic, and motion artifacts in fNIRS signals generally consist of a combination of three steps: (1) motion correction; (2) bandpass filtering; and (3) low-frequency systemic noise removal. While numerous detailed signal processing methods have been developed for each step, there is still no consensus in literature for a standardized preprocessing pipeline (Pinti et al., 2019). Several methods have been proposed in the literature to eliminate the common systemic noise in fNIRS signals. Regarding data-driven approaches, global signal regression (Carbonell et al., 2011; Hocke et al., 2018), time-delayed global signal subtraction (Erdogan et al., 2016), ICA (Kohno et al., 2007; Santosa et al., 2013), and PCA based methods (Zhang et al., 2005) have been the most widely used methods. However, modeling systemic effects with regressors derived from short separation channel measurements that monitor only scalp hemodynamics are considered as the gold standard for global noise removal (Saager et al., 2011). Data-driven methods have limitations and the potential to distort the signals of interest which are extensively reviewed in Hocke et al. (2018) and Pinti et al. (2019). Our system did not have short separation channels for monitoring scalp hemodynamics. Hence; we decided to use minimal preprocessing practices to avoid possible signal distortion. For future work, we plan to compare the impact of the data-driven noise removal strategies with short channel removal methods for a better understanding of the noise characteristics of the signals collected from the prefrontal cortex region during cognitive tasks. We also plan to monitor frontal and parietal areas with a system equipped with a higher number of channels to investigate functional connectivity measures among different brain regions.
Conclusion
Our findings indicate that the right DLPFC is one of the core brain regions activated during mental rotation regardless of the task difficulty. Our results also showed that left hemisphere activity becomes profound with increased task difficulty and the fact that the left hemispheric involvement does not depend on the dimension (2D or 3D) of the stimuli but on task difficulty. The present study showed that fNIRS is a reliable neuroimaging method to be utilized in mental rotation studies, or in general lateralization studies. It is of vital importance to show that fNIRS technology can be used in assessing various cognitive skills for two reasons. First, it allows for investigating the brain dynamics during cognitive tests in daily, natural environments rather than the noisy, stress-inducing artificial MR or PET settings. Secondly, due to its great ease of use when compared with EEG, fNIRS is a good candidate for new generation BCI systems.
Data Availability Statement
The datasets generated for this study are available on request to the corresponding author.
Ethics Statement
The studies involving human participants were reviewed and approved by the Human Research Ethical Committee of Boğaziçi University. The patients/participants provided their written informed consent to participate in this study.
Author Contributions
MCM, SE, RC, and HS designed the experiments. MCM and OÖ carried out the experiments. MCM and SE analyzed the data, wrote the manuscript, and prepared the figures. All authors reviewed the manuscript.
Funding
This research was financially supported by the Boğaziçi University Research Fund Grant Number 14663.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
We thank Asst. Prof. İnci Ayhan for her help in experimental design and during the interpretation of the findings.
References
Aasted, C. M., Yücel, M. A., Cooper, R. J., Petkov, M. P., Boas, D. A., Cooper, R. J., et al. (2015). Anatomical guidance for functional near-infrared spectroscopy: atlasviewer tutorial. Neurophoton 2:020801. doi: 10.1117/1.NPh.2.2.020801
Abrahams, S., Goldstein, L., Simmons, A., Brammer, M., Williams, S., Giampietro, V., et al. (2003). Functional magnetic resonance imaging of verbal fluency and confrontation naming using compressed image acquisition to permit overt responses. Hum. Brain Mapp. 20, 29–40. doi: 10.1002/hbm.10126
Alivisatos, B., and Petrides, M. (1997). Functional activation of the human brain during mental rotation. Neuropsychologia 35, 111–118. doi: 10.1016/s0028-3932(96)00083-8
Artemenko, C., Coldea, A., Soltanlou, M., Dresler, T., Nuerk, H., and Ehlis, A. (2018). The neural circuits of number and letter copying: an fNIRS study. Exp. Brain Res. 236, 1129–1138. doi: 10.1007/s00221-018-5204-8
Baker, J., Bruno, J., Gundran, A., Hosseini, S., and Reiss, A. (2018). fNIRS measurement of cortical activation and functional connectivity during a visuospatial working memory task. PLoS One 13:e0201486. doi: 10.1371/journal.pone.0201486
Banville, H., Gupta, R., and Falk, T. (2017). Mental task evaluation for hybrid NIRS-EEG brain-computer interfaces. Comput. Intell. Neurosci. 2017, 1–24. doi: 10.1155/2017/3524208
Brederoo, S., Nieuwenstein, M., Lorist, M., and Cornelissen, F. (2017). Hemispheric specialization for global and local processing: a direct comparison of linguistic and non-linguistic stimuli. Brain Cogn. 119, 10–16. doi: 10.1016/j.bandc.2017.09.005
Carbonell, F., Bellec, P., and Shmuel, A. (2011). Global and system-specific resting-state fmri fluctuations are uncorrelated: principal component analysis reveals anti-correlated networks. Brain Connect. 1, 496–510. doi: 10.1089/brain.2011.0065
Chevrier, A., Noseworthy, M., and Schachar, R. (2007). Dissociation of response inhibition and performance monitoring in the stop signal task using event-related fMRI. Hum. Brain Mapp. 28, 1347–1358. doi: 10.1002/hbm.20355
Christie, J., Ginsberg, J., Steedman, J., Fridriksson, J., Bonilha, L., and Rorden, C. (2012). Global versus local processing: seeing the left side of the forest and the right side of the trees. Front. Hum. Neurosci. 6:28. doi: 10.3389/fnhum.2012.00028
Cohen, M., Kosslyn, S., Breiter, H., DiGirolamo, G., Thompson, W., Anderson, A., et al. (1996). Changes in cortical activity during mental rotation A mapping study using functional MRI. Brain 119, 89–100. doi: 10.1093/brain/119.1.89
Cooper, L., and Shepard, R. (1975). Mental transformation in the identification of left and right hands. J. Exp. Psychol. Hum. Percept. Perform. 1, 48–56. doi: 10.1037/0096-1523.1.1.48
Corballis, M. (1997). Mental rotation and the right hemisphere. Brain Lang. 57, 100–121. doi: 10.1006/brln.1997.1835
Curtin, A., Sun, J., Zhao, Q., Onaral, B., Wang, J., Tong, S., et al. (2019). Visuospatial task-related prefrontal activity is correlated with negative symptoms in schizophrenia. Sci. Rep. 4:9575. doi: 10.1038/s41598-019-45893-7
Daniel, T. A., Katz, J. S., and Robinson, J. L. (2016). Delayed match-to-sample in working memory: a brainmap meta-analysis. Biol Psychol. 120, 10–20. doi: 10.1016/j.biopsycho.2016.07.015
D’Esposito, M., Postle, B. R., Ballard, D., and Lease, J. (1999). Maintenance versus manipulation of information held in working memory: an event-related fMRI study. Brain Cogn. 41, 66–86. doi: 10.1006/brcg.1999.1096
Deutsch, G., Bourbon, W., Papanicolaou, A., and Eisenberg, H. (1988). Visuospatial tasks compared via activation of regional cerebral blood flow. Neuropsychologia 26, 445–452. doi: 10.1016/0028-3932(88)90097-8
Ditunno, P., and Mann, V. (1990). Right hemisphere specialization for mental rotation in normals and brain damaged subjects. Cortex 26, 177–188. doi: 10.1016/s0010-9452(13)80349-8
Dravida, S., Noah, J., Zhang, X., and Hirsch, J. (2017). Comparison of oxyhemoglobin and deoxyhemoglobin signal reliability with and without global mean removal for digit manipulation motor tasks. Neurophotonics 5:011006. doi: 10.1117/1.nph.5.1.011006
Erdogan, S. B., Tong, Y., Hocke, L. M., Lindsey, K. P., and deB Frederick, B. (2016). Correcting for blood arrival time in global mean regression enhances functional connectivity analysis of resting state fmri-bold signals. Front. Hum. Neurosci. 10:311. doi: 10.3389/fnhum.2016.00311
Frost, J. A., Binder, J. R., Springer, J. A., Hammeke, T. A., Bellgowan, P. S., Rao, S. M., et al. (1999). Language processing is strongly left lateralized in both sexes: evidence from functional MRI. Brain 122, 199–208. doi: 10.1093/brain/122.2.199
Fu, G., Wan, N., Baker, J., Montgomery, J., Evans, J., and Gillam, R. (2016). A proof of concept study of function-based statistical analysis of fNIRS Data: syntax comprehension in children with specific language impairment compared to typically-developing controls. Front. Behav. Neurosci. 10:108. doi: 10.3389/fnbeh.2016.00108
Halari, R., Sharma, T., Hines, M., Andrew, C., Simmons, A., and Kumari, V. (2006). Comparable fMRI activity with differential behavioural performance on mental rotation and overt verbal fluency tasks in healthy men and women. Exp. Brain Res. 169, 1–14. doi: 10.1007/s00221-005-0118-7
Harris, I. M., Egan, G. F., Sonkkila, C., Tochon-Danguy, H. J., and Watson, J. D. G. (2000). Selective right parietal lobe activation during mental rotation. Brain 123, 65–73. doi: 10.1093/brain/123.1.65
Herff, C., Heger, D., Putze, F., Hennrich, J., Fortmann, O., and Schultz, T. (2013). “Classification of mental tasks in the prefrontal cortex using fNIRS,” in 2013 35Th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Osaka.
Hocke, L. M., Oni, I. K., Duszynski, C. C., Corrigan, A. V., deB Frederick, B., and Dunn, J. (2018). Automated processing of fNIRS Data—A visual guide to the pitfalls and consequences. Algorithms 11:67. doi: 10.3390/a11050067
Hugdahl, K., Brønnick, K., Kyllingsbrk, S., Law, I., Gade, A., and Paulson, O. (1999). Brain activation during dichotic presentations of consonant-vowel and musical instrument stimuli: a 15 O-PET study. Neuropsychologia 37, 431–440. doi: 10.1016/s0028-3932(98)00101-8
Hugdahl, K., Thomsen, T., and Ersland, L. (2006). Sex differences in visuo-spatial processing: an fMRI study of mental rotation. Neuropsychologia 44, 1575–1583. doi: 10.1016/j.neuropsychologia.2006.01.026
Huppert, T., Diamond, S., Franceschini, M., and Boas, D. (2009). HomER: a review of time-series analysis methods for near-infrared spectroscopy of the brain. Appl. Opt. 48:D280. doi: 10.1364/ao.48.00d280
Jang, K. E., Tak, S., Jung, J., Jang, J., Jeong, Y., and Ye, J. C. (2009). Wavelet-MDL detrending for near-infrared spectroscopy. J. Biomed. Opt. 14, 1–13. doi: 10.1117/1.3127204
Jasinska, K., and Petitto, L. (2013). How age of bilingual exposure can change the neural systems for language in the developing brain: a functional near infrared spectroscopy investigation of syntactic processing in monolingual and bilingual children. Dev. Cogn. Neurosci. 6, 87–101. doi: 10.1016/j.dcn.2013.06.005
Jordan, K., Heinze, H., Lutz, K., Kanowski, M., and Jäncke, L. (2001). Cortical activations during the mental rotation of different visual objects. Neuroimage 13, 143–152. doi: 10.1006/nimg.2000.0677
Kohno, S., Miyai, I., Seiyama, A., Oda, I., Ishikawa, A., Tsuneishi, S., et al. (2007). Removal of the skin blood flow artifact in functional near-infrared spectroscopic imaging data through independent component analysis. J. Biomed. Opt. 12:062111. doi: 10.1117/1.2814249
Kujach, S., Byun, K., Hyodo, K., Suwabe, K., Fukuie, T., Laskowski, R., et al. (2018). A transferable high-intensity intermittent exercise improves executive performance in association with dorsolateral prefrontal activation in young adults. Neuroimage 169, 117–125. doi: 10.1016/j.neuroimage.2017.12.003
Leung, H., Gore, J., and Goldman-Rakic, P. (2002). Sustained mnemonic response in the human middle frontal gyrus during on-line storage of spatial memoranda. J. Cogn. Neurosci. 14, 659–671. doi: 10.1162/08989290260045882
Mehta, Z., and Newcombe, F. (1991). A role for the left hemisphere in spatial processing. Cortex 27, 153–167. doi: 10.1016/s0010-9452(13)80121-9
Papanicolaou, A., Deutsch, G., Bourbon, W., Will, K., Loring, D., and Eisenberg, H. (1987). Convergent evoked potential and cerebral blood flow evidence of task-specific hemispheric differences. Electroencephalogr. Clin. Neurophysiol. 66, 515–520. doi: 10.1016/0013-4694(87)90098-8
Peirce, J. W., Gray, J. R., Simpson, S., MacAskill, M. R., Höchenberger, R., Sogo, H., et al. (2019). PsychoPy2: experiments in behavior made easy. Behav. Res. Methods 51, 195–201. doi: 10.3758/s13428-018-01193-y
Pfeifer, M., Scholkmann, F., and Labruyère, R. (2018). Signal processing in functional near-infrared spectroscopy (fNIRS): methodological differences lead to different statistical results. Front. Hum. Neurosci. 11:641. doi: 10.3389/fnhum.2017.00641
Pinti, P., Scholkmann, F., Hamilton, A., Burgess, P., and Tachtsidis, I. (2019). Current status and issues regarding pre-processing of fNIRS neuroimaging data: an investigation of diverse signal filtering methods within a general linear model framework. Front. Hum. Neurosci. 12:505. doi: 10.3389/fnhum.2018.00505
Qureshi, N., Naseer, N., Noori, F., Nazeer, H., Khan, R., and Saleem, S. (2017). Enhancing classification performance of functional near-infrared spectroscopy- brain–computer interface using adaptive estimation of general linear model coefficients. Front. Neurorobot. 11:33. doi: 10.3389/fnbot.2017.00033
Ratcliff, G. (1979). Spatial thought, mental rotation and the right cerebral hemisphere. Neuropsychologia 17, 49–54. doi: 10.1016/0028-3932(79)90021-6
Rorden, C., and Brett, M. (2001). Stereotaxic display of brain lesions. Behav. Neurol. 12, 191–200. doi: 10.1155/2000/421719
Saager, R. B., Telleri, N. L., and Berger, A. J. (2011). Two-detector corrected near infrared spectroscopy (C-NIRS) detects hemodynamic activation responses more robustly than single-detector NIRS. Neuroimage 55, 1679–1685. doi: 10.1016/j.neuroimage.2011.01.043
Santosa, H., Hong, M. J., Kim, S.-P., and Hong, K.-S. (2013). Noise reduction in functional near-infrared spectroscopy signals by independent component analysis. Rev. Sci. Instrum. 84:73106. doi: 10.1063/1.4812785
Scholkmann, F., Wolf, M., and Wolf, U. (2013). “The effect of inner speech on arterial co2 and cerebral hemodynamics and oxygenation: a functional NIRS study,” in Oxygen Transport to Tissue XXXV. Advances in Experimental Medicine and Biology, Vol. 789, eds S. Van Huffel, G. Naulaers, A. Caicedo, D. F. Bruley, and D. K. Harrison (New York, NY: Springer).
Seepanomwan, K., Caligore, D., Cangelosi, A., and Baldassarre, G. (2015). Generalization, decision making, and embodiment effects in mental rotation: a neurorobotic architecture tested with humanoid robot. Neural Networks 72, 31–47. doi: 10.1016/j.neunet.2015.09.010
Seydell-Greenwald, A., Ferrara, K., Chambers, C. E., Newport, E. L., and Landau, B. (2017). Bilateral activations for complex visual-spatial functions: evidence from a visual-spatial construction task. Neuropsychologia 106, 194–206. doi: 10.1016/j.neuropsychologia.2017.10.005
Shepard, R., and Metzler, J. (1971). Mental rotation of three-dimensional objects. Science 171, 701–703. doi: 10.1126/science.171.3972.701
Shimoda, N., Takeda, K., Imai, I., Kaneko, J., and Kato, H. (2008). Cerebral laterality differences in handedness: a mental rotation study with NIRS. Neurosci. Lett. 430, 43–47. doi: 10.1016/j.neulet.2007.10.016
Simmonds, D., Pekar, J., and Mostofsky, S. (2008). Meta-analysis of Go/No-go tasks demonstrating that fMRI activation associated with response inhibition is task-dependent. Neuropsychologia 46, 224–232. doi: 10.1016/j.neuropsychologia.2007.07.015
Singh, A. K., Okamoto, M., Dan, H., Jurcak, V., and Dan, I. (2005). Spatial registration of multichannel multi-subject fNIRS data to MNI space without MRI. NeuroImage 27, 842–851. doi: 10.1016/j.neuroimage.2005.05.019
Slotnick, S., and Moo, L. (2006). Prefrontal cortex hemispheric specialization for categorical and coordinate visual spatial memory. Neuropsychologia 44, 1560–1568. doi: 10.1016/j.neuropsychologia.2006.01.018
Sugiura, L., Ojima, S., Matsuba-Kurita, H., Dan, I., Tsuzuki, D., Katura, T., et al. (2015). Effects of sex and proficiency in second language processing as revealed by a large-scale fNIRS study of school-aged children. Hum. Brain Mapp. 36, 3890–3911. doi: 10.1002/hbm.22885
Tomasino, B, and Gremese, M. (2016). Effects of stimulus type and strategy on mental rotation network: an activation likelihood estimation meta-analysis. Front. Hum. Neurosci. 9:693. doi: 10.3389/fnhum.2015.00693
Vingerhoets, G., de Lange, F., Vandemaele, P., Deblaere, K., and Achten, E. (2002). Motor imagery in mental rotation: an fMRI study. Neuroimage 17, 1623–1633. doi: 10.1006/nimg.2002.1290
Watanabe, A., Kato, N., and Kato, T. (2002). Effects of creatine on mental fatigue and cerebral hemoglobin oxygenation. Neurosci. Res. 42, 279–285. doi: 10.1016/s0168-0102(02)00007-x
Yanagisawa, H., Dan, I., Tsuzuki, D., Kato, M., Okamoto, M., Kyutoku, Y., et al. (2010). Acute moderate exercise elicits increased dorsolateral prefrontal activation and improves cognitive performance with Stroop test. Neuroimage 50, 1702–1710. doi: 10.1016/j.neuroimage.2009.12.023
Ye, J. C., Tak, S., Jang, K. E., Jung, J., and Jang, J. (2009). NIRS-SPM: statistical parametric mapping for near-infrared spectroscopy. NeuroImage 44, 428–447. doi: 10.1016/j.neuroimage.2008.08.036
Keywords: fNIRS, lateralization, mental rotation, puzzle, DLPFC, prefrontal cortex
Citation: Mutlu MC, Erdoğan SB, Öztürk OC, Canbeyli R and Saybaşιlι (2020) Functional Near-Infrared Spectroscopy Indicates That Asymmetric Right Hemispheric Activation in Mental Rotation of a Jigsaw Puzzle Decreases With Task Difficulty. Front. Hum. Neurosci. 14:252. doi: 10.3389/fnhum.2020.00252
Received: 26 November 2019; Accepted: 08 June 2020;
Published: 30 June 2020.
Edited by:
Fenghua Tian, University of Texas at Arlington, United StatesReviewed by:
Summer Sheremata, Florida Atlantic University, United StatesGeorge Alexandrakis, University of Texas at Arlington, United States
Copyright © 2020 Mutlu, Erdoğan, Öztürk, Canbeyli and Saybaşιlι. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Hale Saybaşιlι, saybasil@boun.edu.tr