Dalila Burin
Adriana Salatino
Mounia Ziat- 1Smart-Aging Research Centre (SARC), Institute of Development, Aging and Cancer (IDAC), Tohoku University, Sendai, Japan
- 2Institute of Neuroscience, Université Catholique de Louvain Brussels, Brussels, Belgium
- 3Department of Psychology, University of Turin, Turin, Italy
- 4Information Design and Corporate Communication, Bentley University, Waltham, MA, United States
Editorial on the Research Topic
Virtual, mixed, and augmented reality in cognitive neuroscience and neuropsychology
Introduction
Among many endeavors in scientific research, virtual reality (VR) is one of the most compelling and fruitful scenario for cognitive neuroscience and neuropsychology (Bohil et al., 2011; Parsons et al., 2020). Because of its flexibility and adaptability to different scopes, VR technology has advanced from mere display-VR; where the simulation is implemented on a 2D monitor with depthless interaction, to immersive virtual reality (IVR), that mimics and overlaps with the physical world that fully engages the physical body (Slater, 2009).
VR allows the full manipulation of environmental (such as visual scenarios) and body-related (such as somatic features) parameters (Slater et al., 2008; Peck et al., 2013; Chan et al., 2021); as additional advantage, it can be used in combination with other measurements, such as brain-computer interface (BCI) (Nierula et al., 2021) and motion tracking systems (Banakou et al., 2013), making it a noteworthy method for neuroscientists to explore motor and cognitive functions.
While the initial usage of VR in cognitive neuroscience was to provide a sense of presence by properly representing the virtual setting and the virtual body (Sanchez-Vives and Slater, 2005), it has more recently expanded into diagnostic and rehabilitation strategies for clinical populations (Matamala-Gomez et al., 2021). Several studies involving VR ranges from the exploration of basic concepts such as the motor and bodily self-consciousness (Herbelin et al., 2016), to clinical treatments (Ziat et al., 2014; Chin et al., 2021) and approaches for attention and visuo-spatial disorders (Gammeri et al., 2020; De Luca et al., 2021) and clinical.
Research topic content
The aim of this Research Topic “Virtual, Mixed, and Augmented Reality in Cognitive Neuroscience and Neuropsychology” published in Frontiers in Psychology-Neuropsychology, was to highlight the potential of Virtual Reality into developing new tools for research in cognitive psychology and clinical research. The Research Topic features 7 research articles. Most of the contributions examined spatial disorientation, memory distortion, and multisensory effects on attention. Based on their contributions, the research articles were grouped into three main areas: somatic and spatial features of the virtual body (2 papers), Assessing Learning and Memory in VR (2 papers), IVR Assessment of Attentional and Spatial Disorders (3 papers).
Somatic and spatial features of the virtual body
To enhance immersion, the displayed virtual body, known as avatar, must comprise certain characteristics. By combing BCI and VR, Ziadeh et al. measured the sense of body ownership and agency during a motor imagery task by comparing two levels of embodiments: a humanoid virtual hand to induce a strong level of embodiment, and abstract blocks to infer a low level of embodiment. It was shown that the ownership over the humanoid hand led to an increased sense of agency, when controlling for performance. The authors recommended the implementation of anthropomorphic visual feedback on the virtual body, instead of using abstract images, in order to increase the body embodiment. The position of the avatar plays a critical role in the VR interaction. del Aguila et al. looked for the optimal interpersonal distance (IPD) between the avatar and the agent during a facial emotion recognition task. The IPD is an important concept in virtual social interaction as it determines the maintained distance between the avatar and the agent without invading the personal space of the agent. While general emotion recognition in IVR was very effective, all tested IPDs were considered equal in terms of emotional recognition. For specific emotions such as anger and sadness, a 75 cm distance provided the best results.
Assessing learning and memory in VR
Navigation is a key feature in VR that can offer major benefits in studying memory and learning processes in both healthy individuals and neurological patients. Daviddi et al. focused on how mnemonic reactivation can be either enhanced or distorted by visual exposure and vividness of memory by asking healthy subjects to freely explore artworks in a virtual tour of the Louver museum. This encoding phase was followed by a reactivation phase, 2 h later, and a recognition phase, about approximately 24 h later, where participants were asked to recall old and new artworks (lures). The reactivation phase led to high recognition rate of the reactivated targets (previously seen artworks), but also a high false alarm yet where lures were recalled as being part of the virtual tour. This result is consistent with the premise that memory reactivation is an active process. During the recognition phase, higher living rates were highly correlated with viewing time. Old lures (presented during the reactivation phase) had high living rates comparing to new lures. Visitors vividly believed they were part of the initial virtual museum tour suggesting that memory intrusions are more easily incorporated into true memories.
Virtual Learning and Training is getting a lot of attention. In a clinical setting, Biffi et al. used an IVR where children with cerebral palsy (CB) and typical developing (TD) peers had to navigate in a 5-way maze in a playground to find a reward. While typical participants immediately succeeded the task, independently from the strategy they preferred, half of the children with CP needed more attempts with longer time to identify the best navigation method. They also had a preference for egocentric strategies based on body motion comparing to TD children who mainly relied on allocentric strategies based on environmental cues. The efficiency of the dynamic process of learning and adaptation to environmental modifications seem to be the crucial point for children with CP patients. IVR controlled environments can support this adaptability and help with learning.
IVR assessment of attentional and spatial disorders
A very commonly studied disorders in VR is unilateral spatial neglect (USN) that consists of the failure to report, respond, or orient to stimuli presented to opposite side of the lesioned hemisphere, when this failure cannot be attributed to either sensory or motor defects (Heilman et al., 2000). In their systematic review, Kaiser et al. highlighted the efficiency of combination of IVR and eye tracking (ET) as a tool to design tasks and measures for USN patients and help identify those undiagnosed with traditional tests. IVR specifically guarantees a high level of ecological validity, while ET, in particular, can potentially be useful in differentiating subtypes of USN. A practical example comes from Hougaard et al.'s study where they compare the IVR-ET system with conventional test for a USN diagnosis. IVR-ET combination was more sensitive in detecting three egocentric neglect patients that conventional tests failed to detect. Contrariwise, four patients, that were not detected in the VR system, were picked by the traditional tests. This work emphasizes the importance of complementary diagnostics tools, such as IVR-ET as spatial neglect symptoms are highly heterogeneous. Additional testing and assessment programs can benefit from IVR. Dozio et al. proposed an IVR version of the Bell Test (Gauthier et al., 1989), an established neuropsychological test to assess attentional and visuospatial disorders, to investigate the effect of smell (peppermint scent), and audition (piano tones) on visuospatial attention. When single or multisensory stimulations came from the left side, the performance improved; results that can be explained by the pseudoneglect phenomenon (Jewell and McCourt, 2000), where adults tend to overestimate the left side of the visual hemispace. When both stimuli came from the right side, they were significantly more effective in capturing participants' visuospatial attention. This study emphasizes the importance of multisensory stimulation and provides interesting insights about multisensory combination for diagnosis and treatment of neuropsychological disorders, such as USN.
Outlook
In this Research Topic, researchers described a wide range of challenges in using VR in laboratory and clinical settings, ranging from the importance to the avatar features to the importance of using more IVR environments for assessments, scenarios, and treatments. VR can also be an effective investigation tool for understanding cognitive functions such as learning and memory. The evidence produced by this collection of papers collectively highlights the growing trend for using VR in neuropsychology.
Author contributions
All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
The handling editor SP declared a shared affiliation with the author AS at the time of review.
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
Banakou, D., Groten, R., and Slater, M. (2013). Illusory ownership of a virtual child body causes overestimation of object sizes and implicit attitude changes. Proc. Natl. Acad. Sci. U.S.A. 110, 12846–12851. doi: 10.1073/pnas.1306779110
Bohil, C. J., Alicea, B., and Biocca, F. A. (2011). Virtual reality in neuroscience research and therapy. Nat. Rev. Neurosci. 12, 752–762. doi: 10.1038/nrn3122
Chan, S. H. M., Qiu, L., Esposito, G., Mai, K. P., Tam, K. P., Cui, J., et al. (2021). Nature in virtual reality improves mood and reduces stress: evidence from young adults and senior citizens. Virtual Real. 1, 16. doi: 10.1007/s10055-021-00604-4
Chin, K., Thompson, M., and Ziat, M. (2021). “The effect of multimodal virtual reality experience on the emotional responses related to injections,” in VISIGRAPP (2: HUCAPP), 128–134.
De Luca, R., Leonardi, S., Portaro, S., Le Cause, M., De Domenico, C., Colucci, P., et al. (2021). Innovative use of virtual reality in autism spectrum disorder: a case-study. Appl. Neuropsychol. 10, 90–100. doi: 10.1080/21622965.2019.1610964
Gammeri, R., Iacono, C., Ricci, R., and Salatino, A. (2020). Unilateral spatial neglect after stroke: current insights. Neuropsychiatr. Dis. Treat. 16. 131–152. doi: 10.2147/NDT.S171461
Gauthier, L., Dehaut, F., and Joanette, Y. (1989). The bells test: a quantitative and qualitative test for visual neglect. Int. J. Clin. Neuropsychol. 11, 49–54. doi: 10.1037/t28075-000
Heilman, K. M., Valenstein, E., and Watson, R. T. (2000). Neglect and related disorders. Semin Neurol 2000, 463–470. doi: 10.1055/s-2000-13179
Herbelin, B., Salomon, R., Serino, A., and Blanke, O. (2016). Neural Mechanisms of Bodily Self-Consciousness and the Experience of Presence in Virtual Reality. De Gruyter. p. 80–96.
Jewell, G., and McCourt, M. E. (2000). Pseudoneglect: a review and meta-analysis of performance factors in line bisection tasks. Neuropsychologia 38, 93–110. doi: 10.1016/S0028-3932(99)00045-7
Matamala-Gomez, M., Maselli, A., Malighetti, C., Realdon, O., Mantovani, F., and Riva, G. (2021). Virtual body ownership illusions for mental health: a narrative review. J. Clin. Med. 10, 1–19. doi: 10.3390/jcm10010139
Nierula, B., Spanlang, B., Martini, M., Borrell, M., Nikulin, V. V., Sanchez-Vives, M. V., et al. (2021). Agency and responsibility over virtual movements controlled through different paradigms of brain–computer interface. J. Physiol. 599, 2419–34. doi: 10.1113/JP278167
Parsons, T. D., Gaggioli, A., and Riva, G. (2020). Extended reality for the clinical, affective, and social neurosciences. Brain Sci. 10, 1–22. doi: 10.3390/brainsci10120922
Peck, T. C., Seinfeld, S., Aglioti, S. M., and Slater, M. (2013). Putting yourself in the skin of a black avatar reduces implicit racial bias. Conscious Cogn. 22, 779–787. doi: 10.1016/j.concog.2013.04.016
Sanchez-Vives, M. V., and Slater, M. (2005). From presence to consciousness through virtual reality. Nat. Rev. Neurosci. 6, 332–9. doi: 10.1038/nrn1651
Slater, M. (2009). Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philos. Trans. R. Soc. B Biol. Sci. 364, 3549. doi: 10.1098/rstb.2009.0138
Slater, M., Perez-Marcos, D., Ehrsson, H. H., and Sanchez-Vives, M. V. (2008). Towards a digital body: the virtual arm illusion. Front. Hum. Neurosci. 2, 2008. doi: 10.3389/neuro.09.006.2008
Keywords: virtual reality, augmented reality, cognitive neuroscience, neuropsychology, ownership, sense of agency, visuo-spatial disorders, embodied cognition
Citation: Burin D, Salatino A and Ziat M (2022) Editorial: Virtual, mixed, and augmented reality in cognitive neuroscience and neuropsychology. Front. Psychol. 13:1010852. doi: 10.3389/fpsyg.2022.1010852
Received: 03 August 2022; Accepted: 16 August 2022;
Published: 02 September 2022.
Edited and reviewed by: Sara Palermo, University of Turin, Italy
Copyright © 2022 Burin, Salatino and Ziat. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Adriana Salatino, adriana.salatino@unito.it
†ORCID: Adriana Salatino orcid.org/0000-0002-2471-721
Mounia Ziat orcid.org/0000-0003-4620-7886