In this study, we investigated the feasibility of a VR supermarket application presented in the OctaVis, a novel 360°- VR apparatus, for the assessment and training of neuropsychological functions in a sample of healthy young adults and a small clinical sample of patients with focal epilepsy. Healthy participants learned a list of shopping articles and subsequently bought them in a virtual supermarket on three consecutive days. On day 4, participants learned and bought items of a new, distractive list. Finally, on day 5, participants had to buy only the items of the target list, but without a new presentation of this list. Results show increasing levels of learning throughout the task as well as high levels of subjects’ immersion in the VR. Moreover, performance in our task was significantly correlated with a measure of figural-spatial memory. The time needed for completion of the shopping task was significantly longer on day 1 than on day 4 (application of the distractive list B), indicating that visual-spatial familiarity with the structure of the supermarket decreased the immediate distracting effect of list B. Also, we did not observe a decrease of performance from day 3 to day 5, thus indicating that learning was not significantly interrupted by the introduction of a new interfering list. Importantly, immersion was positively correlated with the number of correctly bought products, suggesting that immersive feelings may enhance cognitive performance in everyday-like neuropsychological tasks.
In the course of training in the OctaVis, there was a considerable improvement of cognitive performance across training sessions. This effect of learning was observed for several concurrent measures: Participants successively needed less time and shorter movement paths to accomplish the task and bought a higher number of correct products, which was even valid when correcting for incorrect products and repetitions. Moreover, performance of the ROF was positively correlated with the number of correctly bought products. Finally, our main findings could be replicated in our clinical sample showing (a) comprehensive and stable learning, (b) no negative effect on performance after the introduction of the interference list, and (c) and association of measures of VR performance and the ROF in our patients with epilepsy.
We suppose that these comprehensive results of learning do not only reflect verbal learning as a consequence of the repeatedly presented shopping-list. Rather, improvement on the different but converging measures support the view that learning occurred mainly on verbal, visual-spatial, executive, and familiarity levels . Although our data do not directly allow for conclusions on multi-layered learning, we propose that successful learning in the VR shopping task depends on the integration of at least verbal and visual-spatial modalities of learning, that is, a form of dual-coding. Moreover, the task requires executive abilities, in particular, visual-spatial planning strategies. This view is supported by the correlations between the ROF and learning scores in our VR task. The ROF represents visual-spatial memory functions, which are mostly independent of verbal abilities . Besides visual-spatial memory abilities, the ROF also requires planning and structuring abilities and has correspondingly been used as a measure of planning and organization in previous studies [39–45]. We therefore propose that the correlation between our task and the ROF might represent the task’s requirement of at least verbal learning and non-verbal figural learning, as well as executive abilities, thus supporting our argument of multi-layered learning in the VR supermarket. Furthermore, our idea of a multi-layered learning process is also supported by the strategies participants used to accomplish the task. The use of dual coding, which implies the dual coupling of words of the shopping list with the visual representation of the respective product in the VR supermarket, was the memory strategy that was most frequently reported by our participants. It is most likely that the VR shopping task also makes demands on visual-spatial orientation and way-finding. However, our data do not contribute to this issue such that this assumption remains speculative. In our study, we did not aim at offering a full psychometric validation of our novel instrument. Furthermore, procedural learning and habituation to both technical control of the shopping task and the visual spatial structure of the VR environment may also have contributed to efficient learning. Eventually, our idea of a multi-layered learning that took place in our VR task goes in line with the concept of multiple memory systems that represent different subtypes of memory and learning processes [6, 46, 47].
Interestingly, the data show evidence for further learning from day 1 (first entering the supermarket) to day 4 (entering the familiar supermarket after having heard a new shopping list) for the time required for the shopping task and the LMT, but not for the product-score. This decrease of time and LMT on day 4 (relative to day 1) indicates that figural-spatial learning of the supermarket’s routes and layout occurred, which may be relatively independent of verbal memory of list A articles and their localization in the supermarket. This dissociation between verbal and figural-spatial learning further confirms our aforementioned idea of multi-layered learning. In particular, we suppose that participants created a cognitive map [48, 49] of the virtual supermarket. Previous studies are in accordance with this assumption. The idea that generation of cognitive maps could be enhanced by learning in a VR , was supported by Tong et al.  who found that active performance in a VR-pathfinding task enhanced the accuracy in a subsequent drawing of a cognitive map of the landmarks in the VR. Results thus suggest that visual-spatial learning might have played a central role for task performance in our task besides verbal learning of the products. This assumption is in good accordance with the frequent application of dual-coding strategies reported by the volunteers (i.e., verbal and visual-spatial representation).
Insofar, our results are in line with a study of Brooks et al. , who found a dissociation between object and spatial learning in a VR. While the spatial layout of the VR could be recalled more accurately by participants who actively (vs. passively) navigated through the VR, the mere learning of objects placed in a VR was independent of (active vs. passive) navigation in the VR . Moreover, in a study of patients with traumatic brain injury, Matheis et al.  found a dissociation between patients’ impaired list learning and spared visual memory performance.
We propose that learning of the layout of the VR supermarket took place on the first two days of training since the data show the highest increases of performance from day 1 to day 2. Moreover, the interference list did not affect performance in our VR shopping paradigm. Thus, there were no significant differences between all measures of performance on day 3 and day 5 (i.e., product-score, LMT, and time). Hence, our initial hypothesis that learning will not be interrupted by the introduction of a new shopping list was corroborated by the data, that is, shopping performance of the target list was comparable on days 3 and 5, although we inserted new learning materials in between these trials. It is therefore reasonable to assume that visual-spatial learning supported the emergence of a multi-layered representation of the shopping articles included in the target list. Importantly, we replicated this result in our preliminary study of the small clinical sample of patients with focal epilepsy. This argues in favor of a stable and robust representation of learned multi-layered information after an initial learning phase of the visual-spatial layout of the VR supermarket in both healthy controls and clinical samples.
For our present data, we suppose that interaction with the highly immersive 360°- VR OctaVis may have prompted multi-modal learning. There is considerable evidence from studies in children and adults that multi-modal learning is more efficient, deep and stable over time than unimodal learning [54–58]. Multi-modal learning may also enhance the feeling of presence and learning in VRs [59, 60]. In addition, combined training of multiple cognitive functions could be shown to be an efficient strategy for rehabilitation of memory problems . It is reasonable to assume that both the realistic layout of the supermarket and the real-life like interactive movements (e.g., turning around, reaching out with the arm and hand for an article) may have supported the integration of visual-spatial and motor learning, and therefore facilitated the formation of episodic in contrast to mere semantic memory contents , which makes our task a more precise measure of real-life cognitive performance. Most likely, immersion in the VR is a key player in the emergence of efficient multi-modal learning since it builds up the basis for real-life like integration of visual perception, motor-action and visual-spatial memory [52, 63].
With regard to immersion, we observed increasing levels of subjective immersive feelings during the course of our training program in the 360°- VR supermarket. Insofar, our results are in good agreement with Lee et al.  who could also show participants’ increasing levels of subjective immersion into the VR in the course of a five day intervention. In our study, it is likely that participants became more and more familiar with the supermarket and the task and could thus successively immerse better into the VR during the course of the training intervention [64, 65]. Increase of immersion may well be related to learning success. The correlations between measures of learning (i.e., product-score, time) and intensity of immersion, which both increased during the course of the task further supports this view. Our results provide first evidence that learning success depends at least in part on subjective immersive feelings. Eventually, our results are in accordance with the findings of previous studies showing that intense feelings of immersion may be associated with higher levels of task performance in a more general sense  and better treatment success in psychotherapy settings where VR is used to cure different forms of phobias .
As in previous VR studies using the PQ [22, 66–68] we observed relatively high levels of immersion. On the one hand, this further underlines our apparatus’ technical feasibility for presenting highly immersive VR. We particularly assume that participants’ immersion was at least in parts enhanced by our novel apparatus with its specially designed 360°- view, the multi-sensory (i.e. visual, motor and auditory) design and the intuitive and interactive control. This is in line with several former research showing positive effects of field-of-view size [32, 33], multi-modal integration  and active control on immersion . On the other hand, it can be also supposed that the high levels of immersion found in our study will beneficially contribute to feasible transfer effects as immersion is thought to be a critical factor to enhance transfer from VR to real-life situations .
In contrast to these correlational associations between momentary presence and immersion measured with the PQ and task performance in the OctaVis, we did not find any association between performance in the OctaVis and the scales of the ITQ. Thus, a subject’s individual tendency for immersion was not found to be a critical factor for our VR training paradigm. We therefore conclude that performance in our paradigm mainly depends on the momentary level of immersion into the VR (as assessed with the PQ), rather than on a subject’s general personal trait or capability to get involved or immersed (as assessed with the ITQ). This aspect is of high importance for a future routine clinical application of VR scenarios in the OctaVis since the efficiency of training in the OctaVis should accordingly be independent of a person’s individual trait to get immersed into VR. This distinction allows for successful application of our program to a wider range of participants and does not restrict the application to participants with computer experience. Certainly, a further investigation examining a sample with a wider range of ages would add important information to this issue.
As it is important to use generally valid, “cross-media” measures of presence as well as specific measures that fit the proper and special technical features of the VR and its control devices , we also looked at the way participants subjectively got along with our particular task (represented by the task-subscale) and the technical apparatus (represented by the technique-subscale). We observed high scores in both subscales, which even increased significantly in the course of our program and were correlated with measures of learning in the VR task, thus indicating both an easy-to-handle control of our technical devices as a training program that seems intuitive and easy to handle. However, our questionnaire’s design was of explorative nature and scores may not be interpreted in terms of established measurements.
In summary, our findings provide preliminary evidence that our novel VR supermarket paradigm presented in the OctaVis may efficiently be applied for the assessment and training of real-life cognitive functions in healthy subjects and patients with focal epilepsy. However, we acknowledge some limitations and caveats interpreting our study. First, we did not control for each of the proposed levels of learning in particular. For example, we could not assess a verbal list learning paradigm as this would have interfered with the learning in our virtual paradigm. Although our study does not claim to offer a full-scale validation study, the application of additional specifically related “traditional” neuropsychological tests (e.g., of verbal memory) would presumably have added important information about single cognitive processes involved in performance in our novel VR paradigm. In particular, a labyrinth task needs to be included in future studies to assess the roles of visual-spatial orientation and way-finding in the VR shopping task. Moreover, list learning paradigms like the California Verbal Learning Task  or the Verbal Learning- und Memory Test [German adaptation of the Auditory Verbal Learning Test; , which comprise the most established measures of verbal memory , might offer important information on the role of verbal memory in the VR shopping task. Thus, the question whether both verbal and visual memory processes might have played a role could have been adressed more directly. In future experiments, we will hence offer additional measures of verbal memory performance. Second, our results should be replicated with a sample of older participants to evaluate generalization of our results to a group of participants with less computer experience. We did not directly address the participants’ motivation during shopping in the VR supermarket on each day of training. It is reasonable to assume that besides cognitive capacity, different levels of motivation and effort could have had an influence on the participants’ performance in the VR supermarket. Moreover, participants’ experience of cybersickness needs to be considered as another constraint of our study. Using a questionnaire on experiences of cybersickness we aimed at identifying sources of cybersickness during training in the OctaVis. Most participants reported a flicker in their peripheral field of view as an eliciting factor of nausea. This flicker is mainly technically related to fast movements in the VR, a relatively low frame-rate of the VR system (i.e. a low speed of re-generation of the VR environment), and the large field of view . In our current studies of training in the OctaVis, we have accelerated the frame-rate and limited the maximum speed of movement inside the VR to reduce symptoms of cybersickness elicited by these factors. These changes of technical parameters resulted in a considerable reduction of reports of cybersickness. Finally, our task’s psychometric properties must be further evaluated. Therefore, we will apply the OctaVis to different populations to examine its validity in laboratory and further clinical settings. We currently elaborate different forms of feedback, which can be suitable for different patient groups. In parallel, we compare our VR program with already established training paradigms.