1. Introduction
Neurofeedback (NF) is a technique to teach users how to modulate neuronal signals, i.e., their own brain activation. Most commonly, the electroencephalogram (EEG) is used to record brain activation, which is then pre-processed in real time to extract relevant features of the brain signal that should be modulated in a desired direction during NF training. For instance, the sensorimotor rhythm (SMR) between 12 and 15 Hz, which is strongest over central brain areas, is often used as a feedback frequency in NF applications since this EEG rhythm is associated with optimal cognitive processing [
1,
2]. So, if the aim of the NF training is to improve cognitive functions (e.g., memory functions), the SMR can be extracted in real time from the ongoing EEG of the NF user, and finally, changes in SMR activity are fed back visually, auditorily or in a tactile manner to the users. When obtaining, for instance, visual feedback of changes in one’s own SMR activity, NF users can try different mental strategies to voluntarily increase this brain signal [
3]. If the users are successful in up-regulating their SMR rhythm, cognitive improvements may follow [
1].
Most NF applications use visual feedback. Conventional visual feedback designs are rather simple in their design and restricted to the presentation on a computer screen. Although such simple feedback designs clearly present the training task without any distraction (e.g., the circle in the middle of the screen should become larger), participants might become bored over the course of several training sessions. It has been shown in previous NF studies that psychological factors such as motivation and attention are paramount for NF training success [
4,
5,
6]. Thus, more and more approaches to using more entertaining or visually appealing feedback, such as game-like feedback scenarios or virtual reality (VR)-based 3D feedback, can be found in the literature [
7,
8]. Instead of regulating the height of objects on a 2D screen, participants need to alter features in a whole virtual environment, move through a 3D forest environment [
8], modify adaptive VR games with NF [
9], or be placed in an inviting virtual beach environment [
10]. Results of VR-based feedback are promising, showing higher training motivation in a variety of populations, such as stroke patients or healthy young participants, who show better NF training results in VR groups compared to conventional 2D groups. However, there are still unanswered questions, such as who is the right target group for VR-based feedback [
7,
11], can VR-based NF be used for home-based training [
12], or what is the optimal VR technology (e.g., head-mounted display vs. 3D presentation on a screen). Also, VR proposes problems of transferability of training results to real-world settings on the one hand, and on the other hand, it entails the problem of cybersickness (CS) for the users. Using visually rich VR scenarios as feedback screens, such as a forest environment, might also distract the attention of the NF users from the NF task or lead to cognitive overload, which might be counterproductive for the modulation of the neuronal signals during NF training [
13]. However, the majority of VR-based NF training studies report positive effects [
8,
14]. Also, in BCI training settings, fewer classification errors are shown when VR paradigms are used [
15]. Generally, there are lots of interesting and creative ways to realize BCI and neurofeedback paradigms in a fun and game-like manner. Suggestions are proposed in previous research [
16,
17].
While the number of VR-based NF training studies is steadily increasing, augmented reality-based NF is still very new in this field. Augmented reality (AR) describes the extension of the real world by adding virtual objects [
18]. It is closer to reality than VR approaches [
19,
20], which is why there are fewer visual distractions compared to VR. This can lead to better focus on the NF task at hand, when working memory is not overloaded by irrelevant virtual stimuli, which has been found multiple times in the literature on game-based learning [
13]. AR-based feedback can also be easily implemented in home-based training settings, as many smartphones and tablets can already implement AR, see list:
https://developers.google.com/ar/devices (accessed on 15 April 2024). Some research groups focused on the first development and implementation of AR features. A group focusing on autism spectrum disorder built an AR smartphone app in a dynamic participant-driven process, called Eggly. It works with consumer-grade EEG systems for neurofeedback purposes and has been tested on five children with autism [
21]. It was efficiently used to increase impaired social attention functionalities in children with autism. Other research groups tested AR feedback effects on meditation with an iPhone AR application. Participants would hatch butterflies in the room by meditating [
22], either with or without EEG-NF.
However, there are only a few studies using AR in NF research, comparing it with traditional feedback. One of the previous studies used a webcam to overlay virtual brains on the participants as an AR paradigm, and as a control group, they used a temporal gauge as 2D feedback [
23]. It was reported to be more engaging than traditional feedback but, at the same time, more complex by the participants. Further, a steady-state visual evoked potential (SSVEP) AR-NF set-up has been used to influence affect-biased attention in five adolescent girls. Here, a Gabor patch was projected onto a wall that participants would see through AR-glasses [
24].
Several researchers propose the need for more research in this field, especially when it comes to user experience [
25]. What is still lacking are studies including sham feedback and the inclusion of a broader set of questionnaires to evaluate the subjective user experience more thoroughly.
Prior VR studies showed that a VR-based feedback condition can lead to negative emotional reactions such as fear or lower values in mastery confidence compared to 2D simple feedback screens [
11]. The appearance of such negative emotional responses when using VR as a feedback modality in NF training might be related to the prior experience with this technology, age, or medical conditions. For instance, in the study by Kober et al. stroke patients showed such negative emotional reactions when using VR-based NF training [
11]. Further, CS can also be a problem in AR environments. Previous research could show an association with oculomotor problems with higher experiences of eyestrain, blurred vision, etc. using the HoloLens [
26]. It remains open whether this is transferable to other AR systems as well as other, more static paradigms. Hence, when using a new technology, such as AR as feedback technology in NF applications, it is mandatory to evaluate the subjective user experience in close detail.
Therefore, we conducted a study comparing the user experience of participants undergoing one session of SMR-based NF training, either receiving conventional 2D feedback or AR feedback, as well as real or sham feedback. Here, our main focus was to see whether AR feedback in general would lead to a different user experience than 2D feedback, independently from more possible adapted or complex designs. Therefore, we chose a paradigm related to the traditional 2D paradigms, showing virtual plants growing and shrinking out of actual plant pots.
We expected to observe group differences in the user experience of how the technology and individual performance after training are perceived, subjective feelings of flow or presence, and CS. In line with previous VR-based NF training studies [
27], we assessed mood, CS, flow, participants’ attitude (e.g., technology acceptance, use, fear, curiosity, interest, skepticism) towards VR technology, subjective VR experience (immersion), subjectively experienced level of control and concentration during NF training, perceived success, fun, as well as physical conditions such as pressure on the head that can occur when wearing the VR headset over the EEG electrodes [
27]. In the present AR-based NF study, we also compared the user experience of a group receiving one session of AR-based NF training with that of a group receiving one session of 2D conventional NF training.
Finally, in order to investigate whether the results of subjective questionnaire data are associated with NF performance, we calculated correlations between the subjective experience and the SMR power over the six feedback runs.
3. Results
Table 2 summarizes the means and SD of questionnaire data and the results of the statistical comparisons.
PANAS. Participants from all four groups had comparable values of positive affect. There was a significant group difference between real and sham feedback concerning negative affect, with a lower decrease in values pre- and post-training in the real feedback group (F(1,81) = 5.87, p = 0.018, η2p = 0.04) compared to the sham feedback group.
SSQ. There was a tendential difference between real and sham feedback in nausea values (
F(1,82) = 3.22,
p = 0.077,
η2p = 0.05), with higher values in the real group compared to the sham group. Also, there was a tendential interaction effect between task and condition (
F(1,82) = 3.65,
p = 0.060,
η2p = 0.04), with higher nausea values in the AR real feedback group compared to the AR sham feedback group (see
Table 2).
Flow. Participants in the AR group reported a tendentially higher flow than the 2D group (F(1,85) = 3.37, p = 0.070, η2p = 0.04) and a tendential interaction effect with feedback group and condition, with a higher subjective experience of fluency during the NF training in the AR sham group compared to the 2D sham group (F(1,85) = 3.94, p = 0.050, η2p = 0.04). Participants from the 2D group reported higher concerns during the training compared to the AR group (F(1,82) = 4.24, p = 0.043, η2p = 0.05). Also, there was a tendentially higher total flow score in the AR group (F(1,84) = 3.95, p = 0.050, η2p = 0.04). There were no differences between sham and real feedback.
TUI. Results show that participants in the 2D group experienced a tendency of higher technology anxiety compared to the AR group (F(1,84) = 3.78, p = 0.055, η2p = 0.04). Further, the AR group evaluates a higher usability of the technology than the 2D group (F(1,83) = 9.73, p = 0.003, η2p = 0.10). However, they rated the technology significantly as less accessible than the 2D group (F(1,83) = 5.20, p = 0.025, η2p = 0.06). There were no differences between sham and real feedback.
Questionnaire on Experience. Participants of the AR group reported a significantly higher subjective feeling of control during the NF task (F(1,85) = 10.17, p = 0.002, η2p = 0.11), as well as a higher perceived success compared to the 2D group (F(1,85) = 8.26, p = 0.005, η2p = 0.09). There were no differences between sham and real feedback.
Pain/Discomfort. Participants in the 2D group report higher feelings of discomfort. They experience more headaches compared to the participants of the AR group (F(1,74) = 4.83, p = 0.031, η2p = 0.06). There were no differences between sham and real feedback.
For questionnaire data, where the ANOVAs revealed significant group differences, correlation analyses were performed between the NF performance (SMR regression slopes) and the questionnaire data for all four groups. A positive association in the AR real feedback group was found for NF performance and accessibility (r = 0.57, p = 0.003).
4. Discussion
In the present study, we evaluated the differences in user experience during a 2D vs. AR NF-training session. The user experience of the participants differed in several points. Participants from the AR group reported a higher subjective feeling of flow, perceived a higher technology usability, experienced a higher feeling of control, and perceived themselves as more successful than those from the 2D group. Psychological factors like this are crucial for NF training motivation and success. In the 2D group, participants reported more concern related to their performance, a higher level of technology anxiety, and also more physical discomfort.
Participants from the real feedback group experienced tendentially more nausea symptoms than those from the sham feedback group. However, it is not the case that the participants from the real feedback group showed a strong increase in nausea symptoms but had comparable levels pre- and post-training, whereas the sham feedback group showed a slight decrease in symptoms. Participants receiving real feedback also had a more comparable negative effect pre- and post-training, while those receiving sham feedback showed a slight decrease after the training. This could also be a result of the slightly higher nausea values of participants in the real feedback group. Also, there is a tendential interaction effect with the feedback group (AR vs. 2D), with higher values in the AR real group compared to the AR sham group. This is contrary to previous studies that show task control can reduce sickness symptoms [
38]. It could be that participants from the sham group became more passive during the training because they could not control the paradigm, and participants from the AR paradigm were more engaged and hence more affected by nausea symptoms. However, this interaction is only a tendency and should not be interpreted too much. The groups (AR vs. 2D) did not differ in their experience of CS. CS is a major problem in VR, where up to 80% of the users are affected [
33,
39]. AR has the advantage that CS is less prevalent among users. Here, the symptom profiles of VR and AR differ. For VR Disorientation > Nausea > Oculomotor problems are most apparent; during AR studies, a pattern of Oculomotor problems > Disorientation > Nausea is described by users [
26]. A pattern, which we could also find in our groups, and as both groups wore the AR set-up, the experiences between groups were comparable. This means that symptoms such as eye strain and fatigue are more prevalent than disorientation and sickness in AR systems. AR cameras can vary in their frame rates so that participants experience a small time lag when moving the head, which can be especially straining for the eyes and results in a feeling of sickness [
40]. However, in the present NF task, participants did not have to move their heads. They only had to watch objects moving up and down. This might explain why CS symptoms were relatively low in the present study and also between both groups. However, as it could be the case that the AR plants would wiggle when participants moved their heads, which in turn could lead to eye strain or dizziness, we nevertheless compared both groups. The exposure time was also only moderate, with about 21 min in total (7 × 3 min). Previous studies on AR and CS suggest an exposure time of about 20 min for a good user experience [
26]. In summary, it can be said that only minor CS symptoms have occurred with the current AR set-up.
Participants in the 2D group reported more physical discomfort during the NF training compared to the AR group. The 2D group reported a stronger headache than the AR group. In our study, both groups wore the VR headset above the EEG electrodes during NF training to reach comparable technical conditions. Prior VR-based NF studies using the same VR headset mounted over EEG electrodes also reported some sort of physical discomfort during the NF training. Berger et al. found that mainly female participants report such negative physical conditions while wearing the VR headset during NF training [
27]. In the present study, a lower subjectively experienced physical discomfort in the AR group during NF training compared to the 2D group might be related to a stronger involvement or engagement in the task [
41,
42], which is also reflected in a tendentially higher flow experience in the AR group and a higher feeling of control and perceived success in the AR group compared to the 2D group. Control beliefs have been shown to be important for good NF performance [
43], and flow has also been associated with faster learning progress in NF tasks [
14]. Such increased task engagement may have reduced participants’ attention to the physical conditions [
41,
42]. Increased attention to the NF task has already been shown to have a positive effect on NF task performance [
4], training motivation [
5], and training adherence, suggesting that AR-based NF training could have positive effects on future NF applications.
Although not statistically significant, flow experience during NF training was tendentially higher in the AR group than in the 2D group. The AR feedback was experienced as more fluent than the 2D condition, showing that the movement of the virtual flower could be pursued more smoothly than the movement of the 2D bars. A smooth pursuit of the visual feedback might be beneficial for NF control, which again might be related to the higher level of control subjectively perceived during NF training in the AR group. Previous research could show that this also corresponds to an improved NF training performance. Gruzelier et al. compared a VR to a 2D setting in SMR NF training in order to improve acting performance [
14]. They reported higher subjective feelings of flow in the VR training group and also a quicker SMR increase during the NF training, which in turn had beneficial effects on the acting performance of the participants.
The 2D condition led to more negative reactions, such as tendentially higher technology fear, a reduced usability rating, and a higher concern about the technology. Prior VR-based NF studies report such increased negative reactions, especially in patient populations of higher age with limited prior experience with VR technology. In the present study, we tested healthy young adults with comparable prior VR experience. We might only speculate about the reasons for the differences in fear and concern between groups. The 2D group saw the moving bars on a computer screen through the VR headset and wore only the VR headset to achieve comparable technical conditions between the groups during the NF training. Hence, the VR headset had no function in the 2D condition. Some participants also asked why they were wearing the VR headset, even though it had no function. This could have led to irritation in the 2D group and could be the reason for the increased reporting of fear and anxiety and the reduced usability rating. This may have contributed to inflating the difference observed between groups. Future AR-based NF training studies should compare the AR condition to a traditional 2D feedback set-up without a VR headset. Also, as participants from the 2D group felt less control over the paradigm and tendentially less flow, this might also explain why they rated the technology as less usable than the AR group, who felt more in control. This might also add to the irritation they reported, which could have led to the higher fear and anxiety levels.
Although the 2D group seemed to be irritated by wearing the VR headset, they rated the accessibility of the 2D NF task higher than the AR group. The participants probably found the development and implementation of AR-based feedback more complex and time-consuming than 2D feedback.
In contrast, the AR condition led to more positive reactions, such as a higher degree of subjective control during NF training and a greater perception of success. Our results regarding more positive reactions in the AR group are in line with previous research on this topic. A meditation study with participants having anxiety and/or depression could show an increased positive mood after undergoing an AR meditation intervention [
22]. However, the AR and 2D groups did not differ in positive or negative affect as assessed with the PANAS questionnaire.
For questionnaire data, where the ANOVAs revealed significant group differences, correlation analyses were performed between the NF performance (SMR regression slopes) and the questionnaire data. We found a significant positive correlation between NF performance and accessibility in the AR group with real feedback. It has been shown previously that a positive attitude towards the paradigms used, such as interest and motivation, can be beneficial for NF performance [
11]. The more accessible a technology appears to be, the more relevant it may feel to its users. Hence, it is important to make participants comfortable with the set-up before the training to enable a good NF performance.