🧬 biology

Short-Term Affective Change in Immersive Installations: A Multimodal Analysis of EEG and PANAS

This study found that baseline self-reported affect (PANAS) provided a more stable and consistent predictor of short-term affective change in immersive installations than EEG-derived indicators, suggesting that validated self-reports remain a practical reference when physiological data are limited or difficult to interpret.

Original authors: Yizhen Wang, Xiaowei Chen

Published 2026-08-20

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yizhen Wang, Xiaowei Chen

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Human emotions are not static snapshots; they are flowing currents that shift and swell as we move through our day. In the field of affective science, researchers have long sought to understand how these internal states change, particularly during immersive experiences like virtual reality or interactive art installations. While we can easily ask someone how they feel right now, predicting how a specific experience will alter their mood from start to finish is a much harder puzzle. Scientists have turned to brain activity, measured through electrodes placed on the scalp, hoping to find a biological signature that reveals these emotional shifts before a person is even aware of them. The central question driving this line of inquiry is whether these electrical signals from the brain can tell us more about a person's emotional journey than the person can tell us about themselves.

A team of researchers set out to test this idea by examining a group of thirty people who participated in an immersive, multisensory art installation. This environment was designed to be engaging and interactive, allowing participants to move and interact freely, much like a real-world setting rather than a sterile laboratory. Before entering the installation, each person completed a standard questionnaire asking them to rate their current feelings, such as whether they felt excited, upset, or attentive. They also had their brain activity recorded using a cap of sensors. After the session, they completed the same questionnaire again. The researchers then calculated the difference between the "before" and "after" scores to see exactly how much each person's positive or negative mood had changed. They also analyzed the brain data collected before the session to see if those initial neural patterns could predict the size of the emotional shift that would occur.

The researchers approached this problem by building computer models to predict the mood changes. They tested three different types of information to see which worked best. The first type relied solely on the self-reported feelings from the questionnaire. The second type used only the brain activity data, specifically looking at the strength of different brain waves and the balance of activity between the left and right sides of the brain. The third type combined both the questionnaire answers and the brain data. They used a machine learning method, which is a type of computer program that learns to find patterns in data, to see if it could accurately guess how much a person's mood would change based on their starting point.

The results offered a clear and somewhat surprising answer. The models that relied on the brain data alone showed the least stable performance; they were unable to reliably predict whether a person's mood would improve or worsen. In fact, for certain algorithms, the models using only the brain signals produced negative test-set R² values, indicating they performed worse than a simple baseline prediction. When the researchers combined the brain data with the questionnaire answers, the results did not improve significantly over using the questionnaire alone. The most consistent and accurate predictions came from the models that used only the self-reported feelings. Specifically, a person's initial level of distress or agitation was a strong indicator of how much their negative mood would decrease after the experience. If someone started the session feeling highly agitated or upset, the installation was more likely to help them feel calmer afterward. Predicting an increase in positive feelings proved much more difficult for all methods, but the self-reports remained the most reliable guide.

This finding suggests that in complex, real-world environments, a person's own description of their emotional state is currently a more useful tool for understanding their emotional trajectory than a summary of their brain waves. The brain data did show modest bivariate associations with the mood changes, particularly regarding the balance of activity in the frontal parts of the brain, but these signals were too faint and noisy to be useful on their own. The researchers noted that the immersive nature of the installation, which allowed for movement and shifting attention, likely introduced a lot of background noise into the brain recordings, making it harder to isolate the specific signals related to emotion.

The study concludes that while measuring brain activity is a powerful tool, it is not yet a replacement for asking people how they feel, especially when trying to track short-term emotional changes in dynamic settings. The self-reported feelings provided a stable and consistent baseline that the brain data could not match. This does not mean the brain data is useless, but rather that it captures different aspects of the emotional experience that do not always align perfectly with what a person can consciously report. For now, when trying to understand how an immersive experience changes a person's mood, the most direct and reliable path is to listen to the person themselves.

Technical Summary: Short-Term Affective Change in Immersive Installations

Problem Statement
The study addresses the challenge of characterizing short-term affective change within immersive, multisensory experiences. While affective computing often focuses on recognizing static emotional states, there is a gap in understanding how affect transitions over the course of an engagement session. A specific methodological tension exists between self-report measures (like the Positive and Negative Affect Schedule, PANAS) and physiological indicators (specifically Electroencephalography, or EEG). While EEG offers access to neural dynamics, its utility in predicting change (pre-to-post transition) in ecologically variable, immersive settings remains unclear compared to subjective baseline reports. Furthermore, it is uncertain whether positive and negative affective changes follow similar predictive pathways or if one is more tractable than the other.

Methodology
The research conducted a secondary analysis on a publicly available multimodal dataset involving 30 participants (a subset of an original 50) who engaged with an immersive art therapy installation integrating EEG, VR, and AIGC.

Data Sources: Pre- and post-session PANAS scores and raw EEG recordings (Emotiv Flex, 128 Hz, 10–10 arrangement).
Outcome Variables: Two continuous targets were defined as difference scores:
- $\Delta$ Positive = Positive $_{post}$ − Positive $_{pre}$
- $\Delta$ Negative = Negative $_{post}$ − Negative $_{pre}$
Predictor Sets: Three feature configurations were constructed using baseline (pre-session) data:
1. PANAS-only: Baseline Positive and Negative scores.
2. EEG-only: Baseline spectral features including global and frontal power (theta, alpha, beta), frontal alpha asymmetry (F4–F3), band ratios (theta/beta, alpha/beta), and relative power measures.
3. Hybrid: A combination of PANAS and EEG features.
Modeling Strategy: The study employed supervised regression using Linear Regression, Decision Tree, and Random Forest (RF) algorithms. Performance was evaluated via a 70:30 train/test split and 5-fold cross-validation using Mean Squared Error (MSE), Mean Absolute Error (MAE), and the coefficient of determination ( $R^2$ ). Random Forest was selected as the primary exploratory model to allow for feature-importance analysis.

Key Results

Predictive Performance: The PANAS-only models demonstrated the most stable and highest performance across both outcomes. For $\Delta$ Negative, the PANAS-only Linear Regression model achieved the highest test-set $R^2$ (0.462), followed by PANAS-only Decision Tree (0.358) and Random Forest (0.335). In contrast, EEG-only models consistently yielded negative test-set $R^2$ values (e.g., -0.469 for RF on $\Delta$ Negative), indicating they performed worse than a simple mean predictor. Hybrid models did not consistently outperform PANAS-only models; in some cases, adding EEG features degraded performance.
Asymmetry of Affect: Predicting changes in negative affect ( $\Delta$ Negative) was significantly more successful than predicting changes in positive affect ( $\Delta$ Positive). For $\Delta$ Positive, all models (including PANAS-only) produced negative test-set $R^2$ values, suggesting that short-term positive affective change is less predictable in this dataset.
Feature Importance: In the Random Forest models, baseline self-reported negative affect was the dominant predictor for $\Delta$ Negative, ranking first by a clear margin. For $\Delta$ Positive, importance was more distributed. While EEG features (specifically frontal alpha asymmetry, theta/beta ratio, and alpha/beta ratio) contributed supplementary information, they did not drive the models as strongly as self-reports.
Correlations: Bivariate analysis showed modest associations between EEG features and affective change (e.g., theta/beta ratio with $\Delta$ Positive, global alpha with $\Delta$ Negative), whereas baseline PANAS items (e.g., "Jittery," "Upset") showed stronger correlations with subsequent changes in negative affect.

Key Contributions

Comparative Efficacy: The study provides empirical evidence that, in the context of short-term affective transition within an immersive installation, baseline self-reported affect is a more consistent and robust predictor than resting-state EEG spectral summaries.
Multimodal Feasibility: It demonstrates the feasibility of multimodal affective modeling in ecologically valid settings while highlighting that combining modalities does not automatically improve predictive accuracy if the physiological data is noisy or coarse.
Interpretability in Small Samples: The work validates the use of interpretable machine learning (Random Forest) for small, ecologically variable datasets. It shows that even when predictive power is limited, such models can clarify the relative contributions of different data modalities without the overfitting risks associated with complex deep learning architectures.
Differentiation of Affective Dimensions: The findings suggest an asymmetry in tractability, where reductions in negative affect are more readily characterizable by baseline measures than increases in positive affect.

Significance and Claims
The authors claim that these findings support a complementary rather than competitive view of self-report and EEG. In immersive or applied settings where physiological data may be limited by motion artifacts, environmental variability, or coarse spectral summaries, validated self-report measures (like PANAS) offer a practical and reliable baseline reference for modeling affective change.

The paper emphasizes that the lower performance of EEG features should not be interpreted as EEG being inherently uninformative for affective research. Instead, it reflects the limitations of the specific neural information available (resting-state spectral power) in a naturalistic context. The study concludes that while multimodal modeling is feasible, researchers should prioritize low-burden, interpretable self-reports when physiological data is difficult to interpret or when the goal is to characterize short-term affective transitions in ecologically variable conditions.

Technical Summary: Short-Term Affective Change in Immersive Installations

More like this