🤖 AI

GroupAffect-4: A Multimodal Dataset of Four-Person Collaborative Interaction

This paper introduces GroupAffect-4, a comprehensive multimodal dataset of 40 participants in ten four-person groups performing diverse collaborative tasks, which integrates synchronized physiological, eye-tracking, audio, and self-report data to enable analysis of affect across individual, interpersonal, and group levels.

Original authors: Meisam Jamshidi Seikavandi, Alice Modica, Anna Obara, Shan Ahmed Shaffi, Fabricio Batista Narcizo, Tanya Ignatenko, Ted Vucurevich, Karim Haddad, Daniel Barratt, Daniel Overholt, Jesper Bunsow Boldt

Published 2026-05-20

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Meisam Jamshidi Seikavandi, Alice Modica, Anna Obara, Shan Ahmed Shaffi, Fabricio Batista Narcizo, Tanya Ignatenko, Ted Vucurevich, Karim Haddad, Daniel Barratt, Daniel Overholt, Jesper Bunsow Boldt, Paolo Burelli, Andrew Burke Dittberner

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine trying to understand a group of friends solving a puzzle together. Usually, researchers might only listen to their voices, or only look at their faces, or maybe just ask them how they felt afterward. But to truly understand the "group vibe," you need to see everything happening at once: their heart rates, where they are looking, what they are saying, and how they are feeling, all synchronized to the exact same second.

That is exactly what the GroupAffect-4 paper introduces. It's a new, highly detailed "movie" of human teamwork, but instead of a movie camera, it uses a suite of high-tech sensors.

Here is the breakdown of what they did and what they found, using simple analogies:

1. The Setup: A "Sensor Suit" for a Team

The researchers gathered 40 people and put them into 10 groups of four. They didn't just sit them in a room; they gave everyone a "digital detective kit":

Wrist Sensors: Like a smartwatch, these tracked heart rate, skin sweat (which shows stress), and body temperature.
Smart Glasses: These tracked exactly where each person was looking and how their pupils changed size (a sign of mental effort).
Lapel Microphones: Tiny mics on everyone's chest to hear exactly who was speaking and when.
Tablets: These popped up questions during the tasks, asking, "How stressed are you right now?" or "How happy do you feel?"

2. The Plot: Four Different "Chapters" of Interaction

The groups didn't just chat; they played four specific games designed to trigger different emotions:

The Mystery (Information Pooling): Everyone had different clues about a fake job candidate. They had to share their secrets to find the right answer.
The Negotiation: They had to plan a workshop, but everyone had a secret agenda (e.g., "I must pick the AI topic"). This created tension and debate.
The Brainstorm: They had to come up with creative ideas for a party. This was the "fun" part.
The Trust Game: They had to decide how much of their own "money" (tokens) to put into a shared pot. This tested cooperation and greed.

3. The Big Discovery: What the Data Actually Said

The researchers tried to use computers to predict what was happening just by looking at the sensor data. Here is what worked and what didn't:

The "Talking Over" Signal: The best way to tell if a group was under mental pressure (like during the negotiation) wasn't their heart rate. It was who was talking over whom. When people interrupted each other more, the computer knew the group was stressed. It's like hearing a chaotic kitchen during dinner prep vs. a calm one.
The Eyes Tell a Different Story: While talking patterns showed stress, the pupils (eyes) were better at showing how engaged someone was.
The "Heart Rate" Limit: Surprisingly, the wrist sensors (heart rate) weren't great at guessing if someone was happy or angry. They were too "noisy" and didn't match up perfectly with what the people said they felt.
The "Personality" Puzzle: The researchers tried to guess people's personality traits (like "Are you an introvert or extrovert?") just by watching them work. They failed. The data wasn't clear enough to do this. It's like trying to guess someone's entire life story just by watching them order a coffee for 10 minutes.
The "Group" Mystery: They tried to predict how the whole group would behave based on the sum of their parts. The computer got this wrong too. A group is more than just the sum of four individuals; the "group dynamic" is a complex dance that the current data couldn't fully capture.

4. Why This Matters (According to the Paper)

The paper claims this dataset is special because it is dense, not just big.

Old Datasets: Like a blurry photo where you can see the people but not their expressions.
GroupAffect-4: Like a 4K, slow-motion video where you can see the sweat on their brow, the dilation of their pupils, and hear the exact tone of their voice, all perfectly synced.

5. Important Warnings (What the Paper Doesn't Say)

The authors are very careful to say what this data is NOT for:

Not a Medical Tool: You cannot use this to diagnose mental health issues.
Not a Hiring Tool: You cannot use this to decide who to hire or fire.
Not a Spy Tool: The data is protected so no one can try to identify who the specific people are (though the voice data is kept private for this reason).
Not a Crystal Ball: Because the groups were small (only 10 groups), the results are a "feasibility check." It proves the idea works, but it doesn't mean we can perfectly predict human behavior in every situation yet.

In short: GroupAffect-4 is a high-definition, multi-sensory recording of four people working together. It shows us that who interrupts whom is a huge clue to how stressed a team is, but predicting personality or complex group dynamics is still a very hard puzzle that this dataset helps us start solving, but hasn't finished yet.

Technical Summary: GroupAffect-4

Problem Statement

Existing affective computing and social signal processing corpora typically capture human interaction in single-participant or dyadic (two-person) settings. While these datasets provide valuable insights, they rarely support the analysis of affect in co-located groups as a coupled process occurring simultaneously at three levels:

Within-person: Physiological state, attention, and cognitive load.
Between-person: Personality, trust, and negotiation stances.
Group-level: Floor balance, coordination, and collective outcomes.

Current literature often fragments the necessary signals (per-participant physiology, eye movement, audio, self-reports, task outcomes, and personality) across separate dataset traditions. There is a lack of publicly described corpora that combine all these modalities in a single, synchronized, four-person group setting to enable cross-level analysis (e.g., testing whether physiological arousal tracks subjective dominance or if audio overlap predicts cognitive demand independently of physiology).

Methodology

Dataset Design and Participants

GroupAffect-4 is a multimodal corpus comprising 10 sessions of four-person groups ( $N=40$ unique participants). The study was conducted at a single site (GN Advanced Science, Denmark) with participants recruited from internal staff and university networks. The cohort has a mean age of 35.4 years (SD 9.2) and is predominantly strangers (mean pre-session familiarity rating of 1.68 on a 7-point scale).

Experimental Protocol

Each session lasted approximately 70 minutes and followed a fixed sequence without counterbalancing:

T0 (Free-talk baseline): 5 minutes of unstructured conversation for sensor stabilization.
T1 (Hidden-profile decision): A group reasoning task requiring consensus on a hiring decision with asymmetric information.
T2 (Mini-negotiation): A task involving interdependent interests where groups negotiated a workshop topic and format.
T3 (Idea generation): A creative task using the Nominal Group Technique (NGT) to generate and select ideas.
T4 (Public-goods game): A micro-game measuring cooperative tension and social comparison via token contributions.

Data Acquisition and Modalities

Participants were instrumented with:

Physiology: EmotiBit wrist sensors recording Photoplethysmography (PPG), Electrodermal Activity (EDA), skin temperature, and IMU at ~25 Hz.
Eye Tracking: Tobii Pro Glasses 3 recording head-relative scene-frame gaze and pupil diameter at 50 Hz.
Audio: Close-talk DPA 4060 lapel microphones recorded at 48 kHz.
Self-Report: Tablet-based Self-Assessment Manikin (SAM) probes for Valence, Arousal, and Dominance (VAD) collected at scheduled intervals, plus post-task questionnaires.
Personality: Pre-session Big-Five Inventory (BFI-44) scores.
Synchronization: All modalities were synchronized via a common Lab Streaming Layer (LSL) timebase, with hardware clock drift corrections applied to audio.

Processing and Benchmarks

The release follows a BIDS-inspired structure with Croissant metadata. The authors defined 15 benchmark targets across three analysis levels and established feasibility baselines using Leave-One-Group-Out Cross-Validation (LOGO-CV) to ensure no participant in the test fold shared an interaction context with the training data.

Level 1 (Within-person state): Predicting task labels, valence, arousal, dominance, mental demand, engagement, satisfaction, and trust.
Level 2 (Between-person traits): Predicting BFI personality traits (Extraversion, Openness, Agreeableness) and contribution levels.
Level 3 (Group dynamics): Predicting speaking Gini coefficients (inequality) and speech-overlap fractions.

Preprocessing included quality gating, plausibility checks, winsorization, and within-person robust z-score normalization (for Level 1) or absolute feature retention (for Level 2).

Key Results

Data Quality and Coverage

The dataset demonstrates high signal integrity:

Physiology: 91.0% of expected rows observed; 76.0% usability for PPG/EDA/Temp.
Eye Tracking: 98.0% of rows observed; 87.2% gaze and 95.4% pupil usability.
Audio: 95% pass rate for usability.
Self-Reports: 98.9% overall completeness for post-task items.

Benchmark Performance

Task Classification (B0): Achieved 64.1% accuracy (vs. 26.5% baseline), confirming tasks are behaviorally distinguishable.
Within-Person State:
- Mental Demand (B3a): The strongest signal (AUC 0.719), driven primarily by a single audio feature: overlap fraction. This suggests cognitive load in groups is largely a conversational-floor phenomenon.
- Valence (B1a): Moderate performance (AUC 0.657).
- Arousal (B1b) and Dominance (B2): Near-chance performance, attributed to temporal mismatches between retrospective probes and peak physiological responses, and feature limitations respectively.
- Trust (B3d): Near-chance overall, but improved to AUC 0.679 when restricted to the cooperative context of T4, indicating that trust signals are context-dependent.
Between-Person Traits (Level 2): All personality benchmarks (B4a–B4c) performed near or below chance. The authors attribute this to the small test fold size (4 participants) making AUC estimates unstable, rather than a lack of signal. Full-sample correlations suggest associations exist (e.g., Pupil $\times$ Agreeableness $r=0.51$ ), but the current design cannot robustly predict them.
Group Dynamics (Level 3): Benchmarks for speaking inequality (B6a, B6b) and overlap (B7) failed to beat naive baselines using task-window aggregates. However, a binary classifier on raw speaking-fraction standard deviation achieved AUC 0.952, confirming the signal exists but is lost when aggregated at the task-window level.

Cross-Modal Complementarity

Analysis revealed near-zero correlation ( $|r| < 0.10$ ) between pupil diameter and wrist physiology, confirming they provide distinct information. Audio and pupil features were found to be the dominant predictors for cognitive-state detection, while physiology contributed primarily to arousal-sensitive targets.

Significance and Claims

The paper positions GroupAffect-4 not as a large-scale corpus, but as a compact, high-density resource designed for rigorous dataset characterization. Its primary contributions are:

Integration: It is the first publicly described dataset to combine wearable physiology, egocentric gaze, close-talk audio, in-situ self-reports, explicit task outcomes, and personality measures in a synchronized four-person setting.
Synchronization: It provides a tightly synchronized multimodal stack (sub-10ms alignment for video/audio, linear correction for audio drift) enabling the study of cross-level relationships that are impossible to test in fragmented datasets.
Feasibility Baselines: The paper establishes that while the dataset supports meaningful evaluation of within-person states (particularly mental demand via audio dynamics), it exposes current limitations in predicting between-person traits and group-level dynamics with small cohort sizes.
Transparency: The release includes a datasheet, Croissant metadata, quality reports, and open processing scripts, adhering to FAIR principles and responsible AI guidelines (e.g., gating raw audio under a Data Use Agreement to prevent re-identification).

The authors conclude that the dataset serves as a foundational resource for studying co-located group affect, offering a transparent view of what is currently achievable with multimodal signals in small groups and highlighting where future methodological or data-scale improvements are required.