💬 NLP

Mitigating Data Scarcity in Psychological Defense Classification with Context-Aware Synthetic Augmentation

To address data scarcity and class imbalance in psychological defense mechanism classification, this paper proposes a context-aware synthetic augmentation framework combined with a hybrid classification model that leverages definition-guided generation to significantly outperform existing baselines in the PsyDefDetect shared task.

Original authors: Hoang-Thuy-Duong Vu, Quoc-Cuong Pham, Huy-Hieu Pham

Published 2026-05-15

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Hoang-Thuy-Duong Vu, Quoc-Cuong Pham, Huy-Hieu Pham

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a computer to understand the hidden, unconscious ways people protect their feelings when they are stressed. In psychology, these are called Psychological Defense Mechanisms. They are like invisible shields people put up without realizing it. Sometimes they are healthy, and sometimes they are not.

The problem the authors faced is that there are very few examples of these "shields" in text data to teach the computer. It's like trying to learn how to identify rare birds in a forest, but you only have pictures of three of them.

Here is how the team at VinUniversity solved this, explained simply:

1. The Problem: The "Empty Forest"

The researchers found two main hurdles:

Not enough data: There weren't enough examples of people using these defenses to train a smart computer.
The "Fake Bird" Trap: When they tried to use AI to make up more examples (synthetic data), the AI often created fluent-sounding text that made no psychological sense. It was like the AI drawing a bird that looked real but had wings made of water. This confused the computer and made it worse at learning.

2. The Solution: The "Stress-First" Recipe

Instead of just asking the AI to "write a sentence about a defense," the team gave it a strict recipe based on real psychological rules (called the DMRS).

Step 1: Find the Stressor. They told the AI: "First, identify the stress (like a bad breakup or losing a job). Defenses only happen because of stress."
Step 2: Use a Clinical Dictionary. They gave the AI the official definitions of each defense mechanism. It's like giving a chef a specific recipe card instead of just saying "make a cake."
Step 3: The "Machine Annotator" Filter. They didn't trust the AI blindly. They used a second AI to check the generated text. If the second AI wasn't at least 60% sure the text was correct, they threw it away. This ensured the "fake birds" were actually birds, not water-winged monsters.

3. The Brain: A Two-Track System

To classify the text, they built a hybrid brain with two tracks working together:

Track A (The Linguist): Looks at the words, sentence length, and how often the person says "I."
Track B (The Clinician): Uses the 150 official psychological indicators to build a "profile" of the defense being used.
The Fusion: They combined these two tracks. It's like having a detective who looks at the words spoken and a psychologist who analyzes the intent behind them, then they vote together on the answer.

4. The Results: A Big Leap, But a Catch

The team tested their system on a "blind" test (a set of data the computer had never seen before).

The Win: Their system was a massive improvement over the previous best method. It went from getting about 18% of the answers right to 58%. In terms of a balanced score (which counts how well it handles rare cases), it jumped from 8.6% to 24.6%.
The Catch (The "Sink" Effect): The paper notes a major flaw. The computer still has a habit of guessing one specific answer (Label 7) for almost everything, especially when it's confused. It's like a student who, when they don't know the answer, just writes "C" on every question. Because one type of answer was so common in the test data, the computer leaned on it too hard, making it hard to correctly identify the rarer, more complex defenses.

5. What They Learned

Definitions Matter: The quality of the "recipe" (the definition given to the AI) was the most important factor. Better definitions meant better fake data, which meant a smarter computer.
More Data Isn't Always Better: If they generated too much fake data, the computer got confused by the noise. A little bit of high-quality fake data was better than a mountain of low-quality data.
Context is King: You can't just look at one sentence; you have to understand the stress that triggered it.

The Bottom Line

The authors built a system that successfully used "smart" fake data to teach a computer to spot psychological defenses, doubling its performance. However, they admit the system still struggles with the most common types of answers and needs more work (like human experts checking the work) before it could ever be used in a real doctor's office. For now, it's a powerful research tool, not a diagnostic one.

Technical Summary: VISHC at PSYDEFDETECT

Problem Statement

The automatic classification of Psychological Defense Mechanisms (PDMs) from text presents significant challenges for Natural Language Processing (NLP). Unlike standard NLP tasks, PDMs are unconscious, context-dependent processes that manifest through subtle cues (e.g., narrative inconsistency, emotional framing shifts) rather than explicit lexical markers. This implicit nature creates semantic ambiguity where identical surface text can reflect distinct defensive processes depending on intent.

These challenges are compounded by severe data scarcity and class imbalance in available datasets like PSYDEFCONV. Furthermore, the task includes auxiliary labels ("No Defense" and "Need More Information") that lack corresponding clinical defense items, violating standard multi-class assumptions and skewing distributions. While Large Language Models (LLMs) offer synthetic data augmentation, ungrounded generation often produces fluent but theoretically invalid text ("hallucinating defenses"), introducing noise that erodes model reliability.

Methodology

The authors propose a Context-Aware Synthetic Augmentation (CASA) framework paired with a Hybrid Feature Fusion architecture to address these issues. The pipeline consists of three main stages:

1. Context-Aware Synthetic Data Augmentation

Instead of standard paraphrasing, the system uses Stressor-Anchored Generation and Theory-Driven Prompting:

Stressor Anchoring: Prompts are anchored in specific stressors identified from dialogues (e.g., job loss, rejection) to ensure generated defenses reflect realistic responses to perceived threats rather than generic emotional expressions.
Theory-Driven Prompting: Prompts utilize structured clinical definitions from the Defense Mechanisms Rating Scales (DMRS), including the defense name, formal definition, and behavioral patterns. This steers the LLM (Llama-3-8B-Instruct) to generate samples that express the theoretical role of the defense.
Quality Control: A "machine-as-annotator" filter retains only generated batches where a secondary classifier achieves substantial agreement (Cohen's $\kappa \ge 0.60$ ). A soft balancing scheme caps minority classes at 500 total instances (real + synthetic) to prevent overfitting to generation artifacts.

2. Dual-Domain Feature Extraction

The system extracts two complementary feature sets for each utterance:

Linguistic Heuristics: Six lightweight features capturing surface cues, including utterance length, I-pronoun density, insight density, phatic flags, mature coping flags, and emotion intensity. These help distinguish non-defensive responses from defensive ones.
DMRS Defense Profile: A clinically grounded representation derived from 150 DMRS indicators. An NLI model estimates entailment probabilities for these indicators, which are aggregated into 30 mechanism scores and mapped to 9 defense levels.

3. Hybrid Feature Fusion Architecture

The model employs a late-fusion strategy:

Textual Encoder: MentalRoBERTa encodes the input formatted as [Stressor:S|Turn:T] to condition the representation on both context and response, yielding a 768-dimensional embedding.
Feature Encoders: The heuristic (7-dim) and DMRS (30-dim) features are processed through dedicated MLPs to produce 32-dimensional vectors each.
Fusion: The three vectors (768 + 32 + 32) are concatenated into an 832-dimensional vector, passed through fully connected layers, and classified into 9 labels.

Key Contributions

Psychologically Grounded Augmentation: The paper demonstrates that the quality of definitions in prompting directly governs generation fidelity. Using DMRS-based definitions significantly outperforms shallow class descriptions.
Clinical Feature Engineering: The integration of structured features from all 150 DMRS items with linguistic heuristics bridges the gap between clinical theory and neural classification.
Strong Low-Resource Baseline: The system establishes a competitive baseline for PDM classification in low-resource settings, significantly outperforming existing baselines without requiring massive new human-annotated datasets.

Experimental Results

Experiments were conducted on the PSYDEFCONV dataset (BioNLP@ACL 2026 shared task).

Performance Gains: The proposed system achieved an accuracy of 58.26% and a macro-F1 of 24.62% on the blind test set. This represents a substantial improvement over the DMRS CO-PILOT baseline (Accuracy: 18.01% $\to$ 58.26%; Macro-F1: 8.63% $\to$ 24.62%).
Augmentation Scaling: Performance peaked at an augmentation scale of $\times 2$ (macro-F1 27.99%), indicating an optimal balance between class coverage and synthetic noise. Aggressive scaling ( $\times 10$ ) led to performance degradation due to noise accumulation.
Class Imbalance Issues: Despite improvements, the model exhibited a "Label 7 sink effect," where the "High-Adaptive" class (Label 7) absorbed errors from other classes. For instance, 77% of "Neurotic" (Label 5) instances were misclassified as Label 7. Consequently, while overall accuracy is high, macro-F1 for minority classes remains low (below 0.30 for six of eight classes).

Significance and Limitations

The paper claims that clinical specificity in class definitions is the primary driver of synthetic augmentation effectiveness in this domain. By grounding generation in DMRS theory, the system successfully mitigates data scarcity better than ungrounded LLM approaches.

However, the authors maintain a modest stance regarding the system's clinical readiness:

Augmentation Ceiling: Naive count-balancing is insufficient when the majority class (Label 7) shares high linguistic overlap with adjacent classes.
Temporal Blindness: The current turn-level modeling fails to capture defensive shifts that occur across multiple dialogue turns.
Validity Constraints: While the "machine-as-annotator" filter ensures label consistency, it does not guarantee clinical validity. The authors emphasize that the system is a research prototype and must not be used as a diagnostic tool without expert review.
Task Ambiguity: Labels 0 ("No Defense") and 8 ("Need More Information") lack DMRS indicator structures, making them difficult to synthesize and classify effectively within the current framework.

Future work is suggested to include constraint-based decoding, human-in-the-loop validation, and dialogue-level modeling to address temporal volatility and majority-class bias.