Technical Summary: DiaRelay

Problem Statement

Emotion Recognition in Conversation (ERC) requires models to identify subtle emotional cues often distributed across distant dialogue turns. Existing approaches typically rely on a fixed context window to incorporate dialogue history. This strategy presents a fundamental trade-off: short windows discard potentially useful long-range evidence, while enlarging the window increases computational and memory costs and may introduce irrelevant context.

Furthermore, while Parameter-Efficient Fine-Tuning (PEFT) methods like LoRA enable Large Language Models (LLMs) to adapt to ERC tasks efficiently, they possess a critical limitation: the learned low-rank transformations are shared across dialogue turns and do not explicitly maintain a persistent, dialogue-level state. Consequently, once an utterance leaves the explicit context window, standard PEFT methods cannot preserve useful antecedent cues or condition subsequent transformations on the evolving conversational history.

Methodology

To address these limitations, the authors propose DiaRelay, a lightweight, memory-augmented adapter designed to equip LLMs with an explicit, constant-size dialogue-level memory. DiaRelay operates on top of a LoRA-based backbone and introduces two tightly collaborative components that function under a "read-before-write" paradigm:

1. Selective Relay Memory Transition (SRMT)

SRMT is responsible for updating a bounded, constant-size relay memory ($R_t$) as the dialogue progresses.

• Relay-Space Projection: The target utterance representation is projected into low-dimensional query ($q^m_t$), key ($k^m_t$), and value ($v^m_t$) vectors within a compact relay space of rank $r$.
• Gated Error-Corrective Update: Instead of indiscriminately accumulating the current value, SRMT employs a gated mechanism to selectively update the memory. It calculates an error-corrective signal ($e^m_t = v^m_t - R_{t-1}k^m_t$), representing the component of the current value not recoverable from the previous memory.
• Update Rule: The memory is updated using a retention gate ($\lambda_t$) and an update gate ($\beta_t$):
  $$R_t = \text{Diag}(\lambda_t) R_{t-1} + \text{Diag}(\beta_t) (e^m_t) (k^m_t)^\top$$
  This allows the memory to preserve stable historical information while incorporating new residual evidence without increasing memory size.

2. Dual-axis Relay Memory Read (DaRMR)

DaRMR retrieves information from the propagated memory ($R_{t-1}$) to dynamically modulate the low-rank feature transformations of the backbone before the current utterance is written into the memory.

• Dual-Axis Readout: The memory is read along two directions to generate corrections for the self-attention mechanism:
  • Query-side Correction ($\Delta q^m_t$): Retrieves memory along the value-to-key direction to adjust the attention query.
  • Output-side Correction ($\Delta o^m_t$): Retrieves memory along the key-to-value direction to adjust the attention output.
• Integration: These corrections are added to the original backbone and LoRA residuals for the target utterance span, enabling context-dependent representation adaptation without test-time gradient updates.

Operational Paradigm

The system follows a strict causal flow: for the $t$-th utterance, DaRMR accesses only the memory $R_{t-1}$ (accumulated from turns $1$ to $t-1$) to generate predictions. Only after the prediction is made does SRMT update the memory to $R_t$ using the current utterance's representation. This ensures the current utterance does not influence its own prediction while allowing it to inform future predictions.

Key Contributions

The paper outlines three primary contributions:

1. DiaRelay Architecture: A lightweight adapter that complements fixed local context windows with a bounded, cross-utterance constant-size relay memory. This enables historical information to persist across predictions without expanding input context length or requiring test-time parameter updates.
2. Novel Mechanisms: The introduction of SRMT (using gated error-corrective transitions for selective memory updates) and DaRMR (using dual-axis readouts for query and output corrections). Together, they establish a "read-before-write" paradigm for dialogue-level memory propagation.
3. Empirical Validation: Extensive experiments demonstrating that DiaRelay outperforms strong LoRA-based fixed-window baselines on standard ERC datasets, achieving state-of-the-art results with minimal parameter overhead.

Experimental Results

The authors evaluated DiaRelay on the MELD and IEMOCAP datasets using Qwen3-4B and Qwen3-8B backbones. The experiments were conducted under a "history-only" setting (no future utterances, no external knowledge).

• Performance:
  • On MELD, DiaRelay (Qwen3-8B) achieved a Weighted F1 of 70.06% and Accuracy of 71.15%, setting a new state-of-the-art among methods using backbones $\le$ 8B without external augmentation.
  • On IEMOCAP, it achieved a Weighted F1 of 70.01% and Accuracy of 69.93%, showing competitive performance against methods that utilize external knowledge or complex graph modeling.
• Efficiency: DiaRelay adds only ~7.1M trainable parameters (approximately 0.09% of the Qwen3-8B backbone) while maintaining the original autoregressive training objective and standard inference pipeline.
• Ablation Studies:
  • Removing the cross-window memory propagation (Window-local DiaRelay) resulted in a performance drop, confirming the value of persistent memory.
  • Replacing the error-corrective update with a full value update degraded performance, particularly in Macro F1, validating the residual update strategy.
  • Removing either the query-side or output-side correction path consistently degraded metrics, confirming the complementary nature of the dual-axis read.

Significance and Claims

The paper claims that DiaRelay provides a new lightweight solution for Emotion Recognition in Conversation that effectively bridges the gap between the limited context windows of LLMs and the need for long-range dialogue understanding.

• Generalizability: The method demonstrates effectiveness across different backbone scales (4B and 8B) and datasets, suggesting it is a generalizable enhancement for LLM-based emotional understanding.
• Efficiency: By maintaining a constant-size memory and avoiding test-time gradient updates or external retrieval mechanisms, DiaRelay offers a computationally efficient alternative to methods that rely on expanding context windows or complex multi-stage training.
• Baseline Establishment: The authors position DiaRelay as a strong "history-only" baseline, proving that explicit dialogue-level memory propagation can significantly enhance ERC performance without relying on auxiliary data like speaker biographies, retrieved demonstrations, or commonsense knowledge.

The authors acknowledge limitations, noting that the current framework is text-only and focuses on classification rather than generating natural language explanations for predictions. Future work is proposed to extend the framework to multimodal settings and incorporate explanation generation mechanisms grounded in the relayed memory.