Lacuna Inc. at SemEval-2026 Task 4: Structurally Gated State-Space Models for Disentangling Narrative Similarity

Technical Summary: IVD-SSM for Narrative Similarity

Problem Definition
The paper addresses the computational challenge of evaluating narrative similarity, a task defined by SemEval-2026 Task 4 as distinguishing the "deep structure" of a plot (abstract patterns of causality and progression) from its surface-level realization (specific names, settings, and objects). Standard Transformer architectures often fail at this due to a bias toward lexical and semantic overlap, leading to "spurious correlations" where stories sharing genre-specific vocabulary (e.g., "zombies") are deemed similar despite having fundamentally different causal chains. Furthermore, standard Transformers face quadratic memory and computational bottlenecks ($O(N^2)$) when processing the long-range dependencies inherent in extended narratives.

Methodology: IVD-SSM
The authors propose the Invariant-Variant Disentangled State-Space Model (IVD-SSM), a hybrid architecture designed to explicitly separate invariant plot structures from variant semantic details. The system consists of two primary components:

1. Backbone Encoder (Jamba-1.5-Mini):

   • The system utilizes a Jamba-1.5-Mini model, a hybrid architecture combining Mamba (State-Space Model) layers with Transformer attention.
   • The Mamba layers are selected for their ability to compress sequence history into a dynamically updated hidden state with linear time complexity ($O(N)$), making them ideal for capturing long-range causal chains without the memory overhead of key-value caches.
   • To adapt the 12-billion parameter model under computational constraints, the backbone is loaded in 4-bit NormalFloat (NF4) quantization. The authors employ QLoRA (Parameter-Efficient Fine-Tuning) targeting only projection and embedding modules, while keeping the rest of the network frozen to preserve pre-trained world knowledge.

2. Structurally Gated Alignment (SGA) Head:

   • This is the core novel contribution, a differentiable dual-scale alignment mechanism inspired by the Smith-Waterman algorithm. It processes an anchor narrative and a candidate narrative through two parallel pathways:
     • Macro-path (Invariant Structure): The input sequences are heavily downsampled (stride $k=4$) to capture the coarse structural skeleton (e.g., introduction, conflict, resolution). This path is blind to specific token-level entities.
     • Micro-path (Variant Details): The full-resolution sequences are processed to capture fine-grained, entity-level matches (e.g., specific character names or objects).
   • Gating Mechanism: The Macro-path generates a structural similarity matrix ($M_{macro}$), which is upsampled and passed through a temperature-scaled Sigmoid activation to create a soft structural mask ($G$). This mask gates the Micro-path similarity matrix ($M_{micro}$) via element-wise multiplication ($M_{filtered} = M_{micro} \odot G$).
   • This mechanism actively suppresses semantic noise and superficial keyword overlaps, ensuring that high similarity scores are only assigned when fine-grained details align within structurally valid regions.

Training and Evaluation

• Data: Due to limited human-annotated data (200 development, 400 test triples), the model was trained exclusively on 1,900 synthetic story triples provided by the task organizers, augmented with the human-annotated set for validation.
• Track A (Pairwise Classification): The model uses a contrastive objective to determine which of two candidates is structurally closer to an anchor.
• Track B (Representation Learning): The model generates dense vector embeddings via mean-pooling of the projected backbone outputs, followed by strict L2 normalization to facilitate cosine distance comparisons.

Key Results

• Track A (Pairwise Classification): The IVD-SSM (Gated) achieved an accuracy of 57.25%, outperforming the random baseline (50.00%) and the lexical Jaccard similarity baseline (56.25%). However, it trailed the zero-shot GPT-4o-mini prompt (67.00%).
• Track B (Embedding Similarity): The model achieved 54.50% accuracy, falling short of the all-MiniLM-L6-v2 sentence encoder (58.50%) and story-emb (63.25%).
• Ablation Studies:
  • The Micro-path alone (55.75%) underperformed the Jaccard baseline, confirming that fine-grained token similarity without structural filtering is vulnerable to lexical distractors.
  • The Macro-path alone (57.00%) substantially outperformed Jaccard, indicating that the coarse structural skeleton carries the majority of the signal for narrative equivalence.
  • The Gating mechanism provided a modest but consistent improvement (+0.25%) over the Macro-only variant, validating its role in filtering residual semantic noise.

Significance and Claims
The paper claims that the primary bottleneck in computational narrative evaluation is the conflation of invariant plot structure with variant semantic details. The authors argue that their results demonstrate narrative similarity cannot be reliably measured through "bag-of-words" or generic sentence embeddings. Instead, it requires architectures that explicitly model the sequential course of action.

The significance of the work lies in the IVD-SSM's ability to disentangle structural invariants from lexical variants, providing a principled framework for deep narrative understanding. The authors note that while their approach does not yet match the performance of large zero-shot LLMs (likely due to the LLMs' extensive pre-training on narrative structures and implicit reasoning capabilities), the SGA architecture successfully mitigates "spurious correlations" that cause standard models to fail on hard negative samples.

Limitations
The authors acknowledge several limitations:

• Soft Gating: The Sigmoid-based gate is continuous and never reaches zero, allowing some "leakage" of strong lexical distractors (e.g., identical named entities) to override structural misalignment.
• Computational Complexity: While the backbone is linear, the SGA head reintroduces a quadratic ($O(L_a \times L_c)$) bottleneck when computing full cross-sequence similarity matrices, which may not scale to full-length novels or screenplays without sparse alignment modifications.
• Data Bias: The model was trained primarily on synthetic data generated by LLMs, potentially leading to overfitting on predictable narrative pacing and standard archetypes, leaving its performance on avant-garde or non-linear human-authored narratives unexplored.