💬 NLP

Event Detection with a Context-Aware Encoder and LoRA for Improved Performance on Long-Tailed Classes

This paper addresses the architectural limitations of unidirectional decoder-only LLMs and the bias of Micro-F1 metrics in event detection by demonstrating that incorporating sentence context and applying Low-Rank Adaptation (LoRA) significantly improves Macro-F1 performance, particularly for long-tailed event classes.

Original authors: Abdullah Al Monsur, Nitesh Vamshi Bommisetty, Gene Louis Kim

Published 2026-02-18

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Abdullah Al Monsur, Nitesh Vamshi Bommisetty, Gene Louis Kim

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery in a massive library. Your job is to find specific "events" (like a "Warning" or an "Incident") hidden inside thousands of books. This is what computers do in a field called Event Detection.

However, the researchers in this paper found two big problems with how detectives (AI models) were currently working:

The "One-Way Street" Problem: Most modern AI models (called "decoder-only" models) read like a person reading a sentence from left to right, never looking back. It's like trying to understand a joke by only hearing the punchline without hearing the setup. They miss the full context.
The "Popularity Contest" Problem: When checking how good these detectives are, people usually use a score called Micro-F1. This score is like a popularity contest: if the AI gets the 100 most common events right, it gets a high score, even if it fails miserably at the 100 rare, weird events. It's like a student getting an 'A' because they aced the easy questions but failed the hard ones.

The researchers wanted to fix both issues. Here is how they did it, explained with some everyday analogies:

1. Giving the AI a "Whole Picture" (Context-Aware Encoder)

The researchers realized that to understand a rare event, you need to see the whole sentence, not just the specific word.

The Analogy: Imagine trying to guess if someone is "angry." If you only hear them say "Fine," you might think they are okay. But if you see their whole body language, hear their tone, and see them slamming a door (the full context), you know they are furious.
The Fix: They took these "one-way" AI models and gave them a special tool to look at the entire sentence at once. They tried different ways to do this, but the best one was called FiLM (Feature-wise Linear Modulation).
The Metaphor: Think of FiLM as a smart spotlight. It doesn't just shine on one word; it takes the "vibe" of the whole sentence and uses it to adjust how the AI sees every single word. It says, "Hey, since this whole story is about a storm, that word 'light' probably means 'lightning,' not 'a lamp'."

2. The "Magic Training Cheat Sheet" (LoRA)

Training these giant AI models is like trying to teach a PhD student a new subject by rewriting their entire brain. It's expensive, slow, and sometimes the student forgets what they already knew (a problem called "catastrophic forgetting").

The researchers used a technique called LoRA (Low-Rank Adaptation).

The Analogy: Instead of rewriting the student's entire textbook, you just give them a sticky note with the most important new rules written on it. You stick it on the cover. When the student reads, they look at the main book and the sticky note.
The Result: This "sticky note" method is incredibly fast and cheap. But surprisingly, it also made the AI much better at learning the rare events (the long-tail classes). It acted like a "focus filter," helping the AI pay attention to the difficult, uncommon cases without getting confused by the easy, common ones.

3. Changing the Scoreboard (Macro-F1)

The researchers argued that we need to change how we grade these AI detectives.

The Old Way (Micro-F1): "You got 90% of the answers right!" (But 90% of those answers were the easy, common ones).
The New Way (Macro-F1): "You got 50% of the easy ones right, but you also got 50% of the super-hard, rare ones right."
Why it matters: By using Macro-F1, they showed that their new method (FiLM + LoRA) was actually a much better detective for the whole library, not just the popular section.

The Big Takeaway

The paper proves that:

Context is King: Even the smartest "one-way" AI models can become great detectives if you give them the full sentence context (using tools like FiLM).
Small Changes, Big Gains: You don't need to rebuild the whole AI. Just adding a "sticky note" (LoRA) makes it smarter, faster, and better at handling rare, tricky situations.
Fair Grading Matters: If you only look at the "popularity contest" scores, you miss the fact that the AI is actually struggling with the rare stuff. We need to grade them on how well they handle the whole spectrum of events.

In a nutshell: They took a smart but slightly blind AI, gave it a pair of glasses to see the whole picture, and gave it a cheat sheet to learn the hard stuff quickly. The result? A detective that is finally fair to both the popular and the obscure events.

1. Problem Statement

The paper addresses two critical limitations in current Event Detection (ED) research:

Architectural Bottleneck: The prevailing reliance on decoder-only Large Language Models (LLMs) (e.g., Llama, Qwen) for ED tasks. These models utilize unidirectional attention, which hinders their ability to capture rich, bidirectional context necessary for understanding natural language events. While encoder-only models (like BERT) have dominated ED, they are often based on older architectures.
Evaluation Bias: The field's over-reliance on Micro-F1 scores. Micro-F1 weights every event mention equally, systematically inflating performance by favoring majority (frequent) classes while masking poor performance on long-tailed classes (rare event types). The authors argue that Macro-F1 is a more representative metric for evaluating a model's generalizability across the full spectrum of event types.

2. Methodology

The study investigates how to adapt decoder-only LLMs for ED by introducing sentence-level context and utilizing parameter-efficient fine-tuning.

A. Task Definition

The task follows the MAVEN and RAMS dataset definitions, focusing on Trigger Classification (TC): identifying the event type for a specific word span (trigger) within a sentence.

B. Model Architectures

The authors categorize their approach into two main strategies:

Prompted LLMs (In-Context Learning):
- Zero-shot & Few-shot: Using instruction-tuned models (Llama 3, Qwen 2.5, Gemma, DeepSeek) with JSON-formatted prompts.
- Strategy: The models are queried directly without updating parameters. Few-shot settings include labeled examples (sentence, trigger, event type) to guide the model.
LLM-as-Embedding (Supervised Fine-Tuning):
- The LLM serves as a text encoder to generate token embeddings, which are then processed by a downstream classification head.
- Baseline: BaseTE (Base Trigger Encoder), which linearly projects LLM hidden states without additional context.
- Context-Aware Variations: To mitigate the unidirectional limitation, the authors introduce sentence-level context into token representations via four methods:
  - ConcatPool: Mean pooling of the sequence to create a global context vector, concatenated with each token, then fused via a linear layer.
  - FiLM (Feature-wise Linear Modulation): Generates scaling ( $\gamma$ ) and shifting ( $\beta$ ) vectors from the global context to modulate token representations element-wise.
  - BiSE-BiLSTM: Passes transformer hidden states through a bidirectional LSTM to create bidirectionally contextualized tokens.
  - Gated Context Fusion: Uses a gating mechanism to dynamically weight the contribution of the original token vs. the global sequence vector.

C. Fine-Tuning Strategy: LoRA

The authors employ Low-Rank Adaptation (LoRA) for fine-tuning. Instead of updating all parameters, LoRA injects trainable low-rank matrices into the linear layers of the model.

Hypothesis: LoRA acts as a regularizer, preventing overfitting to the training data (especially beneficial for large models) and improving generalization to low-frequency (long-tail) event classes.

3. Key Contributions

Context Integration: Demonstrated that incorporating sentence-level information into decoder-only models significantly boosts ED performance, with FiLM and BiSE-BiLSTM being the most effective variations.
Metric Re-evaluation: Highlighted the divergence between Micro-F1 and Macro-F1 rankings. The study proves that Micro-F1 hides the model's inability to handle rare classes, establishing Macro-F1 as a critical complementary metric.
LoRA as a Regularizer: Showed that LoRA not only improves computational efficiency but acts as a potent regularizer that specifically enhances performance on long-tailed event classes, leading to substantial gains in Macro-F1 scores.
Comprehensive Benchmarking: Provided a rigorous comparison between encoder-only (BERT, RoBERTa), decoder-only (Llama, Qwen), and hybrid approaches across two major datasets (MAVEN, RAMS).

4. Key Results

Performance Gains:
- Decoder-only models (Llama 1B, Qwen 1.5B) fine-tuned with LoRA and FiLM context surpassed fully fine-tuned BERT models in Macro-F1.
- Qwen 3B with FiLM achieved the best Macro-F1 score (62.01%) across all experimental settings on the MAVEN dataset.
- Llama 1B with FiLM achieved a Macro-F1 of 61.38%, outperforming the baseline BERT models.
Long-Tail Improvement: LoRA significantly improved Macro-F1 scores (which reflect performance on rare classes) compared to full fine-tuning. The performance gap between frequent and rare classes narrowed, indicating better generalization.
Prompting vs. Fine-Tuning:
- Few-shot prompting significantly outperformed zero-shot prompting but generally underperformed compared to supervised fine-tuning with LoRA on the larger MAVEN dataset.
- On the smaller RAMS dataset, few-shot prompting was competitive with fine-tuning, suggesting fine-tuning benefits diminish on very small datasets.
Precision vs. Recall Trade-offs:
- T5 models exhibited high precision (conservative predictions).
- Decoder-only models (especially with context variations) showed high recall, making them suitable for tasks where missing an event is costlier than a false positive.
Statistical Significance: Paired t-tests confirmed that the variations of Llama and Qwen models were statistically significantly different ( $p < 0.05$ ) from fully fine-tuned baselines and BERT/T5 models in terms of Macro-F1.

5. Significance and Conclusion

This paper challenges the notion that decoder-only LLMs are unsuitable for structured NLP tasks like Event Detection. By introducing sentence-level context and utilizing LoRA, the authors successfully adapted these models to outperform traditional encoder-only baselines, particularly on long-tailed classes.

The study underscores the importance of shifting evaluation standards from Micro-F1 to Macro-F1 to ensure models are robust across all event types, not just the common ones. Furthermore, it positions LoRA not just as a memory-saving tool, but as a crucial mechanism for improving generalization and reducing overfitting in large language models applied to specialized, imbalanced datasets. This work provides a roadmap for building more robust, context-aware NLU systems using modern generative architectures.