💬 NLP

LaMSUM: Amplifying Voices Against Harassment through LLM Guided Extractive Summarization of User Incident Reports

This paper introduces LaMSUM, a novel multi-level framework that leverages Large Language Models and voting mechanisms to generate extractive summaries of large collections of code-mixed sexual harassment incident reports, thereby overcoming context window limitations and outperforming existing state-of-the-art methods to aid stakeholders in policy development.

Original authors: Garima Chhikara, Anurag Sharma, V. Gurucharan, Kripabandhu Ghosh, Abhijnan Chakraborty

Published 2026-04-20

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Garima Chhikara, Anurag Sharma, V. Gurucharan, Kripabandhu Ghosh, Abhijnan Chakraborty

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are the mayor of a bustling city. Every day, thousands of citizens send you letters describing what's happening in their neighborhoods. Some letters are about potholes, some about noisy neighbors, and sadly, many are about scary and dangerous harassment incidents.

You want to help, but you can't read 5,000 letters a day. You need a way to read just the top 50 most important letters that give you a perfect snapshot of the city's problems without missing anything crucial.

This is exactly the problem the paper LaMSUM solves, but for online platforms where people report sexual harassment.

Here is the story of how they built a solution, explained simply:

1. The Problem: The "Too Much Noise" Dilemma

Citizen reporting platforms are like a giant, overflowing mailbox.

The Volume: There are too many reports to read one by one.
The Language: People write in "code-mixed" languages (like Hinglish in India, mixing Hindi and English), which is hard for old computer programs to understand.
The AI Trap: We have super-smart AI (Large Language Models or LLMs) that can read fast. But here's the catch:
- They rewrite instead of picking: If you ask a smart AI to "summarize" these letters, it usually tries to rewrite them in its own words (like a news anchor summarizing a story). But for harassment reports, rewriting is dangerous. You need the victim's exact words to keep the truth and emotion intact. You need to pick the best original letters, not rewrite them.
- They have short memories: These AIs can only read a certain amount of text at once. If you feed them a whole year's worth of reports, their "brain" gets full, and they start making things up or ignoring instructions.

2. The Solution: LaMSUM (The "Smart Committee")

The authors created a system called LaMSUM. Think of it not as a single reader, but as a smart committee that uses a clever voting system to pick the best letters.

Here is how it works, step-by-step:

Step A: The "Shuffle" (Breaking the Bias)

Imagine you have a stack of 100 letters. If you ask a human (or an AI) to pick the best ones, they might accidentally pick the ones at the very top just because they saw them first. This is called positional bias.

To fix this, LaMSUM acts like a card dealer. It takes the stack of letters, shuffles them, and asks the AI to pick the best ones. Then, it shuffles them again (in a different order) and asks the AI to pick again. It does this 3 times.

Analogy: It's like asking three different judges to pick the best songs from a playlist, but every time, the songs are played in a different order so the judges don't just pick the first three they hear.

Step B: The "Multi-Level" Funnel (The Bucket Brigade)

Since the AI can't read 5,000 letters at once, LaMSUM uses a funnel approach.

Level 1: It splits the 5,000 letters into small groups (chunks). It asks the AI to pick the best 50 from each small group.
Level 2: It takes those "best 50s" from all the groups and puts them into new, slightly larger groups. It asks the AI to pick the best ones again.
The Final Level: It keeps doing this until it narrows everything down to the final 50 letters that represent the whole collection.

Step C: The "Voting Election" (Choosing the Winners)

This is the magic part. After shuffling and narrowing down, the AI has picked different letters in different rounds. How do we decide which ones stay?

LaMSUM treats the letters like candidates in an election and the AI's choices as votes.

They use a special voting method called Proportional Approval Voting.
Analogy: Imagine a school election. If one candidate (a letter about "stalking") gets voted for in 3 out of 3 shuffles, they win. But if another candidate (a letter about "domestic violence") is only voted for in 1 shuffle but is very unique, the system ensures they still get a chance so the final list isn't just a bunch of identical stories. It balances popularity with diversity.

3. Why is this a Big Deal?

It keeps the truth: Unlike other AIs that rewrite stories, LaMSUM picks the original words of the victims. This is crucial for legal and emotional accuracy.
It handles the "Big Data": It can process massive amounts of text that would normally crash a computer.
It works with messy languages: It handles mixed languages (like Hinglish) better than previous tools.
It beats the competition: In their tests, LaMSUM did a better job than the best existing AI models at picking the right stories.

The Bottom Line

LaMSUM is like a super-efficient, unbiased editor for a city's safety. It takes thousands of chaotic, scary, and messy reports, shuffles them to be fair, and uses a smart voting system to hand the authorities a concise, powerful list of the most important incidents.

This helps authorities see the big picture quickly, so they can make better policies to keep communities safe, without getting lost in a sea of paperwork.

1. Problem Statement

The paper addresses the challenge of processing large volumes of user-generated incident reports on citizen reporting platforms (specifically regarding sexual harassment, domestic violence, and assault).

Data Volume & Complexity: Authorities face an overwhelming amount of data, making manual review impractical. Furthermore, these reports often exist in code-mixed languages (e.g., Hinglish in India), limiting the effectiveness of traditional English-centric summarization tools.
Extractive vs. Abstractive Needs: For sensitive incident reports, extractive summarization (selecting exact sentences from the original text) is preferred over abstractive summarization (paraphrasing). Preserving the user's exact words is crucial for authenticity, legal context, and maintaining the emotional weight of the testimony.
LLM Limitations: While Large Language Models (LLMs) excel at abstractive summarization, they struggle with extractive summarization due to two main factors:
1. Inherent Behavior: LLMs naturally paraphrase text rather than selecting specific subsets, even when prompted to do otherwise.
2. Context Window Constraints: LLMs have finite context windows and cannot process massive collections of text (e.g., >30K tokens) in a single pass.
3. Positional Bias: LLMs tend to select sentences from the beginning or end of a text, ignoring the middle, leading to biased summaries.

2. Methodology: The LaMSUM Framework

The authors propose LaMSUM (Large Language Model based Extractive SUMmarization), a novel multi-level framework that combines LLMs with Social Choice Theory (Voting Algorithms) to overcome the limitations above.

A. Multi-Level Summarization Architecture

To handle text larger than the LLM's context window ( $W$ ), LaMSUM employs a hierarchical, iterative approach:

Level 0 (Input): The total set of posts $T$ is divided into smaller chunks of size $s$ .
Chunk Summarization: Each chunk is summarized by an LLM to produce a smaller set of $q$ units.
Iteration: The summaries from Level 0 are merged to form the input for Level 1. This process repeats until the final summary size $k$ is reached.
Constraint: The authors set $q = k$ to ensure the algorithm can handle worst-case scenarios where all final summary units might originate from a single initial chunk.

B. Tackling Positional Bias via Shuffling

To prevent the LLM from favoring sentences based on their position in the input:

Shuffling: For each chunk, the textual units are shuffled $m$ times to create $m$ different variations.
Multiple Summaries: The LLM generates a summary for each variation. This ensures every sentence has an equal opportunity to appear in different positions (beginning, middle, end) across the $m$ runs.

C. Output Calibration

Since LLMs may hallucinate or slightly modify words when selecting text:

The system performs a verification step using Edit Distance and Keyword Matching.
If the LLM selects a sentence that is not an exact subset of the original text, the system identifies the closest matching original post to ensure the final summary is strictly extractive.

D. Reimagining Summarization as an Election

The $m$ summaries generated from the shuffled variations are treated as "ballots" in a multi-winner election. The goal is to select the top $q$ "winners" (sentences/posts). The paper evaluates three voting methods:

Plurality Voting (Block Voting): Counts the number of times a sentence appears across the $m$ summaries.
Proportional Approval Voting (PAV): Selects sentences based on approval but applies a "diminishing returns" mechanism. If a sentence is approved by many shuffles, its marginal utility decreases, preventing redundancy and encouraging the inclusion of diverse, less frequent but important posts.
Ranked Choice Voting (Borda Count): The LLM ranks all sentences; points are assigned based on rank position.

3. Key Contributions

First LLM-based Extractive Summarization for Large Collections: This is one of the first works to successfully implement extractive summarization on large user-generated text collections (>30K tokens) using LLMs, overcoming the "abstractive default" of LLMs.
Novel Framework (LaMSUM): Introduces a multi-level architecture combining chunking, shuffling, and voting algorithms to mitigate positional bias and context window limits.
Social Choice Integration: Demonstrates the efficacy of applying voting theory (specifically Proportional Approval Voting) to NLP tasks to balance popularity (frequency) with diversity in summarization.
Code-Mixed Language Support: Validates the approach on real-world incident reports containing code-mixed languages (e.g., Hinglish), where traditional models often fail.

4. Experimental Results

Dataset: The study utilized incident reports from five cities in India (A–E), covering various harassment categories (stalking, assault, domestic violence, etc.). The dataset contained between 545 and 867 posts per city.
Models Tested: Four LLMs were evaluated: Llama-3.1-8B, Mistral-Nemo, Claude-3-Haiku, and GPT-4o-mini.
Baselines: Compared against pre-neural models (LexRank, SummBasic), Transformer models (BERT, XLNet), and fine-tuned extractive models (BERTSUM).
Performance Metrics: Evaluated using ROUGE-1, ROUGE-2, and ROUGE-Lsum.
Key Findings:
- Superiority: LaMSUM (specifically using Claude-3-Haiku with Proportional Approval Voting) outperformed all state-of-the-art baselines across all datasets and metrics.
- Voting Efficacy: Proportional Approval Voting (PAV) significantly outperformed Plurality and Ranked Choice (Borda) voting. PAV was found to better balance content coverage and diversity, avoiding the redundancy seen in Plurality voting.
- Shuffling Impact: Using $m=3$ shuffles provided the best trade-off between performance and computational cost.
- Diversity: LaMSUM selected posts with higher word counts (more descriptive) and greater category diversity (higher entropy) compared to other algorithms.

5. Significance and Impact

Policy & Safety: By providing authorities and stakeholders with a concise, authentic, and diverse overview of harassment incidents, LaMSUM enables data-driven decision-making. This can lead to better resource allocation, identification of "hotspots," and the formulation of proactive safety policies.
Scalability: The framework is scalable and generalizable, capable of handling any text length as long as the final summary fits within a context window. It was also validated on public datasets like US-Election and Me-Too.
Ethical Considerations: The authors acknowledge the "black box" nature of LLMs and the risk of data leakage or bias. They emphasize that while LaMSUM improves summarization, it should be used as a decision-support tool rather than a sole proxy for real-world situations, requiring human oversight to ensure ethical application.

In conclusion, LaMSUM represents a significant advancement in NLP by successfully adapting generative LLMs for strict extractive tasks in high-stakes, sensitive domains, leveraging voting theory to ensure robustness, diversity, and accuracy.