💻 computer science

When Less Is Enough: Context Selection and Prompting Strategies for Bengali News Headline Generation

This paper investigates Bengali news headline generation using large language models and demonstrates that selecting salient lead paragraphs, employing cross-lingual prompting strategies, and utilizing few-shot examples (particularly for Gemini) yield superior results compared to feeding full articles, highlighting that context relevance and prompt design are more critical than input length.

Original authors: Muhammad Ashad Kabir, Kawsar Ahmed, Md. Osama

Published 2026-08-18

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Muhammad Ashad Kabir, Kawsar Ahmed, Md. Osama

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the bustling world of digital news, the headline is the most critical piece of real estate. It is the first thing a reader sees, the hook that decides whether a story gets read or scrolled past. For decades, computers have struggled to write these headlines on their own, often producing titles that are either too vague or completely miss the point. Recently, a new generation of powerful computer programs, known as large language models, has shown great promise in writing text. These systems can read vast amounts of information and generate human-like sentences. However, a significant challenge remains: these models have a limited memory for the text they process at one time. When fed a long news article, they can become overwhelmed, losing track of the most important details or getting distracted by less relevant information. This creates a dilemma for researchers: to write a good headline, does a computer need to read the entire story, or is there a smarter way to feed it information?

A team of researchers set out to solve this puzzle specifically for Bengali news, a language with fewer digital resources than English. They wanted to know if these advanced computer programs could write better headlines if they were given less text to read, provided that the text they did receive was the most important part. They tested this idea using three different large language models, feeding them news articles in various ways. Instead of dumping the entire article into the computer's memory, they tried giving it just the first few paragraphs. They also experimented with how they asked the computer to write the headline, sometimes speaking to it in Bengali and other times using English instructions to see which language guided the computer better. They even tested whether showing the computer a few examples of good headlines before asking it to write one would help it learn the task.

The researchers discovered that the common assumption—that more information is always better—was incorrect for this specific task. When they fed the computer the full news article, the quality of the generated headlines did not improve; in fact, for some of the models, it actually got slightly worse. The computer seemed to get lost in the details of the longer text. However, when the researchers restricted the input to just the first paragraph of the article, the results were often better. This finding aligns with how human journalists traditionally write news, placing the most crucial facts at the very beginning of the story. By focusing only on these opening paragraphs, the computer could ignore the noise and concentrate on the core message, producing headlines that were just as accurate as those generated from the full text, but with significantly less data to process.

The study also revealed that the way the computer was asked to perform the task mattered just as much as the amount of text it read. The researchers found that giving instructions in English, even when the news article itself was in Bengali, often led to better results than giving the instructions in Bengali. This suggests that the computer models, which are trained on massive amounts of global data, might understand the structure of a "news headline" task more clearly when the instructions are in English. Furthermore, the researchers tested whether showing the computer examples of previous articles and their headlines would help it learn. They found that for one of the models, seeing just a single example was enough to boost its performance significantly, while showing it more examples did not add any further benefit. Another model, however, did not seem to need any examples at all to perform well.

Ultimately, the work suggests that for generating news headlines, especially in languages like Bengali, the key to success lies in selecting the right information rather than feeding the computer everything available. The most effective approach involves giving the model a concise, relevant snippet of the story—specifically the lead paragraphs—and providing clear instructions, often in a language the model handles best. This approach allows these powerful tools to work efficiently without needing to be retrained on specific tasks. The findings offer a practical path forward for using artificial intelligence in newsrooms and other information-heavy fields, proving that sometimes, less is indeed enough to get the job done right.

Technical Summary: When Less Is Enough: Context Selection and Prompting Strategies for Bengali News Headline Generation

Problem Statement
Automatic news headline generation is a critical abstractive summarization task. While Large Language Models (LLMs) have demonstrated superior performance in zero-shot and few-shot settings compared to traditional fine-tuned models, their effectiveness is sensitive to input context management. LLMs face constraints regarding input context windows, where processing long documents can lead to information loss, semantic dilution, or capacity overflow. Furthermore, prior research indicates a "position bias," where models disproportionately prioritize information from specific parts of the input. In low-resource languages like Bengali, the optimal strategies for selecting context and designing prompts for headline generation remain underexplored. Specifically, it is unclear whether providing full articles improves performance or if selective context conditioning, combined with specific prompting paradigms and in-context learning (ICL), yields better results.

Methodology
The study investigates Bengali news headline generation using three LLMs: Gemini-2.0-Flash, Llama-3.3-70B, and GPT-4o. The research employs two benchmark datasets: BeliN (2,520 religious news articles) and Shironaam (diverse news articles).

Semantic Relevance Analysis: Before experimentation, the authors analyzed the semantic similarity between gold-standard headlines and individual article paragraphs using banBERT sentence embeddings. This analysis aimed to identify where salient information is concentrated within articles.
Selective Context Conditioning: Instead of feeding full articles, the models were conditioned on varying numbers of initial paragraphs ( $k=1$ to $5$). This strategy was motivated by the journalistic "inverted pyramid" structure and the semantic analysis results.
Prompting Strategies: The study evaluated two prompting paradigms:
- Bengali Native Prompting (BNaP): Instructions entirely in Bengali.
- Cross-Lingual Prompting (XLP): Instructions in English with Bengali article content.
- Contextual Enrichment: Prompts were tested in two variants: Baseline (article content only) and MultiGen (article content augmented with metadata such as sentiment, category, and aspect).
In-Context Learning (ICL): The authors compared zero-shot, one-shot, two-shot, and three-shot configurations to determine the impact of demonstration examples on generation quality.
Evaluation: Performance was measured using BERTScore, ROUGE-L, and METEOR. Statistical significance was assessed using paired mean differences with 95% BCa bootstrap confidence intervals and Wilcoxon signed-rank tests.

Key Results

Context Selection: Providing the full article does not necessarily improve performance.
- Gemini-2.0-Flash achieved its best performance using only the first paragraph of the article, outperforming full-document conditioning with a statistically significant improvement in BERTScore (+0.0078).
- Llama-3.3-70B performed best with the first three paragraphs, achieving a score comparable to the full document.
- Selective conditioning reduced input token length by approximately 52.6% (up to 65.1% for longer articles) while maintaining or improving quality.
In-Context Learning (ICL):
- Gemini showed substantial gains from moving from zero-shot to one-shot prompting, with performance plateauing thereafter.
- Llama showed limited benefit from additional demonstrations beyond one-shot, with performance slightly declining in two-shot and three-shot settings.
Prompting Strategies:
- Cross-Lingual Prompting (XLP) generally yielded stronger performance for Gemini and Llama, particularly when combined with contextual enrichment (MultiGen). For Gemini, XLP+MultiGen achieved the highest overall scores.
- Bengali Native Prompting (BNaP) performed better with the simpler Baseline configuration; adding contextual cues (MultiGen) did not significantly benefit BNaP and sometimes degraded performance.
- GPT-4o exhibited the opposite trend, performing slightly better under BNaP than XLP.
Comparison with Fine-Tuned Models:
- On the BeliN dataset, the Gemini-based prompting approach (XLP+MultiGen, 3-shot) outperformed the best fine-tuned transformer models (BanglaT5, mT5, mBART) across all metrics without task-specific fine-tuning.
- On the Shironaam dataset, the fine-tuned BERT2BERT model outperformed the prompting-based Gemini approach on BERTScore, ROUGE-L, and METEOR, though Gemini achieved a higher BLEU score.

Key Contributions and Significance
The paper claims that effective Bengali news headline generation depends more on context relevance and prompt design than on increasing input length. The primary contributions are:

Empirical Evidence for "Less is Enough": The study demonstrates that selectively conditioning LLMs on lead paragraphs can match or exceed full-document conditioning, highlighting that relevance-focused selection reduces noise and input costs.
Model-Dependent Sensitivity: The research characterizes how different LLMs respond to context length and in-context learning. It shows that Gemini benefits significantly from concise, salient inputs and few-shot examples, whereas Llama is more robust to context length but less sensitive to the number of demonstrations.
Prompting Paradigm Interactions: The work identifies that Cross-Lingual Prompting (XLP) combined with contextual enrichment is highly effective for Gemini and Llama, while Bengali Native Prompting (BNaP) functions best with minimal structure.
Practical Insights for Low-Resource Settings: The findings suggest that carefully designed prompting strategies and selective context conditioning can enable general-purpose LLMs to achieve competitive, and in some cases superior, performance for low-resource languages without the need for task-specific fine-tuning.

The authors conclude that while supervised fine-tuning remains strong for specific benchmarks, the proposed approach offers a flexible, training-free alternative that leverages the inherent capabilities of modern LLMs through optimized context and prompt engineering.

Technical Summary: When Less Is Enough: Context Selection and Prompting Strategies for Bengali News Headline Generation

More like this