AutoJourn: Multi-Perspective Summarisation, Bias Detection and Bias Neutralisation for LLM-Generated News in Automated Journalism

Technical Summary: AutoJourn

Problem Statement
The paper addresses critical challenges in responsible automated journalism, specifically the tendency of Large Language Models (LLMs) to flatten diverse viewpoints into a single narrative and reproduce subtle biases present in source data or training signals. While automated journalism offers scalability, existing Natural Language Processing (NLP) tools often operate in isolation: summarization systems typically merge competing viewpoints into a "neutral" account, and bias detection tools are rarely integrated into end-to-end news generation workflows. Consequently, current pipelines lack support for surfacing multiple perspectives, generating viewpoint-aware articles, and evaluating the bias of the resulting text, particularly when input data is drawn from fragmented and polarized social media discussions.

Methodology
The authors present AutoJourn, an integrated, end-to-end demonstration system that connects multi-perspective extraction, summarization, controlled news generation, and bias evaluation. The system pipeline operates as follows:

1. Input & Topic Modeling: Given raw social media discussions (e.g., Reddit threads), the system employs a prompt-based topic modeling pipeline. Using structured zero-shot prompts, an LLM identifies the top five semantically coherent topics, estimates their proportions, and lists representative keywords.
2. Multi-Perspective Extraction: The system extracts distinct viewpoints (Agree/Disagree stances) for each topic. This is achieved using a GPT-4o model prompted to generate concise, criteria-based justifications. To enhance diversity and grounding, the system optionally utilizes a Retrieval-Augmented Generation (RAG) mechanism, retrieving semantically related documents via TF-IDF and sentence-embedding reranking to provide additional argumentative context.
3. Multi-Perspective Summarization: The extracted perspectives are converted into three types of summaries:
   • Stance-specific: Separate summaries for "Agree" and "Disagree" positions.
   • Merged: A balanced summary synthesizing both viewpoints using neutral language, emphasizing equal representation and the inclusion of evidence from both sides while removing emotionally charged phrases.
4. News Article Generation: Full news articles are generated conditioned on a selected topic and the corresponding merged multi-perspective summary. The system uses GPT-4o to expand these summaries into coherent narratives, allowing for the generation of balanced or neutral articles.
5. Bias Detection & Neutralization:
   • Detection: A sentence-level bias analysis module utilizes a fine-tuned RoBERTa classifier (trained on the BABE dataset) to detect biased sentences and assign bias probabilities. A complementary classifier (GusNet) categorizes bias types (political, demographic, socioeconomic).
   • Neutralization: A two-stage mitigation pipeline rewrites biased sentences. First, fine-tuned encoder-decoder models (BART or T5) attempt to rewrite sentences into neutral formulations. If these fail, a fallback GPT-based rewrite is triggered.

Key Contributions

• Unified Framework: AutoJourn is presented as the first prototype to operationalize perspective extraction, multi-perspective summarization, controlled news generation, and bias evaluation jointly within a single demonstrable system for automated journalism.
• Interactive Interface: The system provides a user-facing interface where users can inspect perspective clusters, compare stance-specific summaries, generate news articles, and apply bias-aware rewrites directly. It includes features for highlighting biased sentences with confidence scores and inserting neutralized alternatives.
• Comprehensive Reporting: The system generates a full PDF report aggregating source conversations, extracted perspectives, summaries, generated articles, bias annotations, and neutralized versions.

Results
The system was evaluated using intrinsic metrics across four axes:

• Perspective Extraction: Evaluated on SocialChem-101 and CMV datasets using semantic diversity metrics (pairwise cosine distances). The GPT-4o variants achieved the highest semantic diversity scores among reported models. Interestingly, the RAG-augmented variant showed slightly lower raw diversity than the designed prompting alone, leading the authors to use the prompt-only approach in the demo.
• Summarization: Evaluated via an "LLM-as-judge" protocol for faithfulness and coverage, and surface-level fluency scores (Grammarly). Merged summaries showed slightly lower fluency than stance-specific summaries, a trade-off accepted to ensure balanced representation.
• Bias Detection: The system integrates existing detectors to provide transparent, interpretable bias maps over generated articles.
• Bias Neutralization: Evaluated on a held-out WNC split and test sets from BABE and generated news. The BART + LLM fallback pipeline achieved the highest bias reduction (91.5%) with only a minor drop in lexical similarity (ROUGE-L ≈ 0.96), confirming the utility of the fallback mechanism for difficult cases.

Significance and Claims
The paper claims that AutoJourn offers a practical path toward more transparent and socially accountable applications of LLMs in news production. By combining interpretability, viewpoint diversity, and bias mitigation, the system allows users to trace how viewpoints propagate from source discussions into generated narratives and assess the bias of those narratives through explicit, model-driven analysis. The authors position the work not as a definitive benchmark for every sub-module, but as a demonstration of how these components can be jointly integrated to support journalists and researchers in exploring responsible automated journalism. The system is available as a live, publicly accessible demo to facilitate reproducibility and further research.