Technical Summary: A Multi-View Media Profiling Suite

Problem Definition

The automated detection of political bias and factuality in news outlets is critical for shaping public opinion, yet the field suffers from a lack of unified resources, comprehensive evaluations across diverse approaches, and systematic analysis of representation and fusion strategies. Existing methods often rely heavily on textual content, which can be noisy, difficult to crawl at scale, and expensive to collect. While recent work has explored alternative signals like infrastructure metadata and social context, graph-based approaches frequently suffer from structural sparsity (e.g., disconnected components) that limits representation learning. Furthermore, prior evaluations are often tied to single graph views and benchmarks, leaving open questions regarding which signals and fusion strategies are most reliable under missing or noisy modalities.

Methodology

The authors propose a multi-view profiling suite that integrates three graph views and two textual views to characterize news outlets.

1. Data Construction and Resources

• MBFC-2025: The authors introduce a new large-scale benchmark containing approximately 2,600 outlets labeled by Media Bias/Fact Check (MBFC). Unlike the previous ACL-2020 dataset which used a 3-point scale, MBFC-2025 utilizes a 5-point scale for both political bias (Left, Left-Center, Center, Right-Center, Right) and factuality (Very High, High, Mixed, Low, Very Low).
• Multi-View Representations: For both the ACL-2020 (~900 outlets) and MBFC-2025 datasets, the suite constructs five distinct representations:
  • Alexa Graph: Captures audience overlap based on similar site recommendations.
  • Hyperlink Graph: Derived from crawling external links on outlet homepages and articles, reflecting inter-outlet connections.
  • LLM-Derived Graph: Generated by prompting an LLM (GPT-3.5) to identify five semantically similar websites for a given outlet, encoding semantic similarity.
  • Articles: Textual content capturing tone and framing.
  • Wikipedia Descriptions: Contextual information about the outlets.

2. Modeling Framework

The approach formulates media profiling as a multi-modal classification problem where political bias and factuality are predicted independently.

• Graph Neural Networks (GNNs): Three architectures are employed to learn structural embeddings from the graphs: Residual Gated Graph Convolutional Networks (ResGatedGCN), Graph Convolutional Networks (GraphConv), and GraphSAGE. These are trained using a contrastive objective in an unsupervised setting.
• Textual Models: Pretrained Language Models (PLMs) including BERT, RoBERTa, DistilBERT, and ALBERT are fine-tuned on articles and Wikipedia descriptions. Large Language Models (LLMs) like Llama-2, Mistral, and GPT-4o are also evaluated using few-shot prompting and voting strategies (hard/soft) on summarized articles.
• Fusion Strategies: The study evaluates various methods to combine embeddings:
  • Static Fusion: SVM, Multi-Layer Perceptron (MLP), Self-Attention, and Cross/Co-Attention mechanisms.
  • RL-Based Dynamic Fusion: A novel approach casting fusion as a contextual bandit problem. A reinforcement learning agent (using Proximal Policy Optimization) learns outlet-specific weights for each view to dynamically maximize the probability of the true label, addressing the limitations of static weighting in noisy or incomplete multi-view settings.

Key Contributions

1. MBFC-2025 Benchmark: A large-scale, fine-grained dataset of ~2,600 outlets with 5-point scale labels for political bias and factuality, addressing the scarcity of high-granularity resources.
2. Multi-View Resources: The compilation of five-view representations (three graphs, two text sources) for both ACL-2020 and MBFC-2025, enabling systematic comparison of different signal types.
3. Systematic Evaluation and Analysis: A comprehensive analysis of embedding views and fusion strategies, including the first application of Reinforcement Learning (RL) for multi-view fusion in bias and factuality detection.
4. State-of-the-Art Results: Establishment of new benchmarks on MBFC-2025 and improved performance on ACL-2020, demonstrating the efficacy of dynamic fusion strategies.

Experimental Results

• Political Bias: On the ACL-2020 dataset, the RL-based dynamic fusion strategy achieved a Macro-F1 of 97.60%, outperforming the best static baseline (SVM with combined views) at 95.53%. On the more challenging MBFC-2025, the RL agent achieved a Macro-F1 of 78.00%, significantly surpassing static baselines (69.29%) and ensemble methods (72.41%).
• Factuality: While text-only models (specifically ALBERTBase with hard voting) remained dominant for factuality on MBFC-2025 (71.74% Macro-F1), the RL-based fusion achieved a Macro-F1 of 55.30%, performing on par with other fusion techniques like Co-attention (56.94%). On ACL-2020, the RL approach achieved 76.80% Macro-F1.
• View Analysis: The Alexa graph was identified as the most effective single view due to broad coverage and informative node features. Hyperlink graphs provided complementary signals, while LLM-derived graphs were found to be the weakest due to missing explicit node features and limited structural information.
• Task Difficulty: Empirical findings confirm that political bias is easier to detect than factuality, as bias is more explicit in language, whereas factuality requires deeper contextual understanding.

Significance and Claims

The paper claims that effective outlet profiling depends on reliable representations that go beyond noisy textual content. By introducing MBFC-2025 and a multi-view suite, the authors address the gap in unified resources and systematic analysis. The primary significance lies in demonstrating that dynamic fusion strategies, specifically those driven by Reinforcement Learning, outperform static combination methods, particularly in complex, multi-view settings where signals vary in reliability. The work establishes that while single-view graph representations are suboptimal, combining multiple views captures diverse network dependencies, though adding too many representations can lead to diminishing returns. The authors position this work as a starting point for source-level profiling, noting that future research should extend to article- and claim-level prediction and multilingual contexts.

Limitations

The authors acknowledge several constraints:

• Data Coverage: Some websites could not be parsed, and the dataset remains U.S.-centric, potentially underrepresenting other cultural or political perspectives.
• Graph Construction: Hyperlink and LLM graphs lack rich node attributes, forcing the use of dummy features which may have constrained performance.
• Computational Constraints: The study prioritized efficient models and did not explore fine-tuning of larger LLMs or advanced mixture-of-expert architectures due to time and memory limitations.
• Scope: The framework is designed for research analysis and transparency, not for content filtering or personalization, and findings should be interpreted with caution when generalizing beyond the studied sources.