Identifying the Geographic Foci of US Local News

This paper proposes a novel geo-foci model that combines manually labeled data, Large Language Model-based entity disambiguation, and spatial-semantic feature engineering to accurately identify the central geographic locations in US local news articles, achieving an F1 score of 0.86.

The Gist

Imagine you are a librarian trying to organize a massive, chaotic pile of newspapers. Some stories are about the high school football game down the street (local), some are about the governor's new tax plan (state), some are about the President's speech (national), and some are about a war in a distant country (international).

The problem is that local newspapers are getting squeezed for money. To survive, they are starting to print more national and international stories, pushing out the local news that actually matters to the people living there. The researchers in this paper wanted to answer a big question: "Are these local newspapers still talking to their own neighborhoods, or have they gone global?"

To find out, they built a smart computer assistant called NLGF (News Lab Geo-Focus). Here is how it works, broken down into simple steps:

1. The "Name Game" (Toponym Disambiguation)

First, the computer has to read the news and find all the place names. But here's the tricky part: Names are confusing.

• If the paper says "Paris," is it talking about the romantic city in France, or the small town in Texas?
• If it says "Springfield," is that the one in Illinois, or the one in Missouri?

In the past, computers used rigid rulebooks (like a dictionary) to guess, and they often got it wrong. In this study, the researchers tried using Large Language Models (LLMs)—which are like super-smart AI chatbots trained on the whole internet.

• The Analogy: Imagine a traditional rulebook is like a tourist with a map who only knows major cities. An LLM is like a local guide who knows that "Paris, Texas" is the one mentioned in a story about a county fair, while "Paris, France" is mentioned in a story about fashion.
• The Result: The AI "local guides" (LLMs) were much better at figuring out exactly which place was being talked about than the old rulebooks.

2. The "Spotlight" (Feature Engineering)

Once the computer knows which places are mentioned, it needs to figure out which one is the main character of the story.

• The Analogy: Think of a news article as a stage play. Just because a character walks on stage doesn't mean they are the star.
  • Frequency: How many times does the name appear? (Is it mentioned in every scene?)
  • Position: Is the name in the headline or the first sentence? (Stars usually get the top billing).
  • Context: Is the name surrounded by other local clues?
• The researchers taught the computer to look for these "spotlights." They created a scoring system that gives points to places mentioned early, often, or in the title.

3. The "Judge" (Classification)

Finally, the computer acts as a judge. It looks at all the clues (the disambiguated names and the spotlight scores) and decides: "What is the main geographic focus of this article?"

• Is it Local? (A story about the town council).
• Is it State? (A story about the whole state's education budget).
• Is it National? (A story about federal laws).
• Is it International? (A story about a foreign election).
• Or is it None? (A story about a scientific discovery that could happen anywhere).

Why This Matters

The researchers tested their new "Judge" (NLGF) against two other methods: a popular AI chatbot (GPT-4o) and an old-school rule-based system (Cliff-Clavin).

• The Scoreboard: NLGF won with flying colors (scoring 0.86 out of 1.0), while the others struggled, especially when trying to tell the difference between "State" news and "Local" news.
• The Takeaway: The old rule-based systems were like a robot that just counts how many times a word appears. The AI chatbot was smart but sometimes got lost without a specific map. NLGF was the winner because it combined the AI's ability to understand context with a specific set of rules about where and how places are mentioned in the text.

The Big Picture

This tool is like a thermometer for local democracy. By automatically scanning thousands of local newspapers, researchers can now see if local news is actually staying local or if it's slowly turning into a national news feed.

If the "thermometer" shows that local papers are talking more about Washington D.C. or foreign wars and less about the local school board or the new coffee shop opening, it's a warning sign. It tells us that the community might be losing its voice. This tool helps researchers, journalists, and citizens measure that shift and understand what information is being lost in the process.

Technical Summary

1. Problem Statement

Local journalism is essential for democratic societies, yet economic pressures have led to the "nationalization" of local news, where outlets prioritize state, national, or international stories over local community issues. A critical gap exists in the ability to automatically and accurately identify the geographic focus (geo-foci) of local news articles.

• The Challenge: Determining whether an article is primarily about a specific city/county (local), a state, the nation, or international events.
• Complexity: Local news often contains ambiguous geographic entities (e.g., "Paris" could be Texas or France) and mixes multiple geographic scopes within a single article. Existing methods often lack generalizability across different writing styles and fail to capture fine-grained spatial-semantic clues.

2. Methodology: The NLGF Model

The authors propose the News Lab Geo-Focus (NLGF) model, a hybrid pipeline designed to detect both the geo-focus level (administrative division) and the specific geo-foci (named locations) of US local news articles. The pipeline consists of four main stages:

A. Data Collection and Annotation

• Dataset: A manually annotated dataset of 1,250 US local news articles, evenly distributed across five classes: Local, State, National, International, and None.
• Annotation: Two domain experts labeled each article with its primary geo-focus level and specific geographic entities (toponyms).
• Reliability: High inter-rater reliability was achieved (Cohen's $\kappa$ = 0.83 for levels; Krippendorff's $\alpha$ = 0.81 for entities).

B. Toponym Recognition and Disambiguation

• Recognition: Used spaCy to extract Geographic Political Entities (GPEs), Locations (LOCs), and Facilities (FACs).
• Disambiguation: The authors evaluated six traditional geoparsers (e.g., Mordecai3, Edinburgh Geoparser) against three Large Language Models (LLMs): GPT-4o, LLaMA2-7b, and Phi-3.
  • Result: Traditional tools performed inconsistently, especially on ambiguous entities (Precision < 0.5). GPT-4o significantly outperformed all others (F1: 0.948 for GPEs), correctly resolving entities like "Paris, Texas" vs. "Paris, France" by leveraging context and publisher location.
• Initial Geo-focus Level (IGL): Each disambiguated toponym was assigned an IGL based on the publisher's location (e.g., a county within the publisher's state = Local; a foreign country = International).

C. Feature Engineering

The authors engineered 15 spatial-semantic features to train a classifier, capturing the prominence and distribution of geographic entities:

1. Toponym Counts in Titles: Frequency of toponyms in the headline.
2. Distributions of GPE IGLs: Counts of GPEs categorized by their assigned IGL (local, state, national, international).
3. Positional Emphasis: Counts of "leading toponyms" (appearing in the first five mentions) for each IGL.
4. Toponym Diversity: Unique geographic identifiers per IGL.

D. Classification and Identification

1. Geo-focus Level Classification: An XGBoost classifier (gradient-boosted decision trees) was trained on the 15 features to predict the article's primary geo-focus level.
2. Geo-foci Identification (Algorithm 1): Once the level is predicted, a heuristic scoring algorithm identifies the specific locations:
   • Filters toponyms matching the predicted IGL.
   • Calculates a focus_score based on: frequency in title + frequency in body + leading position count + GPE recognition count.
   • Normalizes scores and applies a threshold ($\alpha = 0.25$) to select the final geo-foci.

3. Key Contributions

1. Expert-Annotated Dataset: Creation of a balanced, high-quality benchmark dataset of 1,250 US local news articles with multi-label geo-foci and single-label geo-focus levels.
2. LLM Superiority in Disambiguation: Demonstrated that LLMs (specifically GPT-4o) significantly outperform traditional geoparsers in resolving ambiguous toponyms, particularly for Facilities and Locations.
3. Spatial-Semantic Feature Set: Designed a novel set of features that capture not just the presence of locations, but their contextual prominence (position in text, title usage, and hierarchy).
4. Open-Source Hybrid Model: Released the NLGF model, which combines LLM-based disambiguation with a traditional ML classifier and heuristic scoring, achieving state-of-the-art performance.

4. Results

The NLGF model was compared against two baselines: GPT-4o (prompted directly for classification) and Cliff-Clavin (a widely used frequency-based heuristic).

• Geo-focus Level Classification:

  • NLGF: Macro-averaged F1 = 0.89.
  • GPT-4o: F1 = 0.75.
  • Cliff-Clavin: F1 = 0.62.
  • Analysis: NLGF excelled in distinguishing "State" and "National" levels, where spatial cues (like publisher location) are critical. GPT-4o struggled with these boundary cases due to a lack of persistent spatial grounding.

• Geo-foci Identification:

  • NLGF: Precision 0.86, Recall 0.89, F1 = 0.86.
  • GPT-4o: F1 = 0.66.
  • Cliff-Clavin: F1 = 0.37.
  • Analysis: The hybrid approach of combining disambiguation with spatial-semantic features yielded significantly higher accuracy than pure LLM prompting or frequency-based heuristics.

• Error Analysis: The most common confusion occurred between "Local," "State," and "National" due to the inherent semantic overlap in news narratives (e.g., a local story with national implications). However, NLGF handled these better than baselines.

5. Significance and Future Work

• Computational Journalism: NLGF provides a scalable tool to quantify the "nationalization" of local news, allowing researchers to track shifts from local to national narratives over time.
• Generalizability: While currently focused on the US, the framework is adaptable to other countries.
• Future Improvements:
  • Moving from single-label to multi-label classification for geo-focus levels (as articles can cover multiple scopes simultaneously).
  • Expanding the framework to non-US contexts.

Conclusion: The paper establishes that a hybrid approach—leveraging LLMs for robust entity disambiguation and traditional machine learning for spatial-semantic classification—is the most effective method for identifying the geographic focus of local news, outperforming both pure LLM prompting and traditional rule-based systems.