The Course of News Events: A Comparison of Bottom-Up and Top-Down Approaches for Collecting Text-Based Data about Disasters

Technical Summary: The Course of News Events

Problem Statement
In socio-environmental studies, news articles serve as a critical source for understanding disaster impacts and adaptation. However, a fundamental methodological challenge lies in selecting a representative data sample. Researchers typically rely on one of two approaches: a top-down method, which queries news databases using existing disaster inventories (e.g., EM-DAT) as ground truth, or a bottom-up method, which uses Natural Language Processing (NLP) to cluster news texts based on temporal and spatial features. Both approaches possess inherent limitations: top-down methods overlook events not recorded in inventories, while bottom-up methods may capture events deemed unnewsworthy by specific media or generate false positives. This paper investigates how these design decisions influence the resulting news sample, particularly regarding event coverage, which affects studies on media inequality, disaster monitoring, and inventory enrichment.

Methodology
The study utilizes a dataset of approximately 55,000 German news articles concerning landslides worldwide, collected between 2000 and 2024. The analysis compares two event matching strategies:

1. Bottom-Up Approach: Based on prior work by Madureira et al. (2026), this method identifies 4,567 news events across 152 countries. A "news event" is defined as a sequence of articles referring to landslides in the same country, starting on the first day of coverage and ending after $\theta = 5$ consecutive days without coverage.
2. Top-Down Approach: This method extracts 2,014 landslide events from the EM-DAT inventory (covering 138 countries). It queries the news database for articles within a specific time window relative to the EM-DAT onset date.

Event Matching Procedure
To compare the two, the authors employ a temporal alignment strategy (illustrated in Figure 2) using parameters $\Delta_b$ (days before) and $\Delta_a$ (days after):

• Bottom-Up Alignment: An EM-DAT entry is considered aligned if its onset date falls within $[\Delta_b, \Delta_a]$ of the news event's start. For this approach, $\Delta_b = 5$ and $\Delta_a = 1$, acknowledging that news may report an event slightly before the official onset date but rarely after the first report.
• Top-Down Query: For each EM-DAT entry, the system queries the news database for articles in the window $[\Delta_b, \Delta_a]$. Here, $\Delta_b = 1$ and $\Delta_a = 5$, allowing for a delay in international news reporting and accounting for potential errors in onset dates.

Key Results
The comparison reveals significant non-bijective overlaps between the two datasets:

• Coverage Discrepancies: The bottom-up approach identified more than twice as many events (4,567) as the EM-DAT inventory (2,014). However, only 16.1% (737) of bottom-up news events aligned temporally with an EM-DAT entry. Conversely, top-down queries successfully retrieved news for 42.2% (851) of EM-DAT entries.
• Mutual Exclusivity:
  • 57.7% of EM-DAT events had no relevant news coverage in the German database (empty queries).
  • 82.9% of bottom-up news events did not align with any EM-DAT entry.
• Spatial and Economic Bias: The resulting samples differ significantly in distribution. The bottom-up approach assigned 54.3% of events to the Global South, whereas top-down queries assigned 81.4% to the Global South. Conversely, 47.7% of bottom-up events referred to high-income countries compared to 20.6% for top-down queries.
• Content Validity: Manual verification of 150+ bottom-up events revealed that many do not represent specific, ongoing disasters. Categories included "concrete but past" (memories, judicial decisions), "indefinite" (general warnings, studies), "vague" (fictional/rhetorical), and "false positives" (misclassified geolocation or figurative language).

Significance and Contributions
The paper does not claim that one approach is superior; rather, it argues that they are complementary and that the choice of method fundamentally shapes research outcomes.

• Methodological Insight: The study demonstrates that relying solely on top-down queries misses the majority of news events (those not in inventories), while relying solely on bottom-up clustering introduces significant noise and captures general topic discussions rather than specific disaster occurrences.
• Bias Quantification: The results highlight that media coverage and inventory inclusion are not random. EM-DAT focuses on severe events, while media attention is influenced by population density, economic relevance, and public interest.
• Best Practices: The authors conclude that automated extraction cannot eliminate false positives. Therefore, diagnostic analyses are imperative to expose shortcomings in big data paradigms. Researchers must make informed choices about which sample to use based on their specific goals: bottom-up approaches are suitable for studying media attention patterns, while top-down approaches are better for enriching existing disaster databases with impact data, provided the inventory's inherent biases are acknowledged.