Dripper: Token-Efficient Main HTML Extraction with a Lightweight LM

This paper introduces Dripper, a lightweight framework that reformulates web main content extraction as a constrained sequence labeling task using Small Language Models, achieving superior efficiency and accuracy compared to both traditional heuristics and massive generative LLMs while enabling the creation of high-quality training corpora.

The Gist

Imagine the internet is a massive, chaotic library. Every day, billions of new books (web pages) are added. But here's the problem: most of these "books" are wrapped in layers of sticky tape, plastic wrapping, and cardboard boxes (ads, pop-ups, navigation menus, and code) that make it impossible to read the actual story inside.

To train smart AI, we need to read the stories, not the packaging. But reading billions of pages one by one is slow, expensive, and often leads to mistakes.

Enter Dripper. Think of Dripper as a super-efficient, high-speed librarian robot that knows exactly how to unwrap these packages without tearing a single page of the story.

Here is how it works, broken down into simple concepts:

1. The Problem: The "Heavy Box" Dilemma

Traditional tools try to read the whole box (the raw HTML code) to find the story. This is like trying to find a specific sentence in a book by reading the entire cardboard box it came in first. It's slow, and the AI gets confused by all the extra stuff.

On the other hand, the "super-smart" AI models (like the giants in the industry) are great at understanding context, but they are like heavyweight champions. They are so expensive to run and so slow that you can't use them to process the entire internet. Plus, they sometimes "hallucinate"—they might make up a story that wasn't in the book at all.

2. The Solution: The "Dual-Branch" Strategy

Dripper solves this with a clever two-step trick, like a sneaky spy and a faithful scribe working together.

• Step A: The Spy (Simplified HTML)
  First, Dripper takes the messy web page and strips it down to its bare bones. It removes the ads, the fancy fonts, and the hidden code, leaving only the skeleton of the page. It's like taking a complex machine and removing all the paint and bolts to just look at the gears. This "skeleton" is tiny and easy to read.

  • The Result: A small, lightweight AI (called Dripper-0.6B) looks at this skeleton and quickly decides: "Is this part of the story? Yes or No?" It doesn't rewrite the story; it just puts a sticky note on every block saying "KEEP" or "TRASH."

• Step B: The Scribe (Mapping HTML)
  While the Spy is making those quick decisions, the Scribe is holding the original, pristine version of the page. Once the Spy says, "Keep block #2," the Scribe grabs the exact original text from block #2.

  • The Result: Because the Scribe never rewrote anything, the final story is perfect. No typos, no missing words, and no made-up facts.

3. Why It's a Game Changer

• Speed: Because the "Spy" only looks at a tiny skeleton, it can process 3 pages every second on a single computer chip. That's like reading a whole library in the time it takes to brew a cup of coffee.
• Accuracy: By using a "Yes/No" labeling system instead of asking the AI to "write the story," Dripper stops the AI from making things up (hallucinations). It's like asking a guard to point at the right door rather than asking the guard to describe the room behind it.
• The "Magic" Benchmark: The team built a new test called WebMainBench (like a final exam for web scrapers). Dripper scored higher than the old, slow tools and came very close to the massive, expensive "super-AI" models, but at a fraction of the cost.

4. The Big Picture: Why Does This Matter?

Think of AI training data as the food for a giant robot brain. If the food is full of plastic wrappers (ads and junk code), the robot gets sick and learns bad habits. If the food is clean and nutritious (pure text), the robot gets smarter.

The authors proved that by using Dripper to clean the data, the resulting AI models actually learned better. They became smarter at reasoning and answering questions than models trained on data cleaned by older, slower methods.

Summary Analogy

Imagine you are trying to copy a recipe from a magazine, but the magazine is full of ads, celebrity gossip, and colorful pictures.

• Old Tools: Try to copy the whole page, getting confused by the ads.
• Big AI: Reads the whole page perfectly but takes 10 minutes to copy one recipe and costs $100.
• Dripper: Quickly scans the page, puts a green dot on the recipe and a red dot on the ads (taking 1 second), and then you just copy the parts with the green dots. It's fast, cheap, and the recipe comes out perfect.

Dripper is the tool that makes the internet readable for the next generation of AI, ensuring they learn from the best parts of human knowledge without the noise.

Technical Summary

Here is a detailed technical summary of the paper "Dripper: Token-Efficient Main HTML Extraction with a Lightweight LM".

1. Problem Statement

Constructing large-scale training corpora for Large Language Models (LLMs) requires converting raw, unstructured HTML from the web into high-quality, structured text. Current extraction methods face a critical trade-off between efficiency and accuracy:

• Traditional Heuristic/Rule-based Methods (e.g., Trafilatura, Readability): These are computationally efficient but lack semantic reasoning. They struggle with the structural heterogeneity of modern web pages (e.g., inconsistent semantic tags, visual layouts prioritized over structure, and complex boilerplate like ads or sidebars), leading to truncated text or corrupted data.
• Generative Large Language Models (LLMs): While they offer superior semantic understanding, they are impractical for web-scale extraction due to:
  • Excessive Computational Cost: Processing raw HTML often exceeds context windows (29.3% of pages exceed 32k tokens; 21% exceed 128k tokens).
  • Hallucinations: Generative models may fabricate content not present in the source.
  • Latency: High inference costs make them unsuitable for processing billions of pages.

2. Methodology: The Dripper Framework

Dripper addresses these bottlenecks by reframing content extraction as a constrained sequence labeling task using a Small Language Model (SLM). The framework employs a dual-branch representation strategy to decouple semantic reasoning from content reconstruction.

A. Dual-Branch Processing Pipeline

1. Pre-processing (Dual-Branch Generation):

   • Simplified HTML Branch: The raw HTML undergoes aggressive pruning (removing <script>, <style>, <header>, etc.) and attribute reduction (keeping only class and id). Inline tags are aggregated into semantic blocks. This reduces the input token count by ~84.4% (from ~44.6k to ~7k tokens on average), enabling a lightweight model to process long documents within a standard context window.
   • Mapping HTML Branch: This branch retains the original DOM fidelity (structure and content) but undergoes the same block segmentation logic as the Simplified branch. It serves as the source for reconstructing the final output.
   • Alignment: Both branches are strictly aligned block-by-block via unique _item_id attributes.

2. Extraction (Constrained Sequence Labeling):

   • The Dripper-0.6B model (based on Qwen3-0.6B) ingests only the Simplified HTML.
   • Instead of generating text, the model performs binary classification on each block, labeling them as either "main" (content) or "other" (boilerplate).
   • Constrained Decoding: The model is forced to output a strict JSON format (e.g., {"1": "main", "2": "other"}) using regex-based constraints during inference. This eliminates generative hallucinations, prevents repetitive loops, and ensures syntactic validity.

3. Post-processing (Reconstruction):

   • The predicted labels are mapped back to the corresponding blocks in the Mapping HTML.
   • Only blocks labeled "main" are retained to reconstruct the final high-fidelity HTML, which is then converted to clean Markdown.

B. Training Data & Benchmark

• Training Data: Curated from 107 Common Crawl snapshots using layout-aware clustering (DBSCAN) to ensure structural diversity. The final dataset contains 986,442 samples with block-level labels generated by DeepSeek-V3.2 and verified.
• WebMainBench: A new, rigorous benchmark of 7,809 human-annotated pages covering 5,434 unique domains and 46 languages. It features multi-round human annotation and metadata (difficulty, content type) to evaluate extraction quality precisely.

3. Key Contributions

1. Novel Extraction Paradigm: Reformulates extraction as a token-efficient sequence labeling task using SLMs, eliminating generative hallucinations and achieving 3.08 pages/second on a single A100 GPU.
2. WebMainBench Benchmark: Establishes a new gold standard for evaluation, outperforming legacy benchmarks in scale, diversity, and annotation rigor.
3. Infrastructural Value: Demonstrates that high-quality extraction directly improves foundation model training. A 1B model pre-trained on a 63B-token corpus curated by Dripper outperformed baselines trained on data from other tools.
4. Open Source: Released the Dripper-0.6B weights, codebase, and the WebMainBench dataset to facilitate high-quality dataset construction.

4. Experimental Results

• Accuracy: Dripper-0.6B achieves an F1 score of 0.8779 on WebMainBench.
  • Significantly outperforms traditional heuristics (e.g., Trafilatura: 0.6402, magic-html: 0.7138).
  • Rivals massive frontier models (DeepSeek-V3.2: 0.9098, GPT-5: 0.9024) despite having only 0.6B parameters.
  • The Dripper_fallback variant (switching to heuristics for oversized inputs) reaches 0.8925, narrowing the gap to GPT-5 to <1%.
• Efficiency:
  • Input Reduction: Reduces input tokens by 84.4%.
  • Compute Reduction: Lowers estimated FLOPs by 95.6% (22.6x faster) compared to a naive generative baseline processing raw HTML.
• Downstream Impact: Models pre-trained on Dripper-curated data showed a 1.08% absolute improvement in average accuracy across 13 downstream benchmarks (MMLU, ARC, etc.) compared to models trained on Trafilatura-processed data, surpassing even established corpora like RefinedWeb and FineWeb.

5. Significance

Dripper bridges the gap between the speed of heuristic tools and the semantic intelligence of LLMs. By proving that a lightweight, specialized model can approximate the performance of massive frontier models when the task is properly reformulated, it offers a scalable, cost-effective solution for the next generation of AI data pipelines. The framework highlights that extraction quality is a critical infrastructure component for training better foundation models, not just a preprocessing step.