Traceable by Design: An LLM Pipeline and Dashboard for EU Regulatory Consultation Analysis
This paper presents a traceable, LLM-based pipeline and interactive dashboard that automates the analysis of large-scale regulatory consultation submissions by extracting grounded topic annotations with verbatim evidence, demonstrated through a case study on the European Commission's Digital Fairness Act that revealed insights beyond predefined taxonomies.
Original paper dedicated to the public domain under CC0 1.0 (http://creativecommons.org/publicdomain/zero/1.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine the European Union is planning a massive new rulebook for the internet, called the Digital Fairness Act. Before they write the final rules, they ask the public: "What do you think?" This is called a "public consultation."
The problem? People sent in 4,322 responses. Some were short notes typed into a web form; others were long, multi-page PDF documents. Trying to read and understand all of these by hand is like trying to drink from a firehose—it's impossible for a human to do without missing things or getting overwhelmed.
This paper introduces a smart digital assistant (an AI pipeline and a dashboard) designed to read all those responses, figure out what people are talking about, and prove exactly where it found that information.
Here is how the system works, explained simply:
1. The "Smart Librarian" Pipeline
Think of the system as a highly organized, super-fast librarian who never gets tired.
- The Input: The librarian takes in two types of books: the messy, scanned PDFs (which need to be "read" using a special camera called OCR) and the clean, typed web forms.
- The Cleaning: Before reading, the librarian sweeps the floor. They remove page numbers, headers, footers, and gibberish text that often happens when scanning documents. They also check if a page is actually readable. If a page is too messy, they flag it but still try to process the rest.
- The Chopping: The librarian cuts the long documents into small, manageable "paragraph chunks." They only keep the chunks that make sense and have enough words to be useful.
- The Reading (The AI Part): This is where the "Large Language Model" (the brain of the operation) steps in. It reads each clean paragraph and asks two questions:
- What topic is this about? (e.g., "Is this about unfair subscriptions?")
- What is the exact sentence that proves it?
2. The Golden Rule: "Show Your Work"
Most AI tools summarize things. If you ask them, "What did people say about passwords?" they might say, "People are worried about security."
This system refuses to summarize. It follows a strict rule called Verbatim Grounding.
- The Analogy: Imagine a detective in a courtroom. If the detective says, "The suspect was at the scene," the judge demands to see the photo evidence. The detective cannot just say, "I think so."
- How it works here: Every time the AI finds a topic, it must copy and paste the exact sentence from the original document that proves it. It cannot make up a summary. If it can't find an exact quote, it doesn't make a claim. This ensures that if a policymaker wants to check the work, they can click a button and see the original sentence in the original document.
3. The Dashboard: The "Control Room"
The results are displayed on a website (the dashboard) that acts like a control room for policymakers.
- The Big Picture: You can see a map of all the topics. Some are "Predefined" (topics the EU already knew about, like "Dark Patterns"). Others are "Emergent" (new topics the AI found that the EU didn't explicitly ask about, like "Age Verification" or "Digital Ownership").
- The Drill-Down: You can click on a specific topic (like "Payment Processor Censorship") and see exactly which companies or countries mentioned it.
- The Traceability: If you see a statistic on the screen, you can trace it back through the system to the specific paragraph in the original PDF where it came from. Nothing is hidden.
4. Why This Matters
The authors tested this on the Digital Fairness Act data.
- The Result: The system processed 4,322 documents and found 15,368 specific topics, backed up by 20,951 exact quotes.
- The Surprise: Because the system wasn't forced to stick to a rigid list of topics, it found new concerns that a human using a simple checklist would have missed. For example, it noticed people were worried about "Payment Processor Censorship" (banks blocking payments to certain sites) and "Digital Ownership" (who actually owns their digital items).
- The Flexibility: The system is "domain-generic." This means if the EU wants to ask about a different law next year, they don't need to rebuild the machine. They just change the "instruction manual" (the prompt) and feed it new documents.
Summary
This paper describes a tool that turns a chaotic mountain of public feedback into a clear, organized, and provable list of concerns. It doesn't just tell you what people think; it shows you exactly where they said it, ensuring that the decision-makers can trust the evidence because they can see the source code of the conversation.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.