SPIRE: Structure-Preserving Interpretable Retrieval of Evidence
SPIRE is a structure-preserving retrieval framework for semi-structured HTML documents that improves citation quality and interpretability by indexing tree-structured subdocuments and applying global and local contextualization mechanisms, outperforming traditional flat-chunk baselines in question-answering tasks.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are trying to find a specific fact in a massive library, but the librarian has a strange rule: they only give you single sentences ripped out of books, with no page numbers, no chapter titles, and no context.
If the sentence is "It was the best of times," you have no idea if it's from A Tale of Two Cities, a history book, or a recipe. If the sentence is "The answer is 42," you don't know what question it's answering. This is exactly the problem current AI search engines (RAG) face when dealing with complex web pages (HTML). They flatten everything into a long, messy list of text chunks, losing the structure that makes the information make sense.
SPIRE (Structure-Preserving Interpretable Retrieval of Evidence) is a new system designed to fix this. Here is how it works, using simple analogies:
1. The Problem: The "Flat" Library
Current AI systems treat a website like a long strip of tape. They cut the tape into equal-sized pieces (chunks) to feed to the AI.
- The Issue: If you cut a piece of tape right in the middle of a list, a table, or a sentence, the piece becomes useless. A table cell without its row and column headers is like a puzzle piece with no picture on it. A bullet point without the heading above it is confusing.
2. The Solution: The "Tree" Map
SPIRE treats documents not as a strip of tape, but as a family tree.
- The Concept: Every piece of text has a specific "address" (a path) in the tree. It knows exactly which "room" (section) it is in, which "parent" (heading) it belongs to, and which "siblings" (other list items) are nearby.
- The Benefit: Instead of grabbing a random chunk, SPIRE grabs a specific "node" in the tree and knows exactly how to reconstruct its surroundings.
3. The Two-Step Magic Trick
SPIRE uses a clever two-step process to give you the perfect answer without wasting space.
Step A: The "Global" Context (The Frame)
When the system finds a relevant sentence, it doesn't just show you the sentence. It first asks: "What is the frame around this picture?"
- Analogy: Imagine finding a photo of a person in a crowd. If you just show the person, you don't know who they are. SPIRE automatically adds the "frame": the Title of the page, the Section Header (like "Key Takeaways"), and the List Structure.
- Why it matters: This ensures that even if you only read one sentence, you know where it came from and what category it belongs to. It's like putting a label on a jar so you know it's "Jam" and not "Jelly."
Step B: The "Local" Context (The Neighborhood)
Once the system has the framed sentence, it looks at the immediate neighborhood.
- Analogy: If you are looking for a specific house on a street, you don't just want the house number; you want to see the house and the two houses next to it to make sure you are in the right block.
- The Filter: SPIRE expands the view to include the surrounding paragraph or list items. Then, it uses a smart AI "editor" to trim away the fluff. It keeps only the parts that actually help answer your question, ensuring the final answer is compact but complete.
4. The "Budget" Analogy
Imagine you have a backpack with a strict weight limit (a token budget) for your search results.
- Old Way: You pack 5 heavy, bulky boxes (large chunks of text). You can only fit 2, and they might contain irrelevant info.
- SPIRE Way: You pack 20 lightweight, precise items (sentences). Because SPIRE knows how to share the "frame" (the title and headers) among all the items, you don't have to carry the heavy frame 20 times. You carry it once, and it applies to all your items.
- Result: You fit way more useful information in your backpack without breaking the weight limit.
5. The Final Result: The "Citation"
When SPIRE gives you an answer, it doesn't just say "Here is the text." It gives you a precise citation.
- Analogy: Instead of saying "I found a quote in a book," it says, "I found this quote on Page 42, Chapter 3, Paragraph 2, Bullet Point B."
- Because it keeps the "tree structure," you can click that citation and go exactly to that spot in the original document, seeing the full context (the title, the headers, the table) exactly as the author intended.
Summary
SPIRE is like a super-smart librarian who refuses to rip pages out of books. Instead, they use a map to find the exact paragraph you need, automatically add the chapter title and section headers so you understand the context, and then trim the fat so you only get the most relevant info. This makes AI answers more accurate, more trustworthy, and much easier to understand.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.