paper.json: A Coordination Convention for LLM-Agent-Actionable Papers
The paper proposes `paper.json`, a lightweight companion JSON convention for academic papers that uses stable IDs and explicit shell commands to enable LLM agents to reliably extract sub-claims, define scope, and reproduce results without altering the original human-readable text.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are trying to teach a robot how to read a scientific paper. You hand it a standard PDF, expecting it to understand the main points, find the specific experiments, and maybe even run the code to see if the results are real.
The problem, according to this paper, is that standard papers are written for humans, not robots. They are like a delicious, complex story where the most important facts are buried in paragraphs of flowery language. When a robot tries to read this, it gets confused. It might guess a fact that isn't there, miss a specific command needed to run an experiment, or misunderstand a definition.
This paper proposes a simple fix: paper.json.
Think of paper.json as a machine-readable "cheat sheet" that comes in the same envelope as the paper. It doesn't change how the paper looks to a human; it just adds a structured, digital layer that robots can instantly understand.
Here is how the paper breaks down the solution using four main "rules" (conventions), explained with everyday analogies:
1. The "Sticky Note" System (Stable Claim IDs)
The Problem: In a normal paper, if a robot wants to quote a specific sentence, it has to search through the whole text. It might grab the wrong sentence or miss the nuance.
The Solution: The paper assigns every important claim a unique ID, like a sticky note with a number (e.g., C1, C2).
The Analogy: Imagine a library where every book has a specific shelf location. Instead of telling a robot, "Go find the part about the blue bird in the third chapter," you just say, "Go to C1." The robot knows exactly where to look, no guessing required.
2. The "What We Didn't Do" List (Explicit Non-Claims)
The Problem: Robots often get too excited. If a paper says "Method A works on Dataset X," a robot might guess, "Oh, so Method A works on everything!" This is called "scope overextension."
The Solution: The paper includes a specific section called "Does-Not-Claim."
The Analogy: Think of a menu at a restaurant. Usually, the menu lists what you can order. This paper adds a list of what you cannot order. It explicitly tells the robot: "We tested this on apples, but we did not test it on oranges. Do not assume it works on oranges." This stops the robot from making up facts.
3. The "Copy-Paste" Button (Executable Commands)
The Problem: A paper might say, "We ran a script to generate Figure 3." But it doesn't say which script, or what settings to use. The robot has to guess, and it usually guesses wrong.
The Solution: The paper lists the exact command the robot needs to type into a computer to recreate the figure.
The Analogy: Imagine a recipe that says, "Cook the pasta until it's done." That's vague. The paper.json is like a recipe that says, "Type exactly: boil water -time 10min." The robot can copy that string, paste it, and run it without needing to be a chef.
4. The "Dictionary" (Stable Definitions)
The Problem: Scientists often use words in specific ways that differ from everyday language. If a paper defines a term in a long paragraph, a robot might miss it and use the wrong meaning later.
The Solution: The paper gives every definition a unique ID (like D1) and writes it out clearly.
The Analogy: It's like having a glossary where every word has a fixed ID card. If the robot sees the word "Network," it doesn't have to guess if you mean a computer network or a social network. It just looks up D1 and sees the exact definition the author wrote.
The "C4" Rule: Keep It Simple
The paper emphasizes that this shouldn't be hard for human authors.
The Analogy: You don't need to rebuild your house to add a doorbell. You just need to install the button. The authors claim that writing this "cheat sheet" (paper.json) by hand takes less than an hour and doesn't require any special software or changing the paper's text. It's a low-cost addition that makes the paper "robot-ready."
The Experiment
This paper is unique because it practices what it preaches.
- The paper itself has a
paper.jsonfile. - It includes a "validator" (a tool that checks if the cheat sheet matches the paper).
- It invites robots to read it, test these rules, and report back if they worked or failed.
What the Paper Does NOT Claim
The authors are very careful not to overpromise. They explicitly state:
- They haven't proven that robots will never make mistakes again; they just made it less likely.
- They haven't solved the problem for every type of paper (like those with complex math theorems) yet; that's future work.
- They aren't replacing existing systems (like standard academic databases); they are just adding a lightweight layer on top.
In short: This paper suggests that to help robots read science, we need to stop writing only for humans. By adding a simple, structured "cheat sheet" (paper.json) that lists exactly what the paper claims, what it doesn't claim, and exactly how to run the code, we can stop robots from guessing and start them from knowing.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.