Co-Authoring with AI: How I Wrote a Physics Paper About AI, Using AI

This paper uses a case study of a physics manuscript co-authored with AI to argue that while Large Language Models can handle structural and syntactic tasks, human authors must remain the ultimate "Principal Investigators" responsible for logical rigor and integrity, necessitating the mandatory publication of full AI interaction transcripts to ensure accountability.

The Gist

Imagine you are a master chef trying to create a complex, 5-star gourmet meal. In the past, you would chop every vegetable, mix every sauce, and plate every dish yourself.

Now, imagine you have a team of incredibly fast, super-smart kitchen robots. They can chop vegetables at lightning speed, mix sauces perfectly, and even plate the food with artistic precision. But here's the catch: The robots don't actually know what "good taste" is. They don't understand the chemistry of the ingredients, and if you just tell them, "Make me a fancy dinner," they might serve you a plate of raw, hallucinated ingredients that look beautiful but taste like soap.

This paper, written by a physicist named Yi Zhou, is a story about how he used these "kitchen robots" (AI) to write a scientific paper about AI, and why he still had to be the Head Chef.

Here is the breakdown of his experience in simple terms:

1. The Big Shift: From Tool to Intern

For years, scientists used computers like calculators: you press a button, and it gives you a number. But recently, AI has become more like a team of eager, hyper-fast interns.

• The Experiment: Yi Zhou set up a "Virtual Research Group." He assigned different AI models specific jobs: one to act as a "Junior Theorist" (finding the math), one as a "Senior Postdoc" (checking the logic), and one as a "Coder" (writing the software).
• The Result: They finished a project in 24 hours that usually takes a human graduate student months.

2. The Trap: Don't Just Ask for the Essay

The biggest mistake people make is thinking, "I'll just ask the AI to write the whole paper."

• The Analogy: If you ask a robot to write a novel without giving it a plot, it will write a story that sounds smooth but makes no sense. It's like a robot writing a recipe for "delicious soup" but forgetting to mention water or salt.
• The Fix: Yi Zhou didn't let the AI start writing immediately. First, he acted as the Principal Investigator (the Boss). He fed the AI all the raw data, the messy chat logs, and the specific goals. He made sure the AI understood the whole story before it wrote a single sentence.

3. The Human Role: The "Quality Control" Inspector

Even with the AI doing the heavy lifting, Yi Zhou had to step in constantly. He wasn't just fixing typos; he was the physics police and the diplomat.

• Catching Physics Errors: The AI once wrote that the math was "continuous." Yi Zhou said, "Wait, in our field, this is actually 'discrete' (like steps on a ladder, not a smooth slide)." The AI corrected itself. Without him, the paper would have been scientifically wrong.
• Academic Diplomacy: The AI wanted to write a sentence saying, "The old software libraries are a messy mess." Yi Zhou stopped it. "Hey," he said, "The people who made those libraries are our friends and colleagues. We can't insult them." He rewrote it to be polite but still accurate.
• Anticipating the "Grumpy Reviewer": In science, papers are reviewed by experts who love to find holes in your logic. Yi Zhou played "Devil's Advocate." He asked the AI, "What if a reviewer says you just copied the code from the internet?" He then guided the AI to write a section proving that the code was actually new and reasoned, not just copied.

4. The Visuals: The Art Director

When it came to drawing diagrams, the AI tried to draw a "neural network" on a whiteboard, but it looked like a chaotic spiderweb.

• The Problem: The AI didn't understand the rules of 1D physics.
• The Solution: Yi Zhou acted as an Art Director. He told the AI, "Stop drawing a spiderweb. Draw a neat line of 5 circles. That's what a 1D quantum system looks like." The AI then wrote a perfect instruction for the image generator to create the right picture.

5. The Big Lesson: Radical Transparency

The most important part of the paper is the conclusion. Yi Zhou argues that we can't just say, "We used AI." That's not enough.

• The Proposal: He suggests that if you use AI to write a paper, you must publish the entire chat log (the transcript) as a public appendix.
• Why? Imagine a magic trick. If you just show the audience the final trick, they might think it's real magic. But if you show them the entire video of the magician setting up the wires, pulling the rabbit out of the hat, and checking the box, they see the human effort behind the illusion.
• By publishing the chat logs, scientists prove: "The AI did the typing, but I did the thinking, the steering, and the correcting."

Summary

This paper is a guide on how to be a Human-in-the-Loop.

• The AI is the super-fast typist, the data organizer, and the draft writer.
• The Human is the Principal Investigator, the logic checker, the diplomat, and the final decision-maker.

The future of science isn't about humans being replaced by AI; it's about humans becoming conductors of a massive orchestra, where the AI plays the instruments, but the human decides the music. And to prove the music is real, we must show the sheet music and the conductor's notes to everyone.

Technical Summary

1. Problem Statement

The rapid integration of Large Language Models (LLMs) into scientific workflows has created a crisis in defining authorship, responsibility, and scientific integrity. Traditional views of computers as deterministic tools are shifting toward viewing AI as "virtual collaborators." However, unguided AI usage leads to:

• Scientific Hallucinations: AI generates generic, enthusiastic, but physically shallow or incorrect content.
• Logical Inconsistencies: AI struggles with domain-specific nuances (e.g., confusing discrete vs. continuous physics, misclassifying topological phases).
• Academic Diplomacy Failures: AI may generate aggressive or inaccurate critiques of established open-source libraries, risking community friction.
• Lack of Transparency: The "black box" nature of AI generation obscures the origin of ideas and the extent of human oversight, making it difficult to verify the validity of the scientific record.

2. Methodology

The author employs a Human-in-the-Loop (HITL) paradigm, treating the AI not as an autonomous writer but as a "Virtual Research Group" managed by a human Principal Investigator (PI). The methodology involves a multi-stage, iterative workflow:

• Role Assignment: The author assigned specific academic personas to different LLM instances to simulate a research team:
  • LLM-0 (Junior Theorist): Extracted raw equations.
  • LLM-1 (Senior Postdoc): Created rigorous LaTeX blueprints.
  • LLM-2 (Coder): Executed Python implementation.
• "Inside-Out" Writing Strategy:
  • Context Loading: Instead of starting with an Introduction, the human PI loaded the AI's context window with the entire project history, including raw theoretical ideas, coding transcripts, and mathematical specifications. This grounded the AI in specific reality rather than parametric generalizations.
  • Conversational Genesis: Core concepts (e.g., "Virtual Research Group," "Universal API") emerged through iterative dialogue where the human provided raw intuition and the AI synthesized formal terminology.
• Active Mentorship & Correction: The human PI acted as a strict editor, intervening at critical junctures to:
  • Correct physics inaccuracies (e.g., distinguishing between "hidden topological order" and "Symmetry-Protected Topological (SPT) order").
  • Enforce academic diplomacy (reframing critiques of libraries like ITensor/TeNPy).
  • Anticipate peer-review skepticism ("Reviewer 2") by explicitly addressing potential loopholes regarding data contamination and model capabilities.
• Multi-Modal Visual Direction: The text-based AI was tasked as an "Art Director" to generate highly constrained design briefs for image generators (e.g., Nano Banana 2), ensuring visual outputs (figures, cartoons) adhered to strict physical laws (e.g., correct 1D Matrix Product State diagrams) rather than hallucinating generic neural networks.

3. Key Contributions

• The "Virtual Research Group" Framework: A novel workflow demonstrating how to orchestrate multiple LLM agents with specific academic roles under human supervision to accelerate complex tasks (e.g., building a tensor network engine in 24 hours vs. months).
• The "Inside-Out" Writing Protocol: A methodological shift arguing that AI writing must begin with deep context loading and logical pillar definition, rather than starting with the Introduction, to prevent narrative drift and hallucination.
• The "Transcript Transparency" Proposal: A concrete policy recommendation that the scientific community must mandate the publication of full, unedited AI interaction transcripts as standard supplementary material. This serves as the "raw data" of the writing process, proving human steering and accountability.
• Demonstration of HITL Necessity: The paper provides empirical evidence that while AI excels at syntax and structure, the human author is indispensable for:
  • Enforcing rigorous physical logic.
  • Managing academic diplomacy.
  • Anticipating and closing logical loopholes before peer review.

4. Results

• Efficiency: The multi-agent workflow successfully translated a dense theoretical review into a scalable Python codebase and a manuscript in under 24 hours, a task typically requiring graduate students months.
• Quality Control: Through human intervention, the manuscript avoided critical errors, such as:
  • Misidentifying the Haldane phase as having "topological order" instead of "Symmetry-Protected Topological (SPT) order."
  • Incorrectly framing tensor networks as operating on continuous space rather than discrete spin lattices.
  • Generating aggressive language that could alienate the creators of referenced software libraries.
• Visual Accuracy: The AI-generated figures and editorial cartoons were rendered physically accurate (e.g., correct 1D quantum diagrams) only after the human PI intervened to correct the image generator's initial hallucinations via the text-based "Art Director."
• Reproducibility: All interaction transcripts, prompts, and raw data were made publicly available on GitHub, allowing independent verification of the human-AI collaboration process.

5. Significance

This paper serves as a foundational case study for the future of scientific authorship in the age of generative AI. Its significance lies in:

• Redefining Authorship: It shifts the definition of the human author from a "text typist" to a "Principal Investigator" who mentors, steers, and validates AI reasoning.
• Preserving Integrity: By advocating for radical transparency (publishing transcripts), it offers a solution to the "black box" problem, ensuring that scientific accountability remains with the human researcher.
• Setting a Precedent: It establishes a replicable standard for how computational physicists (and scientists generally) can leverage AI for acceleration without compromising the rigor, accuracy, or ethical standing of their work.
• Future-Proofing Peer Review: It anticipates the challenges of future peer reviews, suggesting that the "data" of scientific writing now includes the prompt history, which must be scrutinized just like experimental data.