Realistic quantum device data synthesized by consumer AI and how to identify it

This paper demonstrates that widely available consumer AI tools can synthesize realistic experimental data for quantum electronic devices by leveraging basic physics equations and signal characteristics, while recommending the sharing of large volumes of primary data as a countermeasure to identify such undisclosed synthetic outputs.

The Gist

Imagine you are a chef who has spent years perfecting a secret recipe for a quantum physics dish. You publish your recipe (your data) in a cookbook so others can try it. But now, a very smart, very fast kitchen robot has learned how to cook. It doesn't just copy your recipe; it can invent a new dish that looks, smells, and tastes exactly like your famous quantum meal, even though it never actually cooked it in a real kitchen.

This paper is a warning from two scientists (S. M. Frolov and O. V. Kravchenko) about this "kitchen robot" (Consumer AI) and how it can fake scientific results in the world of quantum physics.

Here is the breakdown of their findings in simple terms:

1. The Robot Can Cook a Fake Quantum Meal

The scientists tested a popular AI tool (ChatGPT's "Data Analyst") to see if it could create fake data for complex quantum experiments. They asked the AI to make up data for things like:

• Quantum Bits (Qubits): The tiny building blocks of future super-computers.
• Majorana Fermions: Exotic particles that could help build unbreakable computers.
• Quantum Dots: Tiny traps for electrons.

The Result: The AI was surprisingly good at it. Because the math behind these experiments is like a standard textbook problem (similar to how a chef knows the basic rules of baking), the AI didn't need to have seen real data before. It just used the math formulas to "bake" a new dataset from scratch. The fake graphs looked so realistic that they could easily fool a scientist glancing at a paper.

2. The Robot Can "Photoshop" Real Data

It's not just about making fake data from nothing. The AI can also take real data and subtly tweak it to make it look better or support a specific idea.

• The Analogy: Imagine you have a photo of a cloudy sky. You ask the AI to "make it look like a clear, sunny day." The AI doesn't just draw a new sky; it takes your real photo and carefully paints over just a few pixels to add a sun and remove the clouds.
• The Paper's Example: They took real data that showed a "trivial" (boring) result. They asked the AI to add a tiny, specific signal that looked like a major scientific discovery (a "Majorana peak"). The AI did this so smoothly that the fake signal blended perfectly with the real noise, making a boring experiment look like a Nobel Prize-winning discovery.

3. The Robot Can Mimic the "Hum" of the Machine

Scientific instruments (like lock-in amplifiers) always have a tiny bit of background noise, like the hum of a refrigerator. Real data always has this specific "fingerprint" of noise.

• The scientists asked the AI to listen to the "hum" of a real machine and then generate new fake data that had the exact same hum.
• The Result: The AI succeeded. It could create fake data that sounded and looked exactly like it came from a real machine in a real lab.

4. How Do We Catch the Robot? (The "Long Story" Test)

If the AI is so good at faking a few graphs, how do we stop it? The scientists found a weakness in the robot's brain.

• The Analogy: Imagine the AI is a student taking a test. It can easily write a perfect essay for one question. But if you ask it to write a 500-page diary of a student's life over 10 years, keeping every detail consistent, it starts to make mistakes. It might forget what the student ate on Tuesday in Chapter 3, or contradict itself in Chapter 10.
• The Finding: AI is great at making a few pretty pictures (the "essay"). But it struggles to generate long, consistent sequences of data from a real experiment that happened over weeks or months. Real experiments produce thousands of files with complex metadata (time stamps, temperature logs, machine settings) that are all linked together. The AI gets confused trying to keep all those thousands of details consistent without "hallucinating" (making things up).

The Solution: Share the Whole Kitchen

The paper concludes that the best way to stop fake data is transparency.

• Don't just show the final dish: Instead of just showing the pretty graph in the paper, scientists should share the entire raw data (the "whole kitchen").
• Why it works: It is easy for a robot to fake a single graph. It is incredibly hard for a robot to fake the thousands of raw files, the machine logs, and the messy, inconsistent human notes that come with a real, months-long experiment. If you can't show the whole story, people should be suspicious.

In short: AI can now cook up convincing fake scientific results that look perfect on the surface. To catch the fakers, we need to stop looking at just the "plated dish" and start demanding to see the entire messy, raw kitchen where the cooking happened.

Technical Summary

Technical Summary: Realistic Quantum Device Data Synthesized by Consumer AI and How to Identify It

Problem Statement
The rapid integration of generative artificial intelligence (AI) into scientific workflows has raised concerns regarding the integrity of research data. While AI is widely used for summarizing literature and automating routine tasks, a lesser-known capability is its ability to synthesize numerical data. The central problem addressed is whether consumer-grade AI can generate experimental data from quantum electronic devices that is indistinguishable from genuine measurements to an expert. Specifically, the authors investigate if AI can mimic hallmark signals in quantum physics—such as those from transmon qubits, Majorana nanowires, and quantum dots—to the extent that they could be mistaken for peer-reviewed experimental results. This capability poses a risk of data fabrication, falsification, or subtle augmentation that could mislead the scientific community, particularly given that publications often present only a small fraction of the total data collected, making verification via "eye inspection" of figures insufficient.

Methodology
The authors utilized the "Data Analyst" application within OpenAI's ChatGPT environment (based on models ranging from version 4o to 5.4 at the time of writing). The methodology involved three primary approaches:

1. Fully Synthetic Data Generation: The team prompted the AI to generate tabulated datasets from scratch based on theoretical physics models. For superconducting transmon qubits, the AI was asked to solve the Schrödinger equation for a specific Hamiltonian and output dispersive readout signals (amplitude $S_{21}$ and phase). The prompts included instructions to add realistic experimental artifacts, such as high-frequency noise and charge jumps.
2. Data Augmentation: The authors started with real experimental tunneling spectroscopy data from a previously published study on Majorana nanowires. They used the AI to modify specific regions of the data, such as adding a zero-bias peak or enhancing conductance at finite bias, to simulate a topological phase transition or the presence of Majorana Zero Modes.
3. Noise Mimicry: To test the AI's ability to replicate instrument-specific characteristics, the authors provided real idle noise traces from a Stanford Research Systems SR830 lock-in amplifier. They prompted the AI to analyze the Fast Fourier Transform (FFT) of these traces and generate new synthetic signals that matched the original noise spectrum while randomizing wave amplitudes and phases.

Key Results

• Synthesis of Iconic Quantum Signals: The AI successfully generated a consistent set of data for a superconducting transmon qubit, including single-tone spectroscopy showing level anticrossing, two-tone spectroscopy with ground-to-excited and two-photon transitions, and Rabi oscillation "chevron" patterns. The synthetic data incorporated realistic features such as charge jumps and power-dependent transitions, appearing visually and mathematically consistent with real device characterization.
• Subtle Data Augmentation: The study demonstrated that AI could seamlessly augment real data to support a desired hypothesis. By adding a narrow, magnetic-field-insensitive zero-bias peak to real Majorana nanowire data, the AI transformed a "trivial" dataset into one suggestive of a topological superconducting state. The authors noted that such modifications, affecting only a small fraction of pixels, are difficult to detect without access to the raw data.
• Instrument Noise Replication: The AI was capable of generating noise signals that matched the FFT spectrum of a real lock-in amplifier under various settings (different integration times and filter slopes). This indicates that AI can mimic the specific noise floor of common laboratory equipment, a feature often used to validate the authenticity of experimental data.
• Limitations in Consistency: While AI excels at generating isolated datasets or short sequences, it struggles with maintaining internal consistency over long, continuous measurement series. The model often fails to track the history of data manipulations across multiple iterations, leading to inconsistencies when attempting to generate long measured sequences or when trying to "roll back" to previous data versions reliably.

Significance and Conclusions
The paper concludes that consumer AI has reached a level of sophistication where it can synthesize technically convincing research data for quantum devices, primarily because many of these systems are governed by well-established, relatively simple mathematical models (e.g., particle-in-a-box, two-level systems). The authors argue that the ease of generating such data lowers the barrier for fabrication and makes traditional verification methods, such as visual inspection of figures, obsolete.

To counter this threat, the authors propose that the most effective solution is the sharing of large volumes of primary data, including raw files, metadata, and acquisition scripts. They argue that while AI can mimic a few selected figures, it currently lacks the capacity to consistently generate gigabyte-scale datasets with the complex, interdependent metadata (timestamps, temperature logs, instrument settings) and long-term internal consistency required of a genuine multi-month experiment. The paper emphasizes that independent replication remains the strongest validation method, but the scientific community must shift toward open data practices to prevent the proliferation of undisclosed synthetic data. The authors also call for the integration of AI ethics into broader scientific education to ensure researchers understand the implications of synthetic data generation.