Perfect score on IPhO 2025 theory by Gemini agent

A simple agent built on the Gemini 3.1 Pro Preview model achieved a perfect score on the IPhO 2025 theory problems, though this result may be compromised by potential data contamination due to the model's release occurring after the competition.

The Gist

Imagine the International Physics Olympiad (IPhO) as the "Olympics of the Mind" for high school students. It's the hardest physics test in the world, where the smartest teenagers from over 100 countries compete to solve incredibly complex puzzles involving gravity, light, and energy. Gold medals are awarded to the top one-twelfth (about 8.3%) of contestants — but even among gold medalists, a perfect score is exceedingly rare. In fact, zero human contestants achieved a perfect score at IPhO 2025.

This paper is a report card from a computer program (an AI agent) that took this test in 2025. The result? It got a perfect score, 100%, every single time.

Here is the story of how they did it, explained simply:

1. The Star Player: "Gemini 3.1 Pro"

Think of the AI model used here as a super-intelligent student who has read almost every book in the library. The researchers used a very new version of Google's AI, called Gemini 3.1 Pro Preview.

• The Catch: This AI was released after the test happened. This means the AI might have seen the questions during its training — like a student whose teacher might have accidentally given them the answer key beforehand. The authors acknowledge this is a possibility (called "data contamination"), but note it's not certain — the model's knowledge cutoff actually predates the exam. Either way, they argue that solving these problems perfectly is still a huge deal, especially given the leap from previous models.

2. The Strategy: "The Debate Club"

The researchers didn't just ask the AI to "solve this." They built a special Agent (a smart assistant) that uses a trick called Parallel Thinking.

Imagine you are trying to solve a tricky math problem. Instead of just writing down one answer, you ask four different friends to solve it on their own.

1. Round 1: The AI generates four different solutions for the same problem.
2. Round 2: It acts like a referee. It looks at the four answers, finds the mistakes in the wrong ones, and combines the best parts of the correct ones into a single, perfect solution.

It's like having a team of detectives where they compare notes to make sure no one missed a clue. This parallel thinking and synthesis process is what allowed the AI to fix its own mistakes and get a perfect score. (Note: the author remarks that a consensus approach may also be effective with other base models, but it was not the approach used in this agent.)

3. The Tool: "The Digital Ruler"

Physics problems often have diagrams (pictures of planets, circuits, or springs). Humans can look at a picture and guess the numbers, but computers often struggle to "see" a ruler on a screen.

• The Problem: If the AI guesses the length of a line in a drawing, it might be slightly off, which ruins the whole calculation.
• The Fix: The researchers gave the AI a Python code tool. Instead of just "looking" at the picture, the AI wrote a tiny computer program to measure the pixels on the screen with mathematical precision. It's like giving the AI a digital ruler instead of asking it to guess with its eyes.

4. The "Clean-Up Crew" (Fixing the Test)

One of the most interesting parts of this paper isn't just the AI winning; it's that the AI helped the researchers find mistakes in the test itself.

Before running the AI, the researchers used it to check the official exam questions. They found three errors:

• A Graph Error: One graph showed a curve that decayed faster than was physically possible. The marking scheme was adjusted so that full points were awarded if the calculated mass had a relative error of 25% or less compared to the specific data point the agent used.
• A Frequency Shift Error: A diagram showed a star moving away (redshift), but the frequency shift graph contradicted this by showing the opposite direction. The axis label was corrected.
• A Math Error: The official answer key had a calculation mistake.

The AI spotted these errors, and the researchers fixed the "official" test before grading the AI. This shows that the AI is becoming smart enough to act as a physics professor, not just a student.

5. The Verdict

The AI took the test five times and got 30/30 every time.

• Why is this important? It proves that AI is getting incredibly good at deep reasoning, not just memorizing facts.
• The Caveat: Because the AI was released after the test, there is a possibility it saw the questions in its training data — though this is not certain. But even with that doubt, the fact that it solved these problems perfectly is a massive leap forward. Notably, the previous best score of 87.7% was also achieved using Gemini (via Gemini 3 Deep Think), meaning both systems share the same potential level of data contamination — yet this agent's perfect score represents a significant jump in capability.

In a nutshell: The researchers built a smart AI team that debates its own answers, uses digital tools to measure pictures perfectly, and even found errors in the test questions. The result? A robot that aced the world's hardest high school physics exam.

Technical Summary

Here is a detailed technical summary of the paper "Perfect score on IPhO 2025 theory by Gemini agent" by Yichen Huang.

1. Problem Statement

The paper addresses the challenge of solving International Physics Olympiad (IPhO) 2025 theoretical problems, which represent the pinnacle of pre-university physics reasoning. These problems are multimodal (combining complex text, mathematical derivations, and figures), require deep conceptual understanding of mechanics, electromagnetism, thermodynamics, and relativity, and demand high precision in both reasoning and numerical calculation.

While previous Large Language Models (LLMs) achieved "gold-medal" performance (scoring roughly 70–88% of the total points), they consistently fell short of a perfect score. The primary challenges identified were:

• Multimodal Complexity: Difficulty in accurately interpreting figures and extracting data points.
• Reasoning Errors: Propagation of small algebraic or sign errors through multi-step derivations.
• Evaluation Limitations: Most previous studies relied on automated grading (LLM-as-a-judge), which often inflated scores by rewarding correct final answers despite flawed intermediate reasoning. (One notable exception is Physics Supernova by Qiu et al., which was graded by human experts.)

2. Methodology

The authors developed a specialized AI agent powered by Gemini 3.1 Pro Preview. The methodology focuses on three core pillars: dataset curation, agent architecture, and rigorous human evaluation.

A. Dataset Curation and Pre-processing

• Source: Problems were downloaded from the official IPhO 2025 website.
• Pre-processing: To avoid OCR errors and token inefficiencies, the authors manually converted PDFs to Markdown.
  • Figure Handling: Figures were cropped to remove irrelevant background (reducing token waste) and split if they contained multiple panels (doubling token allocation per panel to improve visual attention).
  • Context Integration: Answer sheet tables referenced in problem statements were manually embedded into the problem text to ensure self-containment.
• Error Correction (Curation): The authors used the model to audit the official problem statements and marking schemes, identifying and correcting three specific errors in the official IPhO 2025 materials:
  1. Problem 1, Fig 1(B): The official curve featured a physically impossible decay profile, meaning that choosing different valid data points mathematically forced different mass values. The marking scheme was adjusted so that full points were awarded if the calculated mass had a relative error of ≤25% compared to the precise reading of the specific data point chosen by the agent.
  2. Problem 1, Fig 3: A contradiction existed between the geometry (redshift) and the frequency shift graph (blueshift); the axis label was corrected.
  3. Problem 3, Sub-problem B.3: A miscalculation in the official solution formula was identified and corrected for evaluation purposes.

B. Agent Architecture

The agent utilizes a parallel thinking workflow combined with agentic vision (code execution):

1. Parallel Reasoning & Synthesis:

   • For each sub-problem, the agent generates four raw solutions independently.
   • These are synthesized in two rounds:
     • Round 1: Pairs of solutions are compared to identify errors and construct a corrected solution.
     • Round 2: The two intermediate corrected solutions are synthesized into a final answer.
   • This parallel thinking and synthesis approach allows the model to self-correct sign errors and algebraic mistakes that occur in individual generations. (The author notes that a consensus approach may also be effective when using other base models, but it was not the method employed in this agent.)

2. Agentic Vision (Measurement Component):

   • For problems requiring measurements from figures (Problems 1 and 3), the agent employs a specialized sub-routine.
   • Workflow: The model first attempts a solution. If it detects a need for measurement, it triggers a Python-based tool using computer vision libraries to measure distances/coordinates in pixels.
   • Robustness: Measurements are taken three times, and the median value is used to filter out outliers.
   • Decision Logic: The model decides whether to use native vision or the code-based tool based on the specific context (e.g., using native vision for grid-aligned points to avoid artificial pixelation errors).

3. Context Management:

   • The agent maintains a dialogue history where solutions to previous sub-problems serve as context for subsequent ones.
   • Thought Signatures Disabled: The authors disabled "thought signatures" (encrypted internal chain-of-thought) to prevent coherence breaks when the agent modifies or replaces previous turns in the conversation.

C. Evaluation Protocol

• Human Grading: Unlike most previous studies using LLM judges, this work employed human experts to grade the solutions against the official marking schemes.
• Marking Scheme: The official IPhO marking scheme provides for partial credit for correct intermediate steps even if the final answer is wrong. However, since the agent achieved a perfect 30/30 score in all runs, every final answer was correct, and partial credit for wrong final answers was never actually awarded.
• Repetition: The agent was run five times on each of the three theory problems.

3. Key Contributions

1. First Perfect Score: This is the first reported instance of an AI agent achieving a 100% score on an International Olympiad theoretical exam.
2. Dataset Curation: The paper provides a rigorously pre-processed and corrected dataset of IPhO 2025 problems, fixing errors in the official competition materials that had gone unnoticed.
3. Agent Design: Demonstrates that combining parallel thinking (for error correction) with agentic vision (for precise figure measurement) is superior to standard prompting or single-pass generation.
4. Critical Analysis of Data Contamination: The authors provide a nuanced discussion on the risk of data contamination, acknowledging that while the model's knowledge cutoff predates the exam, the model's release date post-dates it, potentially allowing for post-training on the exam data.

4. Results

• Performance: The agent achieved a perfect score (30/30) on all three IPhO 2025 theory problems in all five independent runs.
• Error Analysis:
  • While the final synthesized solutions were perfect, the raw solutions (before synthesis) contained errors.
  • Sign Errors: A common error in raw solutions was an incorrect sign in force vectors (e.g., destabilizing forces appearing as stabilizing). The synthesis process successfully identified and corrected these.
  • Measurement: The agentic vision component successfully handled figure measurements where native vision failed or was imprecise.
• Comparison: The result significantly outperforms previous benchmarks (e.g., Gemini 3 Deep Think at 87.7%, Physics Supernova by Qiu et al. at ~21.5/30, PhysicsMinions at lower scores).

5. Significance

• AI in Physics Education: This work marks a milestone where AI has achieved a perfect score on a standardized physics competition, surpassing even gold medalists (top ~8.3% of contestants) who themselves rarely achieve perfect marks.
• Methodological Shift: It highlights that for complex scientific reasoning, workflow design (parallel generation + synthesis + tool use) is as critical as the base model's raw intelligence.
• Benchmarking Standards: The paper advocates for human evaluation over LLM-based grading for scientific problems, as automated graders often fail to penalize incorrect reasoning that leads to correct answers.
• Data Contamination Context: The previous best score of 87.7% was achieved by Gemini 3 Deep Think, which uses the same base model (Gemini 3.1 Pro). Since both systems share the same potential level of data contamination, the jump from 87.7% to 100% more likely reflects genuine capability improvements in agent design rather than data leakage alone.
• Future Directions: The authors suggest that while this specific agent is optimized for IPhO, the principles of domain-specific agents (combining reasoning, tool use, and error correction) are essential for advancing AI in scientific discovery.