💬 NLP

Super Research: Answering Highly Complex Questions with Large Language Models through Super Deep and Super Wide Research

This paper introduces "Super Research," a novel task and benchmark designed to evaluate Large Language Models' ability to solve highly complex questions by integrating structured planning, super-wide retrieval, and super-deep iterative investigation, supported by a graph-anchored auditing protocol to assess performance across five critical dimensions.

Original authors: Yubo Dong, Nianhao You, Yuxuan Hou, Zixun Sun, Yue Zhang, Liang Zhang, Siyuan Zhao, Hehe Fan

Published 2026-03-04

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yubo Dong, Nianhao You, Yuxuan Hou, Zixun Sun, Yue Zhang, Liang Zhang, Siyuan Zhao, Hehe Fan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to solve a massive, world-changing mystery. You can't just ask a librarian for one book; you need to read 1,000 books, interview 100 experts, cross-reference conflicting testimonies, and write a 50-page detective novel that proves your theory.

That is the challenge of Super Research.

This paper introduces a new way to test Large Language Models (LLMs)—the AI brains behind tools like ChatGPT. The authors argue that while current AIs are great at simple questions or even deep dives into one topic, they struggle when asked to do everything at once: go incredibly deep and incredibly wide simultaneously.

Here is the breakdown using simple analogies:

1. The Problem: The "Tunnel Vision" vs. "Information Overload" Trap

The paper compares three types of research, using a visual metaphor of a flashlight in a dark room:

Standard Search (RAG): Like a dim flashlight. It finds the nearest object but misses the rest of the room.
Deep Research: Like a laser beam. It shines very bright and deep into one specific crack in the wall (great for details), but it has tunnel vision. It misses everything happening on the left and right.
Wide Search: Like a floodlight. It illuminates the whole room at once, but it's so bright and scattered that you can't see the details. It leads to information overload.
Super Research (The Goal): This is the "Super Flashlight." It needs to be a laser beam and a floodlight at the same time. It must explore 100+ different angles (width) while digging 100+ layers deep into each one (depth) to solve a problem that requires reading 1,000+ web pages.

2. The Challenge: The "300 Expert Questions"

To test if AI can handle this, the researchers didn't just ask, "Who won the Super Bowl?" They asked questions like:

"How do we balance the immune system's attack on cancer cells without accidentally triggering an autoimmune disease?"

These questions are:

Super Hard: They require synthesizing conflicting evidence.
Super Long: They need 100+ search steps (like a detective following 100 different clues).
Super Complex: They require writing a report that is 50 pages long, with perfect citations.

3. The Solution: The "Graph-Anchored Audit"

How do you grade a 50-page AI report? You can't just ask another AI, "Is this good?" because AIs often lie or agree with each other (the "Yes-Man" problem).

Instead, the authors built a Truth Map (a Research Graph):

The Gold Standard: Human experts and AI agents work together to build a perfect "map" of the facts, logic, and connections for the answer.
The Audit: When an AI writes a report, the system projects it onto this map.
- Did it find the right facts? (Coverage)
- Do the facts actually lead to the conclusion? (Logical Consistency)
- Did it only use one source, or did it check many? (Citation Health)
- Is it biased, or does it show both sides of the argument? (Objectivity)

Think of it like a forensic audit. Instead of just reading the report, the system checks every single claim against a verified database to see if the AI is hallucinating or making things up.

4. The Results: The AI "Ceiling"

The researchers tested 12 of the world's smartest AI models (including Gemini, Claude, and OpenAI's o3).

The Shocking Result: Even the best AI only scored about 28 out of 100.

What this means: Current AI is like a brilliant intern who can read a book quickly but gets lost when asked to write a thesis based on 1,000 different books. They tend to:
- Get stuck in "tunnel vision" (missing the big picture).
- Get "information overload" (collecting facts but not connecting them).
- Rely on too few sources (citation health issues).
- Fake confidence when they are actually guessing.

5. Why Does This Matter?

You might think, "I don't need an AI to write a 50-page scientific thesis."

But the authors argue that Super Research is a stress test.

If an AI can't handle a "Super Complex" question, it will likely fail at any complex, real-world task that requires long-term planning, checking facts, and avoiding bias.
Passing this test proves the AI is robust enough to be a true "reasoning engine" rather than just a fancy autocomplete.

The Takeaway

Super Research is a new "Olympics" for AI. It's not about who can answer the fastest; it's about who can think the deepest, search the widest, and build the most logical argument without getting lost or making things up. Right now, the AI athletes are still in the training phase, but this new benchmark shows us exactly where they need to improve to become true partners in solving the world's hardest problems.

Based on the provided paper, here is a detailed technical summary of Super Research:

1. Problem Statement

While Large Language Models (LLMs) have shown proficiency in "Deep Research" (vertical, iterative reasoning) and "Wide Search" (horizontal, broad data acquisition), they struggle with highly complex, super-complex questions. These tasks require:

Long-horizon planning: Managing research trajectories over extended periods.
Massive evidence gathering: Synthesizing information from 1,000+ web pages.
Heterogeneous synthesis: Reconciling conflicting evidence across diverse sources.
Limitations of current benchmarks: Existing benchmarks (e.g., GAIA, DeepResearch Bench) suffer from performance saturation, rely on shallow fact-recall metrics, or use unreliable "LLM-as-a-Judge" evaluations that fail to capture deep reasoning, logical consistency, and uncertainty expression.

2. Methodology

The authors propose a comprehensive framework consisting of a new task definition, a benchmark construction pipeline, and a novel evaluation protocol.

A. Super Research Task Definition

Super Research is defined as an autonomous research capability integrating three core pillars:

Structured Decomposition: Breaking monolithic queries into multi-layered research plans (DAGs).
Super Wide Retrieval: Horizontal exploration to ensure total coverage of diverse perspectives (avoiding "tunnel vision").
Super Deep Investigation: Iterative follow-up queries to resolve uncertainties and verify data points (avoiding "information overload").

B. Benchmark Construction (SuperResearch Benchmark)

Scale: 300 expert-written questions across 10 specialized domains (e.g., Science, Medicine, Finance, Law).
Complexity: Each task requires up to 100+ retrieval steps and the synthesis of 1,000+ web pages.
Gold Standard Creation: A four-stage Human-AI collaborative pipeline:
1. Task Generation: LLMs generate "super hard" open-ended tasks under strict constraints (no fact lookups, no settled science), refined by domain experts.
2. Research Graph Construction: Unstructured sub-reports are transformed into a structured Research Graph containing Atomic Facts, Key Insights, and Global Conclusions. This involves human-in-the-loop verification and code sandbox fact-checking.
3. Report Synthesis: Agents generate canonical reports, validated by experts for logical soundness.
4. QA Exam Generation: Automated generation of diverse evaluation questions (MCQ, True/False, Confidence Scoring) to test utility and bias.

C. Evaluation Framework: Graph-Anchored Auditing

Instead of relying solely on LLM judges, the paper introduces a Graph-Anchored Auditing Protocol that evaluates reports against the ground-truth Research Graph across five dimensions:

Coverage & Comprehension ( $R_{weighted}$ ): A depth-weighted recall metric that prioritizes the retrieval of high-level insights and global conclusions over trivial facts.
Logical Consistency ( $C_{logic}$ ): Verifies that global conclusions are algorithmically grounded in atomic facts via unbroken citation chains.
Report Utility ( $U_{qa}$ ): Measures actionable value via a closed-context reading comprehension exam (can the report answer questions without external knowledge?).
Objectivity Score ( $O_{bias}$ ): A double-blind audit assessing the model's ability to balance conflicting viewpoints (Thesis vs. Antithesis) and avoid false consensus.
Citation Health: Diagnostic metrics for Source Dominance (over-reliance on one source) and Narrative Monopolization (single source driving the entire narrative).

3. Key Contributions

SuperResearch Benchmark: The first benchmark specifically designed to stress-test the "ceiling" of LLM research capabilities, requiring 100x the retrieval steps of standard benchmarks.
Graph-Anchored Evaluation: A rigorous, reproducible evaluation method that maps generated claims to a verified knowledge graph, solving issues of hallucination and shallow fact-checking found in LLM-as-a-Judge approaches.
Multi-Dimensional Metrics: Moving beyond simple accuracy to measure logical coherence, objectivity, and citation diversity.
Empirical Insights: Identification of specific failure modes in current SOTA models, such as the "Reasoning Bottleneck" (high retrieval but low synthesis) and the "Defensive Summarization Paradox" (high objectivity but low utility).

4. Experimental Results

The authors evaluated 12 representative systems (including Deep Research agents like Gemini Deep Research, Sonar, and OpenAI's o3/o4-mini, as well as search-integrated agents like Kimi and Grok).

Overall Performance: Even the state-of-the-art system (Gemini Deep Research) achieved an Overall Score of only 28.62, indicating that super-complex research remains a largely unsolved frontier.
Performance Gaps:
- Deep Research Systems generally outperformed baselines but still struggled with logical consistency ( $C_{logic}$ ).
- Native Search Agents (e.g., Kimi-k2) showed competitive performance, sometimes surpassing specialized Deep Research systems in logical alignment.
- Baseline Models (e.g., Llama-3.3, Qwen-2.5) performed poorly, highlighting the gap between raw model capability and integrated agentic workflows.
Key Findings:
- Retrieval $\neq$ Reasoning: High coverage does not guarantee logical consistency; models often fail to synthesize retrieved data into coherent arguments.
- Utility-Objectivity Trade-off: Models prioritizing safety (high objectivity) often sacrifice granular detail (low utility), resulting in "defensive" summaries.
- Citation Issues: Many models suffer from "Single-Source Dependency," recycling a small pool of references rather than synthesizing diverse evidence.

5. Significance

Stress Test for AI: Super Research serves as a critical "ceiling protocol" to expose latent weaknesses in reasoning consistency and context management that simpler tasks miss.
Proxy for General Competence: Proficiency in Super Research acts as a powerful proxy for an agent's general research competence; success here suggests the robustness necessary for any subordinate research task.
Future Direction: The work highlights the need for "Green AI" (efficiency in long-horizon retrieval) and better mechanisms for handling uncertainty and conflicting evidence in autonomous agents.
Open Resource: The benchmark and leaderboard are publicly available, providing a standardized testbed for future advancements in autonomous deep research.

Leaderboard: https://cnsdqd-dyb.github.io/Super-Research-Benchmark/