🤖 AI

Collaborative Agent Reasoning Engineering (CARE): A Three-Party Design Methodology for Systematically Engineering AI Agents with Subject Matter Experts, Developers, and Helper Agents

The paper introduces Collaborative Agent Reasoning Engineering (CARE), a systematic, stage-gated methodology that leverages a three-party workflow of Subject-Matter Experts, developers, and LLM helper agents to transform informal domain intent into rigorous, verifiable specifications for engineering reliable AI agents in scientific domains.

Original authors: Rahul Ramachandran, Nidhi Jha, Muthukumaran Ramasubramanian

Published 2026-05-01

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Rahul Ramachandran, Nidhi Jha, Muthukumaran Ramasubramanian

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to build a highly skilled robot assistant to help scientists find specific data in a massive library. In the past, people tried to teach these robots by just talking to them, guessing what to say, and hoping for the best. This is like trying to teach a dog to perform complex math by just shouting random words at it; sometimes it works, but mostly it's a mess of trial and error.

This paper introduces CARE (Collaborative Agent Reasoning Engineering), a new, disciplined way to build these AI assistants. Instead of guessing, CARE treats building an AI like building a bridge: you need a strict blueprint, a team of experts, and a step-by-step inspection process.

Here is how CARE works, broken down into simple concepts:

The Three-Party Team

CARE isn't just a human talking to a computer. It's a three-way conversation between:

The Subject-Matter Experts (SMEs): The scientists who know the rules of the game (e.g., "We only trust data from 2020," or "Never guess the answer").
The Developers: The engineers who know how to build the robot.
The "Helper Agents": These are special AI assistants whose only job is to help the humans talk to each other. Think of them as a super-efficient scribe or a translator. They listen to the scientists, ask clarifying questions, and write down the rules in a clear, structured format that the developers can understand.

The Problem: The "Jagged Frontier"

The paper explains that AI is like a jagged mountain range. Sometimes it's incredibly smart and helpful; other times, it's confused and makes up facts. If you are an expert, you know how to navigate the safe paths. If you are a novice, you might wander off a cliff. The goal of CARE is to build the AI so that it acts like the expert, no matter who is using it.

The CARE Process: A Five-Step Construction Site

Instead of just typing a prompt and hoping for the best, CARE uses a "stage-gated" process. This means you can't move to the next step until the current step is approved by the humans.

Scope and Decompose: The team defines the goal. What exactly is this robot supposed to do? The Helper Agent helps write down the boundaries.
Key Information Elicitation: The team gathers the "tools" and "context." What databases can the robot access? What does a correct answer look like? The Helper Agent turns these answers into a checklist.
Reasoning Policy and Guardrails: This is the most important part. The team decides how the robot thinks. When should it ask for help? When should it say "I don't know"? The Helper Agent drafts these rules, and the humans approve them.
Prompt Architecture: Only after the rules are set does the team write the actual instructions (the "prompt") for the AI. Because the rules are already clear, the prompt is just a translation of those approved rules, not a guess.
Benchmarking and Verification: Before the robot goes to work, it takes a test. The team creates a set of questions to see if the robot actually follows the rules they wrote down.

The "Helper Agent" Analogy

Think of the Helper Agent as a construction foreman.

The Scientist (SME) says, "I need a wall that is safe and strong."
The Developer says, "Okay, I'll build it."
Without a foreman, they might argue about what "strong" means, or the developer might build a wall that looks good but collapses.
With the Helper Agent, it steps in and says, "Scientist, does 'strong' mean it can hold 500 pounds? Developer, does your plan meet that 500-pound requirement?" It writes down the exact specs in a contract (the "artifact"). If the plan changes later, the contract is updated, and everyone knows exactly what changed.

The Real-World Test: The NASA Case Study

To prove this works, the team built an AI agent for NASA to search through a massive catalog of Earth science data (the NASA Common Metadata Repository).

The Contest: They built two agents.
- Agent A (The CARE Agent): Built using the strict 5-step CARE process with the Helper Agent.
- Agent B (The Baseline): Built the "old way" (just using the same AI model and tools, but without the strict CARE process).
The Results:
- On a large, automated test of 621 questions, the CARE agent found the right answer first 71.7% of the time, while the old way only got 69.1%.
- On a smaller, harder test created by real NASA scientists (43 questions), the CARE agent found the right answer in the top 5 results 27.2% of the time, beating the old way's 20.2%.

The Bottom Line

The paper claims that by using a structured process where humans and "Helper Agents" work together to write down clear rules before building the AI, you get a more reliable, safer, and better-performing robot. It turns the chaotic art of "prompt engineering" into a predictable engineering discipline.

Important Note: The paper only tested this on finding scientific data. It does not claim this method works for medical diagnosis, legal advice, or other specific fields unless those fields are explicitly mentioned in future work, which this paper does not cover. The success is strictly limited to the NASA data search example provided.

1. Problem Statement

The paper addresses the "jagged technological frontier" of Large Language Models (LLMs), where performance is highly inconsistent and dependent on user skill. While LLMs offer potential for scientific workflows, a gap exists between novice and expert analysts:

Novices often accept fluent but incorrect results or under-specify constraints.
Experts consistently apply domain constraints, verify outputs, and structure multi-step workflows.
Current Limitations: Building reliable AI agents typically relies on trial-and-error prompt iteration, which leads to systems that fail silently, regress after minor changes, or overfit to narrow test cases. There is a lack of a disciplined engineering methodology to translate tacit expert knowledge into explicit, verifiable, and maintainable agent specifications.

2. Methodology: CARE (Collaborative Agent Reasoning Engineering)

CARE is a stage-gated, artifact-driven methodology that reframes agent development as a collaborative engineering process. It introduces a three-party workflow involving:

Subject-Matter Experts (SMEs): Provide domain knowledge and constraints.
Developers: Implement technical solutions and tool orchestration.
Helper Agents: LLM-based facilitators that act as "specification infrastructure," converting informal SME intent into structured, reviewable artifacts.

Core Design Targets

CARE deconstructs an agent into four design targets, each mapped to specific artifacts:

Interaction Policy & Reasoning Strategy: How the agent decomposes tasks, manages uncertainty, and verifies steps.
Domain Grounding: Authoritative knowledge sources and constraints to prevent hallucinations.
Tool Orchestration: Logic for selecting, invoking, and validating external tools (APIs).
Evaluation & Verification: Metrics and benchmarks to detect regressions.

The Five-Phase Process

The methodology proceeds through five phases with strict stage gates (human approval required to proceed):

Phase 1: Scope and Decompose: Define the workflow, users, and constraints. Helper agents draft initial scope artifacts.
Phase 2: Key Information Elicitation: Structured gathering of Tools (APIs/limits), Context (retrieval strategies), and Output Format (templates/provenance).
Phase 3: Reasoning Policy and Guardrails: Explicitly define failure modes, safety boundaries, and decision logic (e.g., when to ask for clarification vs. when to abstain).
Phase 4: Prompt Architecture & Implementation: Translate approved artifacts into the actual agent prompt using established patterns (e.g., planning, critique, tool-use scaffolding).
Phase 5: Benchmarking and Verification: Define realistic query sets and scoring rubrics to validate performance before deployment.

Key Mechanism: Helper agents maintain continuity by reusing artifacts from previous phases, allowing teams to iterate without losing alignment. All artifacts are versioned (e.g., in GitHub) to support auditability.

3. Key Contributions

CARE Framework: Introduction of a disciplined, three-party (SME, Developer, Helper Agent) methodology that replaces ad-hoc prompt engineering with a specification-first approach.
Artifact-Driven Design: Deconstruction of agentic functions into four distinct targets (Policy, Grounding, Orchestration, Verification) mapped to concrete, reviewable Markdown artifacts.
Helper Agents as Infrastructure: The novel concept of using LLMs not just as the deployed agent, but as facilitation tools to accelerate the elicitation and specification process while keeping humans in the loop for approval.
Empirical Validation: A case study demonstrating that a CARE-engineered agent outperforms a baseline with identical model and tool access, proving that the engineering process itself drives performance gains.

4. Results: NASA Earth Science Case Study

The authors evaluated CARE by building a NASA Earth Science Data Discovery Agent that queries the Common Metadata Repository (CMR) API.

Comparison:
- $cmr\_care\_v1$ : Built using the CARE methodology.
- $cmr\_simple$ : A baseline "LLM + Tool" agent built via standard iterative prompting (no CARE artifacts/gates).
Evaluation Strategy: A two-gate benchmarking approach:
1. Synthetic Gate: 621 queries generated from research papers (fast iteration).
2. Gold Gate: 43 queries formulated by SMEs (high-confidence validation).
Metrics: Recall@K (proportion of ground-truth datasets found in the top K results).

Performance Outcomes:

Metric	Synthetic Gate (n=621)	Gold Gate (n=43)
Metric	Recall@1	Recall@5
CARE Agent	71.7%	27.2%
Baseline	69.1%	20.2%
Improvement	+2.6%	+7.0%

Note: The CARE agent showed statistically significant improvements, particularly in the SME-grounded Gold Gate (Recall@5), indicating better handling of complex, expert-level queries.

5. Significance and Limitations

Significance:

Bridging the Skill Gap: CARE operationalizes expert workflows, allowing agents to perform reliably without requiring every user to be an expert prompt engineer.
Maintainability: By treating prompts as derived outputs of versioned artifacts, CARE makes agents easier to audit, update, and maintain as models or tools change.
Reliability: The stage-gated process forces explicit definition of guardrails and failure modes, reducing "silent failures" common in trial-and-error development.

Limitations & Future Work:

Helper Agent Quality: The methodology relies heavily on the helper agent's ability to ask the right questions; poor elicitation can lead to missing constraints.
Benchmark Sensitivity: Results depend on the specific benchmark construction; there is a risk of "teaching to the test" if the helper agent inadvertently optimizes for the benchmark distribution.
Model Drift: Changes in underlying LLMs can alter compliance with reasoning policies, requiring periodic re-verification.
Future Direction: The authors plan to test CARE across multiple domains and tool ecosystems to identify which phases drive the most significant performance gains.

In conclusion, CARE demonstrates that systematic engineering—rather than just model capability—is the critical factor in building robust, reliable, and high-performing AI agents for scientific and technical domains.