Measuring LLM Trust Allocation Across Conflicting Software Artifacts

1. Problem Statement

Large Language Models (LLMs) are increasingly used in software engineering (SE) tasks like test generation and code review. However, these models often operate on bundles of heterogeneous artifacts (source code, method signatures, Javadoc, and test scaffolding) that may be incomplete, stale, or mutually inconsistent.

Current evaluation methods focus primarily on downstream outcomes (e.g., "Is the generated test correct?"). This approach fails to reveal the model's internal reasoning process. Specifically, existing evaluations cannot determine:

• Whether the model recognized degraded evidence.
• Whether it identified the unreliable source.
• How it calibrated its trust across conflicting artifacts.

Consequently, an LLM might produce a plausible output while silently relying on the wrong artifact (misallocating trust), making such failures difficult to detect. There is a lack of frameworks that explicitly evaluate artifact-level trust reasoning.

2. Methodology: The TRACE Framework

The authors introduce TRACE (Trust Reasoning over Artifacts for Calibrated Evaluation), a framework designed to elicit structured reasoning traces regarding input quality and consistency.

A. Experimental Setup

• Dataset: 456 curated Java method bundles derived from 25 real-world systems (OE25 benchmark). Each bundle includes:
  • Method Under Test (MUT)
  • Method Signature ($\sigma$)
  • Javadoc ($D$)
  • Test Prefix ($\tau_{prefix}$)
• Perturbation Protocol: To test sensitivity, the authors created 6 variants for each base sample (7 total versions including the clean base):
  1. Removal: Removing Javadoc description, @return tags, or both.
  2. Injection: Introducing bugs into the Javadoc, the MUT, or creating contradictions between them.
  3. Severity Levels: Bugs were categorized as Heavy (explicit), Normal (plausible defects), and Subtle (corner-case deviations).
• Models Evaluated: 7 diverse LLMs (Claude Opus/Sonnet/Haiku, GPT-4o/5.2, DeepSeek-V3.2, Grok-4).
• Blind Protocol: Models were prompted to analyze artifacts without knowing which were perturbed or the severity of the fault.

B. Trace Elicitation

Instead of asking for a final answer, TRACE prompts models to generate a structured JSON trace containing:

1. Per-Artifact Quality Assessment: A score [0, 1] for each artifact's completeness and consistency.
2. Inconsistency Detection: Explicit flags for conflicts between artifacts (e.g., Javadoc vs. MUT).
3. Source Prioritization: A ranked list of artifacts based on reliability.
4. Attribution: Identification of which specific artifact is faulty.

3. Key Contributions

1. Formalization of Trust Reasoning: The first framework to treat artifact-level trust reasoning as a first-class output, moving beyond downstream task accuracy.
2. Provenance-Annotated Benchmark: A controlled benchmark of 456 Java bundles with 6 aligned degradation variants, allowing precise measurement of model sensitivity to specific fault types and severities.
3. Large-Scale Cross-Model Analysis: A study of 22,339 traces revealing systematic asymmetries in how LLMs process natural language specifications versus code implementations.

4. Key Results

RQ1: Input-Quality Sensitivity

• Localization: Models generally penalize the specific perturbed artifact rather than downgrading the entire bundle.
• Asymmetry: Models are significantly more sensitive to documentation (Javadoc) degradation than implementation (MUT) degradation.
  • Heavy-to-Subtle Gap: 0.152–0.253 for Javadoc bugs vs. 0.049–0.123 for MUT bugs.
  • Models detect explicit documentation errors well but struggle with subtle code-level drift.

RQ2: Javadoc–MUT Inconsistency Detection

• Detection Rates: Models detect explicit documentation errors with high accuracy (67–94%) and Javadoc-MUT contradictions (50–91%).
• The "Implementation-Only" Blind Spot: When the Javadoc is correct but the MUT is subtly wrong, detection rates drop by 21–43 percentage points across all models. Models tend to trust the plausible documentation over the drifting code.
• Confidence Calibration: Confidence scores are poorly calibrated for 6 out of 7 models; high confidence does not reliably indicate correct detection.

RQ3: Semantic Code Understanding

• Driver of Fidelity: Inter-model variation is largely explained by semantic code understanding.
  • Robust Models: Sonnet, DeepSeek-V3.2, and Haiku maintain stable detection and description fidelity even as perturbations become subtle.
  • Fragile Models: GPT-4o shows a sharp collapse (76% $\to$ 32% detection) as surface cues disappear, suggesting reliance on pattern matching rather than deep semantic reasoning.
• Localization Errors: Some models (e.g., Opus) correctly identify a contradiction exists but frequently misattribute the fault to the wrong artifact (e.g., blaming the Javadoc when the code is at fault).

5. Significance and Implications

• Current Limitations: Current LLMs are effective auditors of natural-language specifications but are unreliable detectors of subtle code-level drift. They tend to over-trust documentation when code and docs disagree.
• Practical Deployment:
  • Triage: TRACE can be used as a pre-flight check to downweight low-quality or inconsistent inputs before downstream generation.
  • Model Selection: For detecting subtle contradictions, Sonnet, DS-V3.2, and Haiku are preferred. For explaining why a contradiction exists (once found), Opus is strong.
  • Caution: Confidence scores should not be used as a gate for automated decisions without per-model recalibration.
• Future Direction: The paper argues for explicit artifact-level trust reasoning mechanisms in SE workflows to prevent silent failures where models rely on incorrect sources.

In summary, the paper demonstrates that while LLMs can reason about input quality, they exhibit a systematic bias toward trusting documentation over code, and their ability to detect subtle implementation errors is heavily dependent on their underlying semantic code understanding capabilities.