Technical Summary: Grounded Optimization for Reducing LLM Hallucination in Automated Personal Document Rewriting

1. Problem Statement

The application of Large Language Models (LLMs) to resume optimization for Applicant Tracking Systems (ATS) introduces specific hallucination risks distinct from general text generation. Unlike chatbots where users can verify outputs interactively, resume optimization often operates in batch mode with minimal human review. The authors identify four distinct failure modes that carry concrete consequences, such as candidate disqualification or termination:

1. Temporal Fabrication (H1): The injection of technologies that did not exist during the candidate's claimed employment period (e.g., citing LangChain in a 2018 role).
2. Cross-Domain Contamination (H2): The introduction of terminology from technology ecosystems not present in the original role (e.g., adding Azure services to an AWS-only role) to match job description keywords.
3. Structural Mutation (H3): The silent merging, deletion, or condensation of bullet points, which removes genuine achievements under the guise of "optimization."
4. Content Fabrication (H4): The invention of company names, inflation of metrics, or addition of unearned certifications.

The ground truth in this domain is the candidate's own career history, which the LLM must enhance without distorting.

2. Methodology: The Grounded Optimization Framework

The authors propose Grounded Optimization, a five-layer defense-in-depth framework implemented as a multi-agent system on LangGraph. The system processes resumes through specialized agents (Summary, Skills, Experience, Projects, Education) with the Experience Agent receiving the full defense stack. The layers operate in a specific sequence: Layer 4 (Prompt Injection) → LLM Call → Layers 1–3 (Output Validation) → Layer 5 (Evaluator Gate).

Layer 1: Temporal Context Validation

This layer prevents anachronistic technology injection by constructing a per-resume timeline. It embeds a curated mapping of technology release dates (e.g., LangChain → 2022) into the agent prompts. The LLM is instructed to verify that any technology mentioned in a specific role existed during that role's time period.

Layer 2: Cross-Domain Contamination Detection

To address cross-domain bleeding, the authors employ a fully deterministic, LLM-free mechanism. They constructed a taxonomy of 257 cloud services across AWS, GCP, Azure, and On-Premise stacks. Detection uses a two-tier word-boundary regex matching system:

• Tier 1: Attributes a provider based on a single explicit keyword match.
• Tier 2: Requires $\ge$2 service-name matches to resolve ambiguity (e.g., distinguishing "lambda" as AWS Lambda vs. a Python keyword).
  The system compares the cloud signature of a role before and after optimization. If new providers are detected, the output is reverted, and a warning is injected for a retry.

Layer 3: Structural Invariant Enforcement

This layer addresses structural mutation by counting semantic units before and after optimization. It validates that the number of experience entries ($|E'| \ge |E|$) and bullet points per entry ($|b'_i| \ge |b_i| - 1$) are maintained within a tolerance. Significant content loss triggers a retry with explicit structural targets.

Layer 4: Prompt-Level Content Grounding

This is the first line of defense, embedding explicit immutability rules directly into the agent prompts. Constraints include:

• Preserving the exact number of bullet points.
• Prohibiting the addition or modification of educational details.
• Preventing the creation of new companies or use of placeholders.
• Ensuring metrics remain realistic for the time period.

Layer 5: Evaluator Agent QA Gate

An independent LLM instance acts as an adversarial quality-control agent. It receives the original resume, the rewritten version, and the target job description. It returns a binary acceptance decision and feedback. If the evaluator rejects the output, the feedback is injected into the next rewrite attempt. If the evaluator fails (timeout/malformed output), it defaults to rejection to prevent silent acceptance of hallucinations.

Fallback Mechanism

If validation fails after three retry cycles, the system employs a deterministic fallback-merge mechanism that combines the best LLM output with the preserved original content to ensure no original data is lost.

3. Key Contributions

1. Hallucination Taxonomy: A classification of four specific hallucination modes (Temporal, Cross-Domain, Structural, Content) unique to personal document optimization.
2. Five-Layer Framework: A functional multi-agent system combining deterministic validation, prompt engineering, and adversarial checking.
3. Deterministic Contamination Detector: A regex-based tool covering 257 cloud services with two-tier confidence scoring, designed to avoid introducing a second LLM hallucination surface.
4. Ablation and Sensitivity Analysis: An evaluation across 16 experimental conditions (3 LLMs, 4 temperatures, 6 layer configurations, 680 LLM invocations) characterizing the contribution of each layer.

4. Experimental Results

The framework was evaluated on 25 synthetic resumes spanning 14 industries and 42 roles.

• Baseline Performance: Undefended baselines produced 2.48–5.36 detected hallucinations per resume, with Cross-Domain Contamination (H2) being the dominant failure mode (79–89% of incidents).
• Ablation Study (GPT-4.1-nano, t=0):
  • Prompt-Level Grounding (L4) alone: Achieved 0.00 detected hallucinations in this specific configuration.
  • Full Framework: Achieved a rate of 0.12 (0.04 when excluding likely false positives in H4).
  • The study noted that L4-only performance did not generalize to weaker models or higher temperatures.
• Multi-Model Generalization:
  • Less capable models (GPT-4o-mini, Llama-3.1-8B) produced 2–4$\times$ more baseline hallucinations.
  • The Full framework reduced detected hallucination rates by 95.2% – 99.3% across all tested models.
• Temperature Sensitivity:
  • As temperature increased from 0.0 to 1.0, the residual hallucination rate under the Full framework increased from 0.12 to 0.24.
  • This increase was driven almost entirely by Temporal (H1) residuals, suggesting that prompt compliance weakens with higher stochasticity, necessitating the deterministic layers (L1–L3) as a complement rather than a replacement for prompt grounding.

5. Significance and Limitations

Significance:
The paper argues that while prompt engineering is effective under ideal conditions (strong models, low temperature), it is insufficient for robust production systems. The deterministic layers (L1–L3) provide critical value precisely where prompt compliance is weakest (high temperature, weaker models). The work characterizes the specific failure modes of personal document optimization and provides a systematic, layered approach to mitigating them without relying solely on external knowledge bases.

Limitations:
The authors explicitly note several constraints:

• H2 Detector-Defense Coupling: The H2 detector and Layer 2 defense share the same underlying function. Consequently, H2 counts in configurations where Layer 2 is active are mechanically zero by construction and cannot be interpreted as independent empirical measurements of elimination.
• Dataset: The evaluation relies on 25 synthetic resumes and a taxonomy of 257 services, which may miss real-world distributions and long-tail platforms (e.g., Salesforce, SAP).
• Detector Scope: The detectors are task-specific and do not compare against general hallucination benchmarks like SelfCheckGPT or FActScore.
• Magnitude: The framework detects the presence of hallucinations but not necessarily the magnitude of fabrication.

The authors conclude that future work should prioritize an independent NLI-based evaluator for cross-domain contamination and human annotation on real resumes.