Think Fast, Talk Smart: Partitioning Deterministic and Neural Computation for Structured Health Text Generation

Technical Summary: Think Fast, Talk Smart (TFTS)

Problem Statement
Large Language Models (LLMs) are increasingly deployed to generate health text from structured records (e.g., wearable time series, biomarkers, care logs). While LLMs excel at fluency, recurring health-generation tasks demand strict adherence to source data, evidence-grounded claims, policy compliance, and machine-readable outputs. The core problem addressed is determining which responsibilities in this pipeline should be handled by deterministic computation versus runtime LLM prompting. Current approaches often rely on single-call prompting (zero-shot or few-shot) to perform the entire analytical and linguistic task, risking numeric hallucinations, policy violations, and higher costs.

Methodology
The authors introduce Think Fast, Talk Smart (TFTS), a partitioning framework for sleep-health insight generation. The system decomposes the task into four distinct stages:

1. Structured Inputs: Ingestion of wearable-derived signals (sleep, recovery, physiology, behavior).
2. Deterministic Analytical Modules: Code-based layers handle recurring, verifiable reasoning tasks, including:
   • Comparison: Numerical comparisons against baselines.
   • Ranking: Selecting which metrics to surface based on rules.
   • Attribution: Gating explanatory claims based on available evidence.
   • Handoff: Compiling selected facts into a compact, schema-constrained interface.
3. Bounded LLM Writer: A single LLM call that takes the deterministic interface and generates user-facing prose, strictly constrained by the provided facts and schema.
4. Deterministic Evaluation: Automated parsing and verification of the final output.

To evaluate the necessity of this partitioning, the authors employ a Layer-Replacement Protocol. Instead of comparing the full system against a monolithic prompt, they systematically replace individual deterministic layers (Comparison, Ranker, Attribution, Handoff) with LLM-generated typed artifacts. This isolates specific failure modes, testing whether an LLM can preserve the intended meaning of an intermediate analytical step beyond merely producing syntactically valid JSON.

Experimental Setup

• Dataset: 280 user-nights from 20 active users.
• Models: Six LLMs were evaluated (GPT-5 nano, GPT-4o mini, GPT-OSS-20B/120B, Haiku 4.5, Sonnet 4.6).
• Baselines:
  • Structured Zero-Shot: Raw data and rules in a single prompt.
  • Structured Few-Shot: Includes metric tables and examples in a single prompt.
• Metrics: Numeric Error Rate (NUMERR), Instruction-Compliance Error (SELERR + ATTRERR), Schema Error (SCHEMAERR), and end-to-end cost.

Key Results

1. Error-Cost Frontier: TFTS consistently occupies the low-cost, low-error region across all six models. It achieved an average NUMERR of 0.7% (worst model 2.0%) and instruction-compliance error below 3%. In contrast, one-call baselines failed to recover this frontier; Structured Few-Shot improved schema adherence but still exhibited high aggregate selection errors (68.6%) and numeric errors.
2. Layer Replacement Findings:
   • Replacing Comparison: Delegates numerical observation to the LLM, causing NUMERR to spike to 16.9%.
   • Replacing Ranker: Keeps facts grounded but degrades adherence to selection rules, resulting in 65.8% selection error.
   • Replacing Attribution: Increases unsupported causal language, with attribution errors rising to 24.5%.
   • Replacing Handoff: Even when upstream facts are deterministic, replacing the compact evidence-to-writer interface reintroduces errors (NUMERR 4.8%, ATTRERR 10.7%), indicating the interface itself is a reliability mechanism.
3. Cost Efficiency: TFTS achieves lower end-to-end costs than baselines by utilizing one bounded LLM call rather than relying on complex, multi-step prompting or iterative refinement within a single call.

Significance and Claims
The paper argues that for structured health workloads with recurring analytical rules, deterministic code should own the analysis, while LLMs should be restricted to expressing verified facts within bounded interfaces.

• Design Rule: "Let code think fast over stable facts and policies, then let the LLM talk smart within verified bounds."
• Failure Modes: The study reveals that specific analytical responsibilities (arithmetic, ranking policy, evidence gating) are prone to failure when delegated to LLMs, even when the LLM produces syntactically correct outputs.
• Scope: The authors position this as a complementary approach to Retrieval-Augmented Generation (RAG) and tool use. While those methods improve context retrieval or learned calls, TFTS addresses which responsibilities should be removed from runtime prompting entirely.
• Limitations: The study focuses on the preservation of a deterministic reference system, not clinical optimality or open-ended diagnosis. The partitioning principle is most applicable to domains with structured inputs, stable rules, and verifiable intermediates (e.g., lab results, vitals monitoring), rather than tasks requiring open-ended clinical reasoning.

The authors conclude that relying on single-call prompting for structured health generation is insufficient for reliability and cost-efficiency, advocating for a hybrid architecture where code handles the "thinking" and LLMs handle the "talking."