Generative AI systems conflate two distinct pregnancy-dating algorithms

Technical Summary: Generative AI Systems Conflate Two Distinct Pregnancy-Dating Algorithms

Problem Statement
The paper identifies a critical discrepancy in how generative AI systems handle clinical calculations, specifically regarding the estimation of the due date (EDD) and gestational age. While the medical community often treats "Naegele's rule" (a calendar-based heuristic) and a "fixed 280-day count" as identical, they are mathematically distinct algorithms. Naegele's rule, formulated in 1812, instructs clinicians to subtract three calendar months from the first day of the last menstrual period (LMP) and add seven days. In contrast, the fixed 280-day count simply adds 280 days to the LMP. Because the Gregorian calendar contains months of varying lengths (28–31 days), these two methods frequently yield different results. The authors argue that generative AI models, trained on textbooks that have historically conflated these methods, may reproduce this confusion by reciting the steps of one algorithm while computing the output of the other, thereby obscuring the calculation pathway.

Methodology
The authors employed a deterministic audit approach to evaluate this hypothesis:

1. Ground Truth Generation: A script enumerated every possible LMP date from January 1, 1812, through December 31, 2025 (N = 78,163 dates). For each date, two EDDs were computed: one using Naegele's calendar rule ($EDD = \text{Date}(year+1, month-3, day+7)$) and one using the fixed 280-day count ($EDD = LMP + 280 \text{ days}$). The difference ($\Delta$) between the two was calculated.
2. AI Audit: On June 17, 2026, five widely used generative AI systems (ChatGPT 5.5, Claude Opus 4.8, Gemini 3.5 Flash, Perplexity Sonar 2, and Open Evidence) were queried with two standardized prompts:
   • Prompt 1 (Convergent): LMP of January 21, 2026 (a date where both methods yield the same result).
   • Prompt 2 (Divergent/Leap-crossing): LMP of January 21, 2024 (a date where the methods diverge due to the inclusion of February 29 in the gestational interval).
3. Evaluation: Responses were analyzed for the returned EDD, the stated calculation method, and whether the system distinguished between the calendar rule and the 280-day count.

Key Results

• Algorithmic Divergence: Across the 78,163 dates, the two methods produced identical results in only 44.3% of cases. In 55.7% of cases, Naegele's rule yielded a date 1–3 days later than the fixed 280-day count. The discrepancy was unidirectional; Naegele's rule never produced an earlier date.
• Leap Year Impact: In intervals crossing February 29 (n = 14,828), the methods never agreed (0.0% agreement). The 280-day count absorbs the intercalary day, while the calendar subtraction does not.
• AI Performance:
  • ChatGPT 5.5 and Claude Opus 4.8 correctly distinguished the methods for the divergent date, providing the 280-day result (Oct 27) while acknowledging the calendar rule yields Oct 28.
  • Gemini 3.5 Flash, Perplexity Sonar 2, and Open Evidence failed to distinguish the methods. They either labeled the 280-day output as "Naegele's rule" (despite showing calendar steps that led to a different date) or asserted that the calendar date equaled the 280-day count.
• Nature of Error: The failure was conceptual rather than arithmetic. Three systems presented the calendar rule and the 280-day count as the same procedure, effectively reproducing the historical conflation found in medical textbooks.

Significance and Claims
The authors assert that while a 1–3 day difference is clinically trivial given the biological variability of human gestation (standard deviation ~13 days), the audit reveals a fundamental flaw in generative AI behavior: the system can return a plausible clinical value while concealing or mislabeling the method used to produce it.

The paper claims this is a "method-label error" where models rely on pattern-matching rather than logical consistency. The authors emphasize that this pregnancy-dating case serves as a "deterministic test case" with a clean ground truth to expose a broader problem. They hypothesize that similar method-label mismatches in higher-stakes clinical scenarios, such as drug dosing or risk scoring, could be far less benign, though this work does not demonstrate such harm directly.

The primary contribution is a call for method transparency. The authors argue that medical AI tools must disclose their calculation pathways (e.g., explicitly stating whether they add 280 days or use calendar arithmetic) along with their assumptions and uncertainty, rather than presenting a single, over-precise date that masks the underlying algorithmic choice.