Auditing Generative AI Error Patterns in Applied Differential Calculus: Evidence from ChatGPT and Gemini

Technical Summary: Auditing Generative AI Error Patterns in Applied Differential Calculus

Problem Statement
While Generative Artificial Intelligence (GenAI) tools like ChatGPT and Gemini are increasingly adopted in higher education for mathematical explanation and problem-solving, their reliability in applied Differential Calculus remains underexamined, particularly regarding the integrity of solution processes. Unlike language tasks where fluency often correlates with quality, mathematical validity depends on the correctness of every symbolic and inferential step. A single error in translating narrative conditions into functions, mishandling integration constants, or failing contextual verification can invalidate an entire solution. Existing literature highlights the pedagogical ambivalence of GenAI—offering immediate support while risking the substitution of reasoning with fluent but inaccurate outputs. This study addresses the specific gap in understanding how these tools fail in applied calculus contexts involving business optimization, particle kinematics, and vertical motion, where the translation of verbal constraints into mathematical models is critical.

Methodology
The study employed a descriptive-evaluative design utilizing qualitative comparative error analysis. The corpus consisted of selected applied Differential Calculus problems from early 2026 across three domains:

1. Business Optimization: Translating price-rebate relationships into demand and revenue functions.
2. Particle Kinematics: Recovering velocity and position functions from acceleration using initial conditions.
3. Vertical Projectile Motion: Deriving motion functions from constant acceleration and solving for impact time.

Two GenAI models, ChatGPT and Gemini, were tested using standardized, neutral prompts ("Solve the following problem step-by-step") without corrective feedback or prompt engineering to simulate typical student usage. Generated outputs were compared against expert-verified solutions. The analysis focused on five specific dimensions of error:

• Semantic-to-algebraic translation
• Calculus procedure execution
• Algebraic and arithmetic execution
• Boundary-condition (constraint) handling
• Contextual relevance and verification

Key Contributions and Results
The study identifies that while both tools demonstrate partial procedural competence in isolated differentiation or integration routines, their reliability significantly declines when tasks require maintaining context, tracking constraints, or verifying physical plausibility. The analysis revealed three dominant error typologies:

1. Semantic-to-Algebraic Translation Errors:

   • Observation: Both models frequently failed to correctly convert verbal problem statements into mathematical functions.
   • Specifics: In the business optimization task, the tools generated flawed demand/revenue setups (e.g., implying a positive slope for demand where it should be negative). Consequently, subsequent derivative work, though procedurally correct, operated on invalid functions, rendering the final optimization results meaningless.

2. Boundary-Condition Tracking Errors:

   • Observation: Both tools struggled to consistently carry integration constants and initial conditions through multi-step solutions.
   • Specifics: In kinematics and vertical motion tasks, ChatGPT tended to default displacement constants to zero when not explicitly tracked, while Gemini occasionally reinterpreted correct constants as errors and recalculated under different initial conditions. This resulted in final functions describing physical trajectories that contradicted the problem's stated starting conditions.

3. Verification and Contextual Relevance Errors:

   • Observation: The tools exhibited arithmetic fragility and a failure to reject impossible results or remove irrelevant information.
   • Specifics: Gemini showed vulnerability in quadratic root calculations for impact time and inserted irrelevant explanatory notes (e.g., unnecessary gravity definitions). ChatGPT often omitted initial height values in vertical motion problems. Both tools produced fluent, confident narratives that masked these underlying mathematical failures.

Comparative Patterns

• ChatGPT: Its primary weakness was constraint retention. It tended to follow recognizable algorithmic templates but frequently lost the specific conditions (such as initial height or displacement constants) required to anchor the solution to the problem statement.
• Gemini: Its primary weakness was inconsistent validation. It sometimes proceeded with structurally invalid functions without recognizing contradictions and demonstrated higher susceptibility to arithmetic execution errors in numerical evaluations.

Significance and Pedagogical Implications
The paper argues that GenAI should not be treated as an unquestioned source of answers in calculus instruction. The study posits that the "danger" of these tools lies not merely in wrong final answers, but in the presentation of superficially coherent solutions that begin with invalid premises (e.g., a wrong revenue function) or lose critical physical constraints.

The authors propose a shift in pedagogy from answer acquisition to mathematical auditing. The significance of the findings lies in reframing AI-generated errors as productive learning artifacts. Instead of using AI to solve problems, instructors should use AI outputs as critique objects where students are tasked to:

• Identify the first invalid step in the solution chain.
• Perform semantic audits of variable definitions and signs.
• Track constraints and integration constants explicitly.
• Verify results through independent computational methods (e.g., estimation, graphing, or computer algebra systems).

Limitations and Scope
The study modestly frames its findings as evidence from specific tools, tasks, and a specific time of data collection (early 2026). It acknowledges that model updates and prompt variations may alter future outputs. The analysis is qualitative and task-level, documenting meaningful failure patterns rather than estimating population-level accuracy rates across all calculus topics. The authors recommend that future research utilize larger problem banks and mixed-methods scoring to further explore how students respond to AI-auditing tasks and whether such practices improve conceptual understanding and verification skills.