What is Human in Judgment? Comparing Automation Bias and Algorithm Aversion Between the United States Military Academy and the General Public

Technical Summary: Human in Judgment? Comparing Automation Bias and Algorithm Aversion Between the United States Military Academy and the General Public

Problem Statement
As militaries increasingly integrate Artificial Intelligence (AI) Decision Support Systems (DSS) into command structures—from predictive maintenance to automatic target recognition—the role of human judgment in conflict escalation and war remains central. However, the interaction between human operators and automated systems introduces specific cognitive risks: automation bias (the tendency to over-rely on algorithmic recommendations, leading to errors of commission or omission) and algorithm aversion (the premature rejection of automated outputs in favor of human judgment). While existing literature highlights these risks in civilian contexts, there is a significant gap in behavioral research regarding how military personnel, specifically future officers, interact with AI DSS. The central problem addressed is whether military training and exposure to AI mitigate these cognitive distortions or if military operators remain susceptible to the same biases as the general public, potentially increasing the risk of miscalculation and accidental escalation in high-stakes environments.

Methodology
The study employs a comparative survey experiment design to test susceptibility to automation bias and algorithm aversion.

• Samples: The authors compare two distinct groups:
  1. West Point Cadets (USMA): 236 cadets from the United States Military Academy, surveyed in May 2025. This group represents a demographically specific subset of future military leaders who receive mandatory training in warfighting, decision-making, and AI fundamentals.
  2. General Public: A demographically matched sample of 702 respondents from a cross-national survey (Horowitz and Kahn, 2024), restricted to individuals aged 25 or younger with university education to ensure comparability with the cadet sample.
• Experimental Task: Participants engaged in a target identification task involving aircraft classification (friendly vs. adversary) based on visual features (wing taper, wing tip shape, tail shape).
• Design: A within-subject, pre-post design where participants made an initial classification, received a recommendation from a Decision Support System (DSS), and then had the opportunity to revise their identification.
• Treatments: The study utilized a 2x4 factorial structure (plus a control condition) manipulating:
  • Source: Human Analyst vs. AI Algorithm.
  • Confidence Level: High (extensive testing/training) vs. Low (undergoing testing/training).
  • Accuracy: Recommendations were randomized to be either accurate or inaccurate.
• Variables:
  • Dependent Variable: The binary decision to "switch" an initial identification after receiving advice.
  • Independent Variables: "AI Background" (an index aggregating knowledge, familiarity, and experience), trust in AI, and sample group (USMA vs. Public).
  • Outcome Coding: "Automation bias" was defined as following incorrect advice; "Algorithm aversion" was defined as rejecting correct advice.

Key Results
The study yields several statistically significant findings regarding the cognitive calibration of military cadets compared to the general public:

1. Baseline AI Competence: West Point cadets demonstrated significantly higher levels of AI knowledge, familiarity, and experience (p < 0.001) compared to the matched general public sample. They also exhibited a more nuanced trust profile: while optimistic about AI's potential, they expressed greater caution regarding its accuracy and limitations.
2. Switching Behavior:
   • General Public: Showed a higher overall rate of switching identifications (0.116) and displayed a general bias toward automation, switching more often when advice came from an AI than a human, regardless of confidence levels.
   • West Point Cadets: Exhibited a significantly lower baseline switching rate (0.065). Crucially, their switching behavior was driven by confidence signals (switching nearly twice as often in high-confidence conditions vs. low-confidence) rather than the source of the advice (AI vs. Human). This suggests a focus on the justification of the recommendation rather than the type of recommender.
3. Automation Bias (Errors of Commission): In trials where participants received incorrect advice that they could visually verify, the general public followed incorrect advice 9.0% of the time. West Point cadets followed incorrect advice only 3.9% of the time. Regression analysis confirmed that being a West Point cadet significantly reduced the probability of automation bias (p < 0.05), even when controlling for AI background, age, and sex.
4. Algorithm Aversion (Errors of Omission): Contrary to the hypothesis that training might reduce aversion, the study found no statistically significant difference in algorithm aversion rates between the two groups (General Public: 45.1%; West Point: 44.4%). Both groups were significantly more likely to reject correct advice than to follow incorrect advice.

Key Contributions

• Empirical Evidence on Military AI: This paper provides the first direct experimental evidence comparing automation bias in a military population (future officers) against a demographically similar general public sample.
• Differentiation of Bias Types: The study empirically demonstrates that automation bias and algorithm aversion are distinct phenomena that may require different mitigation strategies. The data suggests that military training effectively reduces susceptibility to automation bias (over-trust) but does not necessarily reduce algorithm aversion (under-trust).
• Role of Training: The findings challenge the assumption that military operators are inherently more prone to automation bias due to high-pressure environments. Instead, the data suggests that specific education and exposure to AI (as found at USMA) can lead to "better calibrated trust," where operators rely on signals of system reliability (e.g., testing status) rather than blindly trusting the algorithm.

Significance and Claims
The authors claim that their findings offer a "modest" but significant insight into the future of international politics and conflict. The paper argues that:

• Mitigation is Possible: The negative consequences of AI DSS on miscalculation are not inevitable. Military education and training can meaningfully shape how AI influences decision-making, potentially reducing the risk of accidents caused by automation bias.
• Calibration over Blind Trust: The goal of military AI integration should be "justified confidence," where operators are trained to distinguish between high-confidence and low-confidence algorithmic outputs, rather than simply trusting or distrusting the technology.
• Limitations and Future Directions: The authors explicitly note that their results are limited to a specific demographic (young, educated, USMA cadets) and a specific type of bias (errors of commission). They do not claim that all military personnel are immune to bias, nor do they suggest that organizational SOPs (Standard Operating Procedures) cannot override individual training. They call for further research into non-commissioned officers, different branches, and older career officers to determine if the USMA training model is scalable or if other groups remain more susceptible to cognitive distortions.

Ultimately, the paper posits that while AI introduces new risks to crisis decision-making, human agency—specifically when supported by targeted education—remains a critical variable in preventing the erosion of appropriate judgment in war and peace.