🤖 AI

A Deterministic Evidence Layer for Vision-Language Autism Screening from Naturalistic Home Video

This paper proposes a deterministic evidence layer that stabilizes vision-language model outputs for autism screening by converting naturalistic home videos into timestamped, labeled event tables scored via a transparent weight-of-evidence mechanism, achieving robust and interpretable diagnostic performance on preschool children.

Original authors: Wenqi Li, Mindi Ruan, Chuanbo Hu, Shuo Wang, Xin Li

Published 2026-10-08

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Wenqi Li, Mindi Ruan, Chuanbo Hu, Shuo Wang, Xin Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Diagnosing autism spectrum disorder often begins with a specialist watching a child play, looking for subtle signs in how they interact with others or repeat certain movements. This process is highly effective but relies on a scarce resource: trained experts. Because there are not enough specialists to see every child who might need help, families often face long waits for an evaluation, and early intervention is delayed. In recent years, scientists have looked to home videos—recorded by parents on their phones during ordinary play—as a way to bridge this gap. The hope is that artificial intelligence could watch these videos and spot the same patterns a human expert would, offering a quick, accessible first step to decide which children need urgent attention. However, a major hurdle has emerged: even the most advanced AI models, when asked to make the same judgment twice, can give different answers. This inconsistency makes them unreliable for medical screening, where a decision must be the same every time to be trusted.

A team of researchers has tackled this problem by building a new kind of screening system that separates what the computer sees from how it decides. Instead of asking a single artificial intelligence to watch a video and immediately say "autism" or "no autism," they created a two-step process that acts more like a careful human observer followed by a strict accountant. First, a powerful vision model watches the video and writes down a detailed list of events, noting exactly what the child did, when it happened, and what the parent or caregiver did just before. Crucially, this model is forced to stick only to what is visible in the video, avoiding guesses about the child's inner thoughts. It also has to list examples of normal behavior, not just the unusual ones, to provide a complete picture. This list is then passed to a second, text-only model that adjusts the importance of each event based on the child's age, understanding that some behaviors are normal for a toddler but concerning for a preschooler. Finally, a deterministic computer program, which follows fixed mathematical rules without any guesswork, adds up the evidence from this list to produce a final risk score.

The researchers tested this system on 43 home videos of preschool children, recorded by their families without any special instructions or scripts. The videos included children who had been diagnosed with autism by specialists and children who were developing typically. When the system watched these videos, it achieved a high level of accuracy, correctly identifying the risk level in about 86 percent of the cases. More importantly, the system was remarkably stable. When the researchers ran the same videos through the system three times, the final decision remained the same for nearly three-quarters of the clips every single time. In contrast, a standard artificial intelligence model without this special structure changed its mind on nearly a third of the clips between runs. The new system was particularly good at avoiding false alarms; it never flagged a typically developing child as high-risk in every single run, whereas a standard model flagged nine out of thirty-one such children in every run.

The key to this success was not making the artificial intelligence smarter, but rather changing how it was used. The researchers found that the instability in standard models comes from the way they generate answers, which can vary slightly even when the settings are identical. By moving the final decision out of the AI and into a fixed scoring system that simply adds up the evidence the AI collected, they removed that source of error. The system also introduced strict rules for the AI, requiring it to name the specific moment a child failed to respond to a parent's call or a gesture, rather than just guessing that the child was unresponsive. This "grounded" approach meant the AI could only report what it actually saw, and the final score was built on a transparent record of those observations.

While the results are promising, the researchers are careful to note the limits of their work. The system was tested on a relatively small group of children from one location, and it missed three children with autism in every single run, suggesting that the current technology still struggles to detect certain subtle signs. The system works best as a triage tool to help prioritize children for specialist evaluation, not as a final diagnosis. The study demonstrates that by combining the pattern-recognition power of modern AI with rigid, transparent rules for decision-making, it is possible to create a screening tool that is both accurate and reliable. This approach offers a path forward for using home videos to reduce the waiting time for autism evaluations, provided that future work can expand the system to include more diverse children and improve its ability to catch every case.

Technical Summary: A Deterministic Evidence Layer for Vision-Language Autism Screening from Naturalistic Home Video

1. Problem Statement

Autism Spectrum Disorder (ASD) diagnosis relies on specialist observation of social behavior, creating a bottleneck due to scarce expertise and long wait times. While home videos offer a scalable source of behavioral data, current Vision-Language Models (VLMs) used for screening suffer from instability. Even at temperature 0, repeated runs of identical VLM-based screening pipelines yield different verdicts for 16–37% of clips due to non-determinism in the serving stack. A clinical screening tool requires stable correctness: the ability to label a clip correctly in every run, not just on average. Furthermore, existing approaches either fail on fine-grained clinical behaviors when using frozen models or require fine-tuning on clinician-rated data, which limits generalizability to unstructured, protocol-free home videos.

2. Methodology

The proposed pipeline is training-free and separates perception from decision-making into three distinct stages, ensuring the final verdict is a deterministic function of a saved evidence table.

Stage 1: Grounded Perception (Frozen VLM)

A frozen VLM (Gemini 3.1 Pro) acts as a high-recall perception front-end. It processes the raw home video without knowledge of the child's age to prevent expectation bias.

Grounded Perception Constraints: The model is prompted to generate a timestamped event table where every observation must be tied to a specific time interval.
Behavioral Glossary: The model uses a clinically aligned glossary of 11 ASD-related features (e.g., eye contact, joint attention, stereotypies).
Counter-Evidence: Typical behaviors (e.g., responsive smiling) are explicitly logged as counter-evidence rows.
Press Rules: For "probe-gated" features (e.g., response to name), the model only codes a deficit if a specific social "press" (e.g., calling the name) occurred and the child failed to respond. Absence without a press is not coded.
Output: A structured table of events with timestamps, observable descriptions, glossary labels, and confidence levels (certain/possible).

Stage 2: Text-Only Age Calibration

A text-only model (never seeing the video) receives the event table and the child's age.

Never-Erase Rule: This stage cannot delete rows or upgrade confidence. It can only lower confidence (e.g., from "certain" to "possible") if a behavior is age-appropriate (e.g., solitary play in a 2-year-old) or add a pervasive feature (#4) based on the aggregate table.
Output: A calibrated event table where weights are adjusted for developmental norms.

Stage 3: Deterministic Weight-of-Evidence Scorer

A fixed, non-learned algorithm converts the calibrated table into a risk score.

Scoring Mechanism: It sums the log-positive-likelihood-ratio ( $\ln LR+$ ) weights of observed features.
Constraints:
- Social-communication (SC) features are capped at 1.5 nats to prevent saturation.
- Restricted/Repetitive (RRB) features sum fully.
- Confidence factors ( $c_f$ ) scale the weights (1.0 for certain, 0.25 for age-downgraded possible).
Decision Rule: The total score ( $s$ $s$ ) is stratified by two fixed cut-offs:
- Low Risk: $s < 1.5$
- Intermediate/High Risk (Screen-Positive): $s \ge 1.5$ (with High Risk $\ge 1.8$ ).
Auditability: Every decision decomposes into named per-feature contributions, allowing re-scoring without model calls.

3. Key Contributions

Auditable, Training-Free Architecture: The system separates grounded observation from a deterministic, evidence-based decision. It achieves high performance without fine-tuning the backbone or the decision layer, relying instead on a structured event table and a fixed scorer.
Grounded Perception Constraints: By enforcing temporal grounding, explicit counter-evidence logging, and "press" rules for social deficits, the system significantly improves specificity and F1 scores compared to unconstrained VLM prompting.
Stable Correctness Framework: The paper introduces "stable correctness" (percentage of clips labeled correctly in every run) as a critical metric for clinical tools. The proposed pipeline achieves 74.4% stable correctness, significantly outperforming the 60.5% of zero-shot baselines, by removing the decision rule from the stochastic serving stack.

4. Experimental Results

The pipeline was evaluated on 43 caregiver-recorded, protocol-free home video clips (12 ASD, 31 Typically Developing) from preschool children (ages 26–48 months).

Performance Metrics (Full Pipeline):
- AUC: $0.851 \pm 0.012$
- Accuracy: $86.0\%$
- Specificity: $94.6\%$
- Recall: $63.9\%$
- F1 Score: $71.8$
Stability: Across three runs, the full pipeline correctly labeled 74.4% of clips in every run, compared to 60.5% for the zero-shot baseline.
False Positives: No Typically Developing (TD) clip was flagged in every run (0/31), whereas the zero-shot baseline flagged 9/31 TD clips in every run.
Ablation Study:
- Moving from a single-call VLM to a two-stage split improved accuracy from 69.0% to 77.5% (driven by specificity).
- Adding grounded perception constraints further improved accuracy to 81.4%.
- Replacing the LLM verdict with the deterministic scorer yielded the final 86.0% accuracy and 71.8 F1.
Failure Analysis: The system missed 3 ASD clips in every run; these clips had near-empty event tables, indicating the bottleneck lies in the initial perception (Stage 1) rather than the decision logic.

5. Significance and Claims

The paper claims that stability is a prerequisite for clinical utility. A screening decision that fluctuates between identical runs is not a viable clinical tool. By moving the decision logic out of the probabilistic model and into a deterministic, arithmetic layer based on a saved, auditable evidence table, the system achieves:

Reproducibility: Decisions can be re-scored and verified without re-running the VLM.
Transparency: Every "screen-positive" verdict traces back to specific, timestamped behavioral events and counter-evidence.
Tunability: The operating point (recall vs. specificity) can be adjusted by changing the cut-off thresholds on the saved tables, allowing sites to tune for rule-in or rule-out scenarios without retraining.

The authors acknowledge limitations, including the small sample size (43 clips), the single-site nature of the data, and the fact that the current recall ceiling (75%) is limited by the perception stage's ability to detect subtle deficits in unstructured home videos. The work positions this pipeline as a triage step for families waiting for specialist evaluation, rather than a definitive diagnostic tool.