💬 NLP

TRACE: A taxonomy-grounded synthetic dataset for teaching-program generation and session interpretation in Applied Behavior Analysis

This paper introduces TRACE, a taxonomy-grounded synthetic dataset of 2,999 examples designed to overcome HIPAA restrictions by enabling the training of AI models for Applied Behavior Analysis tasks such as teaching-program generation and multi-session behavioral interpretation.

Original authors: Festus Kahunla

Published 2026-05-26

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Festus Kahunla

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot how to be a therapist for children with autism. The robot needs to learn two main things: how to write a lesson plan for a specific skill, and how to read a diary of past sessions to understand if the child is improving, struggling, or acting out.

The problem? Real therapy notes are like sealed medical records. They are protected by strict privacy laws (HIPAA), so researchers can't share them to train the robot. If they try to remove names, they often lose the tiny, crucial details the robot needs to learn.

Enter TRACE. Think of TRACE as a massive, perfectly organized "fake" library created specifically to train this robot. It contains nearly 3,000 examples of these therapy documents, but they aren't real; they are synthetic (computer-generated) and built like a Lego set.

Here is how TRACE works, broken down into simple concepts:

1. The "Recipe Book" (The Taxonomy)

Instead of just guessing what a therapy note should look like, the creators built a strict rulebook (called a taxonomy).

The Analogy: Imagine a master chef's recipe book. Every single ingredient (like "prompt hierarchy" or "reinforcement schedule") is listed, and every recipe cites exactly which famous cookbook it came from (like the standard ABA textbooks).
The Result: The computer doesn't "hallucinate" or make things up. It picks ingredients from this rulebook and combines them according to strict rules. If the rulebook says "You can't mix X with Y," the computer won't do it.

2. The "Factory" (The Generator)

The paper describes a deterministic factory.

The Analogy: Think of a 3D printer that builds a toy car. If you give it the exact same blueprints and the exact same starting button press (a "seed"), it will build the exact same car every single time.
Why this matters: In TRACE, every single example comes with a "receipt" (provenance). If a researcher finds a mistake in one example, they don't have to fix 3,000 files one by one. They just fix the one "blueprint" in the recipe book, hit the button again, and the entire library of 3,000 examples instantly regenerates with the fix applied to everyone.

3. What's Inside the Library?

The library is split into two main sections:

Section A: The Lesson Plans: These are instructions on how to teach skills (like tying shoes or saying "hello") using three different teaching styles.
Section B: The Session Diaries: These are summaries of what happened over many days. The computer learns to spot patterns, like "The child is getting frustrated," "The child is regressing," or "The child is having an explosion of behavior." It also learns to suggest what to do next, including safety plans if things get dangerous.

4. The "Special Shapes"

The paper notes that some behaviors are hard to measure with a simple "count."

The Analogy: If you are counting how many times a child screams, a simple number works. But if you are counting how long a tantrum lasts, or how many times a child tries to eat dirt (pica) and how many times they were stopped, you need a special form. TRACE includes these specific "forms" for different behaviors so the data looks realistic.

5. What This Is (and What It Is NOT)

The authors are very clear about the boundaries of this project:

It IS: A research tool, a dataset for training AI, and a way to test how well computers can understand clinical rules. It is released so other scientists can use it to build better models.
It IS NOT: A real clinical tool. It has not been tested on real patients. The authors explicitly state that no one should use this to make real medical decisions, diagnose patients, or write final treatment plans without a human doctor looking over it first. It is a "training dummy," not a "doctor."

Summary

TRACE is a safe, privacy-friendly, and perfectly organized training ground for AI. It uses a strict rulebook to generate thousands of fake therapy notes that look and feel real, allowing researchers to teach computers how to understand Applied Behavior Analysis without ever needing to see a real patient's private records. It is a "simulation" designed to make the AI smarter, but it is not a replacement for human judgment.

Technical Summary: TRACE – A Taxonomy-Grounded Synthetic Dataset for Applied Behavior Analysis

Problem Statement

Applied Behavior Analysis (ABA) is the dominant evidence-based treatment for autism spectrum disorder, generating high volumes of formulaic clinical documentation, including teaching programs and multi-session behavioral logs. However, the development of language models (LMs) capable of drafting these documents is severely constrained by data access. Real-world session data is protected by HIPAA regulations and professional confidentiality rules, preventing the release of training corpora. De-identified releases often fail to preserve the granular clinical detail necessary for effective model training. Furthermore, while prior work has explored AI in ABA, existing solutions either rely on proprietary, unreleased datasets or fail to address the complex task of multi-session behavioral interpretation. The field also faces ethical concerns regarding hallucination in clinical settings and the lack of validated tools for AI-assisted service delivery.

Methodology

The authors present TRACE (Taxonomy-Referenced ABA Clinical Examples), a synthetic instruction-tuning dataset generated through a deterministic, taxonomy-driven pipeline.

1. Taxonomy and Clinical Grounding

The core of the methodology is a controlled vocabulary encoded in YAML files, organized by clinical area. Every element in this vocabulary is grounded in canonical ABA literature and ethical codes, including:

Textbooks: Cooper et al. (2020) for operational definitions and teaching methods.
Functional Analysis: Iwata et al. (1994) and Hanley et al. (2003) for behavior functions.
Curricula: VB-MAPP and AFLS for skill targets and mastery states.
Ethics: BACB Ethics Code and ABAI Position Statements for crisis planning.

The taxonomy covers two primary tasks:

Teaching-Program Generation: Spanning Discrete Trial Training (DTT), Natural Environment Teaching (NET), and Task Analysis.
Session Interpretation: Covering 12 trajectory patterns and 13 target behaviors across multi-session logs.

2. Deterministic Generation Loop

The dataset generation process is fully deterministic, driven by a configuration file and a random seed. For each example, the system:

Samples a clinically valid configuration from the taxonomy, weighted by clinical frequency.
Applies Compatibility Rules: Rejects inconsistent combinations (e.g., ensuring errorless error correction pairs only with specific prompting hierarchies).
Computes Slots: Fills template slots based on sampled cells.
Renders Messages: Generates user and assistant messages in a chat triple format.
Stamps Provenance: Records the exact taxonomy cells and seed used to generate the example in a meta.provenance field.

This approach allows for "refineability at the taxonomy layer." If an inaccuracy is flagged, the responsible taxonomy cell can be edited, and the entire corpus can be regenerated deterministically to update all affected examples without manual per-example rewriting.

3. Data Structure and Composition

TRACE consists of 2,999 examples formatted as JSONL files. Each example includes:

Chat Triple: System prompt, user task prompt, and structured assistant response.
Meta Block: Contains gold labels (derived deterministically from the taxonomy), example ID (SHA-256 hash), and full sampling provenance.

The dataset is stratified into four splits:

Train: 2,549 examples (85.0%)
Validation: 149 examples (5.0%)
Test: 281 examples (9.4%)
Sanity: 20 examples (0.7%)

The corpus covers four learner profiles (Early Learner, School-age, Adolescent, Adult) and ensures diversity in teaching methods, skill targets, and behavioral trajectories. Notably, the dataset includes behavior-specific measurement shapes (e.g., duration for tantrums, partial-interval recording for stereotypy) rather than generic frequency counts.

Key Results and Validation

The paper reports on the construction and technical validation of the dataset rather than the performance of a specific model trained on it.

Schema Integrity: All 2,999 examples parse correctly as JSON with complete metadata.
Provenance Integrity: 100% of examples carry populated provenance fields, linking every output back to specific taxonomy cells and sources.
Uniqueness: All example IDs are unique within and across splits.
Stratification: The splits are balanced across teaching methods and interpretation patterns, with the test set holding out a representative curation pool.
Determinism: Re-running the generation pipeline with the same seed produces identical corpora, ensuring reproducibility.
Clinical Review: The taxonomy and generation logic were reviewed against published sources. The paper notes that for multi-reviewer validation, specific metrics (quadratic-weighted Cohen's $\kappa$ and Matthews correlation coefficient) are recommended, though the paper itself presents the dataset as a research artifact pending such clinical validation.

Significance and Claims

The authors position TRACE as a research artifact, explicitly stating it has not been clinically validated and is not a clinical tool. Its significance lies in:

Bridging the Data Gap: It provides the first synthetic ABA corpus that covers both teaching-program generation and multi-session interpretation, addressing the barrier of HIPAA-protected real-world data.
Auditability and Refineability: Unlike typical synthetic datasets, TRACE's deterministic, taxonomy-grounded design allows for systematic correction. Errors can be traced to specific source cells, enabling targeted repairs that propagate through the entire dataset.
Domain-Agnostic Pipeline: While the taxonomy is ABA-specific, the generator code and templates are domain-agnostic, offering a blueprint for synthetic data generation in other clinical disciplines with structured conventions.
Ethical and Safety Framework: The dataset generation incorporates ethical constraints (e.g., avoiding specific restraint procedures in crisis plans) and includes "sanity" checks to prevent the release of clinically invalid data.

The paper concludes that TRACE serves as a baseline for future ABA-specific language models, a tool for researching taxonomy-driven synthetic data generation, and a resource for supervised fine-tuning of small language models, provided that any clinical application remains under the responsibility of the user and facility.