Measuring High-Priority Outcomes in Autistic Adults: Initial psychometric assessment of the instruments in the AASPIRE Measurement Toolkit

This study validates the AASPIRE Measurement Toolkit, a set of 19 accessible, co-created outcome measures for autistic adults, demonstrating strong psychometric properties including reliability, validity, and responsiveness to change across a diverse longitudinal sample.

The Gist

Imagine you are trying to fix a leaky roof, but you don't have a ruler, a level, or a flashlight. You're just guessing where the water is coming from and if your patch is working. That is essentially what researchers and doctors have been doing when trying to help autistic adults for a long time. They didn't have the right tools to measure how well their services were actually helping.

This paper is about a team of researchers and autistic adults building a brand-new, high-quality "toolbox" to fix that problem.

The Problem: The Wrong Ruler

For years, scientists used surveys designed for the general population to ask autistic adults about their lives, health, and happiness. But imagine trying to measure a jagged, irregular rock with a straight, rigid ruler. It doesn't fit.

• The Language Barrier: Many existing surveys used complex words, confusing sentences, or asked two questions at once (like "Do you like your job and your boss?"). For an autistic person, this can be like trying to read a map written in a foreign language.
• The "Proxy" Problem: Because the surveys were so hard, researchers often asked parents or caregivers to answer for the autistic adult. But a caregiver can't know exactly how the autistic person feels inside, just like a parent can't know exactly how their child feels about a specific headache.
• The "Stuck" Problem: Even if the surveys worked, they were often just snapshots. They could tell you how someone felt today, but they couldn't tell you if a new therapy actually made things better over six months.

The Solution: The AASPIRE Measurement Toolkit

The team behind this study (AASPIRE) decided to build a new set of tools from scratch. But here is the secret sauce: They didn't build it alone.

They used a method called Community-Based Participatory Research (CBPR). Think of this not as a doctor prescribing a diet to a patient, but as a team of chefs (including autistic people) cooking a meal together. Autistic adults, people with intellectual disabilities, and non-speaking autistic people were equal partners in designing every single question.

They created 19 different "measuring sticks" (surveys) covering things like:

• Health: How is your mental health? Are you experiencing "autistic burnout" (a state of total exhaustion)?
• Life: Are you happy? Do you feel like you have a say in your own life?
• Services: Is your doctor listening to you? Are your disability services actually helping?

The Test Drive: Putting the Tools to Work

Once they built these tools, they didn't just trust their own opinion. They went on a massive road test.

• The Drivers: They recruited 870 autistic adults from all over the US. This wasn't just a small group of easy-to-reach people; it included people with different support needs, different backgrounds, and different communication styles.
• The Trip: These participants took the surveys three times over a year and a half.
• The Check-up: They also asked a small group to take the survey twice in two weeks to make sure the answers were stable (like checking if a scale gives the same weight if you step on it twice).

The Results: A Toolbox That Actually Works

The results were fantastic. Here is what they found, translated into everyday terms:

1. It Was Easy to Use: Over 90% of people said the questions were easy to understand. Participants called the survey "thoughtfully-designed" and "fabulous." One person even said the visual icons (like little cups to show levels) helped them because they have trouble with numbers.

   • Analogy: It's like finally getting a video game controller that fits your hand perfectly, instead of one that's too big and slippery.

2. It Was Reliable: The tools gave consistent answers. If someone was feeling anxious, the survey picked it up. If they were feeling good, it picked that up too.

   • Analogy: It's like a good thermometer that always reads the same temperature for the same fever, rather than a broken one that says "cold" one minute and "boiling" the next.

3. It Could Detect Change: This is the most important part. When people said, "My life got better over the last six months," the survey scores actually went up. When they said, "It got worse," the scores went down.

   • Analogy: This is the difference between a camera that just takes a picture and a video camera that can show you a movie of your life improving. It proves the tools can actually measure if a new service or therapy is working.

4. It Was Honest: The survey could tell the difference between similar feelings. For example, it could distinguish between "feeling lonely" and "feeling anxious," even though they often happen at the same time.

Why This Matters

Before this study, researchers were often flying blind. They couldn't be sure if the programs they funded were actually helping autistic adults.

Now, they have a standardized, accessible, and accurate toolkit.

• For Doctors: They can use these tools to see if a patient is getting better after a new treatment.
• For Policymakers: They can use the data to decide which disability services are actually worth the money.
• For Autistic People: It means their voices are finally being heard directly, without needing someone else to guess what they are thinking.

The Bottom Line

This paper isn't just about statistics; it's about respect. By building these tools with autistic people, the researchers proved that when you listen to the people you are trying to help, you get better results. They created a set of measuring sticks that finally fit the shape of the autistic experience, allowing us to finally measure progress and improve lives in a real, meaningful way.

Technical Summary

1. Problem Statement

Research into services and interventions for autistic adults is significantly hampered by a lack of accessible, valid, and reliable self-reported outcome measures. Existing instruments often fail to account for the unique characteristics of the autistic population (e.g., pragmatic language challenges, sensory sensitivities, or co-occurring intellectual disabilities), leading to:

• Inaccessibility: Instruments that are confusing, offensive, or anxiety-provoking, causing high dropout rates or inaccurate data.
• Validity Issues: A reliance on proxy reporting (caregivers) which excludes autistic voices, or the use of tools validated only on general populations that may not apply to autistic adults.
• Responsiveness: Many existing tools are cross-sectional and lack the sensitivity to detect changes over time, making it difficult to evaluate the effectiveness of interventions.

2. Methodology

The study utilized a Community-Based Participatory Research (CBPR) approach, ensuring autistic adults and caregivers were equal partners in every stage of the research.

• Study Design: A longitudinal cohort study conducted between April 2023 and December 2024.
• Participants: A pragmatic sample of 870 autistic adults recruited from three subcohorts to ensure heterogeneity:
  • Healthcare (HC) subcohort ($n=319$) from two healthcare systems.
  • Disability Services (DS) subcohort ($n=266$) from two government systems.
  • Community subcohort ($n=285$) recruited via social media and community organizations.
  • Demographics: The sample was diverse, with 30% identifying as BIPOC, nearly 50% as LGBTQ+, and a wide range of support needs (including non-speaking individuals and those with intellectual disabilities).
• Instrument Development:
  • Phase 1: A CBPR-nested Delphi process identified 19 high-priority outcome constructs.
  • Phase 2: Teams collaboratively adapted, revised, or created 19 outcome measures and 12 characteristic modules. This involved iterative cognitive interviews with 37 autistic adults and caregivers to refine accessibility (e.g., simplifying vocabulary, adding visual icons, defining terms).
  • Versions: Instruments were created in two formats: Direct Report (DR) for autistic adults (with or without support) and Caregiver Report (CR).
• Data Collection:
  • Surveys were administered at three time points over 12–18 months.
  • A 15% random subset ($n \approx 130$) completed a retest survey 2 weeks after Time 2 to assess test-retest reliability.
• Psychometric Analysis:
  • Accessibility: Measured via completion rates and self-reported ease of understanding.
  • Reliability: Internal consistency (Cronbach's $\alpha$ and $\omega$) and test-retest reliability (Intraclass Correlation Coefficients, ICC).
  • Validity: Convergent and discriminant validity assessed via Pearson's correlations against hypothesized constructs.
  • Responsiveness: External responsiveness to change assessed by comparing score changes over 6 months with participants' self-perceived change (improved, same, worsened).

3. Key Contributions

• The AASPIRE Measurement Toolkit: The creation of a comprehensive, freely available suite of 19 outcome measures covering quality of life, health (depression, anxiety, autistic burnout, flourishing), social functioning (community participation, self-determination, employment), and service adequacy.
• CBPR-Driven Accessibility: The study demonstrates that co-creating instruments with autistic adults results in tools that are not only psychometrically sound but also highly accessible, addressing barriers like complex syntax and ambiguous timeframes.
• Longitudinal Validation: Unlike many cross-sectional studies, this research validates the toolkit's ability to detect change over time, a critical requirement for intervention research.
• Inclusivity: The toolkit includes specific adaptations for individuals with intellectual disabilities and non-speaking individuals, and provides both self-report and caregiver-report versions.

4. Results

• Accessibility & Completion:
  • Over 90% of participants found the survey items easy to understand.
  • 99% of items were answered by participants who started the survey; over 90% of participants completed all applicable sections at each time point.
  • Qualitative feedback described the survey as "thoughtfully-designed," "non-ambiguous," and "fabulous."
• Reliability:
  • Internal Consistency: All scales demonstrated good to excellent reliability ($\alpha > 0.80$), with most exceeding $0.90$.
  • Test-Retest Reliability: Strong stability over a 2-week period, with ICCs ranging from 0.81 to 0.95.
• Validity:
  • Convergent/Discriminant: Correlations aligned with theoretical expectations. For example, mental health scales (depression, anxiety, burnout) were highly intercorrelated ($r = 0.59–0.73$) but distinct from unrelated constructs.
  • Specific Findings: The "Choices and Decisions" scale showed negligible correlation with mental health or social support, confirming it measures a distinct construct (autonomy) rather than psychological distress.
• Responsiveness to Change:
  • Almost all scales showed significant responsiveness to change ($p < 0.001$), where self-reported improvements or declines matched the direction of score changes over 6 months.
  • Exception: The "Barriers to Healthcare Checklist" did not show significant responsiveness in this observational period, though it has shown responsiveness in prior intervention trials.

5. Significance and Implications

• Clinical and Research Utility: The AASPIRE Measurement Toolkit provides the first rigorously validated, accessible set of tools specifically designed to measure outcomes in autistic adults. This allows clinicians and researchers to accurately evaluate the effectiveness of interventions and track individual progress over time.
• Paradigm Shift: The study validates the efficacy of CBPR in instrument development, proving that involving the target population as equal partners yields tools that meet or exceed standard psychometric criteria while being culturally and cognitively appropriate.
• Policy and Advocacy: By enabling the measurement of high-priority outcomes (e.g., autistic burnout, service adequacy), the toolkit empowers advocacy organizations and policymakers to identify gaps in care and advocate for evidence-based improvements in services.
• Future Directions: While the initial psychometric properties are strong, the authors note that further in-depth analyses of construct and structural validity for specific instruments are forthcoming. Additionally, future research is needed to validate the Caregiver Report versions, as the current sample had a low number of proxy respondents.

Conclusion: The AASPIRE Measurement Toolkit represents a major advancement in autism research, offering a scientifically robust, accessible, and community-validated solution to the long-standing problem of measuring outcomes in autistic adults.