Machine Learning Driven 'Therapy Calculator' for Self-Managed Digital Speech-Language Therapy for Individuals with Post-stroke Aphasia

This paper presents a publicly available, machine learning-driven "Therapy Calculator" that utilizes a large dataset of over 3.5 million digital therapy sessions to reliably predict the probability of improvement for individuals with post-stroke aphasia based on their demographics, starting performance, and therapy engagement.

The Gist

Imagine you've just finished a long journey and are now trying to learn a new skill, like playing the piano, but you've had a stroke that makes it harder to speak or understand words. You want to get better, but you're stuck in the dark. You don't know if practicing every day will actually help, or if you're too far along in your recovery for it to make a difference. You're essentially guessing your way through rehabilitation.

This paper introduces a tool called the "Therapy Calculator" designed to turn that guesswork into a clear, data-driven map. Think of it as a GPS for speech therapy.

The Problem: The "Black Box" of Recovery

Usually, when people with aphasia (a language disorder from a stroke) try to improve on their own using apps, they are flying blind. They know they need to practice, but they don't know:

• How much practice is enough?
• Will I actually get better if I do this?
• Is my age or how long ago I had the stroke going to stop me from improving?

The Solution: A Crystal Ball Built from Real Data

The researchers built this calculator using a massive library of real-life stories. They looked at 3.5 million therapy sessions from over 18,000 people who used an app called Constant Therapy. It's like having a time machine that lets you see what happened to thousands of other people who were in your exact shoes.

They fed all this history into a computer brain (a machine learning algorithm) to find patterns. The computer learned to answer a simple question: "If a person starts at this specific skill level and practices this much, how likely are they to reach the next level?"

How It Works: The "Therapy Calculator"

The tool is a website that acts like a personal coach. Here is the step-by-step journey a user takes:

1. The Check-In: You tell the calculator your age and how long it's been since your stroke.
2. The Skill Test: You pick a skill you want to work on (like reading, naming objects, or math). The app then gives you a few quick tasks to see where you stand right now. It's like a coach checking your current running speed before planning a training schedule.
3. The "What-If" Slider: This is the magic part. You can slide a bar to say, "I want to practice 3 days a week for 20 minutes," or "I want to practice every day for an hour."
4. The Prediction: The calculator instantly tells you the probability (a percentage chance) that you will move up to the next "landmark" of ability.

The Analogy: Imagine you are climbing a mountain. The calculator tells you, "If you take steps of this size, 4 times a week, you have an 85% chance of reaching the next camp. If you only take steps twice a week, your chance drops to 40%."

What the Data Actually Said

The researchers tested their "GPS" and found it was very accurate (about 84% reliable) at predicting if someone would move to the next level. Here are the key rules the computer learned from the data:

• Younger is often better: Just like a fresh battery, younger patients tended to have a higher chance of climbing to the next level, especially on harder tasks.
• Sooner is better: People who started practicing sooner after their stroke had better odds of improving.
• Starting strong helps: If you already have a decent grasp of the skill (a high initial score), you are much more likely to get better.
• Consistency is King: This was a huge finding. It didn't matter as much if you practiced for a long total time over a year; what mattered was not taking long breaks. If you practiced every day, you improved. If you took weeks off between sessions, your progress stalled.
• Different skills need different fuel:
  • Thinking skills (like math or attention) relied heavily on your age and how soon you started.
  • Speaking and language skills (like naming things or writing) relied most heavily on how well you were already doing and how frequently you practiced.

The Result

The team built this tool and let nine stroke patients try it out. The patients loved it. They asked for bigger text and audio descriptions, which the team added. The final product is a free, easy-to-use website that gives patients a realistic, data-backed answer to the question: "If I do this, will it work?"

In short: This paper describes a tool that uses the collective experience of thousands of past patients to give a single patient a clear, personalized prediction of their recovery potential, helping them decide how hard and how often they need to practice to see results.

Technical Summary

Technical Summary: Machine Learning Driven 'Therapy Calculator' for Self-Managed Digital Speech-Language Therapy

Problem Statement
Individuals with post-stroke aphasia face long-term disabilities and often lack clarity regarding their potential for recovery in communication and cognitive skills. While Speech-Language Therapy (SLT) is beneficial, clinical access is frequently limited by insurance constraints, mobility issues, and geographic isolation, resulting in patients receiving significantly less therapy (average 8 hours/year) than the dosage associated with positive outcomes (approx. 98.4 hours). Although self-managed digital therapeutics like Constant Therapy (CT) offer high-intensity practice opportunities, patients often lack guidance on the specific dosage (frequency and duration) required to achieve functional improvements. There is a need for a data-driven tool that can assess a patient's current functional level and predict the likelihood of improvement based on specific therapy schedules.

Methodology
The study developed a "Therapy Calculator," a web-based application integrating a machine learning (ML) predictor with a user-friendly interface.

• Data Source and Filtering: The model was trained on a dataset comprising 3,556,321 therapy session records from 18,728 unique users of the Constant Therapy app (collected April 2020–June 2023). To ensure relevance for self-managed therapy, the data was filtered to include only patients who:

  1. Subscribed to the app and were assigned practice schedules.
  2. Completed assessments independently (without clinician supervision).
  3. Demonstrated continuous engagement (minimum 10 sessions per domain, max 14-day gap between sessions, and recorded age/time-since-stroke).
  4. This process yielded a final dataset of 2,269 patients.

• Feature Engineering: The study focused on 13 functional domains (e.g., reading, naming, auditory comprehension, quantitative reasoning). For each domain, patients were grouped by their initial "functional landmark" (a specific skill level derived from a domain score ranging from 0 to 1). The model utilized seven input features:

  1. Patient age (years).
  2. Time since stroke (months).
  3. Initial domain score (subjective/objective assessment baseline).
  4. Initial task accuracy (percentage).
  5. Total number of therapy sessions.
  6. Total length of therapy (days).
  7. Maximum time gap between sessions (days).

• Machine Learning Model: A logistic regression algorithm was selected for its robustness and interpretability. The model performs a binary classification task: predicting whether a patient will improve their domain score to the threshold of the next functional landmark ($y=1$) or not ($y=0$).

  • Features were standardized (zero mean, unit variance).
  • Training utilized 5-fold cross-validation within each domain-landmark grouping.
  • Model performance was evaluated using the F1 score to balance precision and recall, addressing class imbalance in the data.

• System Architecture: The Therapy Calculator is a web application built with Vue.js (frontend) and Python (backend), deployed on Amazon Web Services (AWS). The workflow involves:

  1. User input of demographics.
  2. Subjective and objective assessment tasks to determine the current functional landmark.
  3. User selection of planned therapy frequency and duration via sliders.
  4. Backend execution of the domain-specific ML model to output the probability of improvement.

Key Results

• Predictive Performance: The logistic regression models achieved an average F1 score of 0.84 (standard deviation 0.10) across all domain-landmark combinations.
  • Predictions were most reliable for lower-numbered landmarks (easier tasks), with an average F1 of 0.89 for the first landmark, compared to 0.81 for the highest available landmarks.
• Feature Importance (Coefficients):
  • Demographics: Age and time since stroke consistently showed negative coefficients, indicating that younger patients and those in the acute phase post-stroke have a higher probability of improvement.
  • Baseline Performance: Initial domain score and initial task accuracy were strong positive predictors. Higher baseline performance correlated strongly with progression, particularly in speech/language domains.
  • Therapy Dosage: The total number of therapy sessions was a highly influential positive factor. Conversely, the maximum time gap between sessions showed negative coefficients, reinforcing the need for consistent practice intervals. Total therapy length showed mixed effects and was less predictive than session frequency.
• Domain Specificity: Cognitive domains (e.g., attention, visuospatial skills) showed stronger dependency on demographic factors, while speech/language domains (e.g., reading, production) were more heavily influenced by initial performance and therapy frequency.

Key Contributions

1. Development of a Predictive Tool: The creation of the "Therapy Calculator," a publicly available web-based tool that allows patients to input their demographics, current functional level, and proposed therapy schedule to receive a probability estimate of improvement.
2. Data-Driven Validation: Utilization of a large-scale, real-world dataset (3.5 million sessions) to train and validate ML models for aphasia rehabilitation, moving beyond small-scale clinical trials.
3. Interpretability: The use of logistic regression allows for the interpretation of feature weights, providing clinicians and patients with insights into how age, injury timing, baseline ability, and therapy consistency influence recovery outcomes.
4. User-Centric Design: The system includes a formative user study with stroke patients, leading to interface improvements such as audio descriptions, larger fonts, and the ability to print results.

Significance and Claims
The paper claims that the Therapy Calculator provides a reliable method for estimating the probability of functional improvement for patients beginning self-managed SLT. Unlike existing medical calculators that estimate stroke risk or mortality, this tool is the first of its kind to predict specific therapy outcomes (progression between functional landmarks) based on dosage and frequency.

The authors emphasize that the tool addresses a critical gap: patients with long-term disabilities often do not know if they will improve or how much therapy is required. By providing data-driven estimates, the calculator aims to empower patients and clinicians to make informed decisions regarding therapy planning. The study concludes that consistent practice schedules and early intervention are crucial factors for recovery, and the tool successfully translates these findings into an actionable resource.

Limitations Acknowledged
The authors note that the current model predicts improvement only from one task/landmark to the next within a specific domain, rather than long-term trajectories. Additionally, the model relies on initial session data and aggregate therapy metrics (total sessions, length, gaps) rather than utilizing rich, ongoing session data (e.g., real-time domain scores during therapy) due to the model's input criteria. Future work may explore advanced algorithms like neural networks to refine predictions.