🧬 biology

Machine-Learning Modeling of DARWIN Handwriting Features for Alzheimer’s Disease Classification: A Secondary Analysis

This secondary analysis of the DARWIN dataset demonstrates that a machine-learning model using a reduced subset of 10 handwriting tasks achieves classification accuracy for Alzheimer's disease comparable to the full 25-task set, suggesting that shorter, memory-focused assessments could effectively streamline screening without sacrificing performance.

Original authors: Sanjay Singhvi, Riddhi Singhvi

Published 2026-08-11

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Sanjay Singhvi, Riddhi Singhvi

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine you are a detective trying to solve a mystery, but instead of looking for fingerprints or footprints, you are looking for clues hidden in how someone writes. This field of science is called handwriting analysis, and it's becoming a hot topic for spotting Alzheimer's disease early. You see, writing isn't just a mechanical act of moving a pen; it's a complex dance between your brain's memory, your planning skills, and your hand's muscles. When the brain starts to struggle with Alzheimer's, this dance gets a little clumsy. Scientists use special digital tablets to record every tiny movement, pressure, and pause a person makes while writing, turning their handwriting into a massive cloud of data points. The big question researchers are asking is: Do we need to watch someone write for a long time to find these clues, or can we spot the trouble signs in just a few quick scribbles? Finding a quick, cheap, and easy way to screen for this disease is crucial because catching it early gives doctors the best chance to help.

Now, let's look at the story this paper tells. The researchers decided to test a simple idea: Could they shrink a very long handwriting test down to a tiny one without losing any of the detective work? They used a famous dataset called DARWIN, which contains the handwriting of 174 people—89 with Alzheimer's and 85 healthy controls. Each person had completed a marathon of 25 different writing tasks, ranging from drawing simple shapes to copying words and writing down a phone number from memory. Altogether, this created a mountain of 450 different data features (like how long the pen hovered in the air or how hard they pressed).

The team asked: "If we only looked at a small handful of these tasks, would we still catch the disease?" To find out, they used three different "smart" computer programs (machine learning models) to act as their detectives. They tried to figure out which tasks were the most important. Imagine you have a bag of 25 different keys, and you know only a few of them open the treasure chest. The researchers tried three different methods to find the right keys: one method (RFE) kept removing the worst keys until only the best remained; another (LASSO) weighed every key to see which were heaviest; and a third (Mutual Information) checked which keys shared the most secrets with the disease label.

Here is the twist: While the three methods disagreed on exactly which specific data points (like "pressure on the 15th second") were the winners, they all agreed on which tasks were the stars of the show. They found that the simple tasks, like drawing circles or joining dots (Tasks 1 through 7), were actually the least helpful. The real clues were hiding in the harder tasks that required memory and listening, like writing a phone number from dictation (Task 23) or writing a sentence from memory (Task 19).

The big discovery? The researchers found that they could cut the test down to just 10 specific tasks and get almost the exact same results as the full 25-task marathon. When they tested this "10-task shortcut" using a Random Forest computer model, it achieved an accuracy of 89.14% (with a confidence range of 87.33% to 90.95%). The full 25-task test scored 88.95%. The difference was so tiny that a statistical test said they were essentially the same. In fact, the 10-task version was even slightly better than a different method that tried to pick just 25 individual data points from the whole pile.

The paper suggests that by focusing on the right 10 tasks—specifically tasks 19, 23, 9, 25, 15, 24, 22, 17, 2, and 10—you can use 60% fewer tasks to get the same diagnostic power. This is a huge win because it means a doctor could potentially screen a patient in a fraction of the time. However, the authors are careful to note that this is a "secondary analysis," meaning they are looking at data that already exists. They haven't tested this shortcut on a brand-new group of people yet. So, while the results are very promising and suggest that a shorter test works just as well within this specific group, the final proof will come when they try it on a completely new set of patients in the future. For now, the paper suggests that the brain's struggle with memory and planning leaves a much louder fingerprint in handwriting than simple motor control does, and we might not need to watch someone write for very long to hear it.

Technical Summary: Machine-Learning Modeling of DARWIN Handwriting Features for Alzheimer's Disease Classification

Problem Statement
Alzheimer's disease (AD) screening currently relies on clinician time and cognitive tests that often miss mild cases, while neuroimaging remains expensive. Handwriting dynamics, captured via digitizing tablets, offer a promising, non-invasive alternative. However, the standard DARWIN dataset protocol requires participants to complete 25 distinct writing tasks, generating 450 features (18 features per task). This study addresses two critical gaps: (1) whether a significantly shorter subset of tasks can achieve classification performance comparable to the full 25-task set, thereby improving clinical feasibility, and (2) whether independent feature-selection methods converge on the same informative tasks, suggesting robust signal rather than overfitting artifacts.

Methodology
The study performed a secondary analysis on the DARWIN dataset, comprising 174 participants (89 AD patients, 85 healthy controls) matched for age and education. Each participant completed 25 standardized tasks (graphic, copying, memory, and dictation) on a Wacom Bamboo tablet, yielding 450 features per subject.

The experimental design utilized 30 stratified 80/20 train/test splits with fixed random seeds to ensure paired comparisons. Key methodological components included:

Preprocessing: Features were Winsorized at the 5th and 95th percentiles and standardized (zero mean, unit variance) within training folds to prevent data leakage.
Feature Selection: Three independent methods were applied to the training data: Recursive Feature Elimination (RFE) with Logistic Regression, L1-regularized Logistic Regression (LASSO), and Mutual Information (MI).
Classifiers: Three models were evaluated at varying feature counts ( $k$ ): Random Forest (RF), L1-regularized Logistic Regression (LR-L1), and a small Multilayer Perceptron (MLP).
Evaluation Metrics: Performance was measured via test accuracy, AUC, macro F1, recall, precision, and Brier score, summarized with bootstrap 95% confidence intervals (2,000 resamples).
Saturation Analysis: A Hill function was fitted to the accuracy-versus- $k$ curve to identify points of diminishing returns ( $K_{50}$ , $K_{90}$ ).
Task-Subset Validation: Eight specific task subsets were tested head-to-head against the full 25-task set. These included a candidate 10-task set (TOP10_MEAN) derived from the consensus ranking of the three selection methods, as well as control subsets (e.g., the 10 least informative tasks, a 2-task baseline).
Statistical Testing: Paired Wilcoxon signed-rank tests were used to compare performance differences between subsets across the 30 splits.

Key Contributions

Identification of a High-Performance Reduced Set: The study identified a 10-task subset (Tasks 19, 23, 9, 25, 15, 24, 22, 17, 2, and 10) that retains the classification power of the full 25-task protocol.
Methodological Convergence: While the three feature-selection methods (RFE, LASSO, MI) showed low agreement at the individual feature level (Jaccard overlap 0.06–0.36 at $k=10$ ), they demonstrated strong agreement at the task level, consistently ranking memory and dictation tasks as most informative and simple graphic tasks as least informative.
Efficiency vs. Model Architecture: The analysis quantified that the choice of feature set size is as critical as the choice of classifier. Random Forest demonstrated the highest efficiency, reaching 50% of its total accuracy gain with only 4 features ( $K_{50}=4.0$ ).
Validation of Task-Level Selection: The study demonstrated that feeding a Random Forest a larger pool of features (180 features from 10 tasks) without aggressive per-feature selection outperformed a model trained on only 25 aggressively selected features from the full set.

Results

Performance Equivalence: A Random Forest trained on the 10-task subset (180 features) achieved a test accuracy of 89.14% (95% CI: 87.33%–90.95%) and an AUC of 0.958. This was statistically indistinguishable from the full 25-task set (88.95% accuracy, AUC 0.963), with a paired Wilcoxon $p$ -value of 0.748.
Diminishing Returns: The accuracy curve for Random Forest was strongly concave. The model reached 80% saturation at 16 features and 90% saturation at 36 features.
Task Hierarchy: The most informative tasks (19, 23, 9, 25, 15, 24, 22, 17, 2, 10) were predominantly memory and dictation tasks requiring working memory and executive function. Conversely, simple graphic and copying tasks (1–7) were consistently the least informative.
Control Subsets:
- The 10 least-informative tasks (BOTTOM10) dropped accuracy to 83.43% ( $p=0.0001$ vs. full set).
- A 2-task baseline (Tasks 1 and 25) yielded 75.43% accuracy ( $p<0.0001$ ).
- Smaller subsets (9, 7, and 5 tasks) showed no statistically significant difference from the full set, though the 5-task consensus set approached significance ( $p=0.082$ ).
Robustness: Leave-one-out cross-validation for the $k=10$ per-feature model yielded 83.33% accuracy, aligning closely with the 80/20 split estimate (83.43%), confirming the results were not artifacts of specific data splits.

Significance and Claims
The paper claims that within the specific context of the DARWIN cohort, a focused 10-task subset performs comparably to the full 25-task protocol while utilizing 60% fewer tasks. The authors emphasize that this finding suggests a practical path toward shorter, more efficient clinical screening tools without sacrificing diagnostic accuracy.

Crucially, the authors maintain a modest stance regarding the strength of this claim:

The result is an internal evaluation on a single cohort; it does not constitute proof of equivalence or external validity.
The non-significant $p$ -value ($0.748$) indicates no detected difference in this specific evaluation, not a formal proof that the sets are equivalent.
The observed performance relies on the specific demographics, equipment (Wacom Bamboo), and protocol of the DARWIN dataset.

The primary remaining question identified by the authors is whether this 10-task subset generalizes to independent, external cohorts. The study concludes that the signal for AD in handwriting dynamics concentrates in cognitively demanding tasks (working memory, executive function) rather than basic motor control, a finding that aligns with established clinical understanding of the disease. Future work is planned to validate these findings prospectively on new populations.

Technical Summary: Machine-Learning Modeling of DARWIN Handwriting Features for Alzheimer's Disease Classification

More like this