MAP-DyS: An Interactive Framework for Mapping Analytic Decision Pathways in Subtyping Research

This paper introduces MAP-DyS, an open-source interactive Shiny app designed to enhance transparency and reproducibility in subtyping research by visualizing and comparing the diverse methodological decision pathways found in developmental dyslexia studies, with a framework adaptable to other psychological and behavioral fields.

The Gist

Imagine you are trying to sort a massive, messy pile of puzzle pieces into distinct boxes. Some people say, "Put all the blue pieces in Box A and the red ones in Box B." Others say, "No, put the shiny pieces in Box A and the matte ones in Box B." A third group says, "Let's just throw the big pieces in Box A and the small ones in Box B."

If you look at the final boxes, they will look completely different, even though everyone started with the same pile of puzzle pieces. This is exactly the problem researchers face when studying dyslexia (a learning difficulty with reading). They try to sort people with reading difficulties into different "subtypes" (like "phonological dyslexia" or "surface dyslexia") to understand them better. But because every research team uses a different set of rules to sort the pieces, they end up with different boxes, making it hard to compare their work.

This paper introduces a new tool called MAP-DyS to help solve this confusion. Here is a simple breakdown of what they did and what they found:

1. The Problem: The "Recipe" Chaos

Think of every study on dyslexia subtypes as a chef trying to bake a cake.

• The Ingredients: Some chefs use flour and sugar; others use almond meal and honey.
• The Oven: Some bake at 350°F; others at 400°F.
• The Timer: Some check the cake after 20 minutes; others after 45.

Because everyone uses a different "recipe" (different theories, different tests, and different math), they end up with very different cakes. In the world of dyslexia research, this means one study might say there are two types of dyslexia, while another says there are four. The paper argues that we need to stop just looking at the final cake and start looking at the recipes to understand why they are so different.

2. The Solution: The "Recipe Map" (MAP-DyS)

The authors built an interactive website (a Shiny app) called MAP-DyS. You can think of this as a giant, interactive recipe book or a flight simulator for research.

• How it works: Instead of just reading one study, you can plug in your own "filters." For example, you could ask the app: "Show me only the studies that used the 'Dual-Route' theory and tested children in English."
• The Visuals: The app draws colorful maps and charts. It shows you exactly which "ingredients" (theories, tests, math methods) were used in 63 different studies and how those choices led to different results.
• The Goal: It doesn't tell you which recipe is "right." Instead, it makes the differences transparent. It lets researchers see, "Oh, that study found three subtypes because they used a specific math trick, not because the people they studied were actually different."

3. What They Found in the "Recipe Book"

By mapping out these 63 studies (mostly from 2014 to 2023), the authors discovered some surprising things about how the field operates:

• The "Secret Sauce" is Missing: About 76% of the studies didn't even say what software or computer program they used to do the math. It's like a chef saying, "I baked this cake," but refusing to tell you if they used an oven, a microwave, or a fire. This makes it very hard for others to copy the recipe.
• No Standard Recipe: There is no single "official" way to do this. Some studies use strict rules (like "if your score is below X, you are Type A"), while others use complex computer algorithms to find patterns.
• Small Samples: Most studies used very small groups of people (often fewer than 100). It's like trying to guess the flavor of a whole ocean by tasting a single spoonful. This limits how reliable the "subtypes" can be.
• Lack of Double-Checking: More than half of the studies didn't check if their results were stable. They didn't try to see if the same "recipe" would work if they used a slightly different group of people.
• The "Boxes" are Simple: Despite the complex math, most studies ended up finding only 2 to 4 subtypes. This suggests that the way researchers set up their "sorting rules" might be limiting the number of boxes they find, rather than the people themselves naturally falling into just a few groups.

4. Why This Matters

The paper argues that we shouldn't just argue about which "box" is the correct one. Instead, we need to understand that the box is created by the rules we choose to sort with.

By using MAP-DyS, researchers can:

• See exactly how their choices (like which test to use) change the outcome.
• Stop treating "subtypes" as fixed, unchangeable facts and start seeing them as the result of specific decisions.
• Design better studies that are more transparent and easier to repeat.

In Summary

The paper is a call to stop guessing and start mapping. It says, "We have been trying to sort people with reading difficulties for years, but we've been using different sorting machines. Let's build a tool (MAP-DyS) that shows us exactly how each machine works, so we can finally understand why we are getting different results and how to make our research clearer for everyone."

The tool is currently focused on dyslexia, but the authors say the same "sorting machine" idea could be used for any field where researchers try to group people into subtypes, like autism or ADHD.

Technical Summary

Technical Summary: MAP-DyS – An Interactive Framework for Mapping Analytic Decision Pathways in Subtyping Research

Problem Statement

Subtyping approaches are widely utilized in psychological and cognitive research to classify individuals into subgroups based on shared behavioral or cognitive characteristics, particularly to address heterogeneity in developmental conditions like developmental dyslexia. However, these analyses involve a complex multistage pipeline comprising numerous methodological decisions, including the selection of theoretical models, data preprocessing strategies, performance indices, and statistical techniques.

The core problem identified is that variability in these analytic decisions leads to inconsistent subgroup identification and complicates cross-study comparisons. While the field acknowledges "researcher degrees of freedom," the specific decision space in subtyping research is rarely systematically defined or mapped. Without a systematic map of these decisions, it is difficult to design informed multiverse analyses, interpret variability across studies, or determine if subtyping results are robust or merely artifacts of specific analytic choices.

Methodology

Corpus Construction and Systematic Review

To address this gap, the authors conducted a systematic literature search following PRISMA guidelines (without pre-registration) across four databases: Embase (including MEDLINE/PubMed), APA PsycInfo, ERIC, and OSF Preprints.

• Search Strategy: The search string combined terms for dyslexia/reading difficulties with terms for subtyping/subgrouping.
• Eligibility Criteria: Studies were included if they reported original empirical results identifying cognitive/behavioral profiles or subtypes of readers with reading difficulties. Studies were excluded if they relied solely on neural measures (e.g., fMRI) or genetic data without behavioral measures, or if they did not include individuals with developmental dyslexia.
• Screening: Two raters screened titles and abstracts. Inter-rater reliability (Cohen's $\kappa$) was calculated on a 20% random subset. Initial $\kappa$ was 0.592; discrepancies were resolved through discussion, and the primary rater re-reviewed the full list to finalize the dataset.
• Final Corpus: The resulting dataset comprises 63 studies (from 52 items) published between 2014 and 2023.

Coding Framework

The authors extracted 168 variables categorized into five domains:

1. Publication Information: (e.g., code/data availability).
2. Sample Characteristics: (e.g., sample size, language, comorbidity).
3. Measures: (e.g., constructs, test names, variable types, performance indicators).
4. Methods: (e.g., software, theoretical models, statistical models, cut-off values, data standardization, outlier/missing data treatment).
5. Results: (e.g., subtype names, prevalence, validation status, robustness).

The MAP-DyS Framework

The authors developed MAP-DyS ("Mapping Dyslexia Subtypes"), an open-source, interactive Shiny application designed to visualize these decision pathways. The framework operates at the level of published articles, treating the dataset as a "multiverse" of analytic choices.

The application consists of three primary components:

1. Dataset Exploration: Allows users to filter the dataset by variable categories (publication, sample, measures, methods, results) and generate custom tables ("Create Your Table").
2. Multiverse of Decision Steps:
   • Descriptive Data: Univariate (frequency distributions) and Bivariate (cross-tabulations) visualizations of study features.
   • Subsetting the Multiverse: Users can filter studies based on specific combinations of analytic decisions (e.g., specific theoretical model + statistical method).
   • Decision Map of Individual Article: Visualizes the analytic pipeline of a selected paper, highlighting overlapping decisions with other studies in the database.
3. Subtyping Result Explorer:
   • Theoretical Model and Subtypes: A heatmap linking theoretical frameworks to identified subtypes.
   • Language and Subtypes: A heatmap linking participant languages to identified subtypes.

Key Results and Demonstrations

Using the MAP-DyS framework, the authors demonstrated significant variability across the dyslexia subtyping literature:

• Metadata and Sample Characteristics:

  • Sample sizes are predominantly small (most studies $n < 100$), potentially restricting the stability of complex solutions.
  • The literature is heavily skewed toward Western contexts (English and French), with limited representation of other languages (e.g., Chinese, Arabic, Korean).
  • Participant ages cluster around early adolescence (11–15 years), with fewer studies covering early childhood or adulthood.

• Analytic Decision Variability:

  • Theoretical Frameworks: The Dual-Route Model/Double-Deficit Hypothesis was most common, but ~20% of studies did not specify a theoretical model. The Simple View of Reading was the second most cited.
  • Construct Selection: There is no consensus on construct operationalization; labels for similar constructs (e.g., "word identification" vs. "word recognition") vary, and test battery compositions differ widely.
  • Statistical Approaches: Mean-standard deviation classification (using clinical cut-offs) was the most prevalent method (~46%). Data-driven methods (e.g., cluster analysis, Latent Profile Analysis) were used less frequently and inconsistently.
  • Validation: More than half of the studies did not report any formal validation of their subtypes.
  • Transparency: ~76% of studies did not report the specific software or version used. Among those that did, SPSS dominated; open-source environments (R/Python) were rarely reported.

• Subtyping Outcomes:

  • Most studies identified between two and four subtypes.
  • While some profiles (e.g., phonological and surface dyslexia) appeared consistent across languages, language-specific profiles also emerged (e.g., working memory subtypes in French, RAN/morphological deficits in Chinese). However, the authors note these differences may reflect analytic choices rather than purely linguistic cognitive profiles.

Significance and Claims

The paper claims that MAP-DyS shifts the focus of subtyping research from "identifying subtypes" to "understanding how subtypes are constructed."

• Transparency and Reproducibility: By making analytic variability transparent and navigable, the tool supports researchers in critically evaluating existing practices and designing more reproducible studies. It highlights the lack of standardization in reporting software and validation procedures.
• Multiverse Perspective: The framework provides a foundation for multiverse-aware research, allowing researchers to identify alternative analytic pathways and assess how common they are in the literature.
• Generalizability: While demonstrated on dyslexia, the authors state the framework is adaptable to other areas of psychological and behavioral research employing subtyping or classification approaches (e.g., autism, ADHD).
• Practical Utility: The tool is intended to support evidence-informed reflection for clinicians and educators regarding diagnostic and intervention practices, though it is explicitly not a diagnostic tool.

The authors conclude that subtyping outcomes are contingent on a series of analytic decisions. By visualizing these decision pathways, MAP-DyS enables a more cautious interpretation of heterogeneous findings and supports efforts to enhance the replicability and generalizability of research in behavioral science.