Identification of Suicide-Related Subgroups Using Latent Class Analysis: Complementary Insights to Explainable AI-Based Classification

This study utilizes latent class analysis on a Sri Lankan dataset to identify four distinct suicide-related subgroups with varying prevalence rates and clinical profiles, demonstrating that unsupervised phenotyping offers complementary insights to explainable AI-based classification for better understanding suicide heterogeneity.

The Gist

The Big Picture: Why Are We Doing This?

Imagine suicide and self-harm as a massive, chaotic storm. For a long time, doctors and researchers have tried to predict who is in the eye of the storm using "Supervised AI" (like a weather app that looks at one person at a time and says, "You are at high risk").

While these weather apps are getting better, they sometimes miss the bigger picture. They tell you if someone is at risk, but they don't always explain why different people are in the storm in different ways. Are they all just wet from the rain? Or are some people drowning, some are freezing, and others are being blown away by the wind?

This study asks a new question: Instead of looking at individuals one by one, can we group people into "families" based on how they are experiencing the storm?

The Method: Sorting the Puzzle Pieces

The researchers took a dataset of 1,000 people from Sri Lanka who had come to a hospital for self-harm or suicide-related issues. They used a statistical tool called Latent Class Analysis (LCA).

The Analogy:
Think of the 1,000 people as a giant box of mixed-up LEGO bricks.

• The Old Way (Machine Learning): You pick up one brick, look at its color and shape, and try to guess if it belongs to a castle or a spaceship.
• The New Way (LCA): You dump the whole box out and look for natural piles. You notice that some bricks naturally stick together because they are all "blue and round," while others are "red and square." You aren't guessing; you are letting the bricks sort themselves into their own natural groups.

What Did They Find? The Four "Families"

The analysis sorted the 1,000 people into four distinct groups (or "families"). Two groups were very safe, and two groups were in deep trouble.

1. The "Stable & Supported" Group (Class 1)

• Who they are: Mostly women, employed, and feeling relatively okay emotionally.
• The Vibe: They are like people standing on a sturdy porch during a storm. They might be worried, but they have a roof over their head and a job.
• Suicide Risk: Extremely low (less than 1%).

2. The "Sick but Not in Crisis" Group (Class 3)

• Who they are: Older people, often with physical pain (like bad knees or back pain), but they aren't feeling the intense emotional despair of the others. They are often religious (Muslim, Hindu, Christian) and employed.
• The Vibe: They are like people sitting in a car with a flat tire. They have a problem (pain/age), but they aren't panicking. Their religion and job seem to act as a shield.
• Suicide Risk: Extremely low (less than 1%).

3. The "Desperate but Unseen" Group (Class 2)

• Who they are: Mostly men, unemployed, and feeling intense anger, sadness, and loneliness. Crucially, most of them have NEVER been to a psychiatric hospital before.
• The Vibe: These are the people screaming for help in the middle of the storm, but no one has noticed them yet because they haven't been "diagnosed" or hospitalized yet. They are drowning in emotion but flying under the radar of the medical system.
• Suicide Risk: Very High (91%).

4. The "High-Risk & Hospitalized" Group (Class 4)

• Who they are: Mostly men, unemployed, and they have a history of being in psychiatric hospitals, trying to kill themselves before, and suffering from severe depression or bipolar disorder.
• The Vibe: These are the people who are already in the ER, wearing a life vest, but the storm is still raging. The system knows them well, and they are in extreme danger.
• Suicide Risk: Almost 100% (99%).

The "Aha!" Moment: Comparing Two Maps

The researchers compared their new "LEGO Grouping" map with an old "AI Weather App" map (from a previous study).

• The AI Map said: "The most important things to watch out for are anger, sadness, and loneliness."
• The Grouping Map said: "You're right! Both of our dangerous groups (Class 2 and Class 4) are full of angry, sad, lonely people."

But here is the twist the AI missed:
The AI thought that having been in a psychiatric hospital was a huge warning sign.

• The Grouping Map showed: Only one of the dangerous groups (Class 4) had been hospitalized. The other dangerous group (Class 2) had never been to a hospital.

The Lesson:
If you only look at the "Hospital" sign, you will miss the people in Class 2 who are just as desperate but haven't been caught by the system yet. The "Anger and Sadness" sign is the one that catches everyone in trouble.

Why Does This Matter?

Think of it like treating a fever.

• Old Approach: "If you have a fever over 103, give them medicine." (This misses people who are sick but haven't hit 103 yet).
• New Approach: "Let's look at the types of fevers. Some people have a fever because of a virus, others because of dehydration. We need to treat the type of person, not just the number."

The Takeaway

This study shows that we need two tools to understand suicide:

1. The AI Tool: Great for looking at one person and saying, "This person is at risk right now."
2. The Grouping Tool (LCA): Great for looking at the crowd and saying, "We have two different types of people in crisis. One group needs immediate medical help because they are already in the system. The other group is suffering silently and needs us to reach out before they get to the hospital."

By using both tools together, doctors and society can build a better safety net that catches everyone, not just the ones who have already fallen through the cracks.

Technical Summary

1. Problem Statement

Suicide and self-harm are highly heterogeneous phenomena characterized by diverse demographic, psychosocial, and clinical risk profiles. While Explainable Artificial Intelligence (XAI) and supervised machine learning (ML) models have shown promise in predicting individual suicide risk, they primarily operate at the individual level. These models often fail to capture population-level patterns or the existence of distinct latent subgroups that share specific constellations of risk factors.

The core problem addressed is the lack of integration between unsupervised phenotyping (identifying natural subgroups) and supervised classification (predicting outcomes). Specifically, it remains unclear how latent subgroups derived from population data align with the feature importance patterns generated by XAI models, and whether combining these approaches offers a more comprehensive framework for understanding suicide risk.

2. Methodology

Data Source:

• Dataset: A publicly available, anonymized cross-sectional dataset of 1,000 individuals from Colombo South Teaching Hospital, Sri Lanka.
• Composition: Balanced design with 500 individuals who died by suicide and 500 who did not.
• Variables: Sociodemographic (age, gender, religion, occupation, education), clinical (psychiatric history, past attempts, self-injury), and psychosocial factors (anger, sleep, isolation, sadness).

Analytical Approach:

1. Latent Class Analysis (LCA):
   • Goal: To identify unobserved subgroups (latent classes) based on shared patterns of risk variables, excluding the suicide outcome variable during the modeling phase.
   • Preprocessing: Continuous variables (e.g., age) were categorized; the occupation variable was dichotomized (employed/unemployed) to meet LCA requirements.
   • Model Selection: Models ranging from 2 to 10 classes were estimated using the poLCA package in R. The optimal model was selected based on:
     • Fit Indices: Lowest Bayesian Information Criterion (BIC) and consistent Akaike Information Criterion (cAIC).
     • Statistical Tests: Significant Lo–Mendell–Rubin Likelihood Ratio Test (LMR-LRT).
     • Interpretability: Clinical meaningfulness of the classes and entropy values (>0.80).
2. Comparative Analysis:
   • The resulting latent class profiles were compared against findings from a previously published XAI-based study (Tang et al.) using the same dataset.
   • The comparison focused on SHAP (SHapley Additive exPlanations) feature importance values to see if the variables driving the ML model's predictions aligned with the characteristics of the identified latent classes.

3. Key Results

A. Identification of Four Distinct Latent Classes
The analysis identified a 4-class solution as the optimal model (BIC = 28,398; Entropy = 0.95). The classes exhibited starkly different suicide prevalence rates and clinical profiles:

• Class 1 (Low Risk, n=357, 36%):
  • Suicide Prevalence: 0.6% (95% CI: 0.2–2.0).
  • Profile: High functioning; 95% employed, 51.6% female, predominantly Buddhist. Low levels of emotional distress, psychiatric morbidity, or substance use.
• Class 2 (High Risk, n=330, 33%):
  • Suicide Prevalence: 91.2% (95% CI: 87.7–93.8).
  • Profile: Characterized by severe emotional and social distress (high anger, sadness, isolation, sleep problems). Notably, this class had low psychiatric hospitalization rates (11.3%) and fewer documented diagnoses, yet high suicide prevalence. Predominantly male and unemployed.
• Class 3 (Low Risk, n=115, 12%):
  • Suicide Prevalence: 0.9% (95% CI: 0.2–4.8).
  • Profile: Older demographic with high somatic burden (chronic pain) but low psychosocial distress. Distinct religious composition (majority Muslim, Hindu, Christian; no Buddhists). High employment (94%).
• Class 4 (High Risk, n=198, 20%):
  • Suicide Prevalence: 99.0% (95% CI: 96.4–99.7).
  • Profile: Universal lifetime psychiatric hospitalization (100%). High rates of prior suicide attempts, self-injury, and psychiatric diagnoses (depression, bipolar). Lowest employment rate (60.2%) and highest proportion of males.

B. Comparison with XAI/SHAP Findings

• Convergence: The XAI model identified emotional and social distress (anger, sadness, isolation) as the top global predictors. This aligns perfectly with Class 2 and Class 4, where these variables were universally elevated.
• Divergence/Nuance: The XAI model highlighted lifetime psychiatric hospitalization as a strong predictor. However, LCA revealed this feature is specific to Class 4 only. Class 2 exhibited similarly high suicide risk (91.2%) without significant psychiatric hospitalization history.
• Demographics: Both approaches confirmed that lower education and unemployment are strong risk indicators.

4. Key Contributions

1. Demonstration of Complementarity: The study provides empirical evidence that unsupervised (LCA) and supervised (XAI/ML) approaches offer complementary insights. LCA reveals subgroup heterogeneity (e.g., two distinct high-risk groups with different clinical histories) that a global feature importance metric might obscure.
2. Refinement of Risk Profiles: It challenges the assumption that psychiatric hospitalization is a universal marker for high suicide risk. The identification of Class 2 (high risk, low hospitalization) suggests that a significant portion of the at-risk population is "invisible" to systems relying heavily on prior clinical contact.
3. Population-Level Context: The study situates individual-level predictions within broader population structures, showing how demographic factors (religion, gender, employment) interact with clinical factors to create distinct risk phenotypes.
4. Methodological Framework: Proposes a workflow where latent class membership could be integrated as a feature into supervised ML models to enhance interpretability and potentially improve classification performance.

5. Significance and Implications

• Clinical Utility: Clinicians should recognize that high suicide risk manifests in different ways. One subgroup (Class 4) presents with a clear history of severe psychiatric illness and hospitalization, while another (Class 2) presents with acute social/emotional distress but minimal prior clinical contact. Screening tools must account for both profiles.
• Public Health Strategy: Interventions need to be tailored. For Class 2, community-based interventions addressing social isolation and unemployment may be more effective than traditional psychiatric referrals. For Class 4, intensive clinical management is critical.
• AI Development: Future suicide risk models should move beyond individual prediction to incorporate phenotypic clustering. Integrating latent class labels into XAI frameworks could make models more interpretable by explaining why a prediction was made based on the patient's subgroup profile.

Limitations:
The study is cross-sectional (cannot infer causality) and based on a single-site sample from Sri Lanka, which may limit generalizability to other cultural contexts. Additionally, the local independence assumption of LCA was not fully met, though prior research suggests this has minimal impact on class assignment.