💻 computer science

From Punishment to Protection: Charting Six Decades of U.S. Juvenile Justice Through Topic Modeling and LLM-Assisted Analysis

This paper utilizes topic modeling and LLM-assisted analysis on over 60,000 U.S. appellate opinions from 1970 to 2025 to reveal a dramatic shift from punitive to protective juvenile justice trends, while highlighting critical risks like vocabulary drift and jurisdictional fragmentation that must be addressed for AI tools in this domain.

Original authors: Nia E. George, Simeon Sayer

Published 2026-06-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Nia E. George, Simeon Sayer

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the U.S. juvenile justice system as a massive, bustling library containing 60,470 books (court rulings) written over the last 60 years. For decades, legal scholars have tried to understand how this library has changed by reading a few books at a time. But with so many volumes, it's like trying to understand the history of a whole city by only looking at a few street corners.

This paper, written by Nia George and Simeon Sayer, uses a new kind of "super-reader"—a combination of advanced computer algorithms and Artificial Intelligence (AI)—to read every single book in the library at once. Their goal was to see the big picture: how has the conversation between judges and lawyers shifted from the 1970s to today?

Here is what they found, explained through simple analogies:

1. The Library Split into Two Different Worlds

The most surprising discovery is that the library isn't one big room anymore; it has effectively split into two separate wings that are growing in opposite directions.

The "Crime" Wing: This deals with kids accused of breaking laws. Over the last 50 years, the number of cases here has actually shrunk. Courts are sending fewer kids to adult court, and fewer kids are getting the death penalty. It's like a factory that used to churn out heavy punishments but has now slowed down its production line.
The "Family Safety" Wing: This deals with children who are unsafe at home (neglect, abuse, parental drug use). This wing has exploded in size, tripling its share of the library. It's as if the library added three new floors just for family cases.

Why this matters: If you build a computer tool to help judges make decisions, you can't treat these two wings as the same thing. A tool trained on "crime" cases might give the wrong advice for "family safety" cases because they speak different languages and follow different rules.

2. The Language Changed Completely

The authors found that the words judges used in the 1970s are almost unrecognizable today, even when they are talking about similar problems.

The "Dictionary Drift": Imagine if in the 1970s, people talked about "carnal knowledge" and "sodomy," but by the 2020s, they only talked about "sexual assault" and "registration." Or think about drugs: in the 80s and 90s, the library was full of books about "crack cocaine," but in the 2020s, the shelves are filled with "fentanyl."
The AI Risk: If you train an AI on the old words, it might miss the new cases entirely. It's like trying to find a book about "fentanyl" by searching for "crack"—the computer won't know they are related unless you teach it the new vocabulary.

3. The "Registration" Explosion

One specific topic grew faster than anything else: Sex Offender Registration.

In the 1970s, this topic barely existed.
By the 2020s, it became the single largest topic in the entire library.
The Twist: The laws changed so often (like the Jacob Wetterling Act, Megan's Law, and the Adam Walsh Act) that every decade developed its own unique "dialect." The computer had to work extra hard to realize that a case about "community notification" in the 90s was actually the same type of problem as a case about "GPS monitoring" in the 2020s.

4. The "Punishment" vs. "Protection" Shift

The paper shows a clear trend:

Then (1970s-1990s): The focus was on punishment. Judges were asked, "How do we transfer this kid to adult court?" or "Should we give them the death penalty?"
Now (2010s-2020s): The focus has shifted to protection and limits. Thanks to Supreme Court rulings, judges are now asking, "Is this sentence too harsh for a child?" and "How do we protect the child's future?"
The Result: The "punishment" topics (like the death penalty) have vanished from the library, while "protection" topics (like child welfare and limiting harsh sentences) have taken over.

5. Why This Matters for AI Tools

The authors warn that if we build AI tools to help judges today, we have to be very careful. The paper identifies five "traps" these tools could fall into:

Time Travel Trap: Don't let the AI think old laws (like the death penalty for kids) are still valid just because they were in the library 30 years ago.
Language Trap: Don't let the AI miss new cases just because the words changed (e.g., from "crack" to "fentanyl").
Local Trap: Don't let the AI ignore local rules. Some states have very specific laws about sex offender registration that look different from the national average.
Volume Trap: Don't assume that because a topic is "rising" in the library, the actual crime is rising. Sometimes a topic rises because more people are appealing the decision, not because more crimes are happening.
Split System Trap: Remember that "crime" and "family safety" are two different systems. An AI shouldn't mix them up.

The Bottom Line

This paper proves that we can use computers to read thousands of legal documents to see how justice has evolved. It shows that the system has moved from a "punish the child" mindset to a "protect the child and limit punishment" mindset. However, it also warns that if we use AI to help judges, we must teach that AI to understand these shifts, or it might give advice based on a world that no longer exists.

Technical Summary: From Punishment to Protection: Charting Six Decades of U.S. Juvenile Justice Through Topic Modeling and LLM-Assisted Analysis

Problem Statement
Juvenile justice litigation in the United States encompasses two distinct streams: delinquency cases (youth accused of crimes) and child welfare cases (children at risk within families). Over the past fifty years, these streams have evolved dramatically, yet traditional doctrinal reading struggles to identify long-run patterns across the 60,000+ appellate opinions spanning 1970 to 2025. Existing research relies on administrative data (arrests, caseloads) or policy scholarship, neither of which tracks how the legal questions courts are asked to decide have shifted over time. Furthermore, studying multi-decade legal trends is hindered by uneven data availability, confidentiality norms, and the "temporal mismatch" where legal vocabulary and doctrine shift significantly between eras. This study addresses the gap in systematically analyzing appellate case law to understand the evolution of juvenile justice doctrine and identifies critical risks for AI-based decision support tools trained on such historical corpora.

Methodology
The study employs a six-stage, end-to-end pipeline combining computational topic modeling with Large Language Model (LLM) assistance to analyze 60,470 U.S. appellate opinions from CourtListener (1970–2025).

Data Construction & Preprocessing: Opinions were collected via API, de-duplicated, and cleaned. Text normalization removed boilerplate and citation fragments while preserving doctrinal vocabulary. Multi-word legal phrases were tokenized as single units to ensure consistency across decades.
Global-Local Topic Modeling: A hybrid architecture was used to ensure cross-decade comparability.
- A Global BERTopic model (using MPNet sentence embeddings, UMAP for dimensionality reduction, and HDBSCAN for clustering) was fit to the full corpus to establish a shared topic space, resulting in 182 substantive topics.
- Decade-Local models were fit independently for each of the six decades (1970s–2020s).
- Alignment: Local topics were mapped to the global inventory using Jaccard similarity on top-word sets (threshold $\ge$ 0.20). This allowed for the tracking of stable topics while isolating era-specific "sub-theme" topics that the global model could not capture.
Quality Diagnostics: Topic quality was assessed using UMass coherence scores and nearest-neighbor lexical overlap. Only 182 of the initial clusters were retained as substantive topics, covering 54.7% of the corpus; the remainder were excluded as outliers or boilerplate.
Trend Computation: Prevalence was calculated as a decade-normalized share (opinions assigned to a topic divided by total assigned opinions in that decade) to allow for direct comparison across decades of varying sizes. Linear slopes were fitted to identify rising, falling, and stable trends.
LLM-Assisted Classification & Analysis:
- Theme Induction: An LLM (Claude) analyzed topic metadata (top words, counts, coherence, trends) to propose a 10-theme taxonomy.
- Topic Assignment: The LLM assigned each of the 182 topics to exactly one theme.
- Evidence-Grounded Analysis: For selected high-interest topics, the LLM was provided with "evidence bundles" containing full opinion texts from the global topic, mapped local topics, and unmapped sub-themes. The LLM was instructed to generate legal narratives and doctrinal arcs strictly based on the supplied text, avoiding reliance on pre-trained legal knowledge.

Key Contributions

Methodological: The paper introduces a reproducible, auditable pipeline for cross-decade legal analysis. It combines a global-local BERTopic architecture with built-in quality diagnostics and a two-pass LLM workflow for theme discovery and evidence-grounded trend analysis.
Empirical: It provides the first large-scale, quantitative documentation of doctrinal rebalancing in U.S. juvenile justice across six decades, identifying 182 distinct legal topics organized into 10 themes.
Risk Framework: It establishes an evidence-grounded AI risk framework, identifying five specific guidelines for decision support tools in juvenile justice based on the observed data patterns.

Results
The analysis reveals a fundamental shift in the juvenile justice system from a focus on adjudication and procedure toward supervision and child welfare.

Thematic Shifts:
- Rising: Child Welfare & Dependency (5.6% to 16.7%) and Sex Offense Registration (the single largest topic, rising from 5.0% to 12.9%) are the fastest-growing areas. Probation/Supervision and Detention/Custody also saw significant growth before recent declines in overall caseloads.
- Falling: Procedural & Evidentiary Issues (dropping from 12.6% to 6.7%) declined as due-process rights (established by Kent, Gault, Winship) became settled law. Judicial Transfer (waiver to adult court) fell sharply (4.6% to 1.0%) as states moved to automatic transfer statutes. Juvenile Death Penalty cases declined following the Roper decision.
- Stable: Juvenile Offenses & Adjudication remained the largest theme (~22–27%) but saw a relative decline as other areas grew.
Doctrinal Arcs (LLM Analysis):
- Sex Offense Registration (T0): Evolved from a focus on due process and transfer in the 1970s/80s to a complex regime of registration mandates (Wetterling, Megan's Law, Adam Walsh) in the 1990s/2000s, followed by judicial pushback regarding proportionality and individualized review in the 2010s/2020s.
- Drug Trafficking (T2): Shifted from a rehabilitative model in the 1970s to a punitive "War on Drugs" era (mandatory minimums, accomplice liability) in the 80s/90s, and finally to a focus on the distinct harms of fentanyl and the refusal to merge trafficking charges in the 2010s/2020s.
- Child Welfare (T6): Transformed from a cautious protection of parental rights (requiring actual harm) to a doctrine where methamphetamine addiction is treated as presumptively incompatible with safe parenting, driven by ASFA timelines and the "children cannot wait" standard.
Vocabulary Drift: The study found that legal vocabulary shifts significantly by decade. For example, terms like "carnal knowledge" (1970s) gave way to "sex offender registration" (1990s+), and "crack cocaine" (1980s) gave way to "fentanyl" (2010s+). Global models often miss these era-specific sub-themes, which constitute a significant portion of the litigation.

Significance and Claims
The paper claims that large-scale, reproducible analysis of appellate case law is possible and practically useful for legal scholarship, but it also highlights critical limitations for AI applications in this domain.

Two Parallel Systems: The study demonstrates that delinquency and child welfare are moving in opposite directions and have developed distinct legal vocabularies and logics. AI tools trained on a combined corpus risk blending these systems, leading to inaccurate retrieval or reasoning.
Temporal Mismatch & Vocabulary Drift: Tools trained on historical data without temporal stratification may present superseded legal standards (e.g., mandatory life without parole pre-Miller) as current law. Furthermore, vocabulary drift means tools calibrated on one era's terms may fail to retrieve relevant cases from other eras.
Jurisdictional Fragmentation: The fastest-growing area (sex offender registration) is highly fragmented into state-specific variants that global topic labels cannot capture. AI tools must index these sub-themes to avoid missing critical litigation.
Trend vs. Conduct: The paper emphasizes that trend lines measure legal activity (what courts were asked to decide), not conduct (how often offenses occurred). A decline in transfer cases, for instance, reflects statutory changes removing judicial discretion, not necessarily a drop in serious offenses.

The authors conclude that addressing these risks—temporal calibration, vocabulary alignment, jurisdictional specificity, and the separation of delinquency and welfare systems—is a fundamental requirement for any AI-based decision support tool used in juvenile justice. The pipeline serves as a model for auditing legal reasoning and ensuring AI tools reflect modern standards rather than outdated punitive baselines.