💬 NLP

Children, but not language models, show accelerating returns in word learning

This paper reveals that while children exhibit accelerating returns in word learning by becoming increasingly efficient with each new unit of linguistic experience, language models—even those trained on child-directed speech—show only constant proportional returns consistent with scaling laws.

Original authors: Michael C. Frank

Published 2026-08-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Michael C. Frank

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Human language begins with a slow, stumbling start. In the first year of life, a child might learn only a handful of words, often just names for familiar people or objects. Then, something remarkable happens. The pace of learning shifts dramatically. Within a few short years, that same child acquires hundreds of words, moving from simple labels to complex sentences with a speed that seems almost magical. For decades, scientists have tried to understand the mechanics of this explosion. They have long suspected that the secret lies in how children process the stream of words they hear every day. The prevailing idea was that learning is a steady, linear process: the more a child hears, the more they learn, at a constant rate. If this were true, a child's vocabulary would grow like water filling a bucket at a steady drip.

In recent years, powerful computer programs known as language models have emerged as a new way to study this process. These systems are trained on vast amounts of text, learning to predict the next word in a sentence. Because they can be trained on specific amounts of data, researchers can use them to test theories about how learning works. However, a puzzling gap has appeared. To reach even a basic level of language skill, these computer models require trillions of words of training data. In contrast, a human child learns their first thousands of words after hearing only a few million. This massive difference in data requirements suggests that children and machines are learning in fundamentally different ways, but until now, no one had a clear mathematical description of exactly how that difference plays out over time.

A researcher set out to measure this difference directly. They focused on vocabulary growth, tracking how children learn specific words over time and comparing that pattern to how computer models learn the same words. The researcher gathered data from thousands of children, using detailed records where parents reported which words their children could say at different ages. They also trained computer models on recordings of speech directed specifically at children, ensuring the computer received the same kind of input a toddler would hear. By analyzing these two groups side by side, the researcher discovered that the standard view of learning as a steady accumulation was incorrect for humans.

The study revealed that children do not learn at a constant rate. Instead, their ability to learn accelerates. In the beginning, a child might need to hear a word hundreds of times before they say it. But as they grow older and their vocabulary expands, they need far fewer exposures to learn a new word. A toddler might hear the word "book" thousands of times before speaking it, yet a preschooler might hear the word "ibex" just a few times at a zoo and repeat it the next day. The researcher found that this acceleration is a real, measurable phenomenon. As children get older, each new piece of linguistic experience becomes more valuable. They learn more from each additional hour of conversation than they did from the hour before. The computer models, even those trained on child-directed speech, showed no such acceleration. They learned at a steady, unchanging pace, requiring the same amount of data to learn a new word regardless of how much they already knew.

This difference explains why children can become fluent speakers with so little data compared to machines. The computer models operate on a principle of constant returns: every new word they learn costs the same amount of training data. Children, however, get increasing returns on their investment. The researcher tested whether this acceleration was simply a result of parents changing how they speak to their children as they get older, or if it was an internal change in the child's learning ability. The data suggested it was the latter. The children's brains appeared to be getting better at extracting meaning from the sounds they heard, perhaps because they were learning to use the words they already knew to figure out new ones.

The findings challenge the idea that language learning is just a matter of accumulating evidence over time. Instead, the process is dynamic. The researcher showed that a model where the "value" of each new word increases with age fits the real-world data of children perfectly, while a model where the value stays the same does not. This acceleration is not just a small detail; it is a fundamental difference between human and machine learning. While the computer models eventually learned the words, they did so with a rigidity that human children do not possess. The study suggests that the secret to human language is not just in the amount of data we receive, but in how our ability to use that data improves as we grow. This insight offers a new direction for understanding human cognition and for building artificial systems that might one day learn as efficiently as a child.

Technical Summary: Children, but not language models, show accelerating returns in word learning

Problem Statement
Children acquire vocabulary rapidly during early childhood, exhibiting a "vocabulary spurt" where learning accelerates over time. In contrast, large language models (LMs), even those trained on child-directed speech, require orders of magnitude more data to achieve basic linguistic competence. While prior models describe vocabulary growth as a process of evidence accumulation, it remains unclear whether the underlying mechanism involves constant returns (linear accumulation) or accelerating returns (where each unit of experience yields more learning than the previous one). This study investigates the "data gap" between children and LMs by quantitatively characterizing word learning as a function of training data in both systems.

Methodology
The author developed a psychometric "accelerating accumulator" model to describe vocabulary growth in children and compared it against the learning trajectories of language models.

Child Modeling: The study utilized longitudinal item-level data from the MacArthur-Bates Communicative Development Inventories (CDIs) across five datasets (American English, Norwegian, and Japanese; $N_{total} = 1841$ children, ages 8–36 months). The author formulated a Rasch-based model where a child's latent ability $\theta_i(t)$ at time $t$ is defined as:
$\theta_i(t) = \xi_i + \kappa_i \log(t/a_0) + \log(H)$
Here, $\xi_i$ represents baseline learning efficiency, and $\kappa_i$ is the acceleration exponent. If $\kappa = 1$ , the model represents a "pure accumulator" with linear returns; if $\kappa > 1$ , it represents an accelerating accumulator where later experiences count for more. The author used Bayesian inference (Stan) to fit this model (M3) against ablated versions (M0–M2) lacking individual variation in efficiency or acceleration.
Language Model Analysis: To compare with children, the author trained GPT-2-small models on 24M words of child-directed speech (CHILDES archive). They estimated word learning curves by fitting sigmoidal functions to the trajectory of surprisal (negative log probability) for individual words across training checkpoints. The slope of these sigmoids on a log-scale of experience was treated as the LM's equivalent to the child's $\kappa$ .
Metrics: The study compared the distribution of acceleration exponents ( $\kappa$ ) and calculated "proportional returns" (the ratio of loss reduction to remaining excess loss) for both children and LMs.

Key Results

Accelerating Accumulation in Children: The accelerating accumulator model (M3) provided the best fit to children's growth trajectories across all five datasets, significantly outperforming models without individual acceleration (LOO ELPD improvements > 760). The estimated population acceleration exponent $\kappa$ was robustly greater than 1, ranging from approximately 10.6 to 13.3 across datasets. This indicates that children's learning efficiency increases dramatically with age; the number of exposures required to learn a word decreases exponentially as children develop.
Constant Returns in Language Models: In stark contrast, language models showed no evidence of acceleration. The per-word $\kappa$ estimates for LMs clustered tightly around 1.0 (the pure accumulator value), regardless of whether the models were trained on child-directed speech or standard corpora.
Proportional Returns: Analysis of proportional returns on new data revealed that children exhibit increasing returns (the same experience yields more learning later in development), whereas LMs show constant proportional returns consistent with standard scaling laws ( $L = E + B/D^\beta$ ).
Data Efficiency: The study highlights that children learn hundreds of words with roughly 30–40 million words of input by age three, whereas LMs typically require trillions of words. The accelerating nature of children's learning is proposed as a candidate explanation for this massive data efficiency gap.

Significance and Claims
The paper claims to identify a fundamental disanalogy between human learners and current language models: children learn using an accelerating mechanism where the value of linguistic input increases over time, while LMs learn with constant proportional returns.

Mechanism: The author suggests that children's acceleration may stem from developmental changes in processing speed, memory, and the ability to "learn to learn" (using existing knowledge to infer new meanings more efficiently).
Limitations: The author explicitly states that their evidence is based on statistical model fit and comparison rather than causal manipulation. They acknowledge that the model relies on parent-report data (CDIs) and that the "acceleration" parameter is a statistical description of the growth curve rather than a fully mechanistic account of the cognitive processes involved.
Future Directions: The paper posits that understanding the conditions under which accelerating returns can arise in artificial systems is a critical goal for future research. It suggests that standard pre-training may not provide an analogue to children's growing inferential abilities and that linking hypotheses between children and LMs require further development.

The study concludes that characterizing the nature of children's accelerating returns and determining how to replicate this efficiency in artificial systems should be a primary focus for future investigation in cognitive science and machine learning.

Technical Summary: Children, but not language models, show accelerating returns in word learning

More like this