Before Nonresponse: School-Relative Response Timing and Selective Observability in PISA 2025 Context Questionnaires in Latin America

In the vast world of educational research, scientists often rely on massive surveys to understand how students learn, what motivates them, and where inequalities lie. One of the most famous of these surveys is the Programme for International Student Assessment, or PISA, which tests students from dozens of countries every few years. Alongside the math and reading tests, students fill out long questionnaires about their lives, their habits, and how they study. These questionnaires are crucial because they provide the context needed to make sense of the test scores. However, a persistent problem in such long surveys is that students sometimes stop answering. They might skip the final questions, or they might stop providing useful answers long before they officially quit. Traditionally, researchers have treated this missing information as a sudden event: a student answers a question, and then, suddenly, they do not. But this view assumes that the moment a student stops answering is the moment they stopped caring, ignoring the possibility that their behavior might have been changing silently for a long time before the first blank box appeared.

A new study by Jorge Eliécer Villarreal-Fernández challenges this assumption by looking at the behavior of students in thirteen Latin American countries during the 2025 cycle of PISA. Instead of waiting for a student to stop answering, the researcher watched how fast they were answering as they moved through the questionnaire. The study focuses on a specific behavior called "rapid responding," where a student moves through questions so quickly that it suggests they are no longer reading them carefully, but simply clicking through to finish. The central question was whether this rush to the finish line happens before a student actually stops answering, and if so, what factors predict it. The researcher also wanted to know if this behavior changed the results of the survey. Specifically, the study looked at two important scales located near the very end of the questionnaire: one measuring how well students plan and monitor their own studying, and another measuring how often they ask for help from teachers and classmates. By analyzing the timing data of over 116,000 students, the study reveals that the loss of a student's voice in a survey is rarely a sudden silence, but rather the end of a gradual process that can be seen and measured long before the data disappears.

The investigation began by treating the questionnaire not just as a list of questions, but as a timeline of behavior. The researcher examined the time it took for each student to complete small blocks of questions. If a student was answering significantly faster than the typical pace for that block, it was flagged as rapid responding. To ensure this wasn't just a lucky guess or a momentary speed-up, the study required that this fast behavior persist across a sequence of several blocks. Only when a student maintained this rushed pace for a sustained period were they considered to have "transitioned" into a state of disengagement. This distinction is vital because it separates students who are simply fast from those who have stopped trying. The analysis showed that this transition is not random. About three-quarters of the students who eventually failed to provide a score for the study-planning or help-seeking scales had already entered this sustained rapid-responding phase before they even reached the section containing those specific questions. In other words, the warning signs were visible long before the data was actually lost.

What drives a student to speed through a questionnaire? The study found that the answer lies partly in how a student's pace compares to the students sitting next to them in the same school. The researcher calculated a "relative delay," which measures how much slower or faster a student is compared to their peers in the same classroom or school. The results were striking: for every five minutes a student fell behind the average pace of their schoolmates, their odds of entering that sustained rapid-responding state increased by about 41 percent. This suggests that the social environment of the testing room matters. A student who feels they are lagging behind their classmates may feel pressure to catch up, leading them to abandon careful reading in favor of speed. This effect varied widely across the different countries, but the pattern held true overall. Interestingly, a student's own socioeconomic background did not predict this behavior when compared to their immediate classmates. Instead, the socioeconomic makeup of the school itself was the stronger factor. Students in schools with higher average socioeconomic status were less likely to rush through the questions, regardless of their own family's wealth. This points to the idea that the institutional environment—perhaps the resources available, the testing atmosphere, or the way the survey is administered—plays a larger role in student engagement than individual circumstances.

The study then asked a critical question: does this behavior actually change the results we get from the survey? If the students who rush through are different from those who take their time, then the average scores we calculate might be wrong. To find out, the researcher compared the answers of students who stayed engaged with those who had transitioned into rapid responding but still managed to finish the specific scales. The good news was that for the students who did finish, the way they answered the questions was very similar. The questions about study planning and asking for help seemed to measure the same things for both groups. However, the bad news was that the group of students who remained observable was not a perfect mirror of the whole population. Because the students who rushed through were more likely to drop out entirely before reaching the end, the final group of respondents was slightly different from the original group of all students.

When the researcher adjusted the data to account for this missing group, the results changed in a specific and surprising way. The estimated average for the study-planning scale dropped slightly. This means that the students who stayed to answer the questions were, on average, slightly better at planning their studies than the full population of students, because the students who rushed and dropped out were less likely to be good planners. However, the estimate for the help-seeking scale barely changed at all. Even though both scales were located at the very end of the questionnaire and were equally vulnerable to students dropping out, the impact of the missing data was different for each. This proves that the consequences of missing data are not uniform; they depend entirely on what is being measured. A survey might lose a specific type of student without changing the average for one topic, while significantly skewing the average for another.

The findings of this study offer a new way to think about survey data. It suggests that when a student stops answering a long questionnaire, it is often the culmination of a process that started much earlier. The transition from careful answering to rushing, and finally to silence, is a visible path that can be tracked using the timing data collected by the computer. This changes how researchers should interpret missing information. Instead of assuming that missing data is just a random gap, they can look for the behavioral signals that precede it. For the designers of future surveys, this means that timing information could be used as an early warning system. If a student starts falling behind their peers in speed, it might be possible to intervene or adjust the survey before the student stops answering altogether. The study does not claim to have solved the problem of missing data, nor does it suggest that every survey will have these exact results. But it does show that the loss of a student's voice is often a gradual fading, not a sudden cut, and that understanding this process is key to getting a true picture of the students we are trying to study.