Four Practical Indicators for Real-Time Verbal Autopsy Data Quality Assessment: A Cross-National Validation Study

In many parts of the world, when a person dies, there is no doctor present to sign a certificate stating the cause. In these places, the only way to understand why people are dying is to ask the family members who were there. This process, known as a verbal autopsy, involves a trained interviewer sitting with a grieving relative and asking a long, structured series of questions about the final days of the deceased. The answers are then fed into a computer system that assigns a likely cause of death, such as malaria, heart disease, or complications from childbirth. This information is vital for governments and health organizations. If they do not know what is killing their citizens, they cannot plan the right hospitals, vaccines, or public health campaigns. However, the entire system relies on one fragile thing: the quality of the conversation. If the family member is confused, if the interviewer rushes, or if the answers contain logical contradictions, the resulting data is flawed, and the public health decisions based on it will be wrong.

For a long time, checking the quality of these interviews happened too late. Researchers would wait until the data was collected, aggregated, and processed by algorithms to assign causes of death. Only then, weeks or months later, would they discover that a specific interviewer had been skipping questions or that a dataset contained impossible answers, like a man dying from pregnancy complications. By that time, the interviewers had moved on, the families could not be reached again, and the errors were permanent. A new study published by researchers from institutions in Tanzania, Eswatini, Nigeria, and international partners aims to fix this delay. They have developed a set of four simple, practical tools that can check the quality of a single interview the moment it is finished, allowing supervisors to spot problems immediately and correct them before the data is ever used to count deaths.

The researchers tested these tools on real-world data from three different countries: Tanzania, Eswatini, and Nigeria. These datasets contained thousands of interviews conducted using the standard World Health Organization questionnaires. The team created four specific ways to measure the health of each interview record. The first tool, which they call the Informative Completeness Score, simply checks how many "yes" or "no" questions were answered with a clear decision rather than a "I don't know" or a refusal. A high score means the family member was engaged and knew the answers; a low score suggests the interview was unproductive. The second tool, the Respondent Reliability Score, looks at who was answering the questions. It gives a higher reliability rating if the person speaking was a close family member, like a spouse or parent, if they were present when the person died, and if they had been interviewed relatively soon after the death. It also considers the education level of the deceased as a stand-in for the interviewer's likely education, assuming that spouses or adult children often share similar schooling.

The third tool is an Internal Consistency Index, which acts as a logic check. It scans the answers for biological impossibilities. For example, it flags a record if a man is reported to have had symptoms related to pregnancy, or if the number of days a person had a fever is longer than the total number of days they were sick. These contradictions often happen when an interviewer makes a mistake or when the data entry system fails to stop a question that should have been skipped. The fourth tool measures the time it took to conduct the interview. While the average interview might take around half an hour, the researchers found that the median time—a measure that ignores extreme outliers—is a better indicator of normal performance. If an interview is finished in just a few minutes, it likely means the interviewer skipped large sections. If it takes many hours, it might mean the data was entered over several days or that the interviewer struggled with the respondent.

When the team applied these four checks to the data from the three nations, they found that most of the interviews were of high quality. The vast majority of records showed that respondents were engaged, reliable, and consistent. In Tanzania, for instance, nearly 98 percent of the binary questions were answered with clear "yes" or "no" responses. In Eswatini and Nigeria, the scores were similarly high. This suggests that the field teams in these countries are generally doing a good job. However, the power of these new tools lies in their ability to find the small number of bad records that slip through. The researchers discovered specific patterns that would have been missed by waiting for the final cause-of-death assignment. In Eswatini, they found that a small group of interviews had a specific error where the duration of a cough was recorded as longer than the total time the person was ill. This was not a random mistake but a pattern that appeared in 6.1 percent of the records, suggesting a specific training gap or a misunderstanding of how to ask about symptom duration.

In Nigeria, the tools uncovered a much larger issue with a different kind of logic error. The system flagged that in nearly 6 percent of the records for men, the interview included answers to questions about pregnancy. Since men cannot be pregnant, this indicated that the computer form used by the interviewers was not correctly hiding those questions from male respondents, or that the data was being processed in a way that ignored the gender of the deceased. This is a critical finding because if these records were used to calculate death rates, they would distort the understanding of maternal health. In Tanzania, the tools revealed a period of time where data quality dropped significantly. During a specific window in 2024 and 2025, the number of interviews with low engagement scores and impossible dates—where the interview was recorded as happening before the person died—spiked. This pointed to a temporary breakdown in supervision or data entry oversight that the local team could now investigate and fix.

The study also highlighted that these tools are flexible enough to work with different versions of the questionnaire used in different years, and they do not require a large batch of data to be useful. A supervisor can check a single interview and know immediately if it is trustworthy. The researchers noted that while these tools are powerful, they are not a magic bullet that replaces the need for good training or well-designed forms. Instead, they act as a safety net, catching the errors that happen despite the best efforts of the training. The study confirmed that these indicators can be calculated automatically by the software systems that manage the data, meaning they can be built into the daily workflow of health programs. By moving quality control from a post-mortem analysis to a real-time check, health officials can ensure that the stories told by grieving families are translated into accurate, life-saving knowledge about why people die.