Don't Drop the Singletons: Efficient Inference for Pairwise Experiments with Independent Attrition

In the world of scientific experiments, researchers often face a difficult choice between precision and practicality. To find out if a new treatment works, scientists frequently use a method called randomization, where they assign people to either a treatment group or a control group by chance. A popular and powerful version of this involves pairing up participants who are very similar to one another—perhaps neighbors with similar incomes or students with similar test scores—and then randomly giving the treatment to one person in the pair while the other serves as the control. This "pairwise" approach is like having a built-in control for every single subject, which usually makes the results much sharper and more reliable than just comparing two large, mixed groups. However, a persistent problem in real-world studies is attrition: people dropping out of the experiment before it ends. When this happens, a pair might lose one member, leaving the other person without a partner. For years, standard advice has been to throw away these "orphaned" singletons and only analyze the pairs where both people stayed. The logic was that without a partner, the data was too messy to use. This practice, however, meant that researchers were discarding valuable information, effectively shrinking their study and making it harder to detect real effects.

Two researchers, Simon Heß and Patrick W. Schmidt, have challenged this long-standing rule. They argue that the fear of using data from incomplete pairs is misplaced, provided the reason people drop out is unrelated to the treatment they received. In their new work, they demonstrate that it is possible to keep every single observation, even the lonely ones, and still get highly accurate results. They developed a new way to analyze the data that respects the original pairing structure while intelligently combining the information from perfect pairs with the information from incomplete ones. Instead of discarding the singletons, their method treats them as useful pieces of the puzzle, weighting them appropriately based on how much information they provide. By doing so, they show that researchers can maintain the high precision of the paired design without suffering the statistical penalty of losing data to dropouts.

The core of their discovery is a specific testing procedure that acts like a smart filter. When a study loses a participant, the traditional approach would simply delete the entire pair from the analysis, effectively pretending that person never existed. The authors show that this throws away a significant amount of evidence. Their new method uses a technique called randomization inference, which relies on the fact that the treatment was assigned by a coin flip within each pair. They create a test statistic that looks at the difference in outcomes for the complete pairs and the difference in outcomes between the singletons, then blends these two sources of information together. The blending is done with mathematical precision: the complete pairs, which offer a direct comparison between two similar people, are given more weight, while the singletons, which offer a broader comparison across the whole group, are given a smaller but still significant weight. This combination allows the study to use every single data point that remains, maximizing the power to detect a true effect.

To prove this works, the researchers ran extensive simulations and theoretical calculations. They compared their new method against the two standard ways scientists currently handle this problem. The first standard method is the "paired t-test," which follows the old rule of throwing away incomplete pairs. The second is the "two-sample t-test," which ignores the pairing entirely and just compares the average of all treated people against the average of all control people. The authors found that their new approach consistently outperformed both. In scenarios where matching quality was high—meaning the pairs were very well matched—the old paired method lost a lot of power as soon as people started dropping out. The method that ignored pairing entirely was often too conservative, failing to see effects that were actually there. The new method, however, dominated both, delivering stronger results across every level of attrition and matching quality they tested.

The researchers also addressed a common concern about whether this new method is too complicated to use in practice. They showed that the complex weighting and testing can be performed using a standard statistical tool called a weighted regression, which is available in almost all common software packages. A researcher can simply tell the software to include all the data, assign a specific weight to the singletons, and include fixed effects for the complete pairs. This makes the powerful new method accessible to anyone who can run a basic analysis, removing the barrier of needing specialized, custom-built code. The authors emphasize that this solution works best when the dropouts happen for reasons unrelated to the treatment, such as moving away or losing interest, rather than because the treatment made them feel worse. If the dropouts are related to the treatment itself, the problem becomes much harder, but for the vast majority of cases where dropouts are random, the new method offers a clear path forward.

In a final check, the authors compared their optimized paired design against a different strategy often recommended when dropouts are expected: using larger groups instead of pairs, such as groups of four. The idea is that in a group of four, if one person leaves, three remain, preserving more of the group's structure. The researchers simulated this scenario and found that even with larger groups, their optimized method for pairs still held its own. In many cases, the efficiency gained by the tight matching of pairs, when combined with their method of using all available data, was enough to beat the larger groups. This suggests that researchers do not need to abandon the paired design just because they fear dropouts. Instead, they can stick with the more precise pairing strategy and simply apply the new analysis technique to ensure they are not wasting any of the data they worked so hard to collect.

The implications of this work are straightforward for anyone running a study. It removes the trade-off between keeping a tight, efficient design and losing data to attrition. By refusing to drop the singletons and instead weaving them into the analysis with the right weight, researchers can keep their studies powerful and their conclusions robust. The paper provides a practical toolkit that turns a potential weakness—losing participants—into a manageable part of the process, ensuring that the final results reflect the full scope of the experiment rather than just the survivors.