Non-independence in network-structured Mendelian randomization: a permutation framework and six failure modes

Scientists have long sought a reliable way to predict whether a drug will work before it ever reaches a patient. One powerful method, known as Mendelian randomization, uses nature's own randomization to test cause and effect. It looks at genetic variations that people inherit at birth, which act like natural experiments. If a specific genetic variant causes a person to have higher levels of a certain protein in their blood, and that same person also has a higher risk of a disease, researchers can infer that the protein itself likely causes the disease. This approach has become a standard tool for nominating new drug targets, particularly for heart disease. A common assumption in this field is that if a protein (a ligand) causes a disease, then the protein that receives its signal (its receptor) should tell the same story. If the messenger is dangerous, the logic goes, the receiver must be part of the problem.

However, a recent study challenges this assumption, revealing that treating these two measurements as interchangeable evidence is a mistake. The researchers analyzed thousands of pairs of interacting proteins to see if their genetic signals agreed on the risk of coronary artery disease. They found that the signals from the messenger and the receiver often disagreed, and when they did agree, it was no more often than if the proteins had been paired at random. The study also uncovered a hidden flaw in how scientists count their discoveries: because proteins are connected in a complex web, the same protein appears in many different pairs, making standard statistical corrections invalid. The result is a clearer, more cautious view of how we identify drug targets, showing that we cannot simply assume a pathway works the same way just because two parts of it are genetically linked.

The researchers began by gathering data on 660 different proteins measured in the blood of nearly 50,000 people. They focused on proteins that had a clear genetic switch nearby, which allowed them to use the Mendelian randomization method with confidence. They then looked at 1,129 pairs of proteins where one acts as a signal and the other as a receiver, a relationship mapped out in a database of known biological interactions. For each pair, they calculated the effect of the signal protein on heart disease and compared it to the effect of the receiver protein. If the common assumption were true, these two numbers should move in the same direction. Instead, the team found that the agreement between the signal and the receiver was no better than chance. In one large dataset, the two proteins pointed in the same direction only about 48.5% of the time, a result indistinguishable from randomly pairing proteins together. In a second, larger dataset, the agreement was even lower, at 47%.

This lack of agreement suggests that measuring the signal and measuring the receiver are actually answering two different questions. The study showed that when researchers looked for pairs where the two proteins disagreed significantly, the number of "discoveries" tripled when they used a larger dataset with more heart disease cases, rising from 10 to 31. Crucially, the statistical expectation for random noise also rose in lockstep, from 9 to 30, meaning the increase was consistent with chance rather than a biological signal. The researchers demonstrated that standard methods for correcting these errors were failing because they treated every protein pair as an independent test. In reality, the proteins are connected in a network; a single protein can be part of dozens of pairs. When the same protein appears in multiple tests, the data is not independent, and standard corrections overestimate the number of true discoveries.

To fix this, the team developed a new way to test their results that respected the network structure. They tried two approaches: one that shuffled the connections between proteins while keeping the proteins themselves and their number of connections exactly the same, and another that moved the statistical values to different proteins. They found that moving values to different proteins was unsuitable because it ignored the fact that highly connected proteins tend to have larger effects. However, the method that kept the proteins and their connections fixed, shuffling only the pairings, showed that the apparent "significant" disagreements found by standard methods were likely just artifacts of the network's shape. When they applied this stricter test, the number of observed disagreements was not significantly different from what would be expected by random chance. The study also examined whether proteins that are known to be causal for heart disease, such as PCSK9, showed a special pattern of agreement. While PCSK9 was correctly identified as a risk factor, the broader pattern of agreement between signals and receivers did not hold up. Even when the researchers checked their findings against a different measurement technology to ensure the results were not due to errors in the lab equipment, the lack of agreement between signals and receivers remained.

The implications of these findings are significant for how future drug targets are chosen. The study identified six specific ways that current practices can lead researchers astray, including the failure to account for network connections and the reliance on single genetic signals in complex regions of the genome. One major issue is that many proteins sit in areas of the genome where multiple genetic signals overlap, making it difficult to tell which one is actually driving the disease. The researchers found that when they changed the dataset used to define heart disease, the conclusions about whether a protein and a disease shared a genetic cause often flipped. This instability suggests that many previous claims about shared pathways may not be robust. The study concludes that while Mendelian randomization remains a valuable tool, the assumption that a ligand and its receptor provide interchangeable evidence about a biological pathway is not supported by the data. Scientists must now treat these two measurements as distinct pieces of evidence rather than as a single, unified story.