📊 statistics

Embarrassingly Causal: Causal Use of Associational Data in Magic The Gathering Drafts

This paper introduces the concept of "embarrassingly causal" scenarios, using Magic: The Gathering draft data to argue that when an exposure-outcome relationship is sufficiently uncontroversial, observational data can validly support causal inference despite significant confounding and selection biases.

Original authors: Mark Louie F. Ramos, Ph. D

Published 2026-04-21

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Mark Louie F. Ramos, Ph. D

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: When "Guessing" is Actually Smart

Imagine you are trying to figure out if eating a specific type of candy makes you run faster. You look at a group of people and see that the ones who ate the candy did run faster.

Usually, statisticians would scream, "Stop! That's just a coincidence! Maybe the candy-eaters also happen to be professional athletes who already run fast!" This is the classic rule: Correlation does not equal causation.

However, this paper argues that sometimes, the link between two things is so obvious, so undeniable, and so supported by common sense, that we don't need to be afraid of the "coincidence" argument. The author calls these "Embarrassingly Causal" situations.

Think of it like this: If you see a person pushing a heavy boulder and the boulder moves, you don't need a complex scientific study to prove the push caused the movement. It's "embarrassingly" obvious. The paper suggests that when a situation is this clear, we can safely use observational data (just watching what happens) to make decisions, even if the data is messy.

The Case Study: Magic: The Gathering (The Card Game)

To prove this point, the author uses a video game called Magic: The Gathering. Specifically, a mode called a "Booster Draft."

The Scenario:
Imagine you are at a party where everyone is picking cards from a deck to build a team.

The Choice: You have to pick one card out of three.
The Result: Later, you play a game with the team you built. You either win or lose.

The Problem:
A website called 17Lands collects data from millions of players. They see that players who picked "Card A" won 60% of their games, while players who picked "Card B" only won 40%.

The Skeptic's View:
A strict statistician would say: "Wait! Maybe the people who picked Card A were just better players to begin with. Or maybe they had better teammates. You can't say Card A caused the win. You're just looking at a messy correlation."

The "Embarrassingly Causal" View:
The author says: "No, look at the game mechanics. In this specific game, the cards you pick directly determine the deck you play. If you pick a bad card, your deck is weaker. If you pick a good card, your deck is stronger. There is no way the card didn't influence the outcome."

Because the link between Picking a Card and Winning the Game is so physically and logically direct, it is "embarrassingly causal." Even though the data is messy (people have different skill levels, different opponents, etc.), the direction of the cause is undeniable.

The "Gold Standard" Analogy

The paper discusses how players use simple stats from 17Lands, like "Games in Hand Win Rate" (how often you win when you actually draw a specific card).

The Flaw: This stat is technically biased. If you draw a card, it might be because the game went on longer, or because you played a specific type of deck. It's not a perfect scientific experiment.
The Reality: Despite the flaws, professional players trust this data. Why? Because they know that choosing a card changes the deck, and the deck changes the win rate.

It's like looking at a weather forecast. The forecast isn't 100% perfect (it might rain even if it says sunny), but if the meteorologist says "It's raining," you grab an umbrella. You don't demand a perfect, controlled experiment to prove the rain is real before you get wet. You trust the obvious causal link: Rain → Wetness.

Why Does This Matter? (The "So What?")

The author is trying to fix a problem in science and research.

The Current Problem: Scientists are often too scared to use observational data. They think, "I can't prove causation without a Randomized Controlled Trial (like a drug test where half the people get a sugar pill)." So, they ignore useful data.
The Other Problem: Some people are too reckless. They see a correlation and immediately claim it's a cause without thinking.
The Solution: The "Embarrassingly Causal" rule.

The paper suggests a middle ground:

If the link is "Embarrassingly Causal" (like Radiation causing Cancer, or Picking a Card affecting a Game Win), you should use observational data to make decisions. You don't need to wait for a perfect experiment. The assumption that "A causes B" is so strong that it justifies the study.
If the link is NOT "Embarrassingly Causal" (like "Does drinking coffee cause happiness?"), then you must be very careful. There are too many other factors (maybe happy people just drink more coffee). In these cases, you can't just trust the data; you need stronger proof.

The Takeaway for Everyone

For Researchers: Don't be afraid to use real-world data if the cause-and-effect relationship is obvious. You don't need to apologize for using "messy" data if the direction of the arrow is clear.
For Reviewers (The Judges): When reading a study, ask: "Is this relationship so obvious that we don't need to argue about it?" If yes, the study is likely valid. If no, the study needs much stronger proof.
For You: When you see a headline saying "X causes Y," check if it's "Embarrassingly Causal."
- Example: "Smoking causes lung cancer." (Yes, this is embarrassingly causal. We accept the data.)
- Example: "Drinking red wine causes you to live longer." (Maybe not. Maybe rich people drink wine and also have better doctors. This is NOT embarrassingly causal. Be skeptical.)

In short: The paper gives us permission to trust our gut and our domain knowledge when the cause-and-effect link is too obvious to doubt, allowing us to make better decisions using the data we already have.

1. Problem Statement

The paper addresses a fundamental tension in causal inference: the debate over whether causal structures can be learned from observational data.

The Conflict: Machine learning approaches often argue that causal graphs can be partially discovered from data under strong structural assumptions (e.g., faithfulness, causal sufficiency). Conversely, traditional statistics and epidemiology posit that Directed Acyclic Graphs (DAGs) encode scientific assumptions that cannot be justified by data alone; they require substantive domain knowledge.
The Gap: While it is widely accepted that domain knowledge is necessary to justify causal edges in a DAG, there is no formal guidance on how much domain knowledge is sufficient. Researchers often struggle to communicate when an association can be treated as causal, leading to either an absolute prohibition of causal claims from observational data or the unjustified over-interpretation of correlations.
The Specific Challenge: In many observational settings, estimating causal effects is plagued by confounding, selection bias, and post-treatment conditioning. The paper asks: Under what conditions is it reasonable to collect and analyze purely observational data to estimate causal effects, even when rigorous identification (e.g., via randomization) is impossible?

2. Methodology and Framework

The author introduces a new conceptual criterion called "Embarrassingly Causal" scenarios and validates it using a case study from the digital card game Magic: The Gathering (MTG).

A. The "Embarrassingly Causal" Criterion

The paper defines an "embarrassingly causal" scenario as a situation where the existence of a causal relationship ( $Exposure \rightarrow Outcome$ ) is so uncontroversial and strongly supported by domain knowledge that the assumption of a causal edge in a DAG is considered uncontroversial.

Key Distinction: This criterion does not claim the magnitude or sign of the effect is known, nor does it guarantee the effect is non-zero. Rather, it asserts that the causal direction itself is not in doubt.
Sufficiency: The author argues that this quality is a sufficient condition to justify the construction of causal estimands and the collection of observational data to estimate them, even if the data is heavily confounded.

B. Case Study: MTG Booster Drafts & 17Lands Data

Context: In MTG Booster Drafts, players select cards sequentially (45 picks) to build a deck, which is then used in matches. The goal is to maximize win probability.
Data Source: The study utilizes 17Lands, a massive, crowd-sourced observational dataset containing millions of draft picks and match outcomes from players who opt-in to share logs.
The Causal Question: What is the causal effect of picking a specific card ( $C_t$ $C_{t}$ ) at a specific turn ( $t$ $t$ ) on the probability of winning a future match ( $1_{win}$ $1_{w in}$ )?
- Target Estimands: $P(1_{win} | do(C_t=c))$ and the difference between picking card $c$ vs. card $c'$ .
The Bias Problem: The available statistics from 17Lands (e.g., Games in Hand Win Rate [GIHWR] and Improvement In Hand [IIH]) are naïve associational measures.
- They suffer from selection bias (conditioning on drawing the card).
- They suffer from confounding (deck composition, player skill, game length).
- Formally, they estimate conditional probabilities like $P(1_{win} | C_t=c, Drawn=1)$ , which are biased estimators of the true causal effect.

C. Analysis of Practice

Despite the statistical bias, the paper observes that professional drafters and experts widely and effectively use these naïve metrics (GIHWR, IIH) to make optimal draft decisions. Experts acknowledge the biases but treat the metrics as "informative" regarding the "raw power" of a card because the causal link between picking a card and winning the game is intuitively undeniable.

3. Key Contributions

Formalization of "Embarrassingly Causal":
The paper provides a formal definition for a class of scenarios where the causal direction is so well-supported by domain knowledge that the assumption of a causal edge in a DAG is defensible without further statistical proof. This bridges the gap between "correlation is not causation" and the practical necessity of using observational data.
Justification for Observational Causal Inference:
The author argues that the "embarrassingly causal" quality is a sufficient condition for justifying the pursuit of causal estimands in observational settings. It validates the decision to collect data and construct a causal model, even if the resulting estimates are biased or the exact magnitude is unknown.
Practical Guidance for the Research Community:
The paper offers specific guidelines for three groups:
- Authors: Use this criterion to justify the inclusion of causal edges in DAGs. If a relationship is "embarrassingly causal," the focus shifts to estimating the effect size rather than proving the direction exists.
- Reviewers: Evaluate whether subject-matter experts regard the exposure-outcome relationship as "embarrassingly causal." If not, reviewers should demand that authors address competing causal explanations (e.g., unmeasured confounding) more rigorously.
- Readers: Assess the evidentiary value of a study based on how well the causal assumptions are framed. If the relationship is not "embarrassingly causal," readers should be skeptical of causal claims regardless of statistical significance.
Extension to Other Domains:
The paper demonstrates that this framework applies beyond gaming. Examples include:
- Real-World Evidence (RWE): Using observational data to study interventions already proven causal in Randomized Controlled Trials (RCTs).
- Mechanistic Certainty: Studies on radiation exposure causing cancer or asbestos causing mesothelioma, where the biological mechanism is universally accepted.

4. Results and Findings

Validation of Naïve Metrics: In the MTG context, despite GIHWR and IIH being statistically biased estimators of the true causal effect, they function effectively as decision-making tools. This is because the "embarrassingly causal" nature of the problem (Picking $\rightarrow$ Winning) allows players to ignore the complex confounding structure and rely on the association as a proxy for causality.
Limitations of Causal Discovery: The paper reinforces that without "embarrassingly causal" structure, algorithms attempting to infer causal graphs directly from data (Causal Discovery) remain underdetermined. Multiple distinct causal graphs can fit the same data, making edge orientation reliant on untestable assumptions rather than domain knowledge.

5. Significance

This paper makes a significant contribution to the philosophy and practice of causal inference by:

Reframing the Burden of Proof: It shifts the focus from "proving causality from data" to "justifying the assumption of causality via domain knowledge."
Bridging Theory and Practice: It explains why practitioners in fields like gaming, medicine (RWE), and public health successfully use observational data for causal decisions despite theoretical limitations.
Providing a Heuristic: It offers a concrete, albeit qualitative, heuristic ("Is this embarrassingly causal?") for researchers to determine when it is methodologically sound to proceed with observational causal analysis.

In summary, the paper argues that while observational data alone cannot prove causality, domain knowledge that renders a causal direction "embarrassingly obvious" is sufficient to justify the entire causal inference pipeline, transforming "naïve" associations into actionable, causally-informed decisions.