🤖 machine learning

On design-unbiased algorithmic Machine Learning

This paper proposes a design-based framework for achieving unbiased predictions and classifications in machine learning algorithms by leveraging known sampling probabilities rather than assuming underlying data models, thereby addressing the need for unbiased inference in contexts like official statistics.

Original authors: Li-Chun Zhang, Siu-Ming Tam, Luis Sanguiao-Sande, Wesley Yung, Anders Holmberg

Published 2026-06-30

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Li-Chun Zhang, Siu-Ming Tam, Luis Sanguiao-Sande, Wesley Yung, Anders Holmberg

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a chef trying to create the perfect recipe for a massive banquet of 10,000 guests (the population). You can't taste every single dish before serving it, so you take a small tasting spoonful from the kitchen (the sample) to judge the whole meal.

In the world of Machine Learning (ML), chefs usually try to make their tasting spoonful as "accurate" as possible by minimizing errors. They tweak their recipe until the taste on the spoon is perfect. However, the authors of this paper argue that a perfect taste on the spoon doesn't guarantee the whole banquet will taste right. Sometimes, the spoonful is just lucky, or the way you picked the spoonful was biased, leading to a "delicious" spoon but a "salty" banquet.

This paper is about a new way to cook: Design-Unbiased Machine Learning. Instead of just hoping the spoonful represents the whole pot, they use a strict set of rules (a "design") to ensure that what you learn from the spoon is mathematically guaranteed to represent the whole pot, even if you don't know the "true" recipe of the universe.

Here is the breakdown of their method using simple analogies:

1. The Problem: The "Lucky Spoon"

Standard ML algorithms (like k-Nearest Neighbors or Random Forests) are like chefs who taste a dish and say, "This is good!" based on the ingredients they see. They try to minimize the difference between their guess and the actual taste.

The Issue: If you pick your tasting spoon from the top of the pot (where the cream is), your guess about the whole pot will be biased. You might think the whole soup is creamy, but the bottom is watery. In statistics, this is called bias. Standard ML tries to be "accurate" (low error) but often fails to be "unbiased" (honest about the whole population).

2. The Solution: The "Representative Training" Rule

The authors introduce a concept called Representative Training.

The Analogy: Imagine you have a bag of marbles (the population). You pull out a handful (the sample). To make a fair prediction about the rest of the bag, you need to make sure that the handful you used to train your brain (the training set) is a fair reflection of the handful you are using to test your brain (the test set).
The Rule: If you pick your training marbles and your test marbles using a specific, fair lottery system (called a pq-design), then the "average guess" your algorithm makes for the test marbles will be exactly the same as the "average guess" it would make for the marbles you never saw at all.
Why it matters: This allows you to use the errors you see on your test spoon to fix your prediction for the whole pot.

3. The Fix: "Out-of-Bag" Tuning

Once you have this fair setup, you can fix the bias.

The Analogy: Imagine your chef (the algorithm) tastes a spoonful and says, "I think the soup is too salty." But wait, the chef made a mistake because they tasted the spoon while they were cooking it.
The Trick: The authors suggest using an "Out-of-Bag" (OOB) approach. This is like having a second chef taste the soup without having helped cook that specific spoonful.
- You split your sample into two groups: Group A (Train) and Group B (Test).
- You train the algorithm on Group A.
- You ask the algorithm to predict Group B.
- You compare the prediction to the actual taste of Group B.
- The Magic: If the algorithm consistently overestimates the saltiness on Group B, you know it will likely overestimate the saltiness for the entire pot. You then subtract that "overestimation amount" from your final prediction.
The Result: This "tuning" guarantees that your final prediction for the whole population is unbiased. It doesn't matter if the algorithm is complex or simple; if the sampling rules are followed, the math guarantees the result is fair.

4. Classifying vs. Predicting (The "Yes/No" Menu)

The paper also looks at classification (e.g., "Is this a coffee field or not?" instead of "How much coffee is there?").

The Challenge: If you just say "Yes" or "No" based on a threshold (e.g., "If probability > 50%, it's coffee"), you often introduce bias.
The Fix: The authors suggest using a randomized classifier. Instead of a hard "Yes/No," imagine flipping a weighted coin. If the algorithm says there is a 70% chance it's coffee, you flip a coin that lands on "Coffee" 70% of the time.
Why: This randomness smooths out the errors. When you average these coin flips over the whole population, the math works out to be perfectly unbiased, allowing you to accurately count how many coffee fields exist in the country.

5. Real-World Proof (The Satellite Photos)

To prove this works, the authors used a real dataset of satellite images to identify coffee fields.

They took a sample of images, trained a k-Nearest Neighbor (kNN) algorithm, and then applied their "Out-of-Bag" tuning.
The Result: The standard algorithm (without tuning) made small but noticeable errors in its total count. The tuned algorithm, using their new rules, produced a count that was statistically indistinguishable from the true total (unbiased).
Bonus: They also showed that you can measure how accurate the "Yes/No" classification is without needing to know the true answer for every single image, simply by using the same "Out-of-Bag" logic.

Summary

Think of this paper as a new set of kitchen safety rules for Machine Learning.

Don't just trust the taste: Standard ML tries to minimize error, but that doesn't guarantee fairness.
Follow the lottery: Use specific sampling rules (pq-design) to ensure your training data and testing data are fair reflections of the whole.
Taste the leftovers: Use the "Out-of-Bag" errors (what the algorithm got wrong on the test set) to mathematically correct the final prediction for the whole population.
The Guarantee: If you follow these rules, your final number (whether it's a total count or a classification rate) will be unbiased, meaning it is a truthful representation of the real world, regardless of how complex your "recipe" (algorithm) is.

This is crucial for things like official government statistics, where being "close" isn't enough; you need to be mathematically honest about the whole population, not just the part you happened to look at.

Technical Summary: Design-Unbiased Algorithmic Machine Learning

Problem Statement

Machine Learning (ML) algorithms, such as k-Nearest Neighbors (kNN) and Random Forests, prioritize predictive performance (e.g., minimizing MSE or F-score) over the existence of true data models. While effective for prediction, minimizing these metrics does not inherently guarantee unbiasedness in estimation or classification. This lack of unbiasedness is critical in applications like official statistics, where population totals and proportions must be accurate descriptors of finite populations.

Traditional survey sampling relies on probability designs (p-designs) and estimators like Horvitz-Thompson to ensure design-unbiasedness, independent of any underlying data model. However, when ML algorithms are integrated into this framework—often trained on subsets of the sample (training sets) to predict outcomes for the rest (out-of-sample units)—standard ML practices can introduce bias. Existing literature often relies on asymptotic consistency (as population size $N \to \infty$ ), which may not apply to finite populations as "real entities." Furthermore, previous work on design-unbiased ML (e.g., Zhang et al., 2025) has not fully addressed the general conditions under which ML algorithms yield design-unbiased predictions or how to assess classification accuracy without bias.

Methodology

The paper proposes a framework for Design-Unbiased Algorithmic Machine Learning based on a $pq$-design. This design consists of:

$p$ -design: A probability sampling design selecting a sample $s$ from a finite population $U$ .
$q$ -design: A specific design for splitting the sample $s$ into a training set $s_1$ and a test set $s_2$ (where $s_2 = s \setminus s_1$ ).

The core theoretical contribution is the concept of Representative Training. A $pq$-design yields representative training if the expected predictor $\mu(x, s_1)$ for any unit $i$ is identical whether $i$ is in the test set $s_2$ (in-sample but out-of-training) or in the out-of-sample set $R = U \setminus s$ . Mathematically, this requires:
$E_{pq}(\mu(x_i, s_1) \mid i \in s_2) = E_{pq}(\mu(x_i, s_1) \mid i \notin s)$

The authors derive conditions under which this holds:

General Condition: The covariance between the predictor and the conditional inclusion probability of the test set must be zero.
Sufficient Condition: The conditional probability of a unit being in the test set given the training set, $\pi_{2i} = Pr(i \in s_2 \mid s_1)$ , must satisfy a specific relationship with the overall inclusion probability $\pi_i$ . This is satisfied by designs such as SRS-SRS (Simple Random Sampling for both stages) and Poisson-Bernoulli designs.

Tuning for Unbiasedness

Even with representative training, standard ML predictors may be biased. The paper introduces a tuning mechanism to achieve exact design-unbiasedness:

Subsampling Rao-Blackwell (SRB): The predictor is averaged over all possible training/test splits to reduce variance.
Bias Correction: A correction term $\tau_\mu(s)$ is subtracted from the SRB predictor. This term is derived from the weighted sum of in-sample Out-of-Bag (OOB) prediction errors.
Unbiased Estimation: The paper provides formulas to estimate the bias and Mean Squared Error (MSE) of these tuned estimators using the observed OOB errors and sampling weights, without assuming a true data model.

Classification

The framework extends to binary classification. The authors argue that deterministic classifiers (e.g., thresholding at 0.5) generally fail to be design-unbiased. Instead, they propose randomized classifiers (where the classification decision is probabilistic based on the predicted probability) which satisfy a regularity condition allowing for unbiased estimation of out-of-sample classification accuracy using the OOB errors.

Key Results

The paper validates the theory through analytical examples, simulations, and a real-data application using kNN ( $k=5$ ).

Analytical Example: In a constructed population, standard kNN and SRB-kNN predictors exhibited non-zero bias. However, the OOB-tuned SRB-kNN (using the proposed correction) achieved exactly zero bias.
Simulation Study:
- Direct application of kNN (sample-kNN) and untuned SRB-kNN resulted in significant bias (e.g., $EBias \approx -0.795$ for SRB-5NN in one scenario).
- OOB-tuned SRB-kNN successfully reduced bias to near-zero levels ( $EBias \approx -0.069$ ) while maintaining competitive or lower MSE compared to residual-tuned alternatives.
- The study demonstrated that the proposed method allows for the unbiased estimation of the bias and MSE of the predictors themselves.
Real-World Application (CropHarvest Dataset):
- Applied to satellite image classification (coffee fields vs. non-coffee).
- Results confirmed that OOB-tuned SRB-5NN is essentially unbiased for population means across different feature sets.
- Feature Selection Impact: The study highlighted that feature selection is critical. Poor feature sets (e.g., those selected via LASSO for logistic regression but used for kNN) increased both bias and variance.
- Classification Accuracy Trade-off: While debiasing ensures accurate population estimation, the tuning process can occasionally push predicted probabilities outside $[0, 1]$ , requiring truncation. This can slightly reduce classification accuracy compared to biased predictors. The paper notes that sensible feature selection can mitigate this risk.

Significance and Claims

The paper claims to provide a rigorous theoretical basis for using ML in finite population inference where design-unbiasedness is paramount. Its significance lies in:

Generalizing Representative Sampling: It extends the concept of "representative training" from IID settings (like cross-validation) to finite population sampling with unequal probabilities and without replacement.
Decoupling from True Models: It demonstrates that unbiasedness can be achieved based solely on the known probability design of the sample and training sets, without requiring the ML algorithm to approximate a "true" data model.
Practical Tuning: It offers a concrete, implementable recipe (OOB-tuning) to correct bias in popular algorithms like kNN and Random Forests.
Unified Inference: It enables the unbiased assessment of both prediction errors and classification accuracy for out-of-sample units, addressing a gap in previous literature.

The authors position this work as an extension of Neyman's (1934) foundational work on finite population sampling, providing an intuitive basis for "consistent" ML estimation in official statistics and other finite-population applications. They acknowledge that while unbiased population estimation is achieved, it may come at the cost of slightly reduced unit-level classification accuracy, a trade-off that warrants further research.