💬 NLP

Using Embedding Models to Improve Probabilistic Race Prediction

To address the limitations of standard Bayesian Improved Surname Geocoding (BISG) in predicting the race of individuals with uncommon surnames, the authors propose "embedding-powered BISG" (eBISG), a method that utilizes pre-trained text embeddings and neural networks to significantly improve race probability estimates for populations omitted from Census surname data.

Original authors: Noan Dasanaike, Kosuke Imai

Published 2026-04-27

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Noan Dasanaike, Kosuke Imai

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Problem: The "Mystery Name" Blind Spot

Imagine you are a detective trying to solve a mystery about how different groups of people are treated in a city. To do your job, you need to know the race of the people living there. However, because of privacy laws, the city won't give you a list of names and races.

To get around this, you use a common "detective trick" called BISG. It works like this:

The Name Clue: You look at a person's last name. If their name is "Smith," you know there’s a high chance they are White. If it’s "Garcia," there’s a high chance they are Hispanic.
The Neighborhood Clue: You look at where they live. If they live in a neighborhood that is 80% Black, you use that to help refine your guess.

The Glitch: This trick only works if the name is in your "Detective Handbook" (the Census data). But here’s the catch: about 10% of people have names that aren't in the handbook.

When the detective hits a name they don't recognize, they basically throw their hands up and say, "I have no idea! I'll just guess the national average." This is a huge problem because these "mystery names" aren't random—they are often names from immigrant families (like many Asian or Hispanic names). Because the detective is guessing blindly, they end up making huge mistakes, which makes their whole investigation into fairness and equality inaccurate.

The Solution: "The Name DNA" (eBISG)

The authors of this paper decided to give the detective a superpower: Embedding Models.

Think of an "embedding" like Digital DNA. Instead of just looking at a name as a string of letters, the computer looks at the vibe and structure of the name.

Even if the detective has never seen the name "Zuberi" before, the computer can look at its "DNA" and say: "Wait, this name has a similar linguistic pattern and structure to these other names in my database that are associated with certain racial groups." It’s like recognizing a person's gait or the way they dress even if you've never met them before.

The researchers created three levels of this "Super-Detective" tool:

The Surname Specialist: If the last name is missing, the computer looks at the "DNA" of the last name to make an educated guess.
The Double Agent: It looks at the "DNA" of both the first name and the last name separately.
The Full-Name Expert (The Winner): This is the most advanced version. Instead of looking at the pieces separately, it looks at the entire name as one single unit.

Why is the Full-Name Expert better?
Think of it like music. You can listen to a drum beat and a guitar riff separately, but you get a much better sense of the song if you hear them playing together. Some first names and last names "sound" a certain way when paired up. The Full-Name Expert catches those subtle musical harmonies that the other methods miss.

The Results: Why This Matters

The researchers tested this on real voter data from North Carolina and Florida, and the results were a game-changer:

No More Blind Guessing: For people with "mystery names," the accuracy shot up significantly, especially for Asian and Hispanic voters.
Fixing the "Wealth Bias": Previously, the old detective trick had a bad habit of being wrong in ways that correlated with money (it often misidentified people in rich or poor neighborhoods). The new "DNA" method fixed this, making the data much fairer.
Plug-and-Play: The best part? This isn't a whole new, complicated system. It’s like an upgrade chip for the old detective tool. Researchers can keep using their existing methods but just "plug in" this new, smarter way of reading names.

The Bottom Line

By teaching computers to understand the "linguistic DNA" of names, we can stop guessing and start seeing the true diversity of our communities. This allows scientists to more accurately study fairness in healthcare, lending, and voting, ensuring that no one is "invisible" just because their name wasn't in a standard handbook.

Technical Summary: Using Embedding Models to Improve Probabilistic Race Prediction

1. Problem Statement

The standard method for estimating individual-level race in administrative datasets where race is not recorded is Bayesian Improved Surname Geocoding (BISG). BISG combines a surname-based racial prior $P(R|S)$ (from Census data) with geographic racial composition $P(R|G)$ using Bayes' rule.

However, the authors identify a critical failure mode: the "unmatched surname" problem. The Census surname list only includes names occurring at least 100 times, omitting approximately 10% of the US population. For these individuals, standard BISG reverts to a "generic prior" (the national racial average), which provides zero information about the individual's likely race. This omission is not random; it disproportionately affects Hispanic and Asian voters, whose surnames are more likely to be excluded from Census frequency thresholds. Consequently, standard BISG suffers from poor precision-recall and systematic, income-correlated biases for these groups.

2. Methodology

The authors propose embedding-powered BISG (eBISG). Instead of using an uninformative prior for unmatched names, they use pre-trained text embeddings to represent names as dense vectors. Because these embeddings (specifically E5-Large) capture morphological and linguistic patterns, names with similar etymological roots will have similar vector representations, allowing the model to "infer" race for unseen names.

The paper evaluates five distinct approaches:

Standard BISG: Uses only Census surname probabilities.
BIFSG (Bayesian Improved First name and Surname Geocoding): Adds Census first-name probabilities as an independent Bayesian factor.
Surname Embedding (eBISG): Trains a neural network on Census surname data to predict $P(R|S)$ from surname embeddings for unmatched names.
Surname + First Name Embedding (eBISG): Trains separate models for both surname and first-name embeddings to provide more informative priors for unmatched individuals.
Full-Name Embedding (eBISG): The most advanced method. It concatenates first, middle, and last names into a single string and trains a neural network on voter file data (from four Southern states). This allows the model to capture complex interactions between name components, bypassing the "conditional independence" assumption used in BIFSG.

3. Key Contributions

Introduction of eBISG: A framework that replaces uninformative priors with informative, embedding-based priors without altering the existing BISG architecture.
Mitigation of Systematic Bias: Demonstrates that much of the known income-correlated bias in BISG (where predictions are skewed by neighborhood wealth) is actually a byproduct of the uninformative prior rather than a flaw in the BISG framework itself.
Interaction Modeling: Proves that treating a name as a single semantic unit (full-name embedding) is superior to treating name components as independent signals.
Seamless Integration: The method is designed to be "plug-and-play" with existing downstream statistical tools like the BIRDiE estimator.

4. Results

The models were evaluated using voter registration files from North Carolina and Florida, using self-reported race as the ground truth.

Improved Precision-Recall: The Full-Name Embedding approach yielded the highest gains across all racial categories, particularly for Asian and Hispanic voters who were previously poorly served by standard BISG.
Reduction in Socioeconomic Bias: The full-name embedding significantly reduced the Brier score (prediction error) across the income distribution, effectively neutralizing the income-correlated bias identified in previous literature.
Superior Calibration: The full-name model provided the best calibration, meaning its predicted probabilities most closely matched the actual observed frequencies of race.
Generalization: Even when trained on different Southern states, the full-name model showed significant predictive improvements in the evaluation states.

5. Significance

This research is highly significant for the social, political, and health sciences. Accurate racial imputation is essential for measuring disparities in lending, healthcare access, and political representation. By providing a way to accurately estimate the race of the "missing 10%"—who are often from immigrant-origin communities—eBISG ensures that policy research and legal enforcement (such as fair lending audits) are more equitable and less prone to systematic error.