Marginal Alignment Does Not Guarantee Joint-Distribution Fidelity: An Official-Reference Audit of Nemotron-Personas-Korea with Cross-Locale Replication
This paper introduces the Independence-Assumption Footprint (IAF) audit protocol to demonstrate that while the NVIDIA Nemotron-Personas-Korea dataset aligns with official marginal demographics, it fails to preserve critical joint-distribution structures across attributes like occupation, age, and gender, thereby establishing that marginal alignment alone is insufficient for ensuring synthetic data fidelity.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are trying to bake a perfect replica of a bustling city using a super-smart robot chef. You give the robot a list of rules: "Make sure 50% of the people are men, 50% are women," "Make sure 20% are doctors," and "Make sure 10% are farmers." The robot follows these instructions perfectly. It creates a million fake people, and if you count them up, the numbers match the real city exactly. But here is the catch: the robot was told to pick gender and job independently. It didn't know that in the real world, certain jobs are mostly held by men, while others are mostly held by women. So, while the robot has the right total number of doctors and the right total number of women, it might accidentally make half the doctors women and half the farmers women, creating a city that looks right from a distance but falls apart when you look at the details. This is the core puzzle of "synthetic data"—fake information created by AI to help researchers test software or study society without needing real people. The big question is: just because the fake data looks right on the big numbers, does it actually capture the messy, interconnected reality of how people really live?
This paper, titled "Marginal Alignment Does Not Guarantee Joint-Distribution Fidelity," dives into that exact problem. The author, Joonhyung Bae, investigates a massive dataset called "Nemotron-Personas-Korea," which contains one million fake Korean profiles created by NVIDIA. The dataset's creators claimed it was trustworthy because its big numbers (marginals) matched official government statistics. However, the author argues that matching the big numbers isn't enough; the fake people need to match the combinations of traits (joints) that exist in real life. To test this, the author invented a new audit tool called the "Independence-Assumption Footprint" (IAF). Think of IAF as a detective's magnifying glass that checks if the AI accidentally broke the rules of real life by treating things as separate when they are actually linked.
The investigation reveals that while the fake dataset got the simple counts right—like the total number of men, women, and people in different cities—it failed spectacularly at the complex combinations. For instance, in the real Korean workforce, certain jobs like "plant operators" are overwhelmingly male, and "service workers" are overwhelmingly female. But in the fake dataset, the AI smoothed these differences out, making the gender split in these jobs look almost equal, as if the AI was trying to be "fair" but ended up erasing reality. The paper also found that the fake data got the military service rules completely wrong. In reality, young Korean men are required to serve in the military, creating a huge spike in active-duty numbers for ages 19–22. The fake data, however, showed a flat, tiny percentage of men in the military across all ages, completely missing this crucial life stage.
The author also tested if these problems happened in other countries by looking at similar fake datasets for the USA, Japan, India, and others. The results were mixed: some places showed similar "flattening" of gender roles, while others looked different, suggesting that the errors depend on the specific country's data and rules. The paper concludes that while these fake datasets are useful for some things, like testing if a computer program can read text, they are dangerous to use if you want to understand real-world statistics, like how many women actually work in engineering or how military service affects a person's life. The author warns that if researchers use these fake numbers as if they were real census data, they will draw the wrong conclusions about society. The takeaway is clear: just because a fake city looks right from the sky doesn't mean the streets inside are real.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.