ProfileFoundry: A Synthetic Person-Object Substrate for Privacy, Memory, and Tool-Use Evaluation in LLM Agent
The paper introduces ProfileFoundry, a deterministic generator and fixed reference release of 100,000 synthetic, cross-field consistent "Person Objects" designed to enable responsible evaluation of foundation models in memory, privacy, and tool-use tasks while overcoming the limitations of real user data and inconsistent synthetic alternatives.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are a director trying to film a movie about a complex family drama, but you can't use real people. You can't use real actors' private photos, their actual addresses, or their real family trees because that would be a massive invasion of privacy. If you just make up random names and numbers, the story falls apart because the "brother" might live in a different country than the "sister," or the "husband" might have a job that doesn't match his age.
ProfileFoundry is like a high-tech, digital puppet factory that solves this problem.
Here is how it works, broken down into simple concepts:
1. The "Digital Puppet" (The Person Object)
Instead of just giving you a fake name and a fake address, ProfileFoundry builds a complete, connected digital person. Think of this as a "digital puppet" that comes with:
- A Snapshot: Who they are right now (job, address, age).
- A Family Tree: Who their parents, siblings, and spouse are.
- A Household: Who they live with.
- A History: A timeline of their life (when they moved, when they got married, when they changed jobs).
- A Connection Map: How they link to other "puppets" (e.g., "This person works at the same company as that person").
The magic is that everything is consistent. If the puppet says they are 25, they can't have a PhD (because that usually takes longer). If they are married, the system knows who their spouse is and ensures the spouse's age makes sense.
2. The "Master Blueprint" (The Generator)
The paper describes a deterministic generator. Think of this as a master architect who draws the blueprint before building the house.
- Step 1: The architect decides, "I will build a family of four: two parents and two adult children."
- Step 2: Based on that decision, the system fills in the details. The parents get a shared address; the children get linked to the parents.
- Step 3: The system writes their life stories backwards from their current age to ensure the timeline makes sense (e.g., they couldn't have moved to a new city before they were born).
This ensures that if you ask the system, "Show me all the people who live with their parents," it gives you a list where the math actually works.
3. The "Safe Sandbox" (Why We Need This)
Researchers studying AI (like Large Language Models) need to test things like:
- Memory: Can the AI remember a user's name after 100 conversations?
- Privacy: Can the AI accidentally reveal a fake person's address?
- Tools: Can the AI use a calendar app correctly for a specific person?
To test this, they need data. But they can't use real people's data (it's illegal and unethical). If they use random fake data, the tests fail because the data doesn't look like real life (e.g., the AI gets confused because the "father" is younger than the "son").
ProfileFoundry provides a safe sandbox of 100,000 fake people. It's like a training gym for AI where the "weights" (the data) are heavy and realistic, but no real human is ever at risk.
4. The "Audit Trail" (The Receipt)
One of the coolest parts of this paper is the accountability.
Usually, when you generate fake data, it's a "black box." You get the result, but you don't know how it was made. ProfileFoundry comes with a receipt for every single person.
- It tells you exactly which "seed" (random number starter) created them.
- It proves that the "brother" and "sister" were generated together, not separately.
- It checks to make sure no two fake people accidentally have the exact same birthday and name (a "collision").
- It uses special email addresses (like
name@profilefoundry.example) that are guaranteed to be fake and never belong to a real person.
What It Is NOT
The authors are very clear about what this tool is not:
- It is not a crystal ball that perfectly predicts how real humans behave. It's a simulation, not a census.
- It is not a privacy shield. You still can't use these fake names to call real people or trick banks.
- It is not a finished product for a specific game or movie. It is a raw material (a "substrate") that researchers can use to build their own tests.
The Bottom Line
ProfileFoundry is a reusable, auditable, and consistent set of fake people designed to help researchers test AI safely. It's like giving scientists a box of 100,000 perfectly crafted, interconnected mannequins so they can practice surgery (or in this case, AI testing) without ever needing to touch a real human being.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.