Household Movement Detection in Mixed-Format Occupancy Data Using LLM-Based Entity Resolution

Imagine you are trying to solve a giant, messy puzzle where the pieces are people's lives, but the picture is blurry. In the world of data science, there is a task called "Entity Resolution." Think of it like a super-powered detective trying to figure out if "John Smith at 123 Main St" and "J. Smith at 123 Main Street" are actually the same person. Usually, detectives just look at two pieces at a time, comparing them side-by-side to see if they match. But what if the puzzle pieces are torn, written in different languages, or have typos? What if the real clue isn't that two pieces look alike, but that they moved together?

This is where the story gets interesting. Sometimes, people don't just move alone; they move as a group, like a family packing up a truck to a new city. If you only look at one person's name and address, you might miss the fact that they are part of a household that just relocated. This paper dives into that specific mystery: how do we find these "household movements" when the data is a chaotic mix of handwritten notes, computer glitches, and different formats? The authors suggest that by using a special kind of artificial intelligence (AI) that can read messy text and then looking at the "social network" of who moved with whom, we can solve puzzles that old-school methods simply can't crack.

---

The Mystery of the Moving Household

Imagine you are a detective trying to track a family that just moved across town. You have a stack of old, crumpled notes left on a desk. Some notes say "The Smiths moved to 5th Ave," others say "J. Smith & G. Smith at 5th," and a third just scribbles "5th Ave, Apt 4." The handwriting is bad, the spelling is wild (one says "St," another says "Street," and a third says "Strt"), and some notes are missing dates.

If you try to match these notes two-by-two, you might get stuck. You might think "J. Smith" and "G. Smith" are strangers because their names don't look exactly alike, or because the addresses are written so differently. You miss the big picture: they are a team, and they moved together.

This is the problem the researchers at the University of Arkansas–Little Rock are tackling. They call it Household Movement Detection. In the real world, data about where people live is often a mess. It's "mixed-format," meaning some records are neat lists, while others are just a jumble of words in a single box. There are typos, missing pieces, and duplicate entries. Traditional computer programs, which are like rigid robots, try to match records by comparing them one-on-one. If the spelling is off by a letter, the robot says, "No match!" and throws the clue away.

But the authors realized that households move as a unit. If two people were at "2623 Fiddlestick Circle" last month and are now both at "860105 Post Office Box" in a different city, that's a huge clue that they are related, even if their names are spelled slightly differently or their addresses look nothing alike. The challenge is finding these "indirect links" in a sea of noisy data without getting confused by false alarms.

The New Detective Team: AI, Embeddings, and Graphs

To solve this, the team built a new framework that acts like a smart, three-step detective squad. They didn't just rely on one tool; they combined three powerful techniques to handle the messiness of real-world data.

Step 1: The AI Reader (LLM-Based NER)
First, they needed to make sense of the messy notes. They used a Large Language Model (LLM), which is a type of AI trained to understand human language. Imagine this AI as a super-reading assistant that can look at a jumbled sentence like "Terrie D Zlopez 2623 Fiddlestick Cir Lutz FL" and instantly say, "Okay, the name is Terrie D Zlopez, and the address is 2623 Fiddlestick Cir." Unlike older tools that get confused by weird formatting, this AI can handle the chaos, extracting the names and addresses even if they are mashed together or spelled weirdly.

Step 2: The Semantic Map (Embeddings)
Next, they turned these names and addresses into "semantic embeddings." Think of this as translating every name and address into a secret code (a list of numbers) that represents its meaning rather than just its spelling. In this code, "Fiddlestick Circle" and "Fiddlestick Cir" might be very close to each other, even though they look different. This allows the system to realize that two addresses are the same place, even if one has a typo or an abbreviation.

Step 3: The Social Network (Graph Reasoning)
Finally, they built a "graph." Imagine a giant web where every person is a dot, and lines connect dots that are similar. But here's the magic: instead of just looking at two dots, the system looks at the whole web. It asks, "Did a group of people move from one cluster of dots to another?" If two people who were at Address A are now both at Address B, the system connects the dots between Address A and Address B. This is "indirect linkage." It doesn't just say "Person X matches Person Y"; it says "The group that included Person X and Person Y moved together."

The Test Drive: Synthetic Data and Real Results

To see if this new detective squad actually works, the researchers didn't use real people's private data (which would be a privacy nightmare). Instead, they used a "Synthetic Occupancy Generator" (SOG). Think of this as a video game simulator that creates fake families, fake addresses, and fake moves, but adds in all the real-world messiness: typos, missing info, and duplicate records. They created datasets labeled S8 through S12, with S12 being the messiest and most difficult.

They ran their new framework against an old-school method called the "Data Washing Machine" (DWM), which just compares records two-by-two.

The results were promising. In these simulations, the new framework found 8% to 15% more correct connections (a metric called "recall") than the old method. It didn't just find more matches; it found the right ones. While the old method missed the household moves because the data was too messy, the new AI-powered team caught them. The overall score (called the F1-score, which balances finding everything with being accurate) went up by 6% to 8%.

For example, in one test case, the old method saw two people at different addresses and thought they were strangers. The new framework saw that they had moved together from a previous address, recognized them as a household, and linked them correctly. It even managed to link records where one person's name was spelled "Jorge" and the other "George," or where the address had a typo like "Huston" instead of "Houston," because the "group move" evidence was too strong to ignore.

Why This Matters (And What It Isn't)

This paper shows that looking at people as part of a group is a powerful way to solve data puzzles. By using AI to read messy text and graph logic to see group movements, we can recover information that was previously lost in the noise.

However, the authors are careful to note what this isn't. They didn't prove this works on every possible dataset in the real world; they tested it on simulated data that mimics real life. They also point out that if a household only has one person moving, or if a name changes completely (like a surname change after marriage), the system might still struggle. It's not a magic wand that fixes everything instantly.

But for the messy, mixed-format data that governments and companies deal with every day, this approach suggests a new way forward. Instead of trying to clean up every single typo before starting, we can let the AI handle the mess and look for the patterns of movement. It's a shift from "matching two pieces" to "understanding the whole picture," and in the world of data, that's a pretty big leap.