Opening the Black Box of Eviction Court with Document-Conditioned LLM Analysis
This paper introduces and validates a document-conditioned large language model pipeline that achieves 98.4% accuracy in extracting procedural and legal outcomes from over 195,000 eviction court documents, revealing that while statewide right-to-counsel programs exist, tenant access to legal representation remains low and heavily dependent on early procedural engagement.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are trying to solve a massive, messy puzzle, but the pieces are hidden inside thousands of locked boxes. In the world of law, these boxes are court files. For a long time, researchers trying to understand how the legal system works for regular people had to open these boxes one by one, reading every single page by hand. It was slow, expensive, and meant they could only look at a tiny handful of cases before their eyes crossed. But recently, a new kind of "super-reader" has arrived: Large Language Models, or LLMs. Think of an LLM as a tireless, incredibly fast robot librarian that can read a million pages in the time it takes you to make a sandwich. The big question scientists have been asking is: Can we trust this robot to read the fine print of legal documents accurately enough to tell us the truth about how the system works, or will it just start making up stories? This paper steps right into that debate, using the robot librarian to peek inside the "black box" of eviction court, where landlords and tenants fight over who gets to stay in a home.
The researchers behind this study decided to test if their robot librarian could do the heavy lifting of reading eviction court records in Pierce County, Washington. They didn't just ask the robot to guess; they built a special system where the robot had to look at the actual text of the documents (like summonses, complaints, and hearing notes) and then fill out a strict, pre-made checklist in a specific format. It was like giving the robot a very specific recipe to follow rather than letting it write a poem. They fed the system 195,050 documents from 8,503 eviction cases filed between 2022 and 2024. To see if the robot was telling the truth, they compared its work against a "gold standard" group of 300 cases that human researchers had carefully read and coded by hand.
The results were surprisingly good. The robot got it right 98.4% of the time when compared to the human experts. It was almost perfect at spotting if a landlord had a lawyer or if a hearing was held, and even when it had to do the tricky job of figuring out if a tenant was actually kicked out of their home, it was still right more than 94% of the time. This means the "black box" of the court is no longer a mystery; we can now see what happens inside at a scale that was previously impossible.
Once they trusted the robot's data, the researchers used it to uncover some fascinating patterns about how tenants navigate the legal system. They found that the path to getting a lawyer is full of "checkpoints." To get legal help, a tenant usually has to first write a response to the eviction notice and then show up to a hearing. If they miss these steps, they are far less likely to get a lawyer, even if one is available. The study showed that about 40% of tenants submitted a written response, and of those who did, nearly 80% got a hearing. But here is the twist: just because a tenant showed up to a hearing didn't always mean they had a strong case. In fact, in 2024, tenants who attended hearings were actually more likely to get an eviction judgment against them. The researchers suggest this might be because a local rule in the city of Tacoma forced more hearings to happen, including for tenants who hadn't even written a response, meaning many of these tenants were attending court without having a strong defense ready.
The paper also looked at how having a lawyer changes the outcome. In the early years of the study (2022), having a lawyer was a huge shield; it strongly reduced the chance of being evicted. But as time went on, that shield seemed to get a little weaker. By 2024, having a lawyer still helped, but the connection between "having a lawyer" and "winning the case" wasn't as tight as it used to be. The authors suggest this might be because the emergency money that used to help people pay their rent and stop evictions dried up during this time, making it harder for lawyers to save their clients even when they were in court.
One of the most important things this paper doesn't do is prove that a specific law caused a specific result. The researchers are careful to say they found "associations," not "causes." They can't say for sure that the new laws made tenants behave differently, only that the behavior changed around the same time the laws were passed. They also admit that while their robot is amazing, it's not perfect. It's great for studying thousands of cases to find big trends, but it shouldn't be used to decide the fate of a single person's home in a real courtroom.
In the end, this study is like handing researchers a pair of super-powered binoculars. Before, they could only see the people standing right in front of them in the courtroom. Now, they can see the entire crowd, spotting patterns like how hard it is for tenants to get a lawyer, how local rules change the game, and how the system is shifting over time. The main takeaway is that while the robot librarian is a powerful tool for understanding the big picture of justice, the system still has deep cracks. Tenants face huge barriers just to get a seat at the table, and having a lawyer doesn't guarantee a win, especially when the safety nets of financial aid disappear. The study suggests that if we want to fix the system, we need to stop guessing and start using these big-data tools to find exactly where the bottlenecks are, so we can help the people who need it most.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.