SoK: From Generation to Consumption of Privacy Documents in Software Systems
This Systematization of Knowledge (SoK) provides a unified, lifecycle-oriented software engineering perspective on privacy documents by systematically analyzing 290 papers to map their generation, analysis, and consumption, while identifying key trends, open opportunities, and future research directions including AI-centric challenges and LLM-based policy-code integration.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you just downloaded a new app to track your running routes or chat with a digital friend. Before you hit "start," a giant wall of text pops up: the Privacy Policy. It's the digital equivalent of a fine-print contract that tells you who gets to see your data, how they'll use it, and what they promise to protect. For decades, these documents have been the main way software companies talk to us about our privacy. But here's the catch: they are often so long, confusing, and written in "legalese" that most of us just scroll past them without reading a word. It's like trying to read a novel written in a language you don't speak, while standing on a moving train.
Recently, things have started to change. Instead of just long paragraphs, we're seeing shorter "privacy labels" (like nutrition facts for apps) and pop-up notices that appear exactly when you need them. Researchers in the fields of computer science, law, and psychology have been trying to figure out how to make these documents clearer, how to check if companies are actually telling the truth, and how to make sure the software code matches what the document says. It's a bit like being a detective who has to check if a restaurant's menu matches the food actually coming out of the kitchen. If the menu says "fresh fish" but the kitchen is serving frozen blocks, that's a problem. This is the world of privacy documents: a mix of legal promises, computer code, and human understanding.
The Paper's Big Picture
This paper is a massive "State of the Art" (SoK) review, which is basically a super-organized map of everything researchers have discovered about these privacy documents over the last 15 years. The authors, a team of experts from universities in the US and Germany, didn't just look at one thing; they gathered and analyzed 290 different research papers published between 2010 and 2025. They wanted to see the whole story, from how these documents are born (generated) to how we read them (consumed), and everything in between.
Think of a privacy document not as a static piece of paper, but as a living software component, like a character in a video game that needs to be created, updated, checked for bugs, and tested to make sure it's fun to play. The authors organized their findings around five big questions:
- What are we even talking about? (Definitions and Scope)
- How are they made? (Generation)
- How do we read and understand them? (Analysis)
- Are they telling the truth? (Consistency and Compliance)
- Are they easy to use? (Usability)
What They Found: The Good, The Bad, and The Confusing
1. The "Long Policy" Problem is Still King
The authors found that despite all the new fancy formats, the old-school, long, boring privacy policy is still the most studied thing by far. It appears in 227 of the papers they looked at. While researchers are starting to look at shorter "privacy labels" (mentioned in 28 papers) and other visual formats, the field is still heavily focused on those giant walls of text. Also, almost all the research is done in English and focuses on big markets like the US and Europe. There's a huge gap in understanding how these documents work in other languages or cultures.
2. Making the Documents: A Messy Workshop
When it comes to creating these documents, the research is surprisingly thin. Only 32 papers (about 11% of the total) looked at how these documents are actually generated. Most of the time, it's a messy process. Some tools try to read the computer code and automatically write a privacy policy, but they often miss the big picture or get the details wrong. Others just use templates where developers fill in the blanks, which can lead to vague or inaccurate statements. The authors suggest that we need better tools that help developers write accurate policies without needing to be legal experts, and that these tools should work better with teams, not just solo coders.
3. Reading the Fine Print: AI is Helping, But It's Not Perfect
This is where the most research happens (205 papers). Scientists are using Artificial Intelligence (AI) and Natural Language Processing (NLP) to read these policies and pull out the important bits. Early methods used strict rules (like a checklist), but now, advanced AI models are doing the heavy lifting. However, the authors warn that these AI tools can sometimes "hallucinate" (make things up) or get confused by tricky legal language. They also point out that we still need a lot of human experts to check the AI's work, especially because privacy laws are complex and often vague.
4. The "Say-Do" Gap: When Code and Text Don't Match
One of the most critical findings is that companies often say one thing in their privacy policy but do another in their actual software. The researchers found widespread inconsistencies. For example, a policy might say "we don't collect your location," but the app's code might be secretly asking for location permissions. This is especially common in mobile apps. The research shows that while we have tools to check mobile apps (because they are easier to analyze), we are struggling to check newer things like AI chatbots or desktop software. Also, most studies focus on European laws (GDPR), leaving other parts of the world under-explored.
5. Usability: Still Too Hard to Read
Finally, the authors looked at how easy these documents are for regular people to understand. The verdict? They are still too long, too vague, and too hard to read. Even with new formats like labels, the problem of "vagueness" is the most common complaint. The good news is that researchers are trying to make privacy info "just-in-time" (showing you the info exactly when you need it) and more visual. But the authors argue that many solutions are still designed for the computer system, not for the human user. We need tools that help developers write better policies and tools that help users actually understand them.
Where Do We Go From Here?
The paper doesn't just list problems; it points the way forward with four big ideas for the future:
- AI is Changing the Game: We need to figure out how to handle privacy for AI systems that are constantly learning and changing, not just static apps.
- Better Data: We need more diverse, up-to-date, and multilingual collections of privacy documents to train our AI tools properly.
- Connecting Code and Policy: We need to use AI to look at the code and the policy together, so we can catch lies or mistakes before the app even launches.
- Helping Everyone: We need to stop ignoring the developers. If the tools developers use to write privacy policies are bad, the policies they write for users will be bad too. We need to make privacy engineering easier for the people building the software.
In short, this paper is a call to action. It tells us that while we've made a lot of progress in understanding privacy documents, we still have a long way to go to make them accurate, easy to read, and trustworthy for everyone. The goal isn't just to have a document; it's to have a clear, honest conversation between the software and the person using it.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.