Enabling Multilingual Privacy Policy Audits: Large-Scale Analysis of Spanish Mobile Apps
This paper demonstrates that large language models can effectively extend privacy policy audits beyond English by achieving high cross-lingual performance across EU languages, revealing that English-only audits systematically obscure transparency gaps and discrepancies between declared and observed data practices in multilingual app ecosystems like the Spanish Google Play Store.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine the internet as a giant, bustling library where every app is a book. Before you can read a book, you're supposed to check its "back cover" to see what secrets it might keep about you. In the digital world, this back cover is called a privacy policy. It's a legal promise from the app maker saying, "Here is exactly what data we will take from your phone and how we will use it." For years, researchers have been building robots to read these promises and check if the apps are telling the truth. But there's a catch: most of these robots only speak English.
This creates a problem for places like the European Union, where people speak 24 different official languages. If an app is written in Spanish, French, or Greek, but the robot only understands English, that app becomes invisible to the audit. It's like trying to check the safety of a whole city's buildings, but your inspector can only read the blueprints for the houses built in English. This paper asks a big question: Can we teach our robot inspectors to understand all these languages without needing a human expert to teach them every single one? And if we can, what hidden secrets do we find when we finally look at the apps that were previously ignored?
The Language-Breaking Robot
The researchers in this paper decided to build a "multilingual super-inspector" using a type of artificial intelligence called a Large Language Model (LLM). Think of an LLM as a super-smart student who has read almost everything on the internet. Unlike older robots that needed a specific textbook for every language (which is hard to write), this student can guess the meaning of a Spanish or Polish privacy policy just by looking at it, even if they were mostly trained on English.
To test if this student was actually smart enough to be a legal auditor, the team didn't just guess. They created a massive "practice exam." They took two famous, expert-written sets of privacy policies (originally in English) and used a high-quality translation tool to turn them into all 24 official languages of the EU. They then asked their AI student to read these translated policies and identify what kind of personal data the apps claimed to collect, like your location, your contacts, or your health info.
The results were surprisingly good. The AI student got the answers right about 91% to 94% of the time, no matter which language it was reading. It didn't matter if the policy was in Estonian or French; the robot understood the rules just as well as it did in English. This proved that we don't need a team of 24 different legal experts to audit every language; a smart AI can do the heavy lifting across the board.
The Great Spanish App Audit
With their new multilingual robot ready, the team went on a real-world mission. They downloaded and tested 2,611 Android apps from the Spanish Google Play Store. This was a huge deal because previous audits mostly looked at popular commercial apps, which often write their policies in English. The researchers wanted to see what happened when they included the "other" apps: government services, local town hall tools, and public transport apps, which usually write their policies in Spanish.
They didn't just read the policies; they also watched the apps in action. They ran the apps on test phones and recorded every single piece of data the apps tried to send out into the internet. Then, they compared the "promise" (the policy) with the "reality" (the actual data sent).
Here is what they found, and it's a bit of a plot twist:
1. The Language Barrier Hides the Truth
The audit revealed a clear divide. The popular commercial apps (like games and shopping tools) mostly wrote their privacy policies in English. The public-sector apps (like government services) wrote theirs in Spanish. Because most previous audits only spoke English, they were effectively blind to the public-sector apps. By speaking Spanish, the researchers could finally see what these government apps were doing.
2. The "Omission" Problem
When they compared the promises to the reality, they found that many apps were lying by omission—meaning they didn't say they were collecting data, but they were doing it anyway.
- The Commercial Apps: About 15.1% of the popular commercial apps had policies that didn't match their behavior.
- The Public Apps: The situation was much worse for government apps. Nearly 49.5% of them had policies that missed major data collection activities.
3. What Was Missing?
The most common secret was "device telemetry." This is technical jargon for the app quietly sending info about your phone's model, its software version, and its unique ID to the internet. The apps often didn't mention this in their policies.
- For the public-sector apps, 79.6% of the data flows they observed were not mentioned in their privacy policies.
- Even though these were government apps, they were often sending data to big tech companies (like Google) and servers in the US and UK, without clearly telling the user that this was happening.
Why This Matters
The paper suggests that the problem isn't just that some apps are bad actors; it's that the whole system of checking them is broken because of language. By only auditing English policies, regulators and researchers have been missing a huge chunk of the digital ecosystem, particularly the services that governments provide to their citizens.
The study shows that when you finally look at the Spanish-speaking apps, you see a different kind of transparency gap. It's not just about ads and tracking; it's about how even public services rely on complex, hidden connections to third-party tools that automatically send data without a clear explanation. The researchers conclude that to fix privacy, we need to stop treating language as a barrier. We need tools that can audit every language, because the secrets apps keep don't care about which language you speak. The "English-only" way of checking privacy is like trying to solve a puzzle while only looking at half the pieces.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.