AccretionLink: On-Device Auditing of Exposure-Control Attacks on Attribute Inference

In the digital world, we often assume that privacy is broken only when someone steals our passwords, hacks our accounts, or tricks us into revealing secrets. But there is a quieter, more subtle way our private lives can be exposed, one that requires no theft and no lies. It happens when an observer is allowed to choose which of our public posts to look at and in what order. Imagine a person who wants to guess your political views or your income. Instead of reading every single thing you have ever written, they are given a list of your public posts and a special tool that lets them pick the ten most revealing ones to read first. Even if every single post they choose is real, authentic, and exactly what you wrote, the mere act of selecting the most interesting ones changes the story they can tell about you. This is the core of a new security concern: the power of exposure control. It is the ability to shape the evidence available to an observer without altering the evidence itself.

Researchers have now built a system to measure exactly how much damage this kind of selection can do. They created a controlled environment where they could test how well an artificial intelligence could guess private details about a person, such as their age, income, or relationship status, based solely on their public writing. The team set up a game where an attacker was allowed to pick posts from a pool of authentic content. In one scenario, the attacker had to read the posts in a random order, like flipping through a book page by page. In another, the attacker was allowed to pick the posts that seemed most likely to reveal the answer. The researchers found that this simple act of choosing made a significant difference. When the attacker was allowed to pick the best posts, their ability to guess correctly improved noticeably. The system they built, called AccretionLink, ran these tests on a specific experimental realization of a Pixel 10 device, ensuring that the entire process happened locally on the device without sending any private text to the internet.

The study used two types of test subjects. The first was a set of 52 completely fake profiles created by a computer. These profiles had known, sealed answers for their income, relationship status, education, and age. The researchers ran the selection game on these profiles and found that when the attacker was allowed to pick posts, they reduced their error rate. At the point where eight posts had been read, the attacker's confidence improved by a measurable amount compared to when they read randomly. More importantly, the selection process caused the system to make confident but wrong guesses in specific cases. Out of 109 opportunities to guess a private trait on these synthetic profiles, the selection method led to six instances where the system was highly confident but completely wrong about the person's income or education. This proved that exposure control could not only improve guesses but also manufacture false certainty, turning a correct profile into a confidently incorrect one.

To ensure these results were not just a trick of the computer code, the researchers also tested the system on real data from a public dataset of Twitter posts from 2015. They applied the same selection logic to these real profiles. Even without the ability to see the actual ranking system used by Twitter, the selection method still improved the attacker's ability to guess the user's age group. The improvement was smaller than in the fake profiles, but it was consistent and statistically clear. Crucially, the researchers showed that this effect did not depend on the attacker using the same model to pick the posts and to guess the answer. They used a different, simpler method to pick the posts, and the original model still guessed better. This confirmed that the advantage came from the act of selection itself, not from a specific flaw in one piece of software. Importantly, while the selection method improved the attacker's accuracy on real data, it did not cause any high-confidence false reversals among the 71 eligible real profiles tested.

The researchers also built a way to prove that the test was done fairly and that no data was tampered with. They ran the entire experiment on a specific experimental realization of a Pixel 10 device, using a specialized chip designed for artificial intelligence. The device processed every post exactly once, creating a digital fingerprint of the work done. It then signed this record with a secure key stored in the device's hardware, creating a permanent, unchangeable log of the test. This log proved that the results came from the specific model and the specific data used, with no hidden steps or outside interference. The study found that the time it took to process the text was dominated by the time it took to analyze the meaning of the words, not by the time it took to update the scores or check the results. This means the bottleneck in the system was the intelligence of the model itself, not the speed of the device's processor.

The findings suggest that the order in which we see information matters as much as the information itself. Even if every post we make is honest and unaltered, the ability to cherry-pick which ones are seen first can significantly change what an observer can learn about us. The researchers showed that this is not just a theoretical risk but a measurable reality that can be tested and recorded on a specific experimental realization of a smartphone. They demonstrated that a simple selection strategy could lead to confident, incorrect conclusions about a person's private life. While the study did not claim to solve this problem or to predict how social media companies currently rank posts, it provided a clear, reproducible method to audit how much harm exposure control can cause. The work highlights that in the age of artificial intelligence, the security of our privacy depends not just on what we say, but on who gets to choose which of our words are heard first.