Who gets to verify AI? Provenance, authority and accountability in digital government
This paper argues that effective AI-content governance requires moving beyond isolated labeling or provenance tools by adopting a holistic framework that distinguishes between public legibility, infrastructural verifiability, and institutional answerability across the content lifecycle to ensure genuine accountability rather than mere symbolic compliance.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the modern world, government agencies increasingly rely on artificial intelligence to draft official notices, translate documents, create images, and even power the chatbots that answer citizen questions. This technology offers speed and efficiency, but it brings a unique danger: it makes it possible to create highly convincing fake versions of official information. A synthetic image of a public official or a fabricated emergency alert can spread faster than the truth, leaving people unsure of what is real and what is manufactured. To combat this, regulators and technology companies have turned to a simple idea: labeling. The logic is that if we can clearly mark content as "made by AI," people will know to be cautious. However, this approach assumes that a simple warning sign is enough to restore trust, ignoring the complex web of rules, software, and human decisions required to make such a system actually work.
A new study by Shuai Wang and colleagues investigates whether these labeling systems truly hold governments and technology companies accountable, or if they merely create the appearance of safety. The researchers examined eleven major regulatory documents and technical standards from China, the European Union, and international organizations. They looked at everything from mandatory labeling laws to the technical specifications for digital "watermarks" that track where a file came from. Their analysis reveals that transparency is not a single switch that can be flipped. Instead, they found that a working system requires three distinct layers that must function together: making the content understandable to the public, making the origin of the content verifiable by machines, and ensuring that institutions can be held responsible when things go wrong.
The researchers discovered that simply attaching a visible label to a piece of content often shifts the burden of verification onto the average citizen. If a user sees a warning that an image is AI-generated but has no way to check the source or understand what the warning actually means, the label becomes a form of liability management for the organization rather than a tool for public safety. The study shows that a label without a way to verify it is like a receipt without a store address; it tells you a transaction happened, but it offers no proof of where it came from. Furthermore, the technical systems designed to track the history of a file—known as provenance—often concentrate power in the hands of a few large technology firms and standards bodies. These systems can verify that a file came from a specific server, but they cannot verify if the information inside the file is true. A document can be perfectly tracked from its creation to its publication while still containing false or misleading information.
Perhaps the most critical finding is that even when all the technical pieces are in place, accountability often fails because of how the system is organized. The researchers identified six common ways these systems break down. One major failure occurs when the rules for different types of content are too broad, treating a harmless creative image the same as a critical emergency alert, which wastes resources and leaves dangerous information under-protected. Another failure happens when the visible label and the hidden technical data are not linked, allowing an organization to display a warning while the underlying evidence is lost or inaccessible. The study also highlights that when a file is edited or shared across different platforms, the digital trail often breaks, leaving no way to trace the content back to its source. In these scenarios, responsibility becomes fragmented; the software provider blames the platform, the platform blames the user, and the regulator lacks the power or technical ability to piece the story back together.
The authors argue that for transparency to be meaningful, it must be treated as a piece of public infrastructure rather than a simple technical fix. This means that labeling rules must be connected to accessible verification tools that allow citizens to check the source of information. It also requires that institutions have the authority and the capacity to access the records needed to assign blame and enforce corrections when misinformation spreads. The study suggests that a successful system would separate the question of "who made this?" from the question of "is this true?", ensuring that a verified source does not automatically mean the content is accurate. Ultimately, the research concludes that more disclosure does not automatically lead to more accountability. Without connecting the visible labels to verifiable evidence and ensuring that institutions can act on that evidence, the current wave of AI labeling risks becoming a form of "trust theater," where the signs of safety are visible, but the ability to actually protect the public remains out of reach.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.