Cooperate And Compete: Archives Can Gain More By Expecting Less From AI Training
This paper argues that archival organizations should overcome their hesitation to share collections for AI training by reframing the loss of item-level provenance as a competitive advantage and adopting strategic policies—such as tailored agreements, RAG technology, and advocacy for public-good contributions—to mitigate risks while fostering collaboration.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the author. For technical accuracy, refer to the original paper. Read full disclaimer
In the quiet corners of the world's libraries and archives, millions of unique documents, photographs, and recordings sit waiting to be seen. These are not just old papers; they are the raw materials of human memory, often stored in boxes or on fragile film, unseen by the public eye. Recently, companies building artificial intelligence have begun asking for access to these hidden collections. They want to feed this historical data into their computer models to make them smarter and more knowledgeable. The goal is to teach the machines how humans have thought, written, and created over centuries. However, the people who guard these archives are hesitant. They worry that if they give away their materials, the machines will learn the facts but forget where those facts came from. They fear losing control over their own history, unable to stop the machines from using the information in ways they never intended. This tension sits at the heart of a new debate: should archives say no to protect their past, or say yes to shape their future?
A recent study by Jinfang Niu, a researcher at the University of South Florida, suggests that archives might be looking at this problem the wrong way. The paper argues that archives are too focused on demanding things from artificial intelligence that the technology simply cannot provide, such as a perfect list of every single document used to create a specific answer. The author suggests that by lowering these expectations, archives can actually gain more power and protection. Instead of trying to force the machines to remember every detail of their training, archives should accept that the machines will not be able to trace their answers back to specific pages. In exchange for this acceptance, archives can negotiate better deals, keep their most valuable tools for themselves, and ensure they are not held responsible if the machines misuse the information.
The core of the issue lies in how artificial intelligence learns. When a human reads a book, they remember the story and the ideas, but they do not keep a mental map of every sentence they read to form a thought. Similarly, when an AI model is trained, it absorbs vast amounts of information into its internal structure, known as weights. These weights are not a filing cabinet of documents; they are a complex web of mathematical connections. Once the learning is done, the model cannot point to a specific page in a specific archive and say, "I learned this fact from here." The information is blended together. The paper notes that expecting an AI to provide a citation for every single piece of data it used to generate an answer is like asking a person to recall the exact sequence of memories that led them to write a poem. It is not just difficult; for current technology, it is practically impossible.
Because of this technical reality, the author argues that archives should stop insisting on "item-level provenance," which is the requirement to link every AI output to a specific record. Instead, archives should realize that this inability to trace the source is actually their greatest advantage. If an AI cannot perfectly replicate or cite a specific document, then the original archive remains the only place where that document can be found in its true, unaltered form. The machine might know the facts, but it cannot replace the need for a human to see the original evidence. This means that archives do not lose their value by sharing their data; they become even more essential as the trusted source of truth.
The paper also challenges the fear that archives will lose control over their knowledge once it is inside a machine. The author points out that archives have never truly controlled what happens to information once it leaves their building. When a researcher reads a document and then writes a book or gives a lecture based on what they learned, the archive cannot stop them. The knowledge has entered the human mind, and the archive has no power to make the researcher forget it. The author suggests that AI models are similar. Once the information is absorbed, it is part of the system's knowledge. Trying to force a company to delete specific facts from a trained model is incredibly expensive and technically difficult, often requiring the model to be rebuilt from scratch. Instead of fighting a losing battle to control the machine's memory, archives should focus on controlling the terms of the deal.
To make this work, the paper proposes a new set of strategies for archives to follow. First, they should tailor their agreements based on the legal status of the materials. For example, if the archive owns the copyright, they can charge for the use of the data. If the material is in the public domain, the archive can ask for funding to help digitize the collection in exchange for letting the AI use it. If the material is restricted due to privacy or donor agreements, it should simply be kept out of the training data. Second, archives should use a technology called Retrieval-Augmented Generation, or RAG. This is a method where the AI does not rely on its internal memory for facts but instead looks them up in a database in real-time. This allows the AI to give an answer and immediately show the user the specific document it found, solving the problem of missing citations without needing to change how the AI learns.
Finally, the author suggests that archives should stop trying to block AI companies from the internet and start negotiating with them. Archives can ask for money to help with digitization, better access to AI tools for their own staff, or legal protection. The paper recommends that archives include a clause in their agreements stating that if the AI company or its users misuse the information, the archive is not responsible. This shifts the legal risk to the company, protecting the archive from lawsuits if the AI generates harmful content. The author also calls for government policies that encourage AI companies to contribute to the public good, ensuring that the wealth generated by these technologies helps support the cultural institutions that provide the data in the first place.
The study concludes that archives do not need to be afraid of artificial intelligence. By accepting that machines cannot perfectly trace their sources, archives can stop wasting energy on impossible demands. Instead, they can focus on what they do best: preserving the original records and providing the context that machines cannot. By negotiating smart agreements and using new technologies to link answers back to sources, archives can turn a potential threat into a partnership that secures their future while helping the world learn from its past. The path forward is not to say no, but to say yes on terms that protect the integrity of the archive and the value of the history it holds.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.