How libraries classified physics preprints before arXiv and set the stage for distinguishing insiders from outsiders

In this comment, historian and sociologist Phillip Roth examines the history of how libraries classified physics preprints prior to the establishment of arXiv, highlighting how these early systems helped distinguish between insiders and outsiders in the scientific community.

The Gist

Imagine the world of scientific research as a massive, chaotic library where new books (scientific papers) are being written every single day. Before the internet, if you wanted to know what was happening in a specific field like physics, you had to rely on a very slow, old-fashioned mail system.

This paper, written by Phillip Roth, tells the story of how a few clever librarians in the 1950s and 60s invented a system to organize this chaos. They didn't just sort books; they accidentally created a "VIP club" that separates the people who belong in the room (insiders) from everyone else (outsiders).

Here is the story broken down with some simple analogies:

1. The Problem: The "Lost in the Mail" Dilemma

Before the internet, scientists shared their latest discoveries by mailing handwritten notes to each other. It was like a private group chat. But if you were a scientist visiting a big lab (like CERN in Switzerland) for a few weeks, you might miss the mail coming to your home office. You'd be out of the loop, like someone who missed the group chat while on vacation.

The Librarian's Solution:
A librarian at CERN named Luisella realized this was a problem. She said, "Stop mailing these notes to people's homes. Send them to the library instead!"

• The Analogy: Imagine a town square where everyone used to whisper secrets in private backyards. The librarian built a central bulletin board in the town square and said, "Put your notes here, and everyone can read them." This turned private whispers into public news.

2. The Sorting System: The "Bouncer" at the Door

Soon, so many notes started arriving that the library was drowning in paper. They needed a way to sort them. They couldn't just pile them up; they needed categories.

• The Human Filter: They hired "scientific information officers" (people who knew both physics and library science). These people acted like bouncers at a club. They looked at every paper and decided: "Does this fit our club's vibe?"
• The Categories: They created simple labels like "Theoretical Physics" or "Machine Building."
• The Catch: These labels weren't neutral. They were designed to fit what the CERN scientists cared about. If your paper was about something CERN didn't care about, it might get lost in the shuffle or labeled in a way that made it hard to find.

3. The "Badge of Membership"

Here is where things get interesting. Once these library lists became famous, scientists realized that getting your paper on the list was a status symbol.

• The Analogy: Think of it like getting your song played on the radio. If you are on the playlist, you are a "real" artist. If you aren't, you are just playing in your garage.
• Scientists started racing to get their papers sorted and listed quickly. Being "categorized" by the library meant you were an insider. If you weren't on the list, you were effectively an outsider.

4. The Hidden Bias: "Mainstream" vs. "Weird"

The paper argues that these sorting systems weren't fair.

• The Keyword Game: At another lab (DESY in Germany), they used keywords to sort papers. If you wrote a paper on a popular topic, you got many keywords, making it easy to find. If you wrote about a weird, niche topic, you got few keywords, making you invisible.
• The "General" Trap: Even today, on the famous preprint website arXiv, there is a category called "General Physics" (gen-ph). The author points out that this is actually a junk drawer. It's where papers go if they don't fit anywhere else. The system treats these papers as "uninteresting" to the experts. It's a digital way of saying, "This doesn't belong to our club."

5. The Modern Twist: Robots as Bouncers

Today, we use computers and AI to sort these papers instead of human librarians. The website arXiv uses algorithms to guess which category a paper belongs in.

• The Illusion: The website says this is "empowering" and "transparent."
• The Reality: The author argues that the robot is just following rules created by humans long ago. When the robot moves a paper to the "General" category, it is still enforcing the same old boundary: "You are not an insider."

The Big Takeaway

The paper concludes that the way we organize science isn't just about "finding information." It's a social tool.

• The Metaphor: Classification systems are like fences. They look like helpful fences that keep the garden tidy, but they also decide who is allowed to walk through the gate and who is left standing outside.
• The "correct" category isn't about scientific truth; it's about who holds the power to decide what is important. By sorting papers into "good" categories and "bad" categories, the system quietly decides who gets to be a famous physicist and who gets ignored.

In short: The librarians of the past built the first "VIP list" for scientists. Today, our computers keep that list going, deciding who is an "insider" and who is an "outsider" based on how well their work fits into the boxes we've built for them.

Technical Summary

Based on the article "How libraries classified physics preprints before arXiv and set the stage for distinguishing insiders from outsiders" by Phillip H. Roth, here is a detailed technical summary:

Problem Statement

The paper addresses the challenge of managing the exponential growth of scientific literature, specifically the influx of "grey literature" (preprints) in the field of high-energy physics. The core problem is twofold:

1. Information Overload: As the volume of preprints increased, researchers and libraries faced difficulties in organizing, retrieving, and disseminating this information efficiently.
2. The Illusion of Neutrality: Classification systems are often perceived as technical, neutral tools for organizing knowledge. However, the author argues that these systems inherently encode political and social struggles, creating boundaries between "insiders" (those whose work aligns with dominant institutional interests) and "outsiders." The problem lies in understanding how these classification mechanisms evolved from pragmatic library solutions into gatekeeping tools that define membership in the scientific community.

Methodology

The paper employs a historical and sociological analysis of information science practices.

• Source Material: The author traces the history of preprint handling through archival records, library staff manuals (e.g., CERN's 1965 Staff Manual), and the evolution of specific information systems (CERN library, SLAC, DESY/HEPI).
• Comparative Analysis: The study compares early physical library systems (1950s–1980s) with modern digital platforms (arXiv) to identify continuities in classification logic.
• Critical Discourse Analysis: The author examines the language and policies used by librarians and physicists to categorize papers, analyzing how these categories were constructed and what social functions they served beyond mere retrieval.

Key Contributions

1. Historical Genealogy of Preprint Systems: The paper documents the transition of preprints from private, ad-hoc circulation among physicists to formalized, library-managed public exchange systems. It highlights the pivotal role of specific librarians (e.g., Luisella Goldschmidt-Clermont at CERN, Louise Addis at SLAC) in institutionalizing preprint collection.
2. The "Gatekeeper" Function of Libraries: It identifies how libraries evolved from passive repositories to active "gatekeepers." By selecting which preprints to catalog and how to categorize them, libraries (specifically CERN and DESY) began to curate the scientific discourse, effectively acting as content moderators.
3. Critique of Classification Neutrality: The paper contributes to the sociology of science by demonstrating that classification categories are not derived from an ideal theoretical system but are pragmatic solutions reflecting the specific research interests and trends of the hosting institutions (e.g., CERN's focus on high-energy physics).
4. Continuity to the Digital Age: It establishes a direct lineage between early library indexing practices and modern algorithmic classification on arXiv, arguing that current "user-driven" or "machine-learning" tools still enforce historical boundaries between insiders and outsiders.

Key Results and Findings

• Institutionalization of Preprints: In the late 1950s and 1960s, libraries at CERN, SLAC, and DESY (via the High-Energy Physics Index or HEPI) created the first formal preprint exchange systems. They shifted preprints from private networks to public, cataloged displays to ensure researchers stayed informed.
• Pragmatic Categorization: Early classification systems (e.g., CERN's "theoretical particle physics," "detectors") were designed for hierarchical sorting and manageability rather than scientific purity. These categories were heavily influenced by the research agenda of the host laboratory.
• Social Incentives and Exclusion: Inclusion in these preprint lists became a "badge of membership." Papers that did not fit the dominant keywords or institutional interests were marginalized. For instance, the DESY keyword system assigned more keywords to mainstream papers, increasing their visibility, while niche or "border" material received fewer keywords and less recognition.
• The "General Physics" (gen-ph) Trap: The paper analyzes the modern arXiv category "gen-ph," revealing it as a modern equivalent of historical exclusion. Papers placed here are deemed to have "no specific audience" and are effectively treated as less relevant, replicating the social sorting of the past.
• Arbitrariness in Classification: Whether performed by human "scientific information officers" in the 1960s or by machine learning algorithms today, the classification process remains inherently arbitrary regarding "border" material. The system naturalizes human visions of what constitutes "relevant" physics.

Significance

• Theoretical Impact: The paper challenges the view of classification systems as purely technical or neutral. It posits that they are social constructs that actively shape the scientific landscape by determining visibility and legitimacy.
• Policy Implications: It suggests that current efforts to improve preprint classification (such as arXiv's 2020 user-driven tools) must acknowledge the historical baggage of these systems. Simply automating the process does not remove the underlying bias of separating "insiders" from "outsiders."
• Understanding Scientific Communities: The research provides a framework for understanding how scientific communities define their boundaries. The ability to be "found" by the right audience is not just a matter of search engine optimization but a result of institutional practices that have evolved over decades.
• Historical Context for Modern Tools: It offers essential context for users and developers of modern preprint servers, illustrating that the struggle to categorize and filter scientific information is a long-standing human endeavor, not a new problem created by the digital age.