Computational conceptual history of scientific concepts: From early digital methods to LLMs

Technical Summary: Computational Conceptual History of Scientific Concepts

Problem Statement
The paper addresses the methodological integration of Large Language Models (LLMs) into the computational conceptual history of scientific concepts within the History, Philosophy, and Sociology of Science (HPSS). While HPSS traditionally treats concepts as historically layered, socially situated, and epistemically significant rather than fixed semantic units, recent digital turns have sought to quantify these dynamics. The core problem is determining what LLMs add to existing computational methods (such as co-citation analysis, topic modeling, and static word embeddings) and how they inherit or reshape longstanding methodological challenges. Specifically, the authors investigate how LLMs affect four critical pillars of the field: corpus construction, concept operationalization, model and training data selection, and the linked challenges of evaluation and interpretation.

Methodology
The paper employs a two-part analytical framework to situate LLMs within the field's history:

1. Historical Reconstruction (Pre-LLM Era): The authors reconstruct the trajectory of computational conceptual history by synthesizing three strands of prior work:

   • Early Digital HPSS Methods: Including co-citation analysis (Small, 1973) and co-word analysis (Callon et al., 1983) which map conceptual structures through citation and keyword co-occurrence.
   • Digital Begriffsgeschichte: Approaches using distributional semantics (e.g., Word2Vec, GloVe) to trace semantic neighborhoods and polysemy, often highlighting the tension between lexical proxies and historical concepts.
   • Lexical Semantic Change Detection (LSCD): Computational linguistics techniques utilizing static embeddings to measure vector shifts across time slices, focusing on technical benchmarking and model evaluation.

2. LLM-Era Analysis: The authors analyze the current landscape by distinguishing between two dominant LLM architectures:

   • Encoder-based models (e.g., BERT): Used primarily for measurement-oriented workflows. These generate Contextualized Word Embeddings (CWEs) that represent individual token occurrences based on bidirectional context, allowing for the handling of polysemy through clustering and sense-specific analysis.
   • Decoder-based/Generative models (e.g., Llama, GPT): Used for text production, prompt-based extraction, and hybrid workflows. These assist in generating synthetic data for sparse historical domains, refining coding schemes, or providing interpretive summaries via Retrieval-Augmented Generation (RAG).

The methodology involves reviewing recent HPSS case studies (e.g., Malaterre & Lareau, 2026; Ahmadi, 2026; Zichert et al., 2025) that apply these models to scientific concepts, evaluating how they operationalize concepts and handle issues like domain adaptation and temporal bias.

Key Contributions

• Methodological Taxonomy: The paper provides a structured comparison of pre-LLM and LLM-era approaches, clarifying the shift from static, single-vector representations to dynamic, context-sensitive embeddings.
• Operationalization of Polysemy: It highlights how encoder-based LLMs allow for a more direct modeling of polysemy by treating word occurrences as unique vectors, enabling researchers to track shifts in dominant senses and sense distributions over time, rather than collapsing multiple meanings into a single average vector.
• Hybrid Workflows: The authors identify emerging hybrid methodologies where generative models are used not as standalone analyzers but as tools to generate synthetic training data, refine lexicons, or assist in the interpretation of encoder-based findings.
• Critical Assessment of Limitations: The paper details how LLMs inherit and amplify specific challenges, including the "word-concept gap" (where linguistic proxies cannot fully capture material-semiotic scientific concepts), the opacity of "black box" models, and the risks of projecting contemporary meanings onto historical texts (temporal bias).

Results and Findings

• Current State of HPSS Case Studies: Recent applications in HPSS predominantly rely on encoder-based models (CWEs) for measuring semantic change. Examples include analyzing epistemic markers in biomedicine, comparing semantic uniformity between sociology and astrophysics, and tracing the history of "virtual particles." Decoder-based models play a secondary, supportive role, primarily in data generation and prompt-based extraction.
• Corpus and Data Challenges: While LLMs are more robust to raw text and reduce the need for heavy preprocessing, they introduce new dependencies on pre-training datasets that may contain historical biases. The authors note that domain adaptation (further pre-training on specific scientific corpora) is now a standard and necessary step to capture disciplinary terminologies.
• Evaluation Difficulties: The paper finds that robust evaluation remains a significant hurdle. Due to the lack of ground truth in historical conceptual analysis and the opacity of LLMs, triangulation with other quantitative methods (frequency, co-occurrence, dependency parsing) and qualitative validation remains the default strategy.
• Interpretive Risks: While LLMs can support heuristic discovery and large-scale testing of qualitative claims, there is a risk that their outputs may subtly shift interpretive control away from the scholar, particularly when using commercial generative models with non-transparent updates.

Significance and Claims
The paper claims that LLMs do not replace earlier digital approaches but rather reconfigure and extend them. The significance of this work lies in its argument that LLMs offer richer, context-sensitive representations that better align with the HPSS understanding of concepts as fluid and situated. However, the authors maintain a modest stance, asserting that LLMs are not self-sufficient solutions. Instead, they function as one component in a multi-method toolbox.

The paper emphasizes that historically accountable conceptual analysis still depends on:

1. Triangulation: Combining LLMs with established quantitative techniques and qualitative validation.
2. Transparency: The need for open infrastructures, domain-specific models, and clear documentation to ensure reproducibility.
3. Future Directions: The authors point to the potential of multimodal LLMs to address the "material-semiotic" nature of scientific concepts (integrating text with diagrams and formulas), though they note this requires new corpora and alignment techniques that are currently in early development.

Ultimately, the paper positions LLMs as powerful tools that expand the scale and nuance of conceptual history but insists that the core methodological responsibilities of corpus construction, operationalization, and interpretation remain with the researcher.