Methods, Data, and Conceptual Change: Reflections from Two Quantitative Diachronic Case Studies

Technical Summary: Methods, Data, and Conceptual Change

Problem Statement
The paper addresses a critical methodological gap in diachronic linguistics: the assumption that quantitative analysis can directly recover "inherent" linguistic change without considering how dataset properties shape observable patterns. The authors argue that diachronic change is not merely a property of language to be extracted, but is co-constituted by the interaction between the structural properties of historical datasets (e.g., orthographic variation, genre heterogeneity, syntactic regularity) and the analytical methods employed. Consequently, different methods applied to different datasets render different dimensions of semantic change visible, while obscuring others. The paper seeks to clarify the limits of purely lexical, frequency-based approaches and demonstrate how dataset structure conditions the reliability of detecting specific types of semantic change.

Methodology
The paper employs a comparative, example-based approach using two distinct quantitative methods applied to two different historical English corpora. Rather than proposing a new algorithm, the authors use these "worked examples" to illustrate method-data interactions.

1. Case Study 1: Quad-based Concept Modelling (EEBO-TCP)

   • Dataset: The Early English Books Online Text Creation Partnership (EEBO-TCP) corpus (c. 1470s–1690s; ~765M words). This dataset is characterized by high heterogeneity in genre, register, and substantial orthographic variation.
   • Method: Quad-based concept modelling (from the Linguistic DNA project). This approach identifies sets of four strongly co-occurring lemmas ("quads") within a moving window (typically 100 words).
   • Preprocessing: The pipeline relies on text normalization and lemmatization to handle spelling variations typical of early modern print.
   • Operationalization: Concepts are modeled as "constellations"—aggregated networks of quads around focal lemmas. Meaning is defined by recurrent patterns of lexical association across diverse textual environments rather than fine-grained syntactic relations.

2. Case Study 2: SynFlow Analysis (Royal Society Corpus)

   • Dataset: A subset of the Royal Society Corpus (RSC) 6.0.4, specifically scientific writing from 1750–1799 (~351k tokens). This dataset represents a more coherent genre (scientific prose) with relatively consistent orthography and syntactic structure.
   • Method: SynFlow, a dependency-based analysis tool.
   • Preprocessing: The corpus was re-parsed using Stanza (a state-of-the-art parser for Modern English) to generate dependency trees, despite known issues with OCR noise and historical spelling in the source text.
   • Operationalization: Concepts are represented by the distribution of lexical fillers in specific dependency slots (e.g., adjectival modification, subjecthood, objecthood). Meaning is operationalized through syntactic co-occurrence patterns and functional roles within structured linguistic environments.

Key Results
The comparative analysis yields specific findings regarding how method and data interact:

• EEBO-TCP Findings: In the heterogeneous EEBO-TCP corpus, quad-based modelling successfully stabilized conceptual structures despite orthographic noise. The analysis revealed that near-synonyms like liberty and freedom, while lexically interchangeable, possess distinct "constellations" of association. Liberty clustered around institutional and legal discourse, while freedom clustered around moral and spiritual contexts. This demonstrates that conceptual divergence is observable through aggregated discursive patterns rather than individual word frequency.
• RSC Findings: In the Royal Society Corpus, SynFlow successfully captured the semantic reorganization of concepts like air, water, and acid during the Chemical Revolution. Specifically, the concept of air shifted from a general substance to a chemically differentiated set of "airs," evidenced by a shift in the adjectival slot from general terms to specific chemical modifiers.
• Robustness of Data: Despite the presence of OCR noise and historical spelling variations in the RSC, the use of an off-the-shelf parser (Stanza) yielded reliable results for high-frequency items and main dependency relations. However, the authors note that this reliability is contingent on the linguistic proximity of the 18th-century text to Modern English; the approach may not be reliable for earlier periods with greater linguistic distance.

Key Contributions
The paper makes three primary methodological contributions:

1. Method-Data Interdependence: It establishes that quantitative analysis is not method-neutral. The "observable" change is shaped by how datasets encode context and how methods operationalize structure. Aggregation-based methods suit heterogeneous, noisy data, while dependency-based methods suit regular, syntactically consistent data.
2. Operationalization of Meaning: The paper contrasts two distinct operationalizations of "concept": one as a distributed network of lexical associations (quad-based) and the other as a structured profile of syntactic relations (SynFlow). It argues that these approaches reveal different, complementary dimensions of semantic change.
3. Limits of Frequency-Based Approaches: The study highlights that purely lexical, frequency-based approaches may obscure underlying conceptual organization. More complex patterns of association (quads) or relational structures (dependencies) are often required to detect nuanced conceptual shifts.

Significance and Claims
The authors modestly position this work as a "methodological reflection" rather than a proposal for a new universal method or a full empirical evaluation. The significance of the paper lies in its call for greater methodological transparency and context-sensitivity in diachronic research.

The paper argues that researchers must explicitly acknowledge the assumptions underlying data preprocessing, annotation, and analytical modeling. It contends that interpretations of historical change must be understood in relation to the specific interaction between the chosen method and the dataset's affordances and limitations. By placing different analytical strategies in parallel, the paper advocates for a flexible approach where methods are evaluated based on their fit with the specific datasets they interrogate, rather than seeking a single "optimal" method for all historical linguistic inquiries.