Technical Summary: Meaning Is Not A Metric

Problem Statement

The paper identifies a fundamental limitation in current sociotechnical systems (e.g., social media platforms, healthcare, education, and governance): their inability to represent "human meaning" at scale. These systems rely on thin descriptions—standardized numerical metrics (e.g., engagement rates, GDP, test scores, click-through rates) that enforce uniformity. While effective for processing large datasets, thin descriptions strip away the cultural context and heterogeneity necessary to encode what actually matters to human experience.

The authors argue that the failure to support meaningful human experience is not merely a problem of selecting better metrics, but a failure of representational format. Traditional metrics reduce complex, context-dependent human activities to quantifiable proxies, leading to "proxy failure" where the metric diverges from the underlying value it is meant to represent (e.g., prioritizing "likes" over genuine social connection). The core challenge is how to make human meaning legible (perceivable and processable) within computational systems without sacrificing the contextual richness that defines it.

Theoretical Framework and Methodology

The paper proposes a shift from thin to thick description as the primary representational strategy. Drawing on Clifford Geertz's anthropological framework, thick description is defined as a verbal representation that interprets symbols and behaviors within their specific cultural context, uncovering layers of significance rather than merely recording observable facts.

• Thick vs. Thin: Thin descriptions account for facts and observable behavior without context. Thick descriptions account for context, interpretation, subjective perspectives, and the symbolic dimensions of actions.
• The Role of LLMs: The central methodological proposal is the use of Large Language Models (LLMs) to automate the generation and processing of thick descriptions at scale. Historically, thick description required human experts (anthropologists, literary critics) to interpret cultural context, creating a bottleneck for scalability. The authors posit that LLMs, with their verbal capabilities, can act as "cultural critics at scale," partially automating the generation of these rich, contextual representations.
• Approach: The paper does not present a specific experimental dataset or a single implemented system. Instead, it offers a position paper arguing for a new direction in AI application: developing technical approaches for domain-agnostic qualitative coding. This involves using LLMs to generate bespoke coding schemes for specific contexts in real-time, rather than relying on pre-defined, static quantitative variables.

Key Contributions

The paper makes several conceptual and strategic contributions to the fields of AI, sociology, and the humanities:

1. Reframing the Metric Problem: It argues that "meaning is not a metric." The solution to representing human meaning is not to find a better numerical proxy but to adopt a new representational format based on thick description.
2. Legibility of Meaning: It applies James C. Scott's concept of "legibility" to AI, arguing that current systems make only thin data legible. The paper proposes using LLMs to render cultural context and human meaning legible within sociotechnical systems.
3. Five Key Challenges: The authors articulate five specific challenges that must be addressed to successfully scale thick description:
   • Context Preservation: Meaning is inherently context-dependent; systems must retain information about the specific cultural and situational context of an event, not just proxy variables like country or language.
   • Interpretive Pluralism: There is no single "source of truth" for meaning. Systems must be able to represent multiple, conflicting interpretations of the same symbol or event rather than collapsing them into a single variable.
   • Integration of Perspectives: Robust representation requires integrating experience-near perspectives (lived, subjective experience) with experience-distant perspectives (analytical, scholarly frameworks).
   • Content vs. Magnitude: Systems must distinguish between the content of meaning (qualitative "what is it?") and the magnitude of meaning (quantitative "how much?"), avoiding the reduction of one to the other.
   • Dynamic Meaning-Making: Meaning is not static; it is a dynamic process of negotiation. Systems must adapt to shifting interpretive frames rather than treating cultural meaning as fixed.
4. Domain-Agnostic Qualitative Coding: The paper proposes a concrete technical direction: using LLMs to generate bespoke qualitative coding schemes for specific datasets in real-time, moving beyond the static coding used in traditional social science.

Results and Claims

As a position paper, the authors do not report empirical results from a specific experiment. Instead, the "results" are theoretical arguments and a roadmap for future research:

• Feasibility Argument: The paper argues that LLMs are empirically capable of producing thick descriptions that approximate human-level interpretations, citing recent work on LLMs identifying "dog whistles" (coded language) and word-sense disambiguation as preliminary evidence of their ability to handle interpretive pluralism.
• Limitations of Current Approaches: The authors highlight that current attempts to optimize for "meaning" (e.g., changing social media algorithms to prioritize "meaningful" interactions) often fail because they substitute one thin metric for another.
• Scalability: The paper claims that the bottleneck of human analysis for thick description can be overcome by LLMs, allowing these rich representations to be deployed in large-scale systems.

Significance and Scope

The paper positions itself as a crucial intervention in the application of generative AI. Its significance lies in:

• Moving Beyond Optimization: It challenges the prevailing paradigm of optimizing sociotechnical systems solely for efficiency or engagement metrics, proposing instead that systems should be designed to support "what really matters" to human experience.
• Unifying Framework: It offers thick description as a unifying framework for addressing diverse issues in AI, including the representation of culture, the limitations of generative models, and the need for "full-stack alignment" of AI values.
• Political and Technical Necessity: The authors argue that while the political will to use AI for human well-being exists, the technical capability to represent human meaning is currently missing. This paper aims to provide the technical foundation (thick description via LLMs) necessary to support future political and social solutions.

The paper concludes with a modest call to action: to begin developing technical approaches for domain-agnostic qualitative coding at scale. It acknowledges significant risks, including the potential for bad actors to use these tools for surveillance ("thoughtcrime") or for the systems themselves to distort the cultural phenomena they measure (reflexivity). However, it maintains that developing these capabilities is a necessary prerequisite for any society wishing to use technology to enhance, rather than diminish, human meaning.