Technical Summary: On Defining Erasure Harms for NLP

Problem Statement

The deployment of Natural Language Processing (NLP) systems has raised significant concerns regarding representational harms, including stereotyping and toxic speech. While "erasure" has been recognized as a specific type of representational harm, the field currently lacks a clear and cohesive conceptual foundation for identifying and measuring it. Existing conceptualizations of erasure suffer from two primary limitations:

1. Over-broadness: Definitions such as "lack of adequate representation" are too vague to guide practitioners in establishing what is necessary to measure erasure.
2. Over-specificity: Some definitions are systematized for particular settings (e.g., image tagging or non-binary gender identities), facilitating measurement in those specific contexts but making adaptation to other settings difficult.

Furthermore, because constructs like "erasure" are unobservable, their presence must be inferred from observable system outputs. Without a cohesive conceptual foundation, existing measurement instruments (metrics, benchmarks) are increasingly questioned for their validity, and results are difficult to interpret across different cultural or deployment contexts.

Methodology

The authors employ a methodology grounded in measurement theory from the social sciences, specifically drawing on the work of Adcock and Collier (2001) and Wallach et al. (2025). The process involves distinguishing between a background construct (the broad constellation of meanings associated with a concept) and a systematized construct (a narrower, context-dependent reformulation).

The authors' approach consists of three main phases:

1. Surfacing Components: A purposive sample of literature on erasure was reviewed across NLP, machine learning, media communication, linguistic anthropology, sociology, philosophy, and law. This review identified recurring components in how erasure is conceptualized and noted missing or implicit elements.
2. Grounding in NLP: Collaborative discussion sessions were held to ground these components in specific NLP tasks (e.g., text summarization). The authors deliberated on whether erasure could be established in specific system outputs, considering task constraints (e.g., a summarization system should not be blamed for omitting information not present in the source).
3. Synthesizing and Connecting: The identified components were grouped by function and connected based on their interactions to form a structured definition. This definition was then applied as an analytical framework to examine nine existing NLP papers that explicitly conceptualize or operationalize erasure, surfacing trends and gaps in current literature.

Key Contributions

The paper proposes a structured definition of erasure designed to facilitate the process of systematization—the movement from a broad background construct to a specific, operationalizable one. The definition explicitly identifies six determinant components that practitioners must specify to establish whether erasure has occurred in a given context:

1. Target of Erasure: Who or what is believed to be erased (e.g., specific identities, language varieties, topics, or histories).
2. Object of Analysis: Where erasure is observed (e.g., system outputs, classification labels, or generated text).
3. Background Conditions: The social, historical, or past patterns deemed undesirable to reproduce. These form the normative basis for the analysis (e.g., historical patterns of media minimizing police violence).
4. Base Knowledge: The knowledge or information expected to be reflected in the object of analysis (e.g., the content of the source document in summarization tasks).
5. Alternative: A counterfactual or "better" version of the object of analysis that contributes less to the reproduction of background conditions while still reflecting the base knowledge.
6. De-emphasis: The mechanism by which erasure is observed in the object of analysis compared to the alternative (e.g., removal of information, obfuscation via passive voice, or incorrect addition of information).

The authors define erasure formally as: "a de-emphasis of a target—where this de-emphasis is occurring in an object of analysis that is intended to reflect some base knowledge—with respect to an alternative that reproduces less a set of background conditions than the object of analysis does."

Results and Analysis

By applying this definition to nine existing NLP papers, the authors identified several patterns and gaps in current conceptualizations:

• Contextual Grounding: Some existing works are appropriately grounded in specific contexts (e.g., non-binary identities or cultural erasure), but uncritically adapting these to new settings without re-specifying the six components can threaten validity.
• Distributional Reliance: Many existing operationalizations rely on observing systematic patterns or differences (distributional views) rather than single instances. This often occurs when background conditions are underspecified, forcing practitioners to infer patterns from system outputs.
• Underspecification of Background Conditions: While many works acknowledge that erasure reinforces power structures or historical asymmetries, they often fail to specify the exact entities or patterns involved. This makes it difficult to justify why a specific operationalization (e.g., matching population statistics) is appropriate.
• Lack of Explicit Alternatives: Existing work rarely explicitly articulates "better" alternatives. Practitioners often have to infer what a preferable system behavior would look like, which hinders the development of precise metrics and mitigation strategies.
• Base Knowledge Constraints: In tasks with unconstrained base knowledge (e.g., persona generation), it is difficult to establish if a generated set is representative. The authors suggest future work could focus on tasks with more constrained base knowledge (e.g., machine translation) to better isolate erasure.

Significance and Claims

The paper claims that its primary contribution is providing a foundation for practitioners to systematize the construct of erasure. By clarifying the necessary components (Target, Object, Background Conditions, Base Knowledge, Alternative, De-emphasis), the definition enables practitioners to:

• Explicitly articulate and operationalize erasure in their specific use contexts and normative concerns.
• Develop valid measurement instruments (metrics, benchmarks) that are grounded in clear conceptualizations.
• Map and analyze current conceptualizations of erasure to identify trends and gaps.

The authors emphasize that this work does not claim to solve erasure or provide a single universal metric. Instead, it offers a framework to facilitate the process of systematization, allowing for the creation of context-specific definitions that are rigorous enough to support valid measurement and meaningful interpretation of results. The paper also notes limitations, acknowledging that erasure can occur in system design and deployment processes, not just in outputs, and that the proposed definition currently focuses on the latter.