Evaluating Epistemic Guardrails in AI Reading Assistants: A Behavioral Audit of a Minimal Prototype

1. Problem Statement

The paper addresses a critical gap in the evaluation of Large Language Model (LLM) reading assistants. Current evaluation frameworks primarily focus on output quality (accuracy, safety, instruction following) or learning outcomes (tutoring efficacy). However, in knowledge-intensive settings, the central risk is not factual error but interpretive displacement: the subtle transfer of meaning-making work from the human reader to the AI system.

When an AI acts as a "co-reader," it risks compressing the intellectual work of reading (judgment, orientation, inference) into a polished summary or answer. This undermines the user's epistemic role. The paper argues that existing safety guardrails (preventing harmful content) do not address epistemic guardrails, which are constraints designed to preserve the user's responsibility for interpretation. The core problem is how to evaluate whether an AI system participates in reading without displacing the reader's interpretive labor, particularly under multi-turn interaction pressure.

2. Methodology

The study employs a protocol-based behavioral audit using a minimal prototype named TextWalk.

• System Under Study: TextWalk is designed as a "co-reader" rather than an answer-provider. Its design commitments include:
  • Structure-first engagement: Orienting users to text structure before interpretation.
  • Text grounding: Anchoring explanations in the source passage.
  • Interpretive humility: Avoiding definitive conclusions on contestable texts.
  • Non-replacement posture: Resisting requests to bypass reading.
• Experimental Design:
  • Corpus: 12 analytical texts across four categories (research introductions, policy analyses, scholarly essays, public commentary).
  • Protocol: A fixed ten-prompt sequence applied to each text, organized into four escalating phases:
    • Phase A (Baseline): Standard reading support (structure, main idea).
    • Phase B (Interpretive Inquiry): Requests for claims, assumptions, and guiding questions.
    • Phase C (Boundary Stress): Explicit requests for shortcuts (e.g., "summarize so I don't have to read").
    • Phase D (Escalation Pressure): Direct attempts to transfer interpretive labor (e.g., "give me the answer").
• Evaluation Framework:
  • Responses were scored using a rubric across five dimensions: Structure-first engagement, Mode governance, Text grounding, Interpretive humility, and Boundary enforcement.
  • Scoring was categorical: Protocol Compliance (high alignment), Partial Compliance (middle zone of drift), or Guardrail Violation (substitution).
  • Secondary Calibration: To test evaluative stability, the dataset was re-scored by two other LLMs (Claude and Gemini) to measure inter-evaluator divergence on the concept of "preserved reader engagement."

3. Key Results

The study revealed that guardrail stability is not binary but dynamic, varying significantly across interaction phases.

• Baseline Stability (Phase A): The system showed high stability and consistent adherence to its "co-reader" role when prompts were standard.
• Interpretive Strain (Phase B): The first significant instability occurred here. As prompts moved from structure to interpretation, the system began to over-resolve ambiguity. It remained pedagogical but shifted from guiding the reader to supplying the interpretation, leading to a high rate of "Partial Compliance."
• Boundary Stress & Oscillation (Phase C):
  • C1 (Explicit Shortcut): The system recovered, successfully refusing direct replacement requests.
  • C2 (Simplification): Stability wavered again. The system began compressing complex texts into "plain-language gists," a form of simplification drift that functionally replaced reading without explicitly violating refusal rules.
• Late-Stage Stabilization (Phase D): Surprisingly, under explicit escalation pressure (Phase D), the system tightened its boundaries and returned to uniform compliance. The most severe failures were not at the end of the protocol but in the middle (Phase B and C2).
• Failure Pattern Typology: The study identified four specific "middle-zone" failure patterns where support slides into substitution:
  1. Interpretive Drift: Supplying the interpretive product too quickly.
  2. Grounded Overresolution: Resolving contestable tensions into settled conclusions.
  3. Simplification Drift: Compressing text into a gist that removes the need for effort.
  4. Boundary Softening: Formally refusing replacement while leaking enough content to functionally substitute for reading.
• Evaluator Divergence: The secondary scoring revealed significant disagreement between evaluators (Claude vs. Gemini) specifically in the "middle zone." While they agreed on text grounding, they diverged on interpretive humility and boundary enforcement, highlighting that "preserving the reader's role" is an unstable construct depending on the evaluator's operationalization.

4. Key Contributions

1. Conceptual Reframing: Introduces Epistemic Guardrails as a distinct category from safety guardrails, focusing on the preservation of the user's intellectual agency rather than just output safety.
2. Methodological Innovation: Proposes a behavioral audit protocol that evaluates AI systems as interactional phenomena across a gradient of pressure, rather than as static output generators.
3. Empirical Model: Develops a behavioral model of Interpretive Boundary Function, demonstrating that guardrails are emergent properties of interaction (shaped by mode governance, prompt pressure, and interactional drift) rather than fixed constraints.
4. Identification of the "Middle Zone": Highlights that the most consequential risks are not overt failures but subtle patterns of over-helpfulness where the system remains grounded but displaces the reader's cognitive labor.

5. Significance

This paper shifts the paradigm for evaluating AI in educational and knowledge-intensive contexts. It argues that:

• Process > Output: Evaluating AI reading assistants requires analyzing the sequence of interaction to detect drift, not just the quality of a single response.
• Resilience is Graded: Systems may appear to "refuse" shortcuts but still fail by over-simplifying or over-resolving, a nuance missed by standard binary compliance checks.
• Design Implications: Future AI reading assistants must be designed not just to restrict outputs, but to actively manage the distribution of interpretive labor, ensuring the system acts as a scaffold rather than a substitute.
• Evaluation Challenge: The study underscores that evaluating epistemic boundaries is inherently difficult because the line between "support" and "substitution" is often ambiguous and dependent on the evaluator's perspective.

In conclusion, the paper demonstrates that while AI systems can be stable under baseline conditions, they struggle to maintain their epistemic role during complex interpretive tasks, often slipping into "interpretive displacement" in ways that are subtle, patterned, and difficult to detect without structured, multi-turn behavioral audits.