Thinking-Visibility: An Interventionist Reconceptualisation of Assessment in the Age of AI Co-Production

Technical Summary: Thinking-Visibility: An Interventionist Reconceptualisation of Assessment in the Age of AI Co-Production

Problem Statement
The paper addresses a structural paradox in contemporary education: while educators increasingly integrate Generative AI (GenAI) into professional practice, assessment criteria remain designed for independent human authorship. Current literature focuses heavily on integrity policing, cheating detection, and validity reframing, yet fails to provide empirically grounded accounts of how assessment criteria transform when AI co-production is structurally integrated into student work. Existing solutions, such as process documentation, are shown to be vulnerable to AI generation themselves, leaving practitioners without actionable frameworks and policymakers without evidential grounding. The core problem is the inability of traditional criteria to capture whether students are genuinely appraising their own thinking or outsourcing that judgement to AI, creating a crisis of authenticity and validity.

Methodology
The study employs Cultural-Historical Activity Theory (CHAT) as the analytical lens and the Change Laboratory as an interventionist methodology. This approach treats systemic contradictions not as errors to be eliminated, but as drivers of transformation.

• Participants: 11 educators from nine countries (including the UK, New Zealand, Indonesia, and Saudi Arabia) across primary, secondary, and tertiary sectors. They were positioned as co-researchers rather than subjects.
• Intervention Design: A five-workshop intervention spanning three weeks, structured around Engeström's seven-stage expansive learning cycle (Questioning, Analysis, Modelling, Examining, Implementation, Reflection, Consolidation).
• Data Generation: "Mirror data" consisting of three marked student scripts with varying degrees of AI use was introduced in Workshop 1 to surface contradictions between professional values and existing criteria. Data sources included workshop transcripts, collaborative artefacts (redesigned criteria), chat logs, and pre/post-interview reflections.
• Analysis: The study traced the transformation of the "assessment object" (the purpose of the activity) across the intervention. Disturbances observed in the mirror data were mapped to contradictions (primary, secondary, tertiary, and quaternary) within the activity system, tracing a trajectory from initial questioning to a redefined object.

Key Results
The intervention revealed that assessment contradictions are not merely technical glitches but deep systemic tensions.

1. The Object Trajectory (O1–O7): The study empirically traced the assessment object's evolution through seven stages:
   • O1–O2: Initial attempts to create better criteria for detecting AI and calibrating to cognitive levels (Bloom's taxonomy).
   • O3–O4: A shift toward assessing process rigour, where a failed product resulting from a rigorous process is seen as achievement. However, participants discovered that process documentation itself could be AI-generated, leading to the reinstatement of teacher judgement as a necessary, albeit risky, epistemic tool.
   • O5–O7: The realization that criteria reform alone is insufficient. The object expanded to include dual-track assessment (formative process + summative competence) and ultimately recognized that full transformation requires institutional and political reform.
2. The Process-Evidence Paradox: The study found that process documentation is not a secure validity solution because it is susceptible to the same AI generation as the final product.
3. Displacement of Contradictions: While the intervention partially addressed key contradictions by shifting focus from product to process, most contradictions were displaced rather than resolved. Policy fragmentation and the lack of institutional support remained unchanged.
4. Thinking-Visibility: The study identified a need to make student thinking visible rather than just the product or the process steps. This was operationalized through mechanisms like oral interrogation, prompt annotation, and live accountability dialogues.

Key Contributions
The paper makes four specific contributions to the literature on AI-mediated assessment:

1. Empirical Operationalisation of Expansive Learning: It provides an empirical account of the O1–O7 object trajectory, demonstrating how practitioners' encounters with systemic tensions progressively reshape the assessment object, moving beyond theoretical identification of contradictions to tracing their developmental consequences.
2. Resolution of the Process-Evidence Paradox: It demonstrates that process documentation cannot guarantee assessment validity as it is vulnerable to AI generation. Consequently, teacher judgement is identified as an irreducible requirement—a non-substitutable mechanism for interpreting genuine cognition, despite its inherent risks of inconsistency and bias.
3. Institutional Necessity: It establishes that criteria reform cannot succeed without parallel institutional and political change. The study exposes the depth of institutional resistance and the fragmentation of policy (e.g., varying rules across exam boards) that renders isolated pedagogical changes ineffective.
4. Thinking-Visibility as a Theoretical Construct: The paper proposes "thinking-visibility" as a provisional construct to name what assessment must positively become in AI-mediated contexts. It shifts the focus from what assessment must stop being (policing) to what it must become (making student reasoning and ownership of AI outputs visible).

Significance and Claims
The paper claims that AI integration does not produce a simple linear improvement in assessment but creates a movement between partial stabilisation and regression depending on whether contradictions in criteria, authenticity, fairness, and policy are addressed collectively.

• Modest Scope: The authors acknowledge that the intervention did not fully resolve all contradictions; many were displaced. The study concludes not with a finished solution but at an "opening" for a second intervention.
• Theoretical Anchor: The significance lies in providing a theoretical anchor ("thinking-visibility") for reconceptualising the assessment object itself, moving beyond criterion design principles.
• Future Directions: The paper explicitly states that the definition of "thinking" in this context and the operationalisation of thinking-visibility through mechanisms more resistant to AI substitution (e.g., real-time presence vs. documentation) remain open questions requiring further empirical testing. It recommends a second intervention to address policy, teacher literacy, and Bloom's taxonomy calibration.