Who Designs the Designer? Behavioural Architecture for GenAI in Education

Technical Summary: Who Designs the Designer? Behavioural Architecture for GenAI in Education

Problem Statement
The paper identifies that the current discourse on AI in education is "stuck" between two ineffective paradigms: the "AI-proofing" approach (banning tools and relying on unreliable detectors) and the "AI-as-tutor" approach (content-adaptive systems). Both fail because they ignore decades of research establishing that personality, motivation, and emotional state shape learning outcomes as strongly as cognitive ability. Current systems infer these traits from opaque behavioral traces (e.g., click patterns, response times) without student consent or understanding, leading to unreliable profiling and a lack of student agency. Furthermore, the "designer role"—the entity that defines what the system treats as true about a student—is currently occupied solely by AI vendors, creating a power asymmetry where institutions assess students using systems they do not govern.

Methodology and Proposed Architecture
The paper does not present empirical results but proposes a theoretical framework called Behavioural Architecture. This architecture redefines the design of AI-in-education systems through three core mechanisms:

1. Co-authored Learning Records: Instead of inferring student traits from hidden data traces, the system adapts based on a record co-authored by the student and the system.
   • Student Agency: Students can name patterns in their own learning, accept or reject system proposals, and read/revise the record at any time.
   • Explicit Adoption: No pattern becomes a rule without explicit adoption (either by the student or the system asking for confirmation).
   • Longitudinal Infrastructure: Unlike course-level interventions, this is an institutional infrastructure where the student's understanding is built early and refined continuously over years, similar to a school-issued laptop.
2. Redistribution of the Designer Role: The role of "designer" (defining the student's representation) is distributed among three actors:
   • The Educator: Sets the pedagogical frame, learning objectives, and baseline pathways.
   • The Student: Co-authors the representation of their own learning patterns and motivations.
   • The AI: Executes within the constraints set by the educator and the student.
3. Ethical Constraints: The architecture explicitly forbids the system from drawing psychological conclusions from learning patterns. It distinguishes between learning observations (e.g., "you write better after a break") and psychological claims, preventing the system from acting as an unlicensed clinical tool within an institution that holds power over the student's academic future.

Key Contributions
The paper's primary contribution is naming a vacancy: the "designer role" in AI-in-education is currently unoccupied. The authors argue that occupying this role requires infrastructure that does not yet exist. The paper outlines:

• A Shift in Adaptation: Moving from content adaptation (adjusting material based on hidden traces) to behavioural adaptation (adjusting to personality, motivation, and emotion surfaced through conversation).
• Meta-literacy as a Core Objective: Identifying that co-authorship is a skill not currently taught. The paper argues that "meta-literacy" (the ability to evaluate, correct, and shape information in adaptive systems) must be a core learning objective to prevent a digital divide where only privileged students can effectively participate.
• The "Solaria Risk": The paper identifies a new risk where highly effective private learning environments could dismantle the collective capacity of education, isolating students (analogous to Asimov's Solaria). Current metrics, which measure individual gains, cannot detect this erosion of shared learning.

Governance Requirements
The paper argues that individual institutions cannot provide the necessary structural protections because they are both the operators of the system and the assessors of the students. Therefore, EU-level governance is proposed as the necessary solution, based on three mechanisms:

1. Closing the GDPR Gap: Extending data portability (Article 20) to include inferred and derived behavioural profiles, preventing vendor lock-in.
2. Independent Certification: Extending the EU AI Act to require independent certification that behavioural architectures are pedagogically sound and psychologically safe.
3. Interoperability Standards: Applying the precedent of the European Health Data Space to ensure students can move between systems without losing their longitudinal records.

Results and Claims
The paper does not present empirical results or experimental data. It is a conceptual proposal.

• Claims: The paper claims that current systems fail because they ignore non-cognitive factors and lack student agency. It asserts that the proposed architecture is the only way to ethically deploy behavioural adaptation in education.
• Modesty: The authors are modest about the current state of the field, acknowledging that the research base to determine what makes such an architecture "pedagogically safe" does not yet exist.
• Future Validation: The paper concludes by proposing five empirical questions to test the architecture's validity:
  1. Does it produce better learning outcomes than content-only adaptation?
  2. Does co-authorship genuinely reduce power asymmetry, or is it performative?
  3. Does the "Solaria effect" (loss of collaborative capacity) emerge?
  4. Can the system be designed to never cross the line from learning observations to psychological claims?
  5. Does meta-literacy training change student outcomes, indicating a need for mandatory training?

Significance
The significance of the paper lies in its structural critique of current AI-in-education trends. It argues that without a deliberate design that redistributes the "designer" role and establishes governance at a supranational level, the field risks defaulting to surveillance systems or isolating educational experiences. It frames the issue not as a technical challenge of better algorithms, but as a governance and architectural challenge of who defines the truth about the learner.