💻 computer science

AI to Learn 2.0: A Deliverable-Oriented Governance Framework and Maturity Rubric for Opaque AI in Learning-Intensive Domains

This paper proposes "AI to Learn 2.0," a deliverable-oriented governance framework and maturity rubric designed to address the "proxy failure" of generative AI in learning-intensive domains by ensuring that final outputs remain usable, auditable, and justifiable without reliance on the original AI models, thereby preserving human capability and accountability.

Original authors: Seine A. Shintani

Published 2026-04-23

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Seine A. Shintani

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Polished Fake"

Imagine you are a chef teaching a student how to cook. The student hands you a beautiful, perfectly plated steak. It looks amazing. But, you realize the student didn't cook it; they just ordered it from a high-end restaurant, wrapped it in a fancy box, and handed it to you.

The steak is real, but the skill is fake.

This is the core problem the paper addresses. Generative AI (like the chatbots we use today) can write essays, write code, and solve math problems that look perfect. But in schools and training programs, the goal isn't just to get a "good result"; it's to prove that you understand the material. If AI does the heavy lifting, the "good result" no longer proves you learned anything. This is called "Proxy Failure." The artifact (the essay/code) is a bad proxy for the human's actual ability.

The Solution: "AI to Learn 2.0"

The author, Seine Shintani, proposes a new rulebook called AI to Learn 2.0.

Think of this not as a ban on AI, but as a "Distillation Process."
Imagine you are making a soup. You can use a high-tech, magical blender (the AI) to chop all the vegetables, mix the spices, and simmer the broth. That's great for the cooking process.

However, when you serve the soup to your guests (the final deliverable), you cannot say, "Here is the soup, but you have to keep the magical blender plugged in to keep it warm, and you can't taste it unless the blender is running."

AI to Learn 2.0 says: You can use the magical blender to make the soup, but the final bowl you serve must be self-sustaining. It must be delicious, safe, and understandable without the blender being plugged in.

The Two Main Rules (The "Residuals")

To pass this new rulebook, a project must satisfy two conditions:

1. The Artifact Residual (The "Standalone Soup")
The final product (the essay, the code, the report) must work on its own.

Bad: "Here is my code, but if you run it, you need to call my private AI API to make it work."
Good: "Here is my code. It runs on any computer, and I can explain every line of it without needing the AI."

2. The Capability Residual (The "Chef's Knowledge")
In learning situations (like school or job training), the human must prove they still know how to do the work.

Bad: The student hands in the AI-written essay and says, "I can't explain this paragraph, the AI wrote it."
Good: The student hands in the essay, but then passes a quick oral test where they explain why they chose those arguments. The AI helped draft it, but the student owns the logic.

The Toolkit: The "Deliverable Package"

You can't just hand in the final essay. To prove you followed the rules, you must submit a 5-Part Package (like a safety manual that comes with a machine):

The Distilled Artifact: The final, clean work (no AI dependencies).
The Audit Trail: A receipt showing what AI tools were used, what data was fed into them, and what the human changed.
The "Where It Works" Map: A clear statement of what the work is good for and what it is not good for (e.g., "This code works for small data, but will crash on huge data").
The Emergency Brake: Instructions on what to do if things go wrong (e.g., "If the numbers look weird, stop and ask a human").
The Resource Note: A list of what is needed to run it (e.g., "This runs on a normal laptop, no supercomputer needed").

The Scorecard: The "Maturity Rubric"

The paper introduces a scoring system (0 to 4) to grade these projects. It's like a Driver's License Test for AI workflows.

There are 7 dimensions to score, but 4 of them are "Gatekeepers." If you fail a Gatekeeper, you fail the whole test, no matter how good you are at the other parts.

Gatekeeper 1 (Residual Opacity): Can you use the final product without the AI?
Gatekeeper 2 (Human Sovereignty): Did a human actually say "Yes, this is good" and take responsibility?
Gatekeeper 3 (Validity/Failure): Do you know where the work might break?
Gatekeeper 4 (Privacy): Did you accidentally feed secret data to the AI?

If you pass the gates and have a high score, you get a "License to Operate."

Real-World Examples from the Paper

The paper tests this idea on different scenarios:

The "Fake" Student (Fail): A student submits an AI-written report with no explanation. Result: Fails. The "soup" requires the blender to exist.
The "Smart" Student (Pass): A student uses AI to draft a report but then defends it in an oral exam. Result: Passes the "Capability" test, but might still fail the "Audit" test if they didn't document the process.
The "Math" Example (Pass): A researcher uses AI to find a complex math formula. The formula is written down on paper. Result: Passes! The formula works without the AI. But, if they don't say where the formula stops working, they fail the "Validity" gate.
The "Teacher" Example (Pass): A teacher uses AI to generate 100 practice quiz questions for students. The teacher reviews them, fixes the errors, and puts them on a static website. Result: Passes! The AI helped build the infrastructure, but the final tool is safe, auditable, and doesn't need the AI to run.

The Bottom Line

AI to Learn 2.0 is a shift in mindset.

Old Way: "Is AI cheating?" (Yes/No).
New Way: "What is left behind after the AI is turned off?"

It allows us to use powerful AI tools to speed up work, but it demands that we leave behind a clean, understandable, and human-owned product. It ensures that when we say "I learned this," we actually mean it.

1. Problem Statement

The paper addresses the "proxy failure" problem in learning-intensive domains (e.g., education, research training, professional certification).

The Core Issue: Generative AI (GenAI) can produce polished, high-quality artifacts (reports, code, analyses) that no longer serve as credible evidence of human understanding, judgment, or transfer ability. A submission may be technically correct but educationally "empty" if the human cannot explain or defend it without the AI.
Limitations of Current Frameworks: Existing governance models (e.g., NIST AI RMF, UNESCO guidelines) focus on organizational policy, model explainability, or process design. They lack a formal mechanism to judge the final deliverable itself to ensure it remains usable, auditable, and valid for human learning or certification after opaque AI use.
The Gap: There is no cross-domain formalism to distinguish between a workflow that merely substitutes human work with AI (creating a proxy failure) and one where AI supports learning while preserving human capability evidence.

2. Methodology

The paper proposes AI to Learn 2.0, a governance framework that shifts the unit of analysis from the AI model to the final deliverable package. It introduces a formal definition, a structured deliverable package, and a maturity rubric.

A. Formal Definitions

The framework defines an AI-assisted workflow $w$ and establishes two critical "Residual Requirements":

Artifact Residual ($ArtRes$): The final deliverable must be usable, auditable, transferable, and justifiable without the original opaque AI model or cloud API. The dependency on the "black box" must be eliminated in routine use.
Capability Residual ($CapRes$): In learning-intensive contexts, the human user must retain sufficient explanatory or transfer capability to serve as evidence of learning. This is distinct from the artifact; it is the human's ability to defend the work.

B. The Deliverable Package ( $D$ )

To satisfy the artifact residual, the framework mandates a five-element package for every AI-assisted output:

Distilled Artifact ( $A$ ): The black-box-free output (e.g., code, equation, checklist) intended for routine use.
Provenance/Audit Record ( $P$ ): A trace of AI tools used, data exposed, human modifications, and verification steps.
Validity-Domain Statement ( $V$ ): Explicit boundaries of where the artifact applies, assumptions made, and exclusions.
Failure/Escalation Rule ( $F$ ): Defined failure modes, warning signs, and triggers for human review.
Operational Resource Note ( $R$ ): Dependencies (e.g., cloud vs. local, compute costs) and runtime requirements.

C. The Maturity Rubric

A 7-dimensional scoring system ( $m_1$ to $m_7$ ) on a 0–4 scale evaluates the workflow's maturity:

Gate Dimensions (Critical Thresholds):
- $m_1$ : Residual opacity (Is the final deliverable black-box-free?)
- $m_2$ : Human sovereignty (Is there accountable human release authority?)
- $m_4$ : Validity-domain and failure awareness (Are bounds and failure rules defined?)
- $m_5$ : Information protection (Was sensitive data protected?)
Non-Gate Dimensions: Auditability ( $m_3$ ), Resource proportionality ( $m_6$ ), and Third-party handoff readiness ( $m_7$ ).
Companion Capability-Evidence Ladder ( $C$ ): A separate 0–3 scale for learning-intensive contexts to assess human evidence (e.g., oral defense, no-AI checkpoints).

Compliance Logic: Full compliance requires passing the Gate (all gate dimensions $\ge$ 3), satisfying Core conditions (Artifact Residual + Human Sovereignty + Info Protection + Package Completeness), and meeting the Capability Residual threshold based on the context's stakes.

3. Key Contributions

Deliverable-Oriented Governance: Moves beyond "AI in the loop" or "model explainability" to focus on the state of the final artifact after AI use.
Dual Residual Requirement: Explicitly separates Artifact Residual (the tool must work without the AI) from Capability Residual (the human must understand the tool).
Operational Instrument: Provides a concrete 5-part deliverable package and a 7-dimension maturity rubric designed for structured third-party review, moving from abstract principles to actionable checks.
Teacher-Side AI / Learner-Side Evidence Principle: Proposes that AI can be used extensively for infrastructure (item design, feedback generation) provided that the evidence used for mastery claims remains attributable to the human learner via non-delegable checkpoints.

4. Results (Worked Cases)

The framework was applied to seven contrastive cases to demonstrate its ability to distinguish between "substitution" and "supported learning":

C1 (Chatbot Report): Failed. High opacity, no audit, no human capability evidence. (Score: 6/28, Gate Failed).
C2 (Report + Oral Defense): Partial improvement. Oral defense provided capability evidence, but the package lacked auditability and validity bounds. (Score: 17/28, Gate Failed).
C3 (Symbolic Regression, No Bounds): Failed. The equation was readable (low opacity), but lacked validity domain/failure rules. Proved that interpretability $\neq$ governance sufficiency.
C4 (Symbolic Regression + Bounds): Compliant. Added validity statements and failure rules. (Score: 27/28, Gate Passed).
C5 (Teacher-Audited Practice Forms): Compliant. AI used for infrastructure (scripts, structuring), but the final tool is static and teacher-audited. No capability residual required for the tool itself.
C6 (Course with C5 + No-AI Checkpoints): Compliant. Combined the infrastructure (C5) with human-only mastery checkpoints. Satisfied both artifact and capability residuals.
C7 (Self-Hosted L2Q Pipeline): Compliant. Local LLM generation with deterministic quality control and black-box-free deployment.

Key Finding: High technical scores (e.g., readable code) do not guarantee compliance if gate dimensions (validity, sovereignty, info protection) are missing. Conversely, opaque AI use during development is acceptable if the final deliverable is distilled and bounded.

5. Significance

Bridging Governance and Learning: Provides a practical bridge between high-level AI governance (NIST, UNESCO) and the specific needs of educational assessment and research training.
Preventing "False Mastery": Offers a mechanism to detect and prevent scenarios where polished AI outputs mask a lack of human understanding, which is critical for certification and credentialing.
Enabling Safe AI Adoption: Rather than banning AI, it provides a pathway for its legitimate use by defining where AI can remain (exploration, drafting) and what must be removed (runtime dependency, unexamined assumptions).
Scalable Review: The rubric allows institutions, reviewers, and third parties to systematically evaluate AI-assisted workflows without needing access to the original prompts or the AI model's internal weights.

In conclusion, AI to Learn 2.0 reframes the governance challenge: it is not about whether AI was used, but what remains after its use. It ensures that the final deliverable is a trustworthy, bounded, and human-attributable artifact.