🤖 AI

PedaCo-Gen: Scaffolding Pedagogical Agency in Human-AI Collaborative Video Authoring

PedaCo-Gen is a pedagogically-informed human-AI collaborative system that utilizes an intermediate representation phase and Mayer's Cognitive Theory of Multimedia Learning to empower educators with metacognitive scaffolding, significantly enhancing the quality and instructional efficacy of generated videos compared to traditional one-shot generation methods.

Original authors: Injun Baek, Yearim Kim, Nojun Kwak

Published 2026-03-30

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Injun Baek, Yearim Kim, Nojun Kwak

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you want to build a house. In the past, if you asked a robot to build it, you'd just say, "Build me a house," and the robot would instantly spit out a finished structure. Sometimes it looks great, but often the rooms are the wrong size, the kitchen is in the attic, or the windows are too small. You can't really fix it once it's built; you'd have to tear it down and start over.

PedaCo-Gen is a new system that changes how we ask robots (AI) to build educational videos. Instead of a "one-shot" magic trick, it acts like a collaborative architect and a strict building inspector working together with a human teacher.

Here is the simple breakdown of how it works and why it matters:

1. The Problem: The "Black Box" Generator

Current AI video tools are like fast-food chefs. They are amazing at making things look tasty and pretty (high visual quality), but they don't always know nutrition. If you ask them to make a video about "How a volcano works," they might make a beautiful video with cool lava, but they might skip the important science steps or add confusing, distracting decorations.

Teachers need videos that actually help students learn, not just videos that look cool.

2. The Solution: The "Blueprint" Phase (Intermediate Representation)

PedaCo-Gen introduces a crucial middle step called the Intermediate Representation (IR). Think of this as the blueprint before the house is built.

Step 1: The Draft. You tell the AI what you want to teach. Instead of making a video immediately, the AI writes a script and a description of the scenes (the blueprint).
Step 2: The Inspector. This is the magic part. The system has a built-in "Expert Inspector" (based on a famous learning theory called Mayer's CTML). This inspector reads the blueprint and says:
- "Hey, you put too much text on the screen. Students can't read and listen at the same time. Let's remove the text."
- "You added a funny cartoon cat in the background. That's distracting! Let's take it out so students focus on the volcano."
- "You didn't explain the hard words first. Let's add a quick intro."
Step 3: The Human in the Loop. The teacher looks at these suggestions. They can say, "Yes, fix it," or "No, I like it this way." The teacher remains the boss (the "Pedagogical Gatekeeper").
Step 4: The Build. Once the blueprint is perfect, then the AI generates the actual video.

3. Why This is a Big Deal: The "Scaffold"

The paper calls this system a "Metacognitive Scaffold." That's a fancy way of saying it's like training wheels for a teacher's brain.

Without PedaCo-Gen: A teacher might make a video, realize it's confusing, and have to start over. It's frustrating and slow.
With PedaCo-Gen: The AI acts like a thoughtful co-pilot. It doesn't just do the work; it asks, "Are we sure this helps the student learn?" It helps the teacher remember the rules of good teaching while they work.

4. What the Study Found

The researchers tested this with 23 education experts. Here is what happened:

Better Quality: The videos made with PedaCo-Gen were significantly better at teaching concepts than videos made by standard AI. They followed the rules of how human brains learn best.
Faster Work: Even though there was an extra step (reviewing the blueprint), the teachers felt it was faster overall. Why? Because they didn't waste time fixing bad videos later. It was like "measure twice, cut once."
More Confidence: Teachers felt they were still in control. They weren't just pushing a button and hoping for the best; they were actively shaping the lesson.

The Takeaway

PedaCo-Gen proves that the future of AI in education isn't about letting the robot do everything. It's about partnership.

Think of it like this:

Old Way: You ask a genie for a video, and you get a random gift.
PedaCo-Gen: You and a genius chef (the AI) look at the recipe together. The chef says, "We need less salt," and you say, "Okay, let's try that." Then, you both cook the meal together.

The result is a video that is not only beautiful but actually teaches the student something new.

1. Problem Statement

While Text-to-Video (T2V) generative AI models are rapidly advancing, they currently suffer from a critical misalignment between visual fidelity and instructional efficacy.

The "Black Box" Issue: Existing T2V pipelines typically operate as "one-shot" generation systems where a prompt is directly converted into a video. This lacks granular control, making it difficult for educators to ensure the content adheres to established pedagogical principles.
Cognitive Overload: Without intermediate review, AI-generated videos often violate instructional design principles (e.g., Mayer's Cognitive Theory of Multimedia Learning), inadvertently increasing extraneous cognitive load and reducing learning outcomes.
Lack of Agency: Current tools treat educators as passive consumers of AI output rather than active co-creators, failing to leverage human professional expertise to refine instructional strategies.

2. Methodology

The authors propose PedaCo-Gen, a human-AI collaborative system designed to bridge the gap between generative AI capabilities and pedagogical soundness.

Core Theoretical Framework

The system is built upon Mayer's Cognitive Theory of Multimedia Learning (CTML), operationalizing its 12 principles (e.g., Coherence, Redundancy, Signaling, Pre-training) as structured constraints within the generative process.

System Architecture & Workflow

PedaCo-Gen introduces an Intermediate Representation (IR) phase, transforming the workflow from a linear "one-shot" process into a three-phase iterative loop:

Setup Phase:
- Users input learning content and configure pedagogical constraints (defaulting to CTML principles but allowing customization).
- An LLM generates an initial draft script based on these inputs.
Refinement Phase (The Core Innovation):
- AI Reviewer Module: An expert LLM acts as a "pedagogical gatekeeper," reviewing the draft script against CTML principles.
- Human-in-the-Loop: The system provides explainable feedback and a proposed revised script. Educators can accept all feedback, selectively apply changes, or manually edit the script.
- This phase creates a "metacognitive scaffold," prompting educators to critically evaluate instructional design rather than passively accepting AI output.
Output Phase:
- Once the script is finalized, the system generates visual descriptions and narrations scene-by-scene.
- A video synthesis model (Veo 3.1) renders the final video, which is previewed by the user.

Study Design

Participants: 23 education professionals (teachers, education majors, early childhood educators).
Task: Participants generated and refined instructional videos on three distinct topics (Galaxy Collision, Chaos Theory, DNS Operation) under two conditions:
1. Baseline: LLM-generated scripts refined manually by educators using CTML guidelines as a reference.
2. PedaCo-Gen: Scripts refined through the system's automated, principled human-AI collaborative workflow.
Metrics: Quantitative evaluation of 13 items (12 CTML principles + Overall Validity) on a 5-point Likert scale, plus system usability metrics (Production Efficiency, Guide Validity, Intention to Apply). Qualitative data was gathered via open-ended feedback.

3. Key Contributions

Pedagogical Agency via IR: The introduction of an Intermediate Representation (script/blueprint) phase allows educators to intervene before resource-intensive video synthesis, reclaiming control over the creative process.
Metacognitive Scaffolding: The system functions not just as an automation tool but as a "principled friction" mechanism. It forces a review loop that enhances the educator's instructional design expertise rather than replacing it.
Operationalization of CTML: The paper successfully translates abstract educational theories (12 CTML principles) into concrete, executable constraints and review prompts for Large Language Models (LLMs).
Human-AI Co-Creation Model: It proposes a shift from "one-shot" generation to transparent, curriculum-aligned co-creation, addressing the "black box" nature of current generative AI.

4. Results

The study demonstrated that PedaCo-Gen significantly outperformed the baseline across all measured dimensions.

Pedagogical Quality:
- Statistical Significance: PedaCo-Gen showed significant improvements ( $p < .05$ ) across all 12 CTML principles compared to the baseline.
- Score Improvement: The average CTML score increased from 3.07 (Baseline) to 3.86 (PedaCo-Gen), a mean improvement of +0.79.
- Top Gains: The most significant improvements were in Pre-training (+0.86) and Coherence (+0.84), indicating the system effectively introduced key concepts early and removed distracting elements.
- Topic Consistency: Improvements were consistent across causal, abstract, and sequential explanation types.
System Usability & Perception:
- Production Efficiency: Rated highly ( $M=4.26$ ), as the system reduced the trial-and-error costs associated with unconstrained generation.
- Guide Validity: Rated highly ( $M=4.04$ ), with low variance, indicating strong consensus that the AI's feedback was actionable and pedagogically sound.
- Overall Satisfaction: $M=3.78$ .
- Intention to Apply: $M=3.91$ , though with higher variance, suggesting context-dependent adoption (e.g., constraints in early childhood education).
Limitations Identified:
- Text Rendering: Improvements in Spatial Contiguity and Signaling were lower (+0.53 and +0.51) due to current T2V models' inability to render text accurately without typos.
- Audio Quality: Participants noted occasional unnatural voice synthesis.

5. Significance and Future Implications

Paradigm Shift: The work advocates for a transition from automated generation to principled human-AI co-creation, where AI serves as a partner that augments human expertise rather than replacing it.
Trust and Transparency: By making the generation process transparent (via script review) and explainable (via CTML-based feedback), the system fosters trust among educators, a prerequisite for AI adoption in education.
Future Directions:
- Contextual Granularity: Future iterations should adapt content difficulty and vocabulary based on specific learner demographics (e.g., age, grade level).
- Explainable AI (XAI): Integrating XAI to explain why specific scenes were generated to align with pedagogical goals.
- Technical Improvements: Enhancing text rendering in T2V models and integrating high-fidelity, emotionally expressive Text-to-Speech (TTS).
- Educator Training: Developing "AI literacy" workshops to help educators transition from simple prompting to principled co-creation.

In conclusion, PedaCo-Gen demonstrates that embedding pedagogical theory directly into the generative pipeline, combined with a human-in-the-loop review mechanism, can produce instructional videos that are not only visually plausible but cognitively optimized for learning.