FORGE: A Framework for Orchestrated Role-based Generation in Education
This paper introduces FORGE, a design science framework that structures instructor-AI collaboration in higher education through four roles and seven dimensions, demonstrating via case studies how its guidelines transform improvised prompting into a transferable, role-based workflow for generating educational materials.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the classrooms of higher education, a quiet revolution has been underway, driven by a new kind of computer program known as a large language model. These systems can read vast amounts of text and write in human language, leading many to wonder how they might change the way teachers prepare their courses. For years, most research has focused on how students use these tools, often worrying about academic integrity or how to detect when a student has relied on them. However, a parallel and equally significant shift is happening behind the scenes: instructors are beginning to use these same tools to help them write exams, design projects, and create lesson plans. The challenge is that without a shared method for this work, every teacher is essentially inventing their own way of asking the computer for help. Some might ask for a full draft, others for a single sentence, and few have a clear system for deciding when to trust the machine and when to step in with their own expert judgment.
This uncertainty matters because creating good educational materials is a complex task. It requires not just generating text, but ensuring that a question tests the right concept, that the difficulty is just right, and that the content aligns with what was actually taught in class. If a teacher simply asks a computer to write an exam and accepts the first result, the material might be flawed in subtle ways that a student could exploit or that might fail to measure what the teacher intended. The core question, then, is not whether computers can write these materials, but how a human and a machine can work together effectively to produce something that is both high-quality and trustworthy.
To answer this, researchers Emmanuel Gonzalez and Paul T. Grogan at Arizona State University developed a new way of working called FORGE. They did not just propose a theory; they tested it in a real undergraduate engineering course. They watched how an instructor used a computer program to create two very different types of assignments: a standard multiple-choice exam and a complex, open-ended term project. By recording every single instruction the teacher gave the computer and how the computer responded, they mapped out a clear set of rules for collaboration. Their work suggests that successful human-computer teamwork in education depends on assigning specific roles to the computer, knowing when to ask for ideas versus when to give orders, and understanding that the human must remain the final judge of quality.
The researchers found that the way a teacher interacts with the computer depends heavily on what they are trying to create. When the goal is a convergent material, like an exam where the answers are fixed and the format is known in advance, the process looks like a long, detailed refinement. In the study, the instructor spent nearly all their time—about ninety-four percent of the active work—checking and fixing the computer's drafts rather than asking for new content. The computer would generate a question, and the teacher would spot a subtle error, such as a code snippet that accidentally gave away the answer, and ask the computer to try again. This cycle of generation and correction continued until the material was perfect. The computer acted as a fast writer, but the human acted as a strict editor, ensuring that every detail met the specific standards of the course.
In contrast, when the teacher worked on a divergent material, like the term project specification where the final shape was not yet known, the process was different. Here, the computer and the teacher spent more time figuring out what the project should even be. The teacher asked the computer for options and ideas, engaging in a back-and-forth conversation to decide which topics to include and how to structure the assignment. In this scenario, the computer acted more like a brainstorming partner, offering suggestions that the teacher could accept, reject, or modify. The teacher spent a larger portion of their time, about twenty-six percent of the prompts, asking open-ended questions to explore possibilities, rather than just correcting errors. This showed that the type of material dictates the rhythm of the work: some tasks need a steady hand to polish a known shape, while others need a flexible mind to discover a new one.
A key finding of the study is that the computer cannot be trusted to make the final call on what is pedagogically sound. The researchers identified four distinct roles that can be played in this collaboration: a Reader who gathers information, a Writer who creates content, a Critic who checks for errors, and an Orchestrator who makes the final decisions. In both cases, the human instructor took the role of the Orchestrator. They found that while the computer could handle tasks like formatting text or checking for spelling mistakes, it consistently missed deeper issues. For example, in one instance, the computer wrote a question about a computer program, but the code it included contained the answer right in front of the student's eyes. The computer did not notice this flaw because it lacked the context of what the question was supposed to test. Only the human instructor, who knew the learning goals, could see the problem and direct the computer to fix it.
The study also revealed that the way a teacher speaks to the computer matters just as much as what they ask. The researchers distinguished between two styles of interaction: conversational and directive. When the teacher was still deciding what the assignment should look like, they used a conversational style, asking the computer for options and trade-offs. Once the plan was set, they switched to a directive style, giving clear commands to fix specific problems. The most effective teachers in the study knew when to switch between these modes. They did not stick to one way of talking; they adapted their approach based on whether they were exploring new ideas or refining an existing draft. This flexibility prevented the work from stalling or going down the wrong path.
Perhaps the most important lesson from this work is that the human must remain the integrator of the system. The computer can generate content at incredible speed, but it cannot understand the broader purpose of the course or the specific needs of the students. The researchers observed that when the teacher stepped back and let the computer run the whole process, the results were often flawed. But when the teacher stayed in the loop, constantly checking the work and guiding the computer with specific instructions, the final materials were high quality. The study suggests that the best use of these tools is not to replace the teacher, but to act as a powerful assistant that handles the heavy lifting of drafting, leaving the teacher free to focus on the judgment and expertise that only a human can provide.
The researchers tested these ideas on two specific assignments in a single course, so the findings are based on a limited set of data. They do not claim that this method works for every teacher or every subject, but they do show that a structured approach is possible. By defining clear roles and knowing when to ask questions versus when to give orders, teachers can turn the chaotic process of using artificial intelligence into a reliable workflow. The study concludes that with the right framework, instructors can describe how they work with these tools, keep what works, and carry those lessons forward to the next assignment. This moves the use of artificial intelligence in education from a series of lucky guesses to a practiced, repeatable skill that can improve the quality of learning materials for everyone.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.