PromptDecipher: Supporting AI Tutor Authoring Through Editable Simulated Interactions
To address the gap where educators rarely test AI tutoring bots before deployment, this paper presents PromptDecipher, a system that streamlines authoring by allowing teachers to directly correct undesirable bot responses in a live preview, which automatically generates targeted prompt rewrites and validates them through a QA pipeline.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are a teacher who wants to build a helpful AI tutor for your students. Right now, most tools ask you to do something very difficult: write a giant, complex set of instructions (called a "system prompt") in plain text, hoping the AI will understand exactly what you mean. It's like trying to teach a new employee by handing them a 50-page rulebook they've never seen before, without ever letting them practice.
The paper "PromptDecipher" argues that this is the wrong way to do it. Teachers are experts at giving feedback on student work, but they aren't experts at writing code or complex instructions for computers.
The Problem: The "Black Box" of Instructions
Currently, if a teacher creates a bot and it gives a bad answer, they have to guess which part of their giant instruction book caused the mistake, rewrite that section, and hope for the best. The authors found that in a study of over 100 teachers, almost none of them actually tested their bots before letting students use them. It's like a chef serving a new dish to a restaurant without ever tasting it first.
The Solution: The "Edit-Then-Explain" Approach
PromptDecipher flips the script. Instead of writing instructions first, you start by playing the role of a student.
- The Simulation: You chat with your new bot.
- The Correction: If the bot says something wrong or unhelpful, you don't go back to the instruction book. Instead, you simply edit the bot's response right there in the chat window, rewriting it to say exactly what you wanted it to say.
- The Magic Translator: Once you hit "submit," the system acts like a super-smart translator. It looks at your correction and asks, "Ah, so the teacher wanted the bot to ask a question instead of giving the answer?"
- The Auto-Update: The system then automatically rewrites the hidden instruction book (the prompt) to match your correction. It shows you the changes, like a "track changes" document in Word, so you can see what it did.
- The Safety Check: Before you can publish your bot, the system runs a quick test. It checks: "Did fixing this one mistake break anything else?" If the bot starts acting weird in other scenarios, it stops you and says, "Hey, we need to fix this too."
The Analogy: The Rehearsal Room
Think of building an AI tutor like directing a play.
- Old Way: You write a massive script for the actor (the AI) and hope they get it right. If they mess up a line, you have to guess which part of the script to change.
- PromptDecipher Way: You watch the actor perform. If they say the wrong line, you step in and say, "No, try saying it this way." The system then takes your new line and figures out how to rewrite the script so the actor remembers to say it that way next time. You keep doing this until the performance is perfect, and the system won't let you open the theater doors (publish the bot) until the rehearsal is successful.
Why It Matters
This system forces teachers to act like quality control engineers without them even realizing it. By making the "fixing" part easy and familiar (just editing a chat), it ensures that bots are actually tested and improved before real students ever see them. The authors plan to use this tool in a course for hundreds of college instructors to see if this method helps them build better, safer AI tutors.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.