A Protocol for Evaluating the Accessibility of AI-Generated Educational Materials: Prompt Configuration, WCAG-Derived Criteria, and Content Overload

Imagine the internet as a giant, bustling library. For decades, librarians have followed a strict rulebook called WCAG (Web Content Accessibility Guidelines) to make sure every shelf, sign, and story is reachable by everyone, including people who use wheelchairs, screen readers, or have trouble seeing or understanding complex text. This rulebook ensures that if you can't see a picture, a robot voice can describe it; if you can't use a mouse, a keyboard can still click the buttons. It's the difference between a library with wide aisles and one where the books are locked behind glass doors.

Now, imagine a new, super-fast robot librarian has arrived. This robot, powered by Generative AI, can write a book, draw a map, or record a story in seconds just because you asked it nicely. It's incredibly fast and can do the work of a whole team in minutes. But here's the catch: because the robot learned to read from the old library (the internet), and because the old library had many locked doors and missing signs, the robot tends to build new books with the same locked doors and missing signs. It doesn't know to be inclusive unless you explicitly tell it, "Hey, make sure this is open to everyone!" This paper asks a simple but urgent question: If we teach this robot the right rules, can it suddenly become the most helpful librarian in the world, or is it stuck building walls no matter what we say?

The Robot Librarian's Big Test

This paper is like a detective story where the author, Hector, sets up a massive experiment to test how well these AI robots can build educational materials—like PDF reports, slide decks, infographics, audio clips, and videos—that are actually accessible to people with disabilities.

The Setup: Three Ways to Talk to the Robot
Hector decided to test the robot under three different "personas" or instructions to see which one worked best:

1. The "Just Do It" Prompt (Condition A): This is what most people do today. You ask the robot, "Make me a slide deck about dinosaurs," with no extra instructions. It's the everyday, casual request.
2. The "Super-Strict Teacher" Prompt (Condition B): Here, the user gives the robot a detailed list of rules based on the WCAG rulebook. They say, "Make a slide deck, but you must include descriptions for every picture, make sure the text is high-contrast, and ensure a keyboard can click every button."
3. The "Permanent Skill" (Condition C): This is the "holy grail" idea. Instead of typing the long list of rules every single time, you load a "skill" or a permanent profile into the robot's brain once. From then on, every time you ask for anything, the robot automatically remembers to be accessible, just like a chef who always remembers to add salt without you having to ask.

The Findings: The Robot Needs a Nudge
The results were a mix of "uh-oh" and "wow."

When the robot was left to its own devices (Condition A), it was a disaster for accessibility. The average score for how well the materials followed the rules was a dismal 24.2%.

• The Documents: The robot made PDFs that looked great to the eye but were a mess to a screen reader. It forgot to label the headings or describe the charts, so a blind user would hear a jumbled stream of numbers with no context.
• The Slides: The robot created flashy slides with clickable buttons that couldn't be reached by a keyboard. It was like building a door that only opens if you have a magic wand (a mouse), leaving out anyone who uses a keyboard.
• The Videos: This was the worst offender, scoring 0%. The robot made videos with auto-generated captions that were often wrong, and it forgot to describe what was happening on the screen for people who are blind. It even created "lip-sync" issues where the AI avatar's mouth didn't match the words, which can confuse people who rely on reading lips.

The Magic of the "Super-Strict Teacher"
But then, Hector tried Condition B. When he gave the robot the specific, detailed instructions to follow the accessibility rules, the results skyrocketed. The average score jumped to 96.7%.

• The PDFs suddenly had proper tags and descriptions.
• The slides became navigable by keyboard.
• The videos got accurate captions and audio descriptions.

This proves that the robot isn't incapable of being accessible; it just doesn't know to be unless you tell it exactly how. The paper suggests that the barrier isn't the technology itself, but the way we talk to it.

The Hidden Trap: Information Overload
There was one tricky part the standard rulebook didn't fully cover: Information Overload.
Hector found that some AI tools, especially those that summarize long texts into one pretty picture (like infographics), tend to cram too much information into a tiny space. Even if the robot followed all the technical rules (like having a description for the image), the picture itself was so dense and cluttered that it was impossible for anyone—sighted or not—to understand quickly. It's like a signpost that has every single street name written on it in tiny font; technically, the information is there, but it's useless because it's too crowded. The paper suggests we need a new way to measure this "visual clutter" that goes beyond the standard checklists.

What About the "Permanent Skill"?
The paper also proposed the "Permanent Skill" (Condition C) as a way to make sure the robot never forgets these rules again, rather than having to type them out every time. However, this specific test wasn't fully run in this first experiment. The author suggests it would be the best long-term solution to keep the robot consistent, but they need to test it more to be sure.

The Bottom Line
This paper doesn't claim that AI has solved accessibility. Instead, it shows that right now, AI tends to build walls by default. But if we take the time to give it clear, specific instructions (or load it with a "skill"), it can build ramps and doors for everyone. The robot is a powerful tool, but it needs a human guide to make sure it doesn't leave anyone behind. The paper ends by inviting everyone to use this new "testing protocol" to check their own AI tools, ensuring that as we speed up education with AI, we don't accidentally leave students with disabilities in the dust.