Child-Oriented AIGC Video Risk Reviewing: A Benchmark and Knowledge-Supported Iterative Reasoning Framework

Imagine a world where almost any video you can dream up can be created in seconds by a computer. This is the era of Artificial Intelligence-Generated Content, or AIGC. It's like having a magical paintbrush that can draw moving pictures, tell stories, and create characters without a human artist ever touching a screen. While this sounds like a superpower for creators, it comes with a tricky side effect: sometimes, the computer gets confused. It might draw a cat with three eyes, make a character do something physically impossible, or tell a story that doesn't make sense. For adults, these glitches are just funny or weird. But for children, whose brains are still learning how the world works, these confusing videos can be like a distorted mirror, potentially teaching them wrong ideas about safety, bodies, or how to act.

To keep kids safe, we need a way to check these videos before they reach a young audience. This is where "Large Vision Language Models" come in. Think of these as super-smart computers that can both "see" a video and "read" the story it tells, kind of like a robot that has eyes and a brain. However, just asking a robot to look at a video and say, "Is this safe?" often isn't enough. The robot might miss a subtle clue because it's looking at the whole picture at once, or it might guess wrong because it doesn't have a rulebook for what's safe for a 5-year-old. This paper dives into the corner of computer science dedicated to making these robots better at spotting the hidden, tricky dangers in AI videos specifically for children.

---

The Magic Detective Squad: Catching AI Video Glitches

So, how do you catch a computer glitch that is trying to hide? In this study, a team of researchers built a new system that acts less like a single security guard and more like a team of expert detectives working together. They call their new framework QVRS-E, but let's just think of it as the "Question-View-Review Squad."

The Problem: The "One-and-Done" Mistake

Imagine you are trying to find a hidden toy in a messy room. If you just glance at the room once and say, "I don't see it," you might miss the toy hiding under a rug. That's what happens when current AI video checkers try to do their job. They often look at a video once, make a quick guess, and move on. They might miss a scary sound that only happens for a second, or a weird logic error where a character walks through a wall.

The researchers found that existing tools are great at spotting obvious bad stuff (like violence or nudity), but they struggle with the "invisible" risks that are bad for kids. These include things like:

• Distorted Bodies: A character with too many fingers or a face that melts.
• Bad Logic: A story where a fire puts out water, or a dangerous act is shown without warning.
• Scary Vibes: A video that looks happy but has a creepy, unsettling feeling that a child might not understand but will feel.

The Solution: A Team of Agents

To fix this, the researchers created a system where four different "AI Agents" work together in a loop, kind of like a game of "20 Questions" but with a video.

1. The Question Agent (Q-Agent): This is the curious kid. It looks at the video and asks, "Wait, why is that dog flying?" or "Did that character just disappear?" It generates specific questions to hunt for clues.
2. The Vision Agent (V-Agent): This is the detective with a magnifying glass. It looks at specific frames of the video to answer the questions. "Yes, the dog is flying because the AI messed up the physics."
3. The Review Agent (R-Agent): This is the judge. It checks if the answers are enough. "Do we have enough proof that this is dangerous? If not, let's ask more questions."
4. The Summary Agent (S-Agent): This is the reporter. Once the team is done, it writes a clear report explaining exactly what went wrong and why it's bad for kids.

But here's the secret sauce: this team doesn't just guess. They have two special libraries of knowledge to help them.

• The Expert Library: This is like a rulebook written by child safety experts. It tells the agents, "If a character has a body part in the wrong place, that's a risk."
• The Experience Library: This is like a diary of past cases. It says, "Last time we saw a video with a flying cat, we found a risk by asking about the wings. Let's try asking about the wings again."

By asking questions, checking the video, and using these libraries, the system builds up a pile of evidence before making a final decision. It's the difference between guessing the answer to a riddle and actually solving it step-by-step.

The New "Playground" for Testing

To test if their new detective squad was any good, the researchers first had to build a playground. They couldn't just use old videos; they needed real AI videos that kids might actually watch. They collected 605 real-world videos from popular platforms like TikTok and YouTube.

They didn't just count how many videos were "bad." They created a super-detailed map of risks, called a taxonomy. Imagine a giant tree with 6 big branches (like "Physical Risks" or "Emotional Risks") and 26 tiny leaves (like "Distorted Biological Appearance" or "Dangerous Behavior Imitation"). This map helped them label exactly what was wrong with each video, so they could train their AI to spot these specific issues.

What They Found

When they put their new "Question-View-Review Squad" to the test, the results were promising.

• Better than the Solo Act: When they compared their team-based system to standard AI models that just look at a video once, their system found many more risks. For example, on a test with 26 different types of risks, their system using a standard open-source AI model jumped from a low score to a much higher one, proving that the method of asking questions was the key, not just the size of the AI brain.
• Beating the Giants: In some cases, their system using a medium-sized open-source AI (about 8 billion "brain cells") performed better than some of the most expensive, closed-source super-computers available today. This suggests that you don't need the biggest, most expensive computer to do a good job; you just need the right way of thinking.
• The Power of "Why": The study showed that the biggest boost in performance came from the iterative probing—the act of asking questions, getting answers, and asking again. This helped the AI find evidence it would have missed in a single glance. The "Expert Knowledge" helped the AI understand the answers better, especially for the tricky, fine-grained details.

Where It Still Gets Stuck

The researchers were honest about where their system still struggles.

• The Blink-and-Miss-It Problem: If a scary or weird thing happens for just a split second and the AI doesn't happen to look at that exact frame, the system might miss it. It's like trying to catch a firefly with a net; if you aren't looking at the right moment, it's gone.
• Over-Reacting: Sometimes, the system gets so good at looking for risks that it sees danger where there isn't any. For example, it might think a cartoon pig moving stiffly is "abnormal behavior" just because it's being overly cautious.

The Takeaway

This paper suggests that to keep children safe in the age of AI videos, we can't just rely on a quick scan. We need a system that thinks like a detective: asking questions, checking the evidence, and using what it has learned from experts and past cases. By building a better map of risks and a smarter way to investigate, we can create a safer digital playground for the next generation. The researchers hope their new benchmark (the 605 videos and the risk map) and their detective framework will help other scientists build even better safety tools in the future.