Multi-Agent LLMs as Ethics Advocates for AI-Based Systems

Imagine you are building a robot, but not just any robot—one that helps doctors diagnose illnesses, decides who gets a loan, or translates sign language for the deaf. Before you press the "start" button, you have to write a rulebook for it. This rulebook is called "requirements engineering." It's like the blueprint for a house; if the blueprint says "put a window here," the builder puts a window there. But what if the blueprint forgets to say, "Make sure the window isn't so big it lets in a hurricane"? In the world of Artificial Intelligence (AI), those missing rules are called "ethics requirements." They are the instructions that tell the AI to be fair, transparent, and kind, rather than biased or secretive.

The problem is that writing these ethical rules is hard. It usually requires a big meeting with lots of different people—engineers, lawyers, community members—to argue and brainstorm for hours. It's slow, expensive, and sometimes, because everyone is tired, the most important ethical rules get forgotten. This paper asks a big question: Can we use a team of super-smart computer brains, called Large Language Models (LLMs), to act as a "moral watchdog" and help write these rules faster? Think of it like hiring a team of AI detectives to look at your robot's blueprint and shout, "Hey, you forgot to check if this is fair!" before you even start building.

The Team of AI Detectives

The researchers, Asma Yamani, Malak Baslyman, and Moataz Ahmed, built a system they call MALEA (Multi-Agent LLM Ethics-Advocate). Instead of asking one computer to do all the thinking, they created a virtual team of four AI agents, each with a specific job, working together like a group of friends editing a story.

First, there's the Requirements Engineer, who takes a simple description of a system (like "an app that finds fake reviews") and tries to write the initial rules. Next, a Quality Assurance agent acts like a strict editor, checking if the rules are clear and make sense. Then comes the star of the show: the Ethics Advocate. This agent's only job is to look at the rules and say, "Wait, is this fair? What about privacy? What if someone gets hurt?" Finally, a Documentation Assistant writes everything down neatly.

These agents don't just work in a line; they talk to each other in a loop. The Engineer writes a rule, the Quality agent fixes the grammar, the Ethics agent critiques the morality, and the Engineer goes back to fix it. They keep passing the ball back and forth until everyone agrees the rules are good. The researchers tested this team on two real-world scenarios: a system designed to spot fake online reviews written in Arabic, and an app that translates Saudi Sign Language into text.

What the Team Found

When the researchers compared their AI team to a human expert and a single AI computer, the results were a mix of "Wow!" and "Whoops."

In the case of the Fake Review Detector, the human expert found 8 ethical rules in a 30-minute meeting. The single AI computer found about 6 of them (75% recall), but the MALEA team found 7 of them (87.5% recall). The AI team was better at catching the human's mistakes. For example, the human missed a rule about tracking "false positives" (when a real review is wrongly flagged as fake), but the AI team added a requirement for quarterly reports to track these errors.

In the Sign Language App scenario, the human expert found 13 rules. The single AI found about 8 (62%), while the MALEA team found about 7 (54%). Here, the human actually did a slightly better job at catching the specific technical details, like needing a response time of under 5 seconds. However, the AI team was a master at adding new ideas that the human hadn't thought of. The AI team suggested things like "audited anonymization" (hiding user identities in a way that can be checked) and specific encryption standards (like TLS 1.3) that the human missed entirely.

The paper suggests that the AI team is excellent at brainstorming a wide net of ethical ideas and making vague rules very specific. For instance, instead of just saying "keep data safe," the AI team wrote "use AES-256 encryption." But, the paper also warns that the AI isn't perfect. Sometimes it misses specific cultural nuances or gets the numbers wrong. In one run, the AI noticed that women in Saudi Arabia wear different types of head coverings and suggested the training data should include specific numbers of videos for each type. But in a different run, it forgot this detail entirely. This shows that the AI is a bit like a creative writer who sometimes remembers a detail and sometimes forgets it.

The Verdict: A Helpful Assistant, Not a Replacement

The main takeaway is that this multi-agent system is a powerful tool for getting a "first draft" of ethical rules quickly. It can generate more ideas and be more specific than a single computer, and it often catches things humans might overlook in a short meeting. However, the researchers are very clear: this does not replace humans.

The AI is great at suggesting, but it needs a human to say, "Yes, that's a good idea," or "No, that doesn't fit our culture." The system even leaves little "placeholders" (like [INSERT NUMBER HERE]) to remind engineers that they need to fill in the specific details. The paper argues that we shouldn't try to automate ethics completely; instead, we should use these AI teams to speed up the process, so humans can spend more time discussing the hard, important questions rather than just writing the basic rules.

In short, the paper suggests that a team of AI agents can act as a fantastic "moral brainstorming partner," helping us build AI systems that are safer and fairer, but only if we keep a human in the loop to make the final call.