AI LEGO: Scaffolding Cross-Functional Collaboration in Industrial Responsible AI Practices during Early Design Stages
This paper introduces AI LEGO, a web-based prototype that leverages boundary object theory to scaffold cross-functional collaboration in early-stage Responsible AI design, enabling technical and non-technical practitioners to more effectively identify and address potential harms through interactive blocks and LLM-driven persona simulations.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are building a giant, complex robot with a team of friends. You have the engineer who knows how to code the brain, the artist who designs the look, and the manager who figures out what the robot should actually do for people. Usually, the engineer builds the brain first, writes a bunch of confusing code, and then hands it to the others, saying, "Here, make it work." But often, the artist and manager can't understand the code, so they miss a huge problem: maybe the robot is going to be rude to certain people or make bad decisions. By the time they realize the mistake, the robot is already built, and fixing it is a nightmare. This is a common problem in the world of Artificial Intelligence (AI), where experts from different backgrounds struggle to talk to each other early enough to stop bad ideas before they become real, harmful products.
To solve this, researchers look at a concept called "boundary objects." Think of a boundary object like a universal translator or a shared Lego set. It's a tool that looks different to everyone but means the same thing to the group. To an engineer, a Lego brick might be a structural component; to a designer, it's a color block; but to the whole team, it's just "a brick" they can all hold and discuss. The paper you are about to read explores how to build a better "shared Lego set" for AI teams, specifically one that helps them spot dangerous mistakes before they start coding.
The Paper: AI LEGO
The researchers behind this study, a team from universities like Carnegie Mellon and Northwestern, noticed that when companies try to build "Responsible AI" (AI that is safe and fair), the process often breaks down. Technical people (the coders) and non-technical people (like product managers and designers) just aren't on the same page. The coders speak in complex technical jargon, and the non-coders feel left out, unable to spot potential harms until it's too late.
To fix this, they built a web tool called AI LEGO.
The Big Idea: Building with Blocks
Imagine you are trying to explain your dream vacation to a friend. If you just ramble on about flight numbers and hotel codes, they might get bored or confused. But if you use a map with big, colorful blocks for "Flight," "Hotel," and "Beach," it's easy to see the whole plan.
AI LEGO works the same way for AI projects. Instead of writing long, boring documents in tools like Google Docs, the team uses a digital canvas filled with Lifecycle Blocks.
- The Technical Team starts by dragging and dropping these blocks to sketch out their plan. Each block represents a stage of building an AI, like "Problem Formulation" (What are we solving?), "Dataset Construction" (What data are we using?), or "Model Definition" (How does the brain work?). Inside each block, there are simple prompts that guide them to explain their choices in plain language.
- The Non-Technical Team then steps in. They don't need to know how to code. They look at the blocks and use Stage-centered Checklists. These are like safety inspection cards. For the "Dataset" block, the checklist might ask, "Is there any sensitive data here that could cause bias?" This helps non-coders spot risks they might have missed.
The Secret Weapon: The "Persona" Minifigures
Here is where it gets really fun. AI LEGO has a special feature called Persona-centered Evaluation.
Imagine you are designing a new video game. To make sure it's fair, you don't just think about "players" in general. You imagine specific characters: "Teenager who loves sports," "Grandma who plays on a tablet," or "Someone who doesn't speak English."
AI LEGO lets the team create digital Minifigures (little Lego people) representing these different types of users. The tool even uses a smart AI (a Large Language Model) to simulate how these Minifigures might react to the plan.
- Example: If the team is building a movie recommendation AI, they might create a Minifigure named "Teen Movie Fan." The tool simulates what this teen might say: "Hey, if you only recommend action movies to me because I'm a guy, that's unfair!"
- This helps the team see "edge cases"—weird or specific situations they hadn't thought of—by literally seeing the project through the eyes of a different person.
What They Found: The Results
The researchers tested this tool with 18 industry practitioners (6 AI developers, 6 Product Managers, and 6 UI/UX designers) working in 6 teams. They asked these teams to design AI systems using three different methods:
- Google Docs (the old way, just writing text).
- AI LEGO Lite (using the blocks and checklists, but no Minifigures).
- AI LEGO Full (using the blocks, checklists, and the Minifigures).
The results were pretty clear:
- More Problems Found: When using AI LEGO, the teams found 195% more problematic design choices than when they used Google Docs. That's nearly double the number of potential disasters spotted!
- Better Quality: The problems they found weren't just random guesses; they were more likely to actually happen. The teams using AI LEGO Full found issues that were rated 1.25 points higher on a likelihood scale (out of 4) compared to the Google Docs group.
- The Minifigures Helped: The "Full" version of the tool (with the Minifigures) helped teams find even more problems than the "Lite" version. The simulated personas helped non-technical people think deeper about how real humans would be affected.
- Everyone Liked It: The participants said AI LEGO was easier to use and more fun than Google Docs. They felt like they were actually working together as a team rather than just passing notes back and forth.
Why This Matters
The paper suggests that the way we build AI matters just as much as the AI itself. If we keep building in silos—where coders talk to coders and managers talk to managers—we will keep making mistakes that hurt people.
AI LEGO shows that if we give teams a shared, visual language (like Lego blocks) and a way to step into other people's shoes (like Minifigures), we can catch bad ideas early. And the best part? Catching them early means we can fix them before anyone gets hurt, without having to tear down a finished robot and start over.
The study doesn't claim this is a magic wand that solves every problem in the world. The researchers admit their tool is a prototype and that real-world teams are more complicated than a study session. But they do suggest that changing how we talk about AI design—making it visual, structured, and inclusive—can make our future technology much safer and fairer for everyone.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.