Guardrails versus Gatekeepers: Understanding Product Managers' Ethical Decision-Making in Generative AI
Drawing on interviews and a global survey, this study reveals that while leadership commitment and organizational principles are essential for enabling product managers to effectively operationalize responsible generative AI, individual actors can still enact meaningful ethical actions through low-resource initiatives despite systemic constraints like uncertainty and diffused responsibility.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine a tech company is like a massive, high-speed train factory. The Product Managers (PMs) are the station masters standing on the platform. They don't build the train (that's the engineers), and they don't lay the tracks (that's the executives), but they are the ones who decide: Does this train leave the station today? Is it safe enough for passengers?
For a long time, experts thought these station masters were "Gatekeepers." The idea was that they held the master key to the gate. If they said "No," the train (the AI product) couldn't leave. If they said "Yes," it could go. They were seen as the ultimate judges of what was ethical and what wasn't.
However, this new study suggests that view is wrong. Instead of being Gatekeepers, most Product Managers see themselves as "Guardrails."
Here is what that means, broken down simply:
1. The Problem: Foggy Roads and Broken Rules
The study found that Product Managers are trying to drive these AI trains through a thick fog. They face three big problems:
- The Fog of Uncertainty: No one really knows exactly how the "black box" AI engines work. The people who built the AI (the model makers) often don't share the blueprints. So, PMs are driving blind, not knowing if the train has hidden brakes or if it might suddenly swerve.
- The "Someone Else's Job" Trap: Because the fog is so thick, many PMs assume, "Surely the safety team or the ethics team has checked this already." They think responsibility is shared by everyone, which often means it's actually the responsibility of no one.
- The Speed vs. Safety Race: The company wants the train to leave now to make money. The PMs are rewarded for speed, not for stopping to check if the tracks are safe. If they stop the train to fix an ethical issue, they might get fired or not get promoted.
2. The Two Types of Actions
Even with the fog and the pressure to speed up, the study found that PMs still try to do the right thing. But they can only do two specific types of things, depending on how much help they get from the "head office" (leadership).
Type A: The "Low-Energy" Moves (Individual Actions)
These are things a PM can do alone, without asking for permission or spending extra money. They are like a driver checking their own mirrors or making sure their seatbelt is on.
- Examples: Double-checking that they aren't feeding private customer data into the AI, or asking a simple question about where the AI's data came from.
- The Catch: These are helpful, but they are small. They can't stop a train from derailing if the tracks are fundamentally broken.
Type B: The "High-Energy" Moves (Collective Actions)
These are the big safety checks that require a whole team, special tools, and a lot of time. They are like stopping the whole factory to rebuild the tracks.
- Examples: Running complex tests to see if the AI is biased, hiring a "red team" to try to break the AI, or doing a full audit of the software.
- The Catch: A PM cannot do these alone. They need the CEO to say, "Yes, stop the line, we will pay for this," and they need the company to change how they measure success (so the PM isn't punished for the delay).
3. The Big Discovery: Guardrails, Not Gatekeepers
The study's biggest surprise is about how PMs see themselves.
- The Old View (Gatekeepers): "I have the power to decide if this is ethical. I am the final boss of morality."
- The New View (Guardrails): "I am just here to make sure the train stays on the tracks that leadership laid down. If the tracks are broken, I can't fix them. I can only nudge the train if the rules are clear and the boss says it's okay."
If the company leadership doesn't provide clear rules, clear rewards for being safe, and the resources to do the big checks, the PMs feel powerless. They aren't refusing to be ethical; they are just stuck behind a wall they can't climb.
4. What Makes It Work?
The study found that when PMs do successfully act ethically, it's usually because of three things from the top:
- Leadership Commitment: The boss clearly says, "Safety is more important than speed."
- Clear Principles: The company has written-down rules that aren't just vague slogans.
- Real Incentives: The company actually rewards people for slowing down to check safety, rather than just punishing them for missing a deadline.
The Bottom Line
You can't just tell a Product Manager, "You are the Gatekeeper, make sure this AI is ethical!" and expect them to fix everything. If the company doesn't give them the tools, the time, and the permission to stop the train, they will just become Guardrails—trying to keep the train on the path as best they can, but unable to change the path itself.
To make AI truly safe, the whole organization needs to build a better track system, not just put more pressure on the person standing on the platform.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.