🤖 AI

Informing AI Policy Assessment using Large-Scale Simulation of Interventions

This paper introduces a methodology that combines participatory evaluation, expert cost assessment, and LLM-based harm mitigation analysis within a genetic algorithm simulation to help policymakers identify and prioritize viable AI policy options by exploring diverse trade-offs between implementation costs, stakeholder input, and effectiveness.

Original authors: Julia Barnett, Kimon Kieslich, Natali Helberger, Nicholas Diakopoulos

Published 2026-05-28

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Julia Barnett, Kimon Kieslich, Natali Helberger, Nicholas Diakopoulos

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are the mayor of a bustling city, and a new, powerful tool has arrived: Artificial Intelligence (AI). While this tool can do amazing things, it's also causing trouble. It's spreading fake news, taking people's jobs, and making headlines overly dramatic. You have a massive toolbox filled with hundreds of different wrenches, hammers, and screwdrivers (these are policy options) to fix the problems. But you can't use them all at once; some are too expensive, some are hard to enforce, and some might not actually fix the problem.

The question is: Which combination of tools should you pick to fix the mess without breaking the bank or ignoring what the citizens want?

This paper by Barnett and colleagues introduces a new way to answer that question. They built a "Policy Simulator" that acts like a high-tech, digital trial-and-error machine. Here is how it works, broken down into simple parts:

1. The Three Ingredients of the Recipe

To find the best policy mix, the simulator balances three competing ingredients, like a chef tasting a soup:

The "Harm Reduction" Taste (Effectiveness): Does this policy actually stop the bad AI behavior? The researchers used AI to imagine future stories where these policies were in place and asked: "Did the problem get better?"
The "Price Tag" Taste (Cost): How much money and effort will it take to implement this? A panel of experts (lawyers, tech folks, journalists) rated each policy from "cheap and easy" to "expensive and impossible."
The "Citizen Vote" Taste (Participation): What do regular people want? The researchers asked everyday citizens (lay stakeholders) which policies they felt were most important and agreed with.

2. The "Evolutionary" Search Engine

The problem is that there are billions of possible combinations of policies. If you tried to test every single combination, it would take longer than the age of the universe.

To solve this, the authors used a Genetic Algorithm. Think of this like breeding dogs or growing a garden:

Generation 1: The computer starts with a random mix of policies (a "litter" of puppies).
The Test: It checks which puppies are the "fittest" (the ones that best balance harm reduction, low cost, and high citizen support).
Breeding: It takes the best policies from the first group and "mashes" them together to create a new, hopefully better, generation.
Mutation: Occasionally, it randomly swaps a policy in or out to see if a surprise change makes things better.
Repeat: It does this over and over, generation after generation, until it finds the "champion" policy combinations that are hard to beat.

3. The "Magic" of the AI

Since they couldn't ask thousands of real people to read and rate millions of different future stories, they used a Large Language Model (LLM) as a stand-in.

Imagine the LLM as a very well-trained actor. The researchers first taught this actor how to think and feel like the real citizens by showing it thousands of examples of how real people rated stories.
Once the actor was "aligned" with the public, they used it to read millions of simulated future scenarios and rate how well different policies worked. This saved them from needing to hire an army of people for the initial screening.

4. What They Found (The Results)

When they ran the simulator with different "recipes" (changing the weight of the three ingredients), they got very different results:

If you only care about stopping the harm: The simulator suggests a huge list of policies. It's like trying to fix a leaky roof by nailing a thousand boards to it. It works great at stopping the leak, but it's incredibly expensive and might even be impossible to build.
If you only care about saving money or listening to the public: The simulator suggests a tiny list of 1 or 2 policies. These are cheap and popular, but they might not stop the AI problems very well.
If you balance all three: The simulator finds a "sweet spot." It usually suggests a small, manageable list of 1 or 2 policies that are popular with the public, cost a moderate amount, and still do a decent job of reducing harm.

The Key Takeaway: There is no single "perfect" policy. The best choice depends entirely on what the decision-maker values most. If you prioritize safety above all else, you get a different list than if you prioritize saving money.

5. The Safety Net

The authors are very clear: This tool does not make the final decision. It is not a robot that says, "Here is the law you must pass."

Instead, it is a decision-support map. It shows policymakers the landscape of possibilities. It says, "If you want X, here are your best options. If you want Y, here is a different set of options." It helps leaders see the trade-offs clearly before they spend real money or time on a policy that might fail.

In short, this paper offers a digital sandbox where leaders can play with different policy combinations to see what works best for their specific goals, ensuring they don't pick a policy that is too expensive, too unpopular, or simply ineffective.

Technical Summary: Informing AI Policy Assessment using Large-Scale Simulation of Interventions

Problem Statement

The rapid proliferation of AI systems has created a complex governance landscape where policymakers face an overwhelming abundance of potential policy options to mitigate harms. Prioritizing among these competing strategies is challenging due to the difficulty of quantifying risks, the scarcity of resources, and the lack of inclusive assessment methods that integrate diverse stakeholder perspectives. Existing approaches often rely heavily on expert knowledge, potentially leading to biased risk assessments that ignore the lived experiences of lay stakeholders who are most affected by AI harms. Furthermore, traditional policy evaluation methods, such as randomized control trials, are often too expensive or time-consuming to be used early in the decision-making process to orient policymakers. There is a critical need for a methodology that can efficiently explore a vast solution space of policy combinations while balancing anticipated harm mitigation, implementation costs, and public perception.

Methodology

The authors propose a computational framework that integrates participatory evaluation, expert assessment, and Large Language Model (LLM) simulation to identify viable policy options. The methodology is grounded in a genetic algorithm (GA) designed to navigate an intractably large solution space (e.g., over 2 billion combinations for political manipulation alone).

Core Components

Stakeholder-Action Pairs (SAPs): The foundational units of policy are SAPs, which link a specific stakeholder (e.g., news publishers, tech companies) to a specific action (e.g., "be transparent about data collection"). These were originally brainstormed and evaluated by lay stakeholders in prior work.
Optimization Function: The system optimizes a weighted function $F(S, P')$ $F (S, P^{'})$ that balances three competing factors:
- Harm Mitigation ( $\alpha$ ): The perceived reduction in harm severity and magnitude. This is evaluated by rewriting original harm scenarios ( $S$ ) under specific policy conditions ( $P'$ ) and assessing the change in risk dimensions.
- Expert Cost ( $\beta$ ): The estimated implementation cost assigned by an interdisciplinary expert panel (policy, law, journalism, AI deployment). Costs are rated on a scale of 1 (Low) to 3 (High), with 4 indicating "Nonviable" (legally or technically infeasible).
- Participatory Rating ( $\gamma$ ): The degree to which lay stakeholders prioritize and agree with specific SAPs, derived from prior survey data.
LLM Alignment and Simulation:
- To scale the evaluation of millions of policy combinations, the authors align an LLM (specifically o3-mini with low reasoning effort) with human ratings.
- The LLM is prompted to act as a panel of five diverse judges to evaluate rewritten scenarios ( $S'$ ) across three dimensions: Severity, Magnitude, and Plausibility.
- The model was trained and validated against a dataset of 234 lay-evaluated scenarios, achieving Pearson correlation coefficients of 0.794 (severity), 0.707 (magnitude), and 0.616 (plausibility) against human ratings.
Genetic Algorithm (GA):
- Initialization: A population of random policy subsets (chromosomes) is generated.
- Selection & Crossover: Parents are selected based on fitness (the optimization function score) using roulette-wheel sampling. Two-point crossover is used to create offspring.
- Mutation: A low mutation rate (0.03) introduces variation.
- Elitism: The top 3 performing policies are preserved in each generation to prevent the loss of optimal solutions.
- Termination: The algorithm runs until a maximum number of "stall generations" is reached without finding new optima.

Case Studies

The method was applied to three specific harms in the generative AI media ecosystem:

Political Manipulation (31 possible SAPs).
Unemployment (16 possible SAPs).
Media Sensationalism (26 possible SAPs).

Key Results

The study demonstrates that varying the weights assigned to harm mitigation, cost, and participatory components yields significantly different policy outcomes:

Balanced Weights (All Equal): Results in a highly selective policy set (1–2 SAPs) that prioritizes low-to-moderate cost and high participatory value. The anticipated harm reduction is modest.
Emphasis on Harm Mitigation ( $\alpha=1$ ): Leads to the largest number of mandated SAPs (40–50% of the total pool) and the highest anticipated harm reduction. However, this approach frequently includes nonviable SAPs (rated as cost 4 by experts) and incurs the highest costs.
Emphasis on Cost or Participation: When cost is prioritized, the model selects fewer, low-cost SAPs. When participation is prioritized, the model selects SAPs with high public agreement, sometimes including high-cost items.
Nonviable SAPs: When the cost component is excluded from the optimization ( $\beta=0$ ), the algorithm frequently selects nonviable SAPs (e.g., government bans on AI-generated news), highlighting the necessity of expert constraints.
Policy Diversity: The GA identified a diverse set of locally optimal solutions. While some SAPs (e.g., "News publishers should be transparent about AI use") were consistently selected across different weightings, many other combinations varied, suggesting that a single "optimal" policy does not exist without explicit trade-off preferences.

Significance and Claims

The authors position this work as a proof-of-concept methodology rather than a definitive policy prescription. Their primary claims include:

Operationalizing Participatory AI: The method successfully integrates lay stakeholder perspectives directly into practical policy development pipelines, complementing expert knowledge rather than replacing it.
Exploration of Trade-offs: By allowing policymakers to adjust weights ( $\alpha, \beta, \gamma$ ), the tool makes explicit the trade-offs between effectiveness, cost, and public acceptance. It reveals that prioritizing one factor (e.g., harm mitigation) often necessitates sacrificing others (e.g., cost or feasibility).
Scalable Simulation: The approach demonstrates that LLMs, when aligned with human data, can serve as a cost-effective proxy for large-scale scenario evaluation, enabling the exploration of solution spaces that are computationally intractable for human-only evaluation.
Decision Support, Not Replacement: The authors emphasize that the output is a "starting point for deliberation." The method is designed to support policymakers in identifying viable areas for further investment and assessment, not to automate decision-making or replace human judgment.

The paper concludes that the diversity of viable policy combinations found by the genetic algorithm provides a useful basis for deliberation, helping stakeholders understand how different knowledge sources (expert vs. lay) and priorities shape the landscape of potential AI governance.