💻 computer science

Open-Domain Safety Policy Construction

This paper introduces Deep Policy Research (DPR), a minimal agentic system that autonomously drafts comprehensive content moderation policies from seed domain information by iteratively searching the web and distilling rules, demonstrating performance that surpasses existing baselines and rivals expert-written policies across diverse benchmarks.

Original authors: Di Wu, Siyue Liu, Zixiang Ji, Ya-Liang Chang, Zhe-Yu Liu, Andrew Pleffer, Kai-Wei Chang

Published 2026-04-03

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Di Wu, Siyue Liu, Zixiang Ji, Ya-Liang Chang, Zhe-Yu Liu, Andrew Pleffer, Kai-Wei Chang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you run a massive, bustling digital town square. In this square, people post ads, share stories, and chat. But sometimes, people say mean things, show scary images, or try to trick others. To keep the town safe, you need a Rulebook (a safety policy) that tells your security guards exactly what is allowed and what isn't.

Usually, writing this Rulebook is a huge, expensive headache. You have to hire experts who know every possible "edge case" (like, "Is a cartoon knife okay? What about a joke that sounds like a threat?"). They have to write thousands of rules, update them constantly, and argue over the wording.

This paper introduces a new robot assistant called "Deep Policy Research" (DPR) that writes this Rulebook for you.

Here is how it works, using a simple analogy:

🕵️‍♂️ The Detective vs. The Librarian

Think of the old way of making rules as hiring a Librarian who sits in a quiet room, tries to remember every rule they've ever read, and writes a list based on their memory. Sometimes they forget the tricky details.

DPR is more like a Super-Detective.
You give the Detective a one-sentence clue, like: "We need rules about 'Violence'."
Instead of guessing, the Detective:

Goes to the Library (The Internet): It doesn't just guess; it actually searches the web. It looks at news sites, government documents, and forums to see how real people define violence.
Asks Specific Questions: It asks things like, "What counts as violence in a cartoon?" or "How do we handle jokes about fighting?"
Takes Notes: It reads the answers and turns them into clear, simple rules.
Organizes the Filing Cabinet: It doesn't just dump a pile of notes on your desk. It sorts them into neat folders (like "Physical Violence," "Verbal Threats," "Gore") so the security guards can find the right rule quickly.

🏗️ How the Robot Builds the Rulebook

The paper describes a three-step loop that the robot repeats a few times:

The Search: The robot looks at what it has so far and asks, "What am I missing?" It then searches the web for those missing pieces.
The Translation: It finds messy, long articles on the web and translates them into short, punchy rules.
- Web Article: "There are many nuances to hate speech involving identity tourism..."
- Robot Rule: "Do not allow content where someone pretends to be part of a group they aren't, just to mock them."
The Organization: It takes all these new rules and groups them together, giving each group a clear title so the final document is easy to read.

🧪 Did It Work?

The researchers tested this robot in two ways:

The Text Test: They asked the robot to write rules for five tricky topics: Sex, Hate, Violence, Harassment, and Self-Harm. They then gave these robot-written rules to a standard AI "Security Guard" to see if it could spot bad content better.
- Result: The robot-written rules made the Security Guard much smarter. It caught more bad stuff than when the guard was just given a vague definition or a few examples. In fact, the robot did almost as well as a team of human experts.
The Ad Test: They tested it on a real-world advertising company (Taboola) that has to check thousands of ads with pictures and text.
- Result: When they replaced a human-written section of their rulebook with the robot's version, the system worked almost just as well! The robot was especially good at catching tricky, visual tricks (like ads that use shocking images to sell products).

🌟 The Big Takeaway

The most exciting part isn't just that the robot is smart; it's that it's simple.

It doesn't need a super-complex brain or a million tools.
It just needs one tool (a web search) and a simple plan (Search -> Summarize -> Organize).

The Metaphor:
Imagine you are trying to build a fence around a garden.

The Old Way: You hire a master builder to design the fence from scratch. It takes weeks and costs a fortune.
The DPR Way: You give a smart robot a shovel and a map. The robot runs around the garden, checks the terrain, finds the best spots for fence posts, and builds a sturdy fence that fits the garden perfectly.

This paper shows that we don't need to be the experts to write the rules anymore. We just need to build a robot that knows how to research like an expert. This could save companies millions of dollars and make the internet safer much faster.

1. Problem Definition

The paper addresses the high cost and complexity of drafting and maintaining domain-specific safety policies for content moderation systems. While modern products rely on these policies to enforce safety guidelines (e.g., filtering hate speech, violence, or misinformation), creating them typically requires significant human expertise, iterative refinement, and frequent updates to handle edge cases.

The authors challenge the assumption that a fully human-written policy is a prerequisite for effective moderation. They frame the task as Open-Domain Policy Construction:

Input: A concise, one-sentence domain specification (defining the scope and intent) and access to a web search engine.
Output: A structured, indexed policy document consisting of actionable rules.
Success Metric: Downstream utility, specifically the improvement in accuracy (F1 score) of a fixed content moderation model when the generated policy is provided in-context.

2. Methodology: Deep Policy Research (DPR)

The authors propose Deep Policy Research (DPR), a minimal agentic system designed to autonomously synthesize safety policies. Unlike complex multi-agent frameworks, DPR relies on a single external tool (web search) and a lightweight scaffolding scheme.

Core Architecture:
DPR operates as an iterative loop over $k$ iterations, maintaining a policy draft ( $P$ ) and an index ( $I$ ). The process consists of three distinct steps per iteration:

Query Generation:
- The agent analyzes the current policy index to identify coverage gaps, ambiguous definitions, or high-risk subtypes.
- It generates targeted search queries to find webpages containing definitions, edge cases, and enforcement cues.
- It retrieves top search results (titles, snippets, URLs) as evidence.
Rule Extraction and Consolidation:
- The agent prompts an LLM to extract candidate rules from the retrieved evidence. Rules are formatted as short, predicate-style statements with clear decision boundaries.
- Self-Critique: The agent performs a self-critique pass to filter noise. It removes irrelevant rules, merges redundant rules expressing the same boundary, and resolves conflicts by prioritizing rules supported by multiple or higher-quality sources.
Indexing (Structuring):
- New rules are merged into the draft.
- The agent organizes the full rule set into coherent sections using keyphrase-based clustering.
- It extracts keyphrases for each rule, clusters them (using k-means), and generates section titles and summaries. This creates a structured, readable document that aids downstream "reader" models in processing long contexts.

3. Key Contributions

Task Formulation: Introduces "Open-Domain Policy Construction" as a distinct task where the goal is not just to classify content, but to generate the policy rules that guide classification.
Minimal Agent Design: Demonstrates that a simple, single-tool agent (web search + LLM) with a structured research loop can outperform generic deep research systems and few-shot learning baselines.
Structured Synthesis: Proposes a specific workflow (Query $\to$ Extract $\to$ Critique $\to$ Index) that transforms unstructured web data into actionable, indexed policy predicates.
Reproducibility: Releases the full codebase, prompts, and domain specifications to facilitate future research on policy drafting agents.

4. Experimental Results

The authors evaluated DPR in two distinct settings:

A. OpenAI Undesired Content Benchmark (Text-Only)

Setup: Five domains (Sexual, Hate, Violence, Harassment, Self-Harm) tested on two compact reader LLMs (Llama 3.1 8B and Qwen2.5 7B).
Baselines: Seed Information (one-sentence definition only), In-Context Learning (few-shot examples), and OpenAI's general-purpose Deep Research (OAI DR).
Results:
- DPR consistently outperformed all baselines.
- Llama 3.1 8B: Average F1 increased from 0.752 (Seed) to 0.792 (DPR).
- Qwen2.5 7B: Average F1 increased from 0.810 (Seed) to 0.831 (DPR).
- Comparison to OAI DR: DPR outperformed the general-purpose deep research agent (e.g., 0.792 vs. 0.776 on Llama 3.1), suggesting that task-specific structuring is superior to generic web research for policy drafting.
- Gains: The most significant improvements were in subjective categories (Violence, Harassment, Self-Harm) where definitions are ambiguous.

B. In-House Multimodal Advertisement Moderation

Setup: A real-world benchmark involving ad text and thumbnail images. The task was to replace specific expert-written policy sections with DPR-generated sections.
Results:
- Replacing expert sections with DPR-generated sections recovered most of the performance benefit lost by removing the section entirely.
- In the "Misrepresentative" domain, DPR achieved 98% of the human policy's performance under majority voting.
- Limitation: DPR struggled with "Finance Claims," a domain heavily reliant on proprietary, organization-specific compliance language not found in public web sources.

5. Significance and Conclusion

Feasibility of Automation: The paper proves that high-quality, structured safety policies can be synthesized from minimal human input (one sentence) and public web data, significantly reducing the cost of policy maintenance.
Structured Research vs. Generic Search: The results highlight that for policy drafting, a constrained, iterative research loop with explicit rule extraction and indexing is more effective than general-purpose deep research agents.
Human-in-the-Loop Potential: While DPR can automate the bulk of drafting, human experts remain crucial for domains requiring proprietary knowledge (e.g., specific financial regulations). The system serves as a powerful "first draft" generator that accelerates the policy refinement cycle.
Future Work: The authors suggest extending this approach to include reader-model-specific policy presentation and generating illustrative examples to clarify decision boundaries.

In summary, Deep Policy Research offers a scalable, reproducible framework for automating the creation of safety guidelines, demonstrating that agentic systems can effectively translate high-level domain definitions into actionable, high-performing moderation policies.