💻 computer science

FormIDEAble: Safe and Socially-aware Autonomous Systems

FormIDEAble is a formally grounded framework that synthesizes socially-aware cooperation strategies for autonomous agents in socio-critical settings by modeling human-agent interaction as a Priced Timed Markov Decision Process to ensure safety guarantees while navigating uncertainty.

Original authors: Livia Lestingi, Amel Bennaceur, Marcello M. Bersani, Carlos Gavidia-Calderon, Anastasia Kordoni, Mark Levine, Bashar Nuseibeh, Matteo Rossi

Published 2026-07-01

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Livia Lestingi, Amel Bennaceur, Marcello M. Bersani, Carlos Gavidia-Calderon, Anastasia Kordoni, Mark Levine, Bashar Nuseibeh, Matteo Rossi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a chaotic emergency scene, like a building on fire. In the middle of the smoke, you have a robot trying to help people escape. The robot faces a tough choice: should it ask a nearby stranger to help carry an injured person, or should it call a professional firefighter?

If it asks the stranger, it might be fast, but the stranger might panic and run away. If it calls the firefighter, it's reliable, but the firefighter might be far away, and the injured person could get hurt while waiting.

This is the problem FormIDEAble tries to solve. It is a new "brain" for robots that helps them make these split-second decisions. It does two things at once that other robots struggle to do:

It understands people: It knows that humans aren't just machines; they act based on their social groups and feelings (like "we are all in this together").
It guarantees safety: It mathematically proves that whatever decision it makes, it won't break the "safety rules" (like "the person must be helped within 10 minutes").

Here is how it works, broken down into simple parts:

1. The "Game Board" (The PTMDP)

The researchers built a special kind of game board to model the situation. Think of it like a board game where:

The Robot is a player who can choose moves (like "Ask for help" or "Call a pro").
The Humans are other players, but they are unpredictable. Sometimes they help, sometimes they don't. The robot doesn't know for sure what they will do, but it can guess the odds based on how people usually act in groups.
The Clock and the Score: Every move takes time and uses up resources (like battery or the limited number of firefighters).

2. The "Social Identity" Crystal Ball

The robot uses a tool called an "Identity Predictor." Imagine the robot has a crystal ball that looks at a person's clothes, what they are saying, or who they are standing with. It uses this to guess: "Is this person likely to feel like part of a team and help out?"
If the robot thinks, "Yes, this person feels like a teammate," it might ask them for help. If it thinks, "No, they seem scared and alone," it might skip them and call a pro.

3. The "Safety Net" (Formal Guarantees)

This is the most important part. Many robots try to be smart, but they can't promise they won't make a mistake. FormIDEAble is different. Before the robot makes a move, it runs a super-fast mathematical check.

It asks: "If I choose this path, is there any chance the person waits too long?"
If the answer is "Yes, there's a risk," the robot cannot choose that path.
It only picks a path that is mathematically proven to be safe, while still trying to be as fast as possible.

4. The "Referee" (The Strategy Synthesis)

The system acts like a referee that designs a playbook for the robot. It looks at all the possible scenarios (what if the fire spreads? what if the survivor runs?), calculates the best move for each, and creates a "strategy."

Without FormIDEAble: The robot might guess and hope for the best.
With FormIDEAble: The robot follows a pre-approved playbook that says, "In situation A, do X. In situation B, do Y," ensuring that safety rules are never broken.

How They Tested It

The researchers tested this in a computer simulation of a mass evacuation (like a fire drill with hundreds of people).

They compared their robot against:
- Robots that did nothing.
- Robots that always called the pros (wasting time).
- Robots that always asked survivors (risky).
- Robots using an older, less safe method.
The Result: The FormIDEAble robot was faster at getting people out than the "do nothing" or "always ask" robots. Crucially, it saved the professional firefighters for the people who really needed them, while still making sure no one waited too long. It was also very good at predicting whether its plan would actually work in the simulation.

The Bottom Line

FormIDEAble is like giving a robot a socially smart brain wrapped in a mathematical safety suit. It allows autonomous agents to work with humans in dangerous situations without guessing, ensuring that while they try to be efficient, they never cross the line into danger.

The paper notes that this is currently a "prototype" tested in simulations. It is not yet a robot running in a real burning building, but the math shows it works in theory and in computer models. The researchers plan to test it in more areas (like healthcare or software teams) in the future, but for now, they have proven the concept works for emergency evacuations.

Technical Summary of FormIDEAble: Safe and Socially-aware Autonomous Systems

Problem Statement
Autonomous agents operating in socio-critical settings (e.g., emergency response) must coordinate with humans under conditions of uncertainty while strictly adhering to safety constraints. Existing approaches face a dichotomy: they either model social dynamics and human behavior without providing formal safety guarantees, or they offer formal assurance by abstracting away complex human behavior. Furthermore, while the "Identity-Aware Architecture for Autonomous systems" (IDEA) leverages social identity theory to foster cooperation, it relies on equilibrium reasoning that lacks formal guarantees that safety constraints (such as time bounds or resource prioritization) will always be satisfied. The core challenge is to synthesize decision-making strategies that are simultaneously socially aware, formally verified, and capable of handling the uncertainty inherent in human responses.

Methodology
FormIDEAble addresses this challenge by modeling the interaction between humans and autonomous agents as a Priced Timed Markov Decision Process (PTMDP). This formalism integrates four critical aspects:

Control: The autonomous agent selects from controllable actions.
Uncertainty: Human responses are modeled as probabilistic, uncontrollable environment reactions, capturing socially driven behaviors.
Time: Actions incur delays, and deadlines are enforced via timing constraints.
Cost: Decisions accumulate costs representing performance objectives (e.g., evacuation time, resource usage).

The decision-making problem is formulated as a cost-bounded reachability problem. The goal is to synthesize a strategy $\sigma$ that minimizes an expected cost (e.g., time) subject to a safety constraint: the probability of reaching a goal state $G$ (satisfying safety requirements) within a cost bound $B$ must exceed a threshold $p$ .

The framework is implemented as a MAPE-K (Monitor, Analyze, Plan, Execute, Knowledge) loop:

Monitoring: Detects decision points and gathers contextual data (distances, resource availability) and social identity estimates from an "Identity Predictor."
Analysis: A "Formal Game Builder" constructs a parametric PTMDP model using reusable patterns for controller decisions and adversarial (human) decisions (observable and unobservable). A "Formal Game Solver" (using Uppaal Stratego) computes the optimal strategy.
Planning & Execution: The "Best Decision Selector" chooses the action with the highest probability (lowest expected cost) prescribed by the strategy, which is then executed by the agent.

Key Contributions
The paper introduces three primary contributions:

Formalization: A formalization of socially-aware human–autonomous agent cooperation using PTMDPs, explicitly capturing social identity uncertainty.
Strategy Synthesis: A novel approach that integrates social identity uncertainty with explicit safety guarantees by formulating the problem as a cost-bounded reachability problem over PTMDPs.
Empirical Evidence: Initial experimental evidence demonstrating the approach's effectiveness in an emergency evacuation scenario, highlighting the trade-offs between optimization and safety.

Experimental Results
The authors evaluated FormIDEAble using the IMPACT+ simulation environment and Uppaal Stratego against four baselines: no support, always calling first responders, always asking survivors to help, and the non-formal IDEA approach.

Performance vs. Baselines: In scenarios with scarce first-responder resources ( $N_{fr}=9$ ), FormIDEAble significantly reduced total evacuation time ( $T_{evac}$ ) compared to IDEA and survivor-only support, while remaining comparable to the "always call staff" baseline. Crucially, FormIDEAble consistently reduced the number of first-responder calls compared to both the staff-support and IDEA baselines.
Safety Prediction: The formal model demonstrated high accuracy in predicting whether safety requirements would hold, with precision and recall consistently above 93% across various configurations.
Optimization vs. Safety: In Uppaal Stratego experiments, FormIDEAble successfully synthesized strategies that minimized task duration. When specific safety goals (e.g., child prioritization or capping first-responder calls) were enforced, the system maintained high probabilities of satisfying these reachability properties, often at the cost of a modest increase in task duration.
Overhead: Strategy synthesis introduced a non-negligible but modest wall-clock time overhead (averaging <0.5s for small/medium configurations and ~2.25s for complex ones), suggesting suitability for scenarios allowing limited deliberation time.

Significance and Claims
The paper positions FormIDEAble as a principled foundation for formally assured, socially-aware decision-making. Its significance lies in bridging the gap between social science (social identity theory) and formal methods (strategy synthesis). Unlike prior work that treats safety and social dynamics in isolation, FormIDEAble integrates them to ensure that autonomous agents can cooperate with humans under uncertainty without violating explicit safety constraints.

The authors are modest regarding the current scope, noting that the evaluation is preliminary and relies on simulation. They acknowledge that the approach currently requires manual effort for model instantiation and safety formalization. The paper claims to provide the foundational modeling and synthesis principles, leaving comprehensive scalability analysis, broader empirical validation across diverse socio-critical domains, and the development of higher-level tooling for future work. The approach is presented as a step toward autonomous systems that are not only safe but also socially competent and trustworthy.