Empirically Derived AI Harm Profiles and the Case for Context-Sensitive Moral Responsibility

Artificial intelligence is no longer a distant promise; it is a daily reality making decisions that affect our safety, our jobs, our privacy, and our access to services. When these systems work, they streamline our lives. When they fail, the consequences can be severe, ranging from physical injury to the denial of civil rights. For years, experts have tried to govern these technologies using broad ethical principles like fairness, transparency, and safety. However, these high-level ideas often struggle to translate into practical rules for the messy, varied ways AI is actually used in the real world. A key challenge has been understanding how different types of AI fail in specific situations. Is a self-driving car that crashes the same kind of problem as a chatbot that spreads misinformation, or a facial recognition system that misidentifies a suspect? To answer this, researchers have begun looking at documented failures not as isolated accidents, but as data points that might reveal deeper patterns.

A team of researchers at the University of the Cumberlands set out to find those patterns by examining a vast collection of real-world AI incidents. They gathered 849 documented cases of AI failures from a public database known as the AIAAIC Repository, which tracks incidents involving artificial intelligence, algorithms, and automation. Instead of treating each event as a unique story, the researchers analyzed how these failures clustered together based on what the technology was, where it was used, what it was supposed to do, and what kind of harm it caused. Their goal was to see if the chaos of individual accidents could be organized into a few distinct categories, or "harm profiles," that would help regulators and companies build better, more specific rules for oversight.

The analysis revealed that AI failures are not random. They tend to group around two main structural themes. The first theme distinguishes between systems that interact directly with the physical world and those that primarily process information. The second theme separates systems designed to identify and monitor people from those designed to generate content. By mapping these relationships, the researchers identified three clear, recurring profiles of harm that emerge when things go wrong.

The first profile is what the researchers call Autonomous or Embodied AI. These are systems that have a physical presence and can cause direct physical harm, such as self-driving vehicles or robots. In the data, these failures almost always happened in the transportation sector. When these systems fail, the result is typically a safety failure: a car crashes, a robot malfunctions, or a human operator loses control. The risks here are immediate and physical, affecting the safety of passengers, pedestrians, and emergency responders. Because the consequences are tangible and dangerous, the researchers suggest that governance for this group must focus heavily on operational safety, rigorous testing of hardware and sensors, and clear rules for who is responsible when a physical accident occurs.

The second profile is Vision Recognition. This category includes systems that use cameras and software to identify people, such as facial recognition technology used by law enforcement or security agencies. The data showed that failures in this area are concentrated in the justice and law enforcement sectors. Unlike the physical crashes of the first group, the harms here are deeply tied to civil rights and discrimination. The most common issues involve the system misidentifying people, particularly from specific demographic groups, or using surveillance in ways that exceed legal boundaries. The researchers found that these failures create a different kind of risk: a threat to privacy, fairness, and the right to be free from wrongful accusation. Governance for this profile, therefore, needs to focus on preventing bias, ensuring transparency in how the systems are used, and providing clear paths for people to challenge incorrect identifications.

The third profile is Generative AI. This is the technology behind tools that create text, images, and other media. The data showed that failures here are most common in the media and information sectors. The harms associated with this group are distinct from the others; they rarely involve physical injury or direct surveillance. Instead, the problems revolve around the quality and truthfulness of information. Failures include the creation of convincing but false content, the impersonation of real people, and the spread of misinformation that can damage reputations or manipulate public opinion. Because the risk is to the integrity of information rather than physical safety or immediate surveillance, the researchers argue that oversight for this group should focus on verifying the source of content, monitoring for deceptive outputs, and ensuring that creators and platforms are accountable for what their systems produce.

Interestingly, the researchers found that one common type of AI capability, predictive monitoring, did not form its own separate group. Predictive monitoring refers to systems that analyze data to forecast future events, such as predicting crime, loan defaults, or health outcomes. The study showed that these systems did not cluster together because their risks depend entirely on where they are used. A predictive system used in healthcare faces different ethical challenges than one used in hiring or finance. This finding suggests that predictive monitoring is not a single category of risk but a tool that takes on the character of the environment it enters. If a predictive system is used in a law enforcement context, it inherits the risks of the Vision Recognition profile; if used in finance, it inherits the economic risks of other profiles. This means that rules for these systems cannot be one-size-fits-all; they must be tailored to the specific context in which the prediction is being made.

The researchers propose a new way to govern AI based on these findings. Instead of applying the same broad rules to all artificial intelligence, they suggest that regulators and companies should first identify which harm profile a system fits into. Once a system is categorized, the specific rules for monitoring, auditing, and assigning responsibility can be applied. For example, a self-driving car would be subject to strict safety audits and immediate reporting requirements for physical near-misses. A facial recognition system would be required to undergo regular checks for racial bias and provide clear avenues for citizens to appeal misidentifications. A generative AI platform would need to prove the authenticity of its content and have mechanisms to quickly remove harmful misinformation.

This approach moves AI governance from abstract principles to concrete actions. It acknowledges that a failure in a robot is fundamentally different from a failure in a chatbot, and that the people responsible for fixing them must be different as well. By looking at the actual history of how AI has failed, the researchers have provided a map for building a more effective, context-sensitive system of oversight. They argue that this method allows society to address the specific dangers of each type of technology without getting lost in generalities, ensuring that the rules we create are as practical and precise as the technology itself. The study concludes that by treating documented incidents as a resource rather than just a record of past mistakes, we can build a future where AI is governed by an understanding of its real-world impact, making the technology safer and more accountable for everyone.