Bridging Cybersecurity Expertise Gaps through Human-LLM Collaboration: Trust, Reliance, and Design Imperatives from a Mixed-Methods Study
This mixed-methods study of 58 cybersecurity professionals reveals that while human-LLM collaboration improves triage accuracy, the degree of reliance on AI advice varies significantly based on task transparency, leading to specific design imperatives for building trustworthy, context-aware cybersecurity assistants that mitigate automation bias and support calibrated human judgment.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the digital world, security teams face a constant flood of alarms. Every day, they must sort through thousands of suspicious emails and network alerts to decide which ones are harmless noise and which are genuine attacks. This process, known as triage, is often done by people who are tired, under pressure, and sometimes lacking deep expertise. As the number of cyber threats grows faster than the number of trained experts, organizations are turning to artificial intelligence to help. Specifically, they are testing large language models—computer programs that can read, write, and reason with human language—to act as assistants. The hope is that these tools can spot dangers that humans miss. However, a new question has emerged: if a computer gives advice, will a human trust it too much, or not enough? The real challenge is not just whether the computer is smart, but whether the human can understand the computer's reasoning well enough to make the final call.
A team of researchers from the Commonwealth Scientific and Industrial Research Organisation in Australia set out to explore this relationship between human workers and artificial intelligence. They did not simply test how well a computer could find bad emails or network attacks on its own. Instead, they studied how people and computers worked together. They recruited 58 participants, many of whom had technical knowledge but were not professional cybersecurity analysts. These volunteers were asked to perform two very different types of security checks. The first task involved reading emails to see if they were phishing attempts—fraudulent messages designed to steal information. These emails contain visible clues, such as strange sender addresses or suspicious links, that a person can see and check for themselves. The second task involved looking at network records, which are tables of data showing how computers are talking to each other. These records are much harder to read; the clues are hidden in complex numbers and patterns that are not obvious to the naked eye.
In the study, participants first made their own decisions about whether an email or a network record was safe or dangerous. Then, they were given the chance to work with an AI assistant. The assistant could explain its reasoning, point out specific details, and offer a recommendation. The researchers watched closely to see how the humans reacted. They found that the way people used the AI depended entirely on how easy it was for them to check the evidence for themselves. When the task was the email check, where the clues were visible, the participants used the AI as a helpful second opinion. They could look at the email, see the suspicious link the AI mentioned, and decide for themselves whether to agree. The AI helped them catch a few more mistakes, but the humans remained in charge.
The situation changed dramatically when the participants moved to the network records. Because the clues were hidden inside complex data, the humans could not easily verify the AI's claims on their own. In this setting, the AI's confidence became the most powerful factor. When the AI spoke with certainty, the humans tended to follow its advice, even if the AI was wrong. The researchers observed that confident but incorrect advice led to more errors than hesitant advice did. This created a dangerous gap: in the tasks where people needed the most help, they were also the most likely to blindly trust the machine. The study showed that while the AI improved the overall accuracy of the group, it did so by changing how people thought. In the difficult network task, the AI acted less like a tool and more like a guide, leading the human through a path they could not see alone.
The researchers also looked at how people asked questions. When checking emails, people asked direct questions like, "Is this a phishing email?" But when looking at network data, they asked more exploratory questions, such as, "What makes this connection unusual?" or "Can you compare this to normal traffic?" This shift showed that for complex problems, people did not just want a yes-or-no answer; they wanted to understand the reasoning behind it. The study found that the most helpful AI responses were those that pointed to specific evidence and explained why that evidence mattered. When the AI simply gave a label without proof, or when it spoke with too much confidence about uncertain situations, the humans made worse decisions.
The findings suggest that for artificial intelligence to be truly useful in cybersecurity, it must be designed differently depending on the job. For simple tasks where humans can see the evidence, the AI should be a quiet helper that highlights details. For complex tasks where the evidence is hidden, the AI must be a careful partner that explains its uncertainty and encourages the human to double-check. The study concluded that the goal is not to replace human experts with machines, but to create a partnership where the machine supports human judgment without overriding it. By teaching the AI to show its work and to admit when it is unsure, security teams can bridge the gap between the growing number of threats and the limited number of experts available to stop them. The research indicates that the future of security lies not in smarter algorithms alone, but in smarter ways for humans and machines to talk to each other.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.