💻 computer science

A Sociotechnical, Practitioner-Centered Approach to Technology Adoption in Cybersecurity Operations: An LLM Case

Through a six-month ethnographic study and sociotechnical co-creation process grounded in Nonaka's SECI model, this paper demonstrates that embedding researchers with cybersecurity practitioners to iteratively develop LLM companion tools effectively overcomes traditional adoption barriers like trust and workflow misalignment, leading to sustained technology integration in security operations centers.

Original authors: Francis Hahn, Mohd Mamoon, Alexandru G. Bardas, Michael Collins, Daniel Lende, Xinming Ou, S. Raj Rajagopalan

Published 2026-04-24

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Francis Hahn, Mohd Mamoon, Alexandru G. Bardas, Michael Collins, Daniel Lende, Xinming Ou, S. Raj Rajagopalan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Building a Better Co-Pilot for Cybersecurity

Imagine a Security Operations Center (SOC) as a high-stress, 24/7 air traffic control tower for a massive airport. The "analysts" are the controllers. Their job is to watch thousands of screens, spot incoming storms (cyberattacks), and guide planes (data) safely to the ground.

For years, these controllers have been overwhelmed. They are drowning in paperwork, juggling too many different radio frequencies (software tools), and trying to find needles in haystacks. They are tired, and they are skeptical of new gadgets. Every time a new "smart" tool is dropped in from the outside, they worry it will break their workflow, give them wrong directions, or make them look foolish if it fails.

The Problem: Researchers often build fancy AI tools in a lab, hand them to the controllers, and say, "Here, use this!" But the controllers usually reject them because the tools don't fit the messy reality of their day-to-day jobs.

The Solution in this Paper: Instead of handing over a finished product, two researchers (PhD students) moved into the control tower for six months. They didn't just watch; they put on the headset, sat in the chairs, and did the job alongside the analysts. They used a method called Co-Creation to build AI tools with the analysts, not for them.

The Journey: How They Did It (The "SECI" Recipe)

The researchers used a recipe for turning "gut feelings" into "working tools" called the SECI Model. Think of this like turning a secret family recipe into a cookbook that everyone can use.

Socialization (The "Coffee Chat"):
- The Metaphor: The researchers hung out with the analysts, watched them work, and listened to their complaints. They learned the "unwritten rules" and the hidden frustrations that never make it into official manuals.
- What happened: They realized the analysts were wasting hours copying and pasting data between five different screens just to answer one simple question.
Externalization (The "Whiteboard Session"):
- The Metaphor: The researchers asked, "If you could wave a magic wand, what would you change?" They turned those vague complaints ("I hate this part") into clear, written problems ("I need a tool that pulls data from Screen A and Screen B automatically").
- What happened: They identified three main pain points: writing complex search queries, finding missing computer assets, and writing incident reports.
Combination (The "Kitchen"):
- The Metaphor: This is where they started cooking. They built three specific AI "assistants" (tools) to solve those three problems.
- The Tools:
  - The Query Builder: A smart assistant that helps write complex search codes (like a spell-checker for hackers).
  - The Asset Detective: A tool that automatically checks which computers are "missing" from the security list.
  - The RCA Writer: A tool that takes messy logs and writes a clean, structured report on what went wrong (Root Cause Analysis).
Internalization (The "Test Drive"):
- The Metaphor: They didn't just launch the tools and leave. They watched the analysts use them, saw where they got stuck, and tweaked the tools.
- The Result: Slowly, the analysts stopped saying "This is annoying" and started saying, "Hey, this saves me 20 minutes." The tools became part of their muscle memory.

The Twist: Why These AI Tools Were Different

Usually, when you give a human a tool, the tool just does what it's told. But these LLM (Large Language Model) tools were different. They were like creative interns rather than robots.

Generative Recombination: Instead of just following a strict rule, the AI could look at past reports, search the company's internal database, and create a brand new summary or search query that didn't exist before. It was like a chef who doesn't just follow a recipe but invents a new dish based on the ingredients in the fridge.
Trust Calibration: Because AI can sometimes "hallucinate" (make things up), the analysts didn't blindly trust it. They treated the AI's output as a draft. They would read it, check the facts, and then sign off on it.
- The Metaphor: Imagine a junior lawyer writing a contract. The senior lawyer (the analyst) doesn't just throw it away; they review it, fix the errors, and then use it. The AI became a "junior partner" that made the senior partner faster, not a replacement.

The Key Takeaways

Don't Build in a Vacuum: You can't build a tool for a specific job unless you sit in the chair of the person doing that job. The researchers succeeded because they became "insiders."
Trust is Earned, Not Given: The analysts were skeptical at first. Trust was built slowly by showing that the tools actually saved time and didn't break anything.
AI is a Co-Pilot, Not the Pilot: The goal wasn't to replace the security experts with AI. The goal was to give them a "super-power" so they could focus on the hard thinking while the AI handled the boring data crunching.
The "Human-in-the-Loop" is Essential: The AI generates the ideas, but the human validates them. This "check-and-balance" system is what made the technology safe and reliable enough for a high-stakes environment.

The Bottom Line

This paper proves that the best way to bring advanced AI into a workplace isn't to drop a shiny new gadget on their desk. It's to roll up your sleeves, work alongside the team, understand their daily struggles, and build a tool that fits perfectly into their life. When you do that, the "magic" of AI stops being scary and starts being helpful.

1. Problem Statement

Security Operations Centers (SOCs) face significant challenges in adopting new technologies, particularly Artificial Intelligence (AI) and Large Language Models (LLMs). Key issues include:

Trust and Reliability: Analysts are skeptical of AI due to hallucinations, inconsistent outputs, and a lack of transparency.
Workflow Disruption: New tools often disrupt established, high-stress workflows without providing tangible efficiency gains.
The Research-Practice Gap: Academic AI solutions often fail in practice because they are designed without accounting for the tacit knowledge, operational constraints, and iterative reasoning of SOC analysts.
Data Fragmentation: Analysts struggle with siloed data, repetitive manual tasks (e.g., log correlation, query building), and fragmented tooling across multiple platforms.

The core problem is how to co-create and integrate LLM-based tools into live SOC environments in a way that builds trust, aligns with operational workflows, and improves efficiency without compromising security or privacy.

2. Methodology

The study employed a sociotechnical, practitioner-centered approach grounded in embedded ethnography and the Tool-Oriented SECI model (Socialization, Externalization, Combination, Internalization) by Nonaka.

Fieldwork: Two PhD researchers were embedded within a multinational company's SOC for six months. They engaged in participant observation, attended meetings, and performed analyst tasks to gain "insider" perspectives.
Co-Creation Process: Instead of delivering a pre-built solution, researchers collaborated directly with 18 SOC analysts (including CISO and directors) to identify pain points and iteratively develop tools.
SECI Framework Application:
1. Socialization (Tacit $\to$ Tacit): Researchers built trust and understood implicit workflows through sustained interaction and shadowing.
2. Externalization (Tacit $\to$ Explicit): Pain points (e.g., repetitive log analysis) were articulated through interviews and workflow walkthroughs.
3. Combination (Explicit $\to$ Explicit): Researchers developed LLM-based tools incorporating these explicit requirements.
4. Internalization (Explicit $\to$ Tacit): Tools were deployed, and analysts iteratively refined them, eventually internalizing the tools into their daily routines.
Technical Stack:
- Architecture: A central LLM Security Platform using the LangChain framework and ReAct (Reasoning + Acting) agents for tool dispatching.
- Retrieval-Augmented Generation (RAG): Used to ground LLM outputs in organization-specific data (playbooks, query examples, RCA reports) stored in vector databases (Chroma).
- Models: Comparative testing of Llama 3.2, Mistral 7B, Qwen3, and GPT-OSS. Mistral 7B was selected for its balance of performance and resource constraints (16GB GPU slice).
- Agents: Four agents were designed: Asset Discovery, Query Builder, MCP (Model Context Protocol) Agent, and a ReAct Agent for orchestration.

3. Key Contributions

The paper makes four primary contributions:

Structured Co-Creation Framework: A methodology for developing LLM tools in live operational environments by integrating ethnographic fieldwork with iterative software development.
Thematic Analysis of Pain Points: Systematic identification of specific SOC bottlenecks (Root Cause Analysis, Query Building, Asset Discovery) that are amenable to LLM augmentation.
Staged Integration Strategy: A demonstration of how trust is built incrementally through feedback loops, moving from skepticism to sustained adoption.
Extension of the SECI Model: The proposal of an LLM-Oriented SECI Model that introduces two new dynamics:
- Generative Recombination: LLMs do not just store explicit knowledge; they actively generate new structured artifacts (e.g., RCA reports, queries) by recombining existing data.
- Trust-Calibrated Internalization: Explicit LLM outputs do not automatically become tacit knowledge; they must pass through human validation (interpretability, reliability, workflow fit) before being internalized.

4. Results and Findings

The six-month engagement resulted in the development and deployment of three specific tools and an integrated platform:

Root Cause Analysis (RCA) Tool:
- Function: Converts heterogeneous incident artifacts (JSON, logs) into structured narratives.
- Outcome: Significantly reduced time for new recruits and experienced analysts to draft initial investigation reports. It enforced constraints to prevent hallucinations (e.g., stating "Not available in provided data" rather than inferring).
Query Builder:
- Function: Assists in constructing complex SIEM queries using a RAG system populated with analyst-specific examples.
- Outcome: Initially struggled with generic examples; success came only after analysts contributed their own workflow-derived examples to the vector store, creating a "living database" of shared experience.
Asset Discovery Tool:
- Function: Automates the detection of assets failing to report to the security vendor by correlating API data with internal databases.
- Outcome: Overcame initial policy hurdles regarding autonomous API access. It successfully identified "blind spots" in asset reporting.
Adoption Metrics:
- Shift in Perception: Analysts moved from skepticism to viewing the tools as "companions" rather than replacements.
- Usage Patterns: Tools were used on-demand for specific tasks (e.g., "I don't use this every day, but when I need it, it helps").
- Trust Calibration: Adoption was driven by verifiability. Analysts trusted the tools because they could inspect the evidence (citations, retrieved logs) behind the LLM's suggestions.

5. Significance

Paradigm Shift in Tool Development: The paper argues against top-down, vendor-driven AI deployment. It demonstrates that sociotechnical co-creation is essential for overcoming the "trust gap" in cybersecurity.
Redefining the Role of LLMs: LLMs in SOCs should not be viewed merely as automation engines but as dynamic collaborators in knowledge production. They act as a "generative recombination layer" that accelerates the conversion of tacit expertise into explicit, actionable artifacts.
Human-in-the-Loop Necessity: The study confirms that for generative AI to be adopted in high-stakes environments, the internalization process must be mediated by human judgment. The "trust-calibrated internalization" phase is critical; analysts must validate outputs before they become part of standard operating procedures.
Practical Viability: The research proves that LLMs can be successfully integrated into live, remote SOCs using modest hardware (e.g., 16GB GPU) and local hosting (Ollama) to maintain data privacy and governance compliance.

In conclusion, the paper establishes that successful LLM adoption in cybersecurity is not solely a technical challenge but a sociotechnical one, requiring deep collaboration, iterative refinement, and a framework that respects the human element of threat detection.