💻 computer science

Exploring Individual Factors in the Adoption of LLMs for Specific Software Engineering Purposes

This study investigates how individual cognitive and behavioral factors, analyzed through the UTAUT2 framework, differentially influence software engineers' adoption of Large Language Models for specific tasks, revealing that distinct purposes are driven by unique and sometimes conflicting determinants.

Original authors: Stefano Lambiase, Gemma Catolino, Fabio Palomba, Filomena Ferrucci, Daniel Russo

Published 2026-04-15

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Stefano Lambiase, Gemma Catolino, Fabio Palomba, Filomena Ferrucci, Daniel Russo

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine software development as a massive, bustling kitchen where chefs (software engineers) are constantly cooking up complex dishes (code, documentation, and apps). For a long time, they've had to chop, mix, and season everything by hand.

Now, enter LLMs (Large Language Models). Think of these as super-intelligent, hyper-fast sous-chefs that can instantly chop vegetables, suggest recipes, or even taste-test the sauce.

But here's the big question the paper asks: Just because these sous-chefs are available, do the chefs actually use them? And if they do, why do they use them for some tasks but not others?

The researchers didn't just ask, "Do you like AI?" They dug deeper, asking, "Do you use AI to chop onions? To invent a new recipe? To check if the soup is salty? Or to learn how to bake a cake?"

Here is the breakdown of their findings, served with some tasty analogies.

1. The "Habit" is the Real Boss

The biggest surprise? Habit is the strongest driver of all.

The Analogy: Think of your morning coffee routine. You don't sit there calculating the "performance expectancy" (will this coffee make me faster?) or the "social influence" (do my friends like coffee?). You just do it because it's what you do.
The Finding: If a developer has made using AI a habit—like checking their phone first thing in the morning—they will use it for almost everything. If they haven't built that habit, they won't use it, no matter how good the tool is. Habit is the engine; everything else is just the fuel.

2. Not All Tasks Are Created Equal

The study found that the "reasons" for using AI change depending on what you are trying to do. It's like having a Swiss Army Knife: you use the screwdriver for screws, the scissors for paper, and the bottle opener for beer. You don't use the same tool for the same reason every time.

Here is how the "reasons" (factors) changed for different tasks:

A. Making New Things (Generating Code/Docs)

The Task: Writing a new file or a paragraph of text.
The Driver: Habit.
The Analogy: This is like using a power drill. Once you get used to the drill, you just grab it. You don't need a friend to tell you to use it, and you don't need a special permit. If you're in the habit of using the drill, you just use it.
Takeaway: To get people to write code with AI, just make the tool so easy to access that it becomes part of their daily routine (like a plugin in their code editor).

B. Making Variations (Generating Alternatives)

The Task: "Give me three different ways to write this function."
The Driver: Habit + Ease of Use.
The Analogy: Imagine asking a sous-chef for three different ways to slice a carrot. If the knife is dull or the process is clunky, the chef will just slice it themselves. They need the tool to feel effortless.
Takeaway: If the AI is hard to use for creative variations, people will stop using it. It must be frictionless.

C. Looking Up Facts (Information Retrieval)

The Task: "How do I fix this specific error?" or "What does this library do?"
The Driver: Habit + Social Influence.
The Analogy: This is like asking for directions. You might know the way, but if your best friend says, "Trust me, take this route," you are much more likely to listen.
Takeaway: For looking up facts, developers need to see their peers using it. If the team lead or a respected colleague says, "I use AI to check my facts," others will follow. It's about trust through community.

D. Making Big Decisions (Decision Support)

The Task: "Should we use Database A or Database B for this project?"
The Driver: Habit + Ease of Use + Social Influence.
The Twist: Interestingly, having too much external support (like a huge team of human experts) actually made people less likely to use AI for decisions.
The Analogy: If you are in a room full of 10 brilliant experts, you might ask them for advice rather than asking a robot. But if the experts are busy, and the robot is easy to talk to, you'll ask the robot.
Takeaway: AI works best for big decisions when it's easy to use and when the team trusts it. But if there are too many human alternatives, the AI gets ignored.

E. Learning New Skills (Training)

The Task: "Teach me how this new framework works."
The Driver: Habit.
The Twist: This was the hardest category to explain. Even having a supportive environment (like a great training department) sometimes made people avoid using AI to learn.
The Analogy: If your company has a fantastic library and a great teacher, you might prefer the human teacher over a chatbot. But if you are in the habit of chatting with the bot, you'll use it anyway.
Takeaway: We still don't fully understand why people use AI to learn, but making it a habit is the only thing that seems to work consistently.

3. What Doesn't Matter as Much

Social Pressure: For most tasks, it doesn't matter if your boss says "You must use AI." If you don't want to use it, you won't.
Performance Expectancy (Will it make me faster?): Surprisingly, knowing that AI can make you faster didn't always make people use it. They needed to feel the habit first.

The Big Picture: A Recipe for Success

The paper concludes that you can't treat all software engineers the same. You can't just say, "Here is a magic wand, use it!"

For Coders: Just integrate the tool into their workflow so it becomes a habit.
For Fact-Checkers: Get the team leaders to endorse it so people trust it.
For Decision Makers: Make sure the tool is reliable and easy to use, but don't force it if they have great human experts.

In short: AI in software engineering isn't a "one-size-fits-all" solution. It's a toolbox. Some tools are used because we are used to them (Habit), some because our friends use them (Social Influence), and some because they are easy to grab (Ease of Use). To get the most out of AI, you have to match the tool to the specific job and the specific person holding it.

1. Problem Statement

While Large Language Models (LLMs) are rapidly transforming software engineering (SE), existing research predominantly focuses on general adoption trends or broad technology acceptance. There is a significant gap in understanding how individual cognitive and behavioral factors influence the adoption of LLMs for specific SE purposes (e.g., code generation vs. decision support vs. training).

Current studies often treat LLM adoption as a monolithic behavior, failing to account for the nuance that a developer might be highly motivated to use an LLM for generating boilerplate code but hesitant to rely on it for critical architectural decisions. This lack of granularity hinders the development of tailored tools (e.g., specialized GenAI agents) and prevents team leaders from devising effective, purpose-specific adoption strategies.

2. Methodology

The study employs a quantitative, cross-sectional survey design analyzed using Partial Least Squares Structural Equation Modeling (PLS-SEM).

Participants: The study surveyed 188 software engineers recruited via Prolific. The sample was heterogeneous, including roles such as software developers, data engineers, and team leaders, with varying levels of experience (from <1 year to >10 years) and global representation (primarily UK, US, Canada, Germany, Spain).
Theoretical Framework:
- Independent Variables (Predictors): The study utilizes the UTAUT2 (Unified Theory of Acceptance and Use of Technology 2) model. Constructs include:
  - Performance Expectancy (PE)
  - Effort Expectancy (EE)
  - Social Influence (SI)
  - Hedonic Motivation (HM)
  - Facilitating Conditions (FC)
  - Habit (Hb)
  - Behavioral Intention (BI)
- Dependent Variables (Outcomes): Based on the taxonomy by Khojah et al. [12], LLM usage was categorized into five distinct purposes:
  1. Manipulate Artifacts (UB1): Generating/modifying code, docs, etc.
  2. Generate Alternatives (UB2): Creating alternative versions of artifacts.
  3. Information Retrieval (UB3): Extracting/synthesizing info from external sources.
  4. Decision Support (UB4): Facilitating decision-making via recommendations.
  5. Training (UB5): Learning new concepts via conversation.
Data Analysis:
- Measurement Model: Validated for reliability (Cronbach's $\alpha$ , Composite Reliability) and validity (Convergent Validity via AVE, Discriminant Validity via HTMT).
- Structural Model: Tested hypotheses regarding direct, indirect (mediated by BI and general Usage Behavior), and total effects.
- Technique: PLS-SEM was chosen for its ability to handle complex models with multiple dependent variables and non-normal data distributions.

3. Key Contributions

Purpose-Specific Granularity: The study moves beyond "general LLM adoption" to map specific individual factors to five distinct usage scenarios, revealing that drivers of adoption are not uniform across different tasks.
Refinement of UTAUT2 in SE: It validates and extends the UTAUT2 framework within the context of Generative AI, demonstrating how specific constructs (like Social Influence) behave differently depending on the task complexity and risk.
Identification of Counter-Intuitive Dynamics: The research uncovers that factors like "Facilitating Conditions" (organizational support) can sometimes have a negative relationship with adoption for high-stakes tasks (Decision Support/Training), suggesting that abundant traditional resources may reduce reliance on AI.
Actionable Guidelines: Provides specific, evidence-based recommendations for tool designers and managers on how to foster adoption for specific use cases (e.g., integrating into IDEs for artifacts vs. peer-validation for information retrieval).

4. Key Results

The analysis revealed distinct patterns for each usage purpose:

Habit (Hb) is the Universal Driver: Habit was the strongest predictor across all five purposes. Once developers form a habit of using LLMs, they are likely to use them for any purpose.
Manipulate Artifacts (UB1):
- Drivers: Primarily driven by Habit and General Usage Frequency (UB).
- Insight: This is the most "intuitive" use case; it requires the least amount of specific persuasion and is deeply embedded in workflows.
Generate Alternatives (UB2):
- Drivers: Habit and Effort Expectancy (EE).
- Insight: Perceived ease of use is critical here. If the tool is seen as difficult, developers revert to manual generation of alternatives.
Information Retrieval (UB3):
- Drivers: Social Influence (SI) plays a unique, direct, and unmediated role.
- Insight: Adoption for retrieving information is heavily dependent on peer endorsement and team norms. Developers trust LLMs for facts only when their peers validate the tool.
Decision Support (UB4):
- Drivers: Habit, Effort Expectancy, and Social Influence (all direct effects).
- Surprising Finding: Facilitating Conditions (FC) had a negative direct relationship. In environments with strong external support/resources, developers were less likely to use LLMs for decision-making, preferring traditional resources over AI.
Training (UB5):
- Drivers: Habit and General Usage.
- Insight: This category had the lowest explanatory power ( $R^2 = 0.289$ ), suggesting that the UTAUT2 model alone is insufficient to explain learning behaviors; other factors (e.g., learning motivation) are likely at play.

Mediation Effects:

For routine tasks (Artifacts), adoption is often mediated through general usage frequency.
For complex tasks (Decision Support, Information Retrieval), factors like Social Influence and Effort Expectancy often have direct effects, bypassing general intention.

5. Significance and Implications

For Tool Designers:
- Artifact Tools: Focus on seamless integration (IDE plugins) to reinforce habit.
- Alternative Generation: Prioritize usability and low-friction interfaces.
- Info Retrieval/Decision Support: Focus on trust mechanisms (source attribution, confidence scores) and design for social validation (e.g., features that allow sharing/reviewing LLM outputs with peers).
For Managers & Organizations:
- Strategy: Do not apply a "one-size-fits-all" adoption strategy.
- Knowledge Sharing: To boost Information Retrieval and Decision Support adoption, organizations should leverage "boundary spanners" (influential peers) to endorse the technology.
- Resource Management: Be aware that providing too many traditional resources might inadvertently suppress LLM adoption for critical decision-making tasks.
For Research:
- Highlights the need for interpretivist approaches to understand the "situated" nature of LLM use, as quantitative models alone cannot fully capture the complexity of trust and hallucination management.
- Calls for future studies to investigate the impact of hallucinations on long-term adoption and the role of organizational policies.

In conclusion, the paper establishes that LLM adoption in software engineering is a heterogeneous, context-dependent phenomenon. Success in integrating these tools requires understanding that the factors driving a developer to generate code are fundamentally different from those driving them to make architectural decisions or learn new concepts.