💻 computer science

Prediction Model of Motivators and Demotivators of Integrating Large Language Models in Software Engineering Education: An Empirical Study

This study develops and validates a cost-aware prediction model that integrates stakeholder perceptions of LLM motivators and demotivators with probabilistic analysis and genetic algorithm optimization to guide the strategic, governance-focused integration of Large Language Models in software engineering education.

Original authors: Maryam Khan, Muhammad Azeem Akbar, Jussi Kasurinen, Estefanía Martín-Barroso

Published 2026-05-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Maryam Khan, Muhammad Azeem Akbar, Jussi Kasurinen, Estefanía Martín-Barroso

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are the principal of a school, and a new, incredibly powerful tool has just arrived: a "Super-Brain" (Large Language Model, or LLM) that can write code, solve math problems, and explain complex ideas instantly. You want to let your students use it, but you're worried. If you let them use it too freely, they might cheat or stop thinking for themselves. If you ban it, you're missing out on a huge learning opportunity.

This paper is like a strategic blueprint for school leaders trying to figure out the perfect balance. Instead of just guessing, the authors built a mathematical "GPS" to help them decide where to spend their limited budget and energy.

Here is how they did it, explained simply:

1. The Map: What Do People Actually Think?

First, the researchers didn't just guess what teachers and students wanted. They asked 126 experts from universities around the world (like Finland, Saudi Arabia, and China) to fill out a survey.

Think of this as asking a group of experienced hikers: "What are the best trails (Motivators) and what are the dangerous cliffs (Demotivators) on this mountain?"

The Good Trails (Motivators): The hikers said the Super-Brain is amazing for helping with debugging code (fixing mistakes), giving personalized help (like a private tutor), and acting as a learning partner.
The Dangerous Cliffs (Demotivators): The biggest fears were cheating/plagiarism, students relying too much on the tool so they stop thinking, and the tool lying (giving wrong answers or biased info).

2. The Compass: The Prediction Model

The researchers took these opinions and fed them into a computer program. They didn't just want to know what people thought; they wanted to know: "If we focus on fixing these specific problems, how likely is it that the school will successfully adopt this technology?"

They used two types of "guessing engines" (Naïve Bayes and Logistic Regression) to create a probability map. Imagine this map as a weather forecast. It doesn't tell you exactly what the weather will be, but it tells you the chance of rain based on current conditions. Here, it calculates the chance of successful adoption based on how much effort you put into different areas.

3. The Budget: The Genetic Algorithm (The "Smart Shopper")

Here is the clever part. Schools have limited money and time. You can't fix every single problem at once. You have to choose.

The researchers used a computer technique called a Genetic Algorithm. Think of this as a super-smart shopper who is trying to buy the best groceries for a dinner party with a strict budget.

The "groceries" are the different factors (like "fixing cheating policies" or "teaching students how to code").
The "budget" is the school's effort and cost.
The "dinner party success" is the likelihood of the students actually learning well with the new tool.

The computer tried thousands of different combinations of "groceries" to find the one that gave the best "dinner party" (high success) for the lowest "cost" (effort).

4. The Results: What Should Schools Do First?

The "Smart Shopper" gave a very specific shopping list. It didn't tell the school to buy everything. It told them to prioritize in a specific order:

Step 1: Build the Fence (Governance First)
Before you let students run wild with the Super-Brain, you must build a strong fence. The model said the most cost-effective first step is to fix rules about cheating and ethics.

Analogy: You wouldn't open a swimming pool to the public without putting up a fence and hiring a lifeguard first. The study says schools must establish integrity policies and ethical guidelines before trying to teach with the tool.

Step 2: Focus on Deep Thinking, Not Just Speed
Once the rules are set, the model suggested focusing on deep learning (understanding concepts and problem-solving) rather than just using the tool to speed up homework.

Analogy: Don't just use the Super-Brain to write the essay for the student. Use it to help the student understand how to write a better essay.

Step 3: Ignore the "Nice-to-Haves" for Now
The model suggested that things like "making the course more fun" or "changing the whole curriculum structure" are less urgent. They are good, but they don't give you as much "bang for your buck" as fixing the cheating rules first.

The Bottom Line

This paper argues that integrating AI into education isn't about buying the most expensive software or changing every class immediately. It's about strategy.

The study concludes that if a school wants to use AI successfully without spending a fortune, they should:

Start with the rules: Stop the cheating and set ethical boundaries first.
Focus on the mind: Use the AI to help students think deeper, not just to do the work for them.
Skip the fluff: Don't waste money trying to overhaul the whole school system at once; start with the high-impact, low-cost fixes.

It's a recipe for a "staged" approach: Build the foundation (rules), then add the furniture (teaching methods), and only then worry about the decoration (fun activities).

Technical Summary: Prediction Model of Motivators and Demotivators of Integrating Large Language Models in Software Engineering Education

Problem Statement
While Large Language Models (LLMs) are rapidly transforming software engineering practice, their systematic integration into software engineering education remains underexplored. Existing research largely focuses on technical performance evaluations, classroom case studies, or descriptive discussions of benefits and risks. There is a significant gap in empirical, cost-aware models that guide institutions in making strategic decisions about LLM adoption. Specifically, there is a lack of frameworks that quantify the trade-offs between the perceived likelihood of successful integration (familiarity) and the institutional effort or cost required to implement specific motivators and mitigate demotivators.

Methodology
The study employs a four-phase empirical and computational methodology to address this gap:

Taxonomy Development: Building on a prior systematic literature review, the authors operationalized 19 validated factors (9 motivators and 10 demotivators) derived from a literature survey. These factors were categorized into four main themes for motivators (e.g., Enhancing Learning Experiences, Skill Development) and four for demotivators (e.g., Learning Challenges, Academic Integrity).
Data Collection: A structured survey was administered to 126 stakeholders in software engineering education across multiple countries (primarily Finland, Saudi Arabia, Pakistan, and China). Respondents rated the perceived impact and implementation cost/effort of each factor using a 9-point Likert scale.
Probabilistic Modeling: The survey data was used to train two probabilistic classifiers: Naïve Bayes (NB) and Logistic Regression (LR). The target variable was a binary indicator of "high LLM familiarity" (ratings 7–9) versus moderate/low familiarity. The models were not optimized for high classification accuracy but to generate stable probability estimates ( $\hat{P}(Y=1|s)$ ) for any given configuration of factor intensities. An aggregated probability estimate was computed by averaging the outputs of both models to reduce dependency on a single modeling assumption.
Optimization Framework: A Genetic Algorithm (GA) was employed to solve a single-objective optimization problem. The objective function maximized the difference between the aggregated probability of high familiarity and the normalized implementation cost:
$F(s) = \hat{P}_{agg}(s) - C_{norm}(s)$
where $s$ represents a configuration of factor levels (1–9), and cost is linearly proportional to the selected intensity level. The GA performed optimization at three levels: global (across all 19 factors), category-level (across the 8 main themes), and factor-level (within categories).

Key Results

Descriptive Findings: Respondents identified "Programming Assistance and Debugging Support" and "Personalized and Adaptive Learning" as the strongest motivators. Conversely, "Plagiarism and Intellectual Property Concerns," "Over-Reliance on AI," and "Reduced Critical Thinking" were the most significant demotivators.
Predictive Modeling: The probabilistic models achieved a classification accuracy of 0.461, which is below the majority-class baseline. The authors interpret this not as a failure, but as evidence that perception patterns regarding LLM benefits and risks are broadly similar across respondents regardless of their familiarity level. Consequently, the models serve as a continuous probability surface for optimization rather than a predictive tool for individual classification.
Global Optimization (GC): The GA identified that the "average" strategy (based on mean survey ratings) is not cost-efficient. The optimal global configuration prioritizes governance and integrity mechanisms. Specifically, "Plagiarism and Intellectual Property Concerns" (Level 8) and "Ethical Concerns in AI-Assisted Learning" (Level 6) received the highest intensity allocations. Most pedagogical factors (e.g., Programming Assistance, Personalized Learning) were minimized (Level 1) in the global model to control total cost, suggesting that governance foundations must precede broad pedagogical expansion.
Category-Level Optimization: Pedagogically oriented categories, specifically "Collaboration and Peer Learning" and "Enhancing Learning Experiences," yielded the highest marginal efficiency gains ( $\Delta$ Fitness) with moderate effort. In contrast, structurally intensive domains like "Assessment and Academic Integrity" and "Integration and Practical Implementation" required higher effort for smaller relative gains.
Factor-Level Optimization: Within categories, the model prioritized deep cognitive factors (e.g., "Conceptual Understanding," "Software Engineering Process Understanding") over surface-level engagement. For demotivators, technical reliability (e.g., "Bias and Hallucination") and security were prioritized over behavioral regulation mechanisms.

Key Contributions

Unified Framework: The study introduces a hierarchical, optimization-informed decision-support framework that links stakeholder perceptions with probabilistic modeling and cost-effort analysis.
Methodological Integration: It demonstrates a novel pipeline combining literature-based taxonomies, multi-stakeholder empirical validation, and evolutionary optimization (GA) to guide educational technology integration.
Cost-Aware Strategy: The research provides a data-driven mechanism to identify cost-efficient strategies, moving beyond descriptive lists of pros and cons to actionable, prioritized implementation plans.
Staged Implementation Logic: The results support a staged approach where governance and integrity safeguards are established first, followed by cognitively meaningful pedagogical interventions, rather than uniform adoption across all domains.

Significance and Claims
The authors claim that this study advances the field from conceptual taxonomy development to data-driven validation and computational decision support. The significance lies in providing institutions with a structured mechanism to balance predicted outcomes with implementation effort. The paper asserts that effective LLM integration is not achieved by uniformly addressing all factors but by selectively prioritizing high-impact, trust-oriented, and cognitively substantive factors.

The study modestly acknowledges that the predictive models are not intended for high-accuracy individual classification but serve as the analytical foundation for the optimization framework. The findings suggest that under resource constraints, institutions should prioritize governance stability (plagiarism policies, ethical guidelines) and deep learning processes (conceptual understanding, process literacy) over broad, resource-intensive structural reforms or superficial engagement tools. The proposed framework offers a replicable approach for future research on AI adoption and governance modeling in computing education.