🤖 AI

GenAI Integration into Engineering Education: A Case Study of an Introductory Undergraduate Engineering Course

This case study of an introductory undergraduate engineering course reveals that while integrating Generative AI initially boosted student performance and provided valuable insights for instructors, the technology's usage remained superficial and declined over time, suggesting that encouraging faculty experimentation is crucial for navigating the unknowns of emerging educational tools.

Original authors: Kadir Kozan, Ozgur Keles, Sihan Jian, Serkan Ayvaz, Krzysztof Sierszecki, Sewon Joo

Published 2026-03-17

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Kadir Kozan, Ozgur Keles, Sihan Jian, Serkan Ayvaz, Krzysztof Sierszecki, Sewon Joo

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are the captain of a ship (the Engineering Professor) sailing through uncharted waters with a crew of junior officers (the Teaching Assistants) and a group of eager cadets (the Students). For years, you've navigated using a paper map and a compass. Suddenly, a new, high-tech GPS system (Generative AI) appears on the horizon.

This research paper is essentially a logbook of that captain and crew's first voyage using this new GPS. They didn't just turn it on and hope for the best; they tried to figure out how to use it, what it could do, and where it might lead them off course.

Here is the story of their journey, broken down into simple concepts:

1. The Experiment: "Let's Try It Out"

The professor decided to let the students use an AI coding assistant (called Copilot) during their practice sessions. Think of this like giving the cadets a smart calculator that doesn't just do math, but also explains why the answer is what it is.

The Goal: To see if this new tool helps the students learn better, or if it just makes them lazy.
The Crew: One experienced captain (the professor) and seven junior officers (TAs) who were also figuring out the technology as they went.

2. The Results: Did the Ship Go Faster?

Yes, but with a catch.
When the students used the AI, their scores on practice problems went up significantly. It was like the GPS helped them find the shortest route to the treasure faster.

The Catch: The excitement was like a new toy. At first, everyone was thrilled to use the GPS. But as the semester went on, the "wow" factor faded, and some students used it less. They realized that while the GPS was fast, they still needed to know how to steer the ship themselves.

3. The Captain's Journey: From "Wow" to "Wait a Minute"

The professor's feelings about the AI changed over time, much like someone trying a new diet:

Month 1 (The Honeymoon Phase): "This is amazing! It's just another tool, like a better textbook. It saves time and answers questions instantly."
Month 2 (The Reality Check): "Hmm, the GPS is giving me directions to places that don't exist (hallucinations). Sometimes the code it writes is wrong. If I let the AI drive the whole car, I'm not learning how to drive."
Month 3 (The Balanced View): "It's a great tutor, not a replacement for the student. It's like having a co-pilot who can point out the clouds, but the student still has to hold the yoke."

4. The Crew's Perspective (The TAs)

The teaching assistants had mixed feelings, too.

The Good: It helped them answer student questions faster.
The Bad: They were worried the AI would eventually replace human jobs. They also noticed that if a student just asked the AI for the answer, they didn't learn anything.
The Lesson: They realized that critical thinking is the most important skill. You can't just trust the GPS; you have to double-check the map.

5. How Deep Did They Go? (The Three Levels of Tech)

The researchers used a simple framework to describe how the AI was used:

Replacement (Level 1): Using the AI instead of a textbook. (The AI is just a faster book).
Amplification (Level 2): Using the AI to do things faster or better, but the task is the same. (The AI helps you write code 50% faster, but you are still writing code).
Transformation (Level 3): Changing the entire way learning happens. (The AI changes the curriculum so students learn entirely new skills).

The Verdict: This study stayed at Level 1 and Level 2. They used the AI to replace old tools and speed things up, but they didn't completely reinvent the course yet. That's normal for a first try!

6. The Big Takeaway

The main lesson from this paper is that experimentation is good.

Don't fear the new tool: The professor and TAs didn't ban the AI; they let the students use it and watched what happened.
Reflection is key: By talking about what worked and what didn't, the teachers learned how to use the tool responsibly.
The Human Element: The AI is a powerful assistant, but it cannot replace the human need to struggle, make mistakes, and learn from them. If you let the AI do all the work, you aren't learning to be an engineer; you're just learning to be a prompter.

In a nutshell: This paper is a story about a group of teachers trying out a super-smart robot assistant in their classroom. They found that the robot made homework faster and scores higher, but they also learned that the robot can lie, get confused, and shouldn't be allowed to do the thinking for the students. The best approach? Use the robot as a tutor, not a cheat sheet.

1. Problem Statement

The rapid emergence of Generative Artificial Intelligence (GenAI) tools (e.g., ChatGPT, GitHub Copilot) presents a significant paradigm shift for Engineering Education (EE). While existing literature acknowledges potential benefits (e.g., personalized learning, efficiency) and risks (e.g., plagiarism, over-reliance, hallucinations), there is a critical gap in understanding how engineering faculty and teaching staff initially discover, experiment with, and integrate GenAI into their instruction.

Specifically, the paper addresses:

The lack of qualitative research on the process of initial GenAI integration by teaching staff.
The uncertainty regarding the impact of intentional GenAI integration on student formative performance in introductory coding courses.
The need to distinguish between general AI in education and specific AI-assisted coding education.
The question of whether GenAI integration leads to superficial efficiency gains or deep pedagogical transformation.

2. Methodology

The study employs an embedded single-case study design (Yin, 2018) focusing on a single introductory undergraduate computer systems course over one academic semester (12 weeks).

Participants:
- Course Instructor (CI): A software engineering faculty member with a Ph.D., 10 years of academic experience, and 25 years of professional engineering experience.
- Teaching Assistants (TAs): Seven TAs with varying levels of experience (ranging from high school diploma to B.A. in software engineering) and an average of 4 months of TA experience.
Intervention:
- Tool: GitHub Copilot (powered by GPT-4).
- Context: Students were explicitly allowed and encouraged to use Copilot during weekly exercise sessions to solve coding and conceptual problems. The CI modeled usage and discussed pros/cons.
- Pedagogy: A practice-oriented approach emphasizing "learning by doing," teamwork, and critical evaluation of AI outputs.
Data Collection:
- Qualitative: Four semi-structured interviews (CI at start, middle, and end of semester; TAs at the middle).
- Quantitative: Student performance data from pre- and post-formative practice exercises (9 exercises total) covering factual and inference-based coding tasks.
Data Analysis:
- Quantitative: Statistical analysis using R (Wilcoxon signed-rank tests for non-normal data; paired t-tests for normal data) to measure gain scores and effect sizes.
- Qualitative: Thematic analysis (Braun & Clarke, 2006) aligned with Hughes's (2005) Technology Integration Levels (Replacement, Amplification, Transformation).

3. Key Contributions

Empirical Evidence of Initial Integration: Provides a rare, longitudinal account of the evolution of an instructor's and TAs' attitudes toward GenAI, moving from initial optimism to ambivalent realism.
Theoretical Alignment: Maps the integration process onto Hughes's framework, demonstrating that initial adoption in EE tends to remain at the Replacement and Amplification levels rather than achieving Transformation.
Pedagogical Conditions for Success: Identifies specific conditions required for effective GenAI use in coding, such as the necessity of "prompt engineering" knowledge, the danger of offloading cognitive effort, and the importance of critical verification of AI outputs.
Student Performance Correlation: Offers data linking GenAI integration to statistically significant improvements in formative exercise performance, while noting the potential role of the "novelty effect."

4. Key Results

A. Quantitative Findings (Student Performance)

Performance Increase: Students showed statistically significant improvements ( $p < .05$ ) in formative exercise scores after GenAI integration.
Effect Sizes: Large effect sizes were observed in 6 out of 9 exercises (Exercises 1, 2, 4, 6, 7, 8). Medium and small effects were found in Exercises 3 and 5, respectively. Exercise 9 showed negligible improvement.
Interpretation: The improvement suggests GenAI can enhance immediate task performance and understanding of coding concepts, though the authors caution that the "novelty effect" (excitement of new technology) may have contributed to short-term gains.

B. Qualitative Findings (Instructor & TA Perspectives)

Evolution of Attitudes:
- Start: High optimism, viewing GenAI as a "tool" for efficiency and a "tutor."
- Middle/End: Shift toward ambivalence. Instructors recognized benefits (speed, scaffolding) but grew concerned about "hallucinations," code quality, privacy, and the risk of students "offloading" the learning process (writing code without understanding it).
Integration Levels (Hughes, 2005):
- Replacement: GenAI replaced textbooks or search engines for finding syntax and basic concepts.
- Amplification: GenAI increased the efficiency of solving coding problems and generating boilerplate code.
- Transformation: Not achieved. The study did not reach the level of fundamentally changing learning processes or cognitive goals. The core pedagogy remained traditional (practice-based), with GenAI serving as an accelerator rather than a transformer of the curriculum.
Critical Conditions for Success:
- Prompt Quality: Students needed prior knowledge to write effective prompts; without basics, prompts were ineffective.
- Critical Verification: TAs and the CI emphasized that students must double-check AI outputs, as hallucinations were common.
- Teamwork: Peer discussion helped students compare AI outputs and validate solutions, mitigating individual over-reliance.
- Assessment Shift: The CI suggested a move toward oral exams or controlled environments to ensure genuine understanding, as written code could be easily generated by AI.

5. Significance and Implications

For Higher Education Institutions: The study argues that institutions should encourage experimental integration by faculty even in the absence of formal guidelines. First-hand experience is crucial for developing realistic expectations and effective policies.
For Faculty Development: Reflective practice and peer collaboration (communities of practice) are essential for navigating the "ambivalent" phase of technology adoption. Faculty need support to move from simple replacement to more sophisticated pedagogical integration.
For Engineering Pedagogy: The findings suggest that while GenAI boosts efficiency, it cannot replace the cognitive effort required for deep learning. The role of the engineer is shifting from "writing code" to "verifying and architecting," necessitating a curriculum that emphasizes critical assessment and prompt engineering.
Future Research: The authors call for multi-site studies, research on domain-specific GenAI models, and deeper investigation into student perspectives and long-term skill retention.

Conclusion

The paper concludes that while GenAI integration in introductory engineering courses currently operates at the replacement and amplification levels, it yields immediate performance benefits. However, the transition to transformational learning requires a shift in assessment strategies and a pedagogical focus on critical thinking and verification. The "experimental" approach taken by the faculty provided valuable, context-specific insights that can inform broader institutional strategies for AI in engineering education.