An Empirical Study of Generative AI Adoption in Software Engineering
This empirical study reveals that while Generative AI is widely adopted in software engineering and offers significant productivity and quality benefits, practitioners face persistent challenges regarding output reliability, security, and governance, ultimately expecting the technology to redefine rather than replace their roles.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine the world of software engineering as a massive, bustling construction site. For decades, the workers (programmers) have been building skyscrapers (software) using hammers, saws, and blueprints. They've been very good at it, but it's been hard, slow work.
Now, a new kind of super-intelligent robot assistant has arrived on the site. This is Generative AI (GenAI). It can instantly sketch blueprints, cut wood, and even lay bricks just by listening to a voice command.
This paper is like a site-wide survey where the researchers asked 204 construction workers from 37 different countries: "How are you using this new robot? Is it helping? Is it causing trouble?"
Here is what they found, translated into everyday language:
1. The Robot is Everywhere (Adoption)
The Vibe: The robot isn't just a toy anymore; it's part of the crew.
- The Stat: About 80% of the workers are using this AI daily. It's no longer an experiment; it's a standard tool, like a smartphone or a power drill.
- The Analogy: It's like if 8 out of 10 chefs suddenly started using a smart oven that could chop vegetables and season soup while they talked to it. They aren't waiting for it to become popular; they are already using it.
2. What Are They Actually Doing? (Usage)
The Vibe: The robot is mostly doing the "heavy lifting" of writing code.
- Top Job: The most common use is Implementation (writing the actual code). It's like the robot is the apprentice who writes the first draft of the blueprint, and the human engineer just tweaks it.
- Other Jobs: It also helps check for mistakes (testing), fixes old broken parts (maintenance), and acts as a personal tutor to help workers learn new skills.
- The Catch: It's rarely used for the very beginning (planning the whole building) or the very end (final inspections). It's best at the "middle work."
3. The Good Stuff (Benefits)
The Vibe: The work is getting faster and, surprisingly, sometimes better.
- Speed: Workers say they can finish an 8-hour task in just 2 or 4 hours. It's like having a turbo button on your work.
- Quality: Many feel the final product is higher quality because the robot catches small typos or logic errors they might have missed.
- Brain Power: It acts as a "second brain," helping workers brainstorm ideas or find answers to tricky problems without spending hours searching Google.
4. The Bad Stuff (Challenges)
The Vibe: The robot is smart, but it can be a bit of a "know-it-all" who sometimes lies.
- The "Hallucination" Problem: The robot sometimes confidently makes things up. It might invent a library that doesn't exist or write code that looks right but crashes the building.
- The "Trust but Verify" Burden: Because the robot can make mistakes, the human has to spend extra time double-checking its work. It's like having a fast typist who makes up words; you have to read every sentence carefully to catch the errors.
- The "Secret Sauce" Fear: Workers are worried that if they ask the robot for help with their company's secret code, the robot might accidentally leak that secret to the public internet.
- The "Prompt" Puzzle: To get good results, you have to know exactly how to ask the robot (called "prompt engineering"). If you ask vaguely, you get vague answers. It's like trying to order a complex meal at a restaurant; if you don't describe exactly what you want, you might get a sandwich instead of a steak.
5. The Bosses Are Playing Catch-Up (Institutionalization)
The Vibe: The companies are letting workers use the robot, but they haven't taught them how to use it safely yet.
- Access vs. Rules: Most companies (81%) just say, "Here is the robot, go use it." But fewer than half have written rules on how to use it, and even fewer have hired experts to teach the staff.
- The Gap: It's like giving everyone a Ferrari but no driving lessons or traffic laws. People are driving fast, but the company hasn't built the guardrails yet.
6. Will the Robot Steal Our Jobs? (Future Impact)
The Vibe: The workers are mostly calm, but a little nervous about the future.
- Redefine, Don't Replace: Most workers (79%) believe the robot won't fire them. Instead, it will change their job. They won't be "brick layers" anymore; they will be "architects" who direct the robot.
- The Job Market Shrink: However, many worry that because the robot is so efficient, companies won't need to hire as many new people. The total number of jobs might shrink, even if the current workers keep their seats.
- Confidence: Despite the fear, the workers are confident they can learn the new skills needed to work with the robot.
The Big Takeaway
Generative AI is a powerful new engine for software engineering. It's making work faster and helping people learn, but it's also introducing new risks like mistakes, security leaks, and the need for new rules.
The paper concludes that we are in a "Wild West" phase. Everyone is riding the horse, but we need to build the fences, write the traffic laws, and teach everyone how to ride safely before we can truly enjoy the speed. The technology is here to stay, but how we manage it is the real challenge.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.