The Competence Crisis: A Design Fiction on AI-Assisted Research in Software Engineering
This vision paper utilizes Design Fiction to explore the potential erosion of researcher competence, responsibility, and trust in software engineering caused by rising publication pressures and the unchecked integration of generative AI tools, ultimately inviting the community to critically redefine proficiency and accountability in a future research landscape.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine the world of software research as a massive, bustling library where scientists write books about how to build better computers. For years, this library has been growing fast, with more and more people trying to get their books published. Recently, a new kind of "super-scribe" (Generative AI) has arrived. This scribe can write sentences, do math, and organize ideas incredibly fast. Everyone is excited because it makes writing books much easier.
But this paper asks a scary question: What if we get so used to letting the super-scribe do the work that we forget how to write ourselves?
Here is the story of the paper, broken down into simple parts:
1. The Current Mood: Happy but Nervous
The authors first looked at what software researchers are feeling right now.
- The Good: The community is friendly and supportive. People like helping each other.
- The Bad: Everyone is under huge pressure to publish more books, faster. This has created a "peer review crisis," where there are so many books to check that the quality of checking is dropping. It's like a judge trying to read 1,000 novels in a day; they can't possibly catch every mistake.
- The Fear: Mentors (the teachers) are worried about their students. They fear that if students rely too much on the AI super-scribe, they will become great at asking the AI questions but terrible at understanding the answers. They might lose the ability to think deeply about the basics.
2. The Story: A Warning from 2035
To show what could happen if we don't change our habits, the authors wrote a fictional story set in the year 2035.
The Scene:
A senior researcher named Dr. Jane Doe is reviewing a new scientific paper. The paper looks perfect. The writing is smooth, the math looks right, and the AI "Contribution Score" is 98% (meaning the AI did almost all the work).
The Mistake:
The paper suggests using a computer rule called "Paxos" to connect human brains to computers.
- The AI's Error: The AI saw the words "distributed nodes" and thought, "Oh, Paxos is used for that!" It didn't realize that human brains are messy, continuous, and biological, while Paxos is a rigid, digital rule. Using it on a brain would be like trying to run a digital video game on a wet sponge—it just doesn't work and could cause seizures.
- The Human Failure: The authors didn't catch this. They trusted the AI completely. They didn't know enough about biology to realize the AI was making a dangerous guess.
The Horror:
Jane realizes this isn't just one bad paper. It's the third one this month. The AI is making up "plausible-sounding" nonsense, and because the authors don't have the deep knowledge to check it, they are publishing it. Soon, these fake papers will be cited by other fake papers, creating a "poisoned well" of knowledge where no one knows what is true anymore.
The Verdict:
Jane rejects the paper. But her rejection letter isn't just about the math error. It's a warning to the whole community: "We have become operators of black boxes, not engineers. We know how to push the buttons, but we don't know how the machine works inside."
3. The Big Lesson: The "T-Shape" vs. The "Dash"
The paper uses a cool analogy to explain the problem:
- The T-Shape (Good): A good researcher should be like the letter T. They have a broad understanding of many things (the top bar) but also have deep, vertical expertise in one specific area (the long leg). This deep leg lets them dig into the messy details and catch errors.
- The Dash (Bad): If we rely too much on AI, we might become like a dash (—). We will be very wide and fast, knowing a little bit about everything, but we will have no depth. We won't have the "vertical leg" to dig deep and verify if the AI is lying.
4. What Should We Do?
The paper doesn't say "ban AI." It says we need to be smarter about how we use it.
- Don't just ask, check: Researchers need to keep their "muscle memory" for the basics. They must be able to verify the AI's work manually, like a pilot who knows how to fly the plane even if the autopilot is on.
- Embrace the struggle: Learning often comes from getting stuck, debugging, and failing. If AI removes all the struggle, we might stop learning how to solve hard problems.
- Take responsibility: We can't just say, "The AI did it." If the AI makes a mistake, the human is still the one responsible.
Summary
This paper is a "Design Fiction"—a made-up story used to hold up a mirror to our current habits. It warns that if we let AI do all the heavy lifting without keeping our own deep knowledge sharp, we risk building a future where our research looks perfect on the surface but is fundamentally broken underneath. The goal is to stay competent, not just efficient.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.