No Two Developers Think Alike: How Problem-Solving Styles and Experience Shape Needs in Conversational Interaction with Copilot
Through a mixed-methods study of 27 developers, this paper identifies five distinct interaction modes and ten underlying needs in GitHub Copilot usage, demonstrating how cognitive diversity in problem-solving styles and experience shapes individual requirements for conversational programming assistants.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you have a super-smart, all-knowing robot assistant sitting next to you while you build a complex Lego castle. You can talk to it, ask it to build parts, or ask it to explain why a wall is falling down. This is what GitHub Copilot is for software developers: a conversational AI that helps write code.
But here's the twist: No two people build with Lego the same way. Some people want the robot to build the whole tower for them. Others want the robot to just hand them a specific brick. Some want to understand how the tower works, while others just want it done fast.
This paper is a study of 27 real-world builders (developers) to figure out how different types of people use this robot assistant and why they do it that way. The researchers found that a person's experience level and their personal "thinking style" completely change how they talk to the AI.
Here is the breakdown of their findings, using simple analogies:
1. The Five "Personalities" of Interaction
The researchers watched how people used the chat and realized everyone falls into one of five distinct "modes" or styles, like different driving styles on a highway:
- The Navigator (The Tour Guide):
- What they do: They treat the AI like a GPS. They ask, "Where is the red box in this code?" or "Show me where the login button is defined."
- The Vibe: They are often looking at the wrong file and need the AI to point them in the right direction. They use this a lot when they are just getting started or exploring.
- The Autopilot (The "Do It All" Commander):
- What they do: They give the AI a huge, vague command like, "Fix the navigation and do everything else," and then hit the "Apply" button. They want the robot to do the whole job in one go.
- The Vibe: "I have a goal; you handle the details." This is a "one-and-done" strategy.
- The Deputy (The Collaborative Partner):
- What they do: This is the most common style. They work with the AI step-by-step. "Change this part," then "Explain why you changed that," then "Fix this error."
- The Vibe: It's a conversation. They are building together, checking the work, and asking questions. They copy-paste small bits of code back and forth.
- The Technician (The Precision Mechanic):
- What they do: They ask for very specific, tiny instructions. "Add a debounce function here" or "Load this specific variable."
- The Vibe: They don't want the AI to write the whole block; they want a specific tool to fix a specific screw. They usually type the code in themselves rather than letting the AI auto-insert it.
- The Scholar (The Student):
- What they do: They use the AI as a textbook. "What does this React code do?" or "How does a string sort work?"
- The Vibe: They are there to learn. They often paste code into the chat just to get an explanation, treating the AI like a tutor.
2. The 10 Hidden "Needs" Driving the Behavior
Why does someone choose to be a "Technician" instead of an "Autopilot"? The study found that 10 different internal "needs" drive these choices. Think of these as the hunger, thirst, and safety needs of the developer:
- Need for Ability: "I can't do this alone; I need help." (Leads to more delegation).
- Need for Agency: "I need to prove I can do this myself." (Leads to less delegation, more control).
- Need for Experimentation: "I want to see what this robot can actually do." (Leads to trying wild things).
- Need for Assurance: "I need to trust that this code won't break." (Leads to checking everything manually).
- Need for Productivity: "I need to finish this fast." (Leads to asking the AI to do more).
- Need for Ease: "I don't want to think too hard or type too much." (Leads to delegating tasks).
- Need for Learning: "I want to get better at this." (Leads to asking for explanations).
- Need for Understanding: "I need to know why this works." (Leads to asking for details).
- Need for Delegation: "I want to outsource the work." (The desire to let the AI take over).
- Need for Control: "I want to be the boss of the solution." (The desire to keep the reins tight).
The Catch: These needs often fight each other. For example, if you want Ease (don't want to work hard), you might delegate everything. But if you also have a strong Need for Control (I need to know exactly how it works), you might refuse to delegate, even if it's harder.
3. How Experience and Personality Change the Game
The study found a clear link between who you are and which "mode" you use:
- Experience Matters:
- Novices (less experienced) tend to be Navigators (looking for help) or Autopilots (hoping the AI fixes everything). They often feel they need the AI to do the heavy lifting.
- Experts (more experienced) tend to be Technicians. They know exactly what they want. They don't ask the AI to "build the house"; they ask it to "install this specific window." They keep more control because they trust their own judgment more.
- Thinking Styles Matter:
- People who like to tinker (mess around and see what happens) didn't necessarily use the AI to "experiment" in the way the researchers expected. Instead, they used it to tweak their own specific solutions.
- People who are risk-tolerant (don't mind if things go wrong) were expected to let the AI do more, but the study showed it's complicated. Sometimes, being willing to take risks means you trust the AI more, but other times it means you are willing to fail while trying to solve it yourself.
4. The Big Picture: One Size Does Not Fit All
The main takeaway is that cognitive diversity (how different people think and solve problems) is huge.
If you build a tool that only supports the "Autopilot" style, you might frustrate the "Technician" who wants control. If you build a tool that only supports "Scholar" style, you might bore the "Autopilot" who just wants speed.
The researchers conclude that to make AI assistants truly helpful for everyone, we need to stop thinking of them as one-size-fits-all tools. Instead, we need to design them to adapt to the user's current need (Do they want to learn? Do they want to save time? Do they want to feel in control?).
In short: Developers are like different types of drivers. Some want a self-driving car (Autopilot), some want a co-pilot (Deputy), and some just want a GPS (Navigator). The best AI assistant is one that knows which driver it's talking to and adjusts accordingly.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.