How Software Engineers Engage with AI: A Pragmatic Workflow

1. Problem Statement

Despite the rapid adoption of AI tools (e.g., GitHub Copilot, ChatGPT) in Software Engineering (SE) for tasks like code generation, testing, and documentation, significant challenges remain:

• Uncertainty in Trust: Engineers struggle with when to trust, refine, or discard AI-generated artifacts.
• Accountability Crisis: AI-generated code often undergoes less scrutiny than human-written code, leading to potential quality and maintainability issues.
• Lack of Structured Models: Existing literature and practitioner workflows (e.g., "vibe coding") are often linear, single-pass, and lack explicit decision points, fallback strategies, or quality reasoning. Current models fail to capture the iterative, decision-driven behavior engineers actually exhibit when interacting with AI.

2. Methodology

The authors developed a pragmatic workflow and decision model using a mixed-method approach based on grey literature and field observations:

• Rapid Grey-Literature Review: A systematic review of practitioner-authored reports, blog posts, and workflow diagrams (8 distinct workflows analyzed). This identified common limitations in existing models, such as linear structures without fallbacks and vague iteration guidance.
• Field Observations: The researchers conducted non-intrusive shadowing of 11 software engineers across three industrial settings (two in Türkiye, one in Azerbaijan) over approximately 90–120 minutes per session.
  • Context: Teams working on web-based enterprise software (backend development, QA).
  • Data Collection: Recorded sequences of prompt creation, AI responses, inspection, refinement, and fallback to manual work.
  • Timing: Data collected in early Summer 2025 using contemporary AI tools.
• Synthesis: Insights from the review and observations were combined to formalize a decision-driven process model.

3. Key Contributions

The paper introduces two primary artifacts: a Pragmatic Workflow and a Four-Quadrant Decision Model.

A. The AI-SE Activity Process Model (Workflow)

This is an iterative, decision-driven workflow (Figure 1a) that formalizes the interaction between engineers and AI. It explicitly marks six key decision points (D1–D6):

• D1 (Artifact Type): Distinguishes between generating boilerplate (e.g., class skeletons) vs. nearly-complete artifacts (e.g., complex SQL queries). Boilerplate requires minimal inspection; complex artifacts require rich prompts and rigorous oversight.
• D2 (Continuation): Decides whether to continue with AI or abandon it early for manual work.
• D3 (Boilerplate Quality): Initial quality judgment for simple templates.
• D4 (Complex Artifact Quality): Core quality assessment for complex outputs, determining if they meet thresholds for acceptance, refinement, or rejection.
• D5 (Refinement Worthiness): Decides if further prompt engineering is viable or if the effort outweighs the benefit.
• D6 (Reuse Strategy): Distinguishes between total abandonment of AI output vs. partial reuse (retaining fragments for manual repair).

B. The Four-Quadrant Decision Model (Figure 1b)

A heuristic visual guide (Q1–Q4) used at evaluation points (D3, D4) to balance Effort Saved vs. Artifact Quality:

• Q1 (High Quality, High Effort Saved): Accept with minimal edits.
• Q2 (Low Quality, High Effort Saved): Appears productive but requires careful inspection, refinement, or rejection due to hidden risks.
• Q3 (Low Quality, Low Effort Saved): Discard immediately; provides little value.
• Q4 (High Quality, Low Effort Saved): Manual development might have been equally efficient; AI use may not be justified.

The model emphasizes that these judgments are qualitative and dynamic; an artifact can shift quadrants as inspection reveals defects or refinement reduces perceived effort savings.

4. Results and Validation

The model was applied in three industrial settings with over 12 engineers for several months.

Real-World Scenarios

1. Scenario 1 (Success): A backend developer used GitHub Copilot for REST Controller boilerplate.
   • Outcome: The output was high quality and high effort-saving (Q1). It followed the "fast-loop" path, accepted immediately with minimal review.
2. Scenario 2 (Fallback): A QA engineer used ChatGPT for Selenium test scripts.
   • Outcome: Initial outputs were brittle (wrong selectors, redundant waits). Despite two refinement attempts, quality remained low (Q2). The engineer decided to fallback to manual development, illustrating the model's utility in recognizing when AI is not cost-effective.

Practitioner Feedback

A survey of the participating engineers yielded high ratings for the model:

• Clarity: 4.5/5
• Applicability: 4.2/5
• Usefulness: 4.3/5
• Key Insight: Engineers reported that the four-quadrant diagram helped them explicitly articulate why they accepted or rejected AI outputs, making decision-making more transparent.

5. Significance and Implications

• Bridging Theory and Practice: The model fills a gap between abstract decision theory and the micro-level, real-time decisions developers make during AI-assisted coding.
• Quality-Aware Adoption: It moves beyond "vibe coding" (exploratory prompt-driven coding) by enforcing structured quality checkpoints and fallback mechanisms, mitigating the risk of "automation bias."
• Pragmatic Guidance: It offers a lightweight, non-prescriptive framework that externalizes the informal reasoning engineers already use, helping teams navigate trade-offs between speed and quality.
• Future Direction: The authors identify limitations, including the need for quantitative validation, expansion to non-code tasks (requirements, design), and integration into IDEs as tool support.

In conclusion, this paper provides a critical framework for the deliberate and quality-aware integration of AI into software engineering, emphasizing that human oversight and structured decision-making are essential to harnessing AI's potential without compromising software reliability.