Technical Summary: Planning on Paper: Problem Decomposition with Diagrams in Introductory Computing

Problem Statement

Problem decomposition is a foundational metacognitive skill in computing, involving the breakdown of complex problems into manageable parts. With the rise of Generative AI (GenAI), there is a growing pedagogical shift in introductory computing (CS1) courses away from code writing and toward higher-level planning and decomposition. However, there is a significant gap in understanding how novices perform decomposition in large, multifunction tasks without pre-specified planning elements. Prior research has largely focused on small, code-based problems where planning elements correspond to localized code structures. It remains unclear how novices conceptualize and represent decomposition in unstructured, complex contexts, particularly when they are not constrained by the syntax of a programming language.

Methodology

The authors conducted a qualitative study within a large introductory Python CS1 course at the University of California, San Diego, which had recently been redesigned to emphasize decomposition and GenAI tools.

• Participants and Context: The study involved 133 students from a course of 181. The course utilized the textbook Learn AI-Assisted Python Programming, which taught decomposition primarily through hierarchical function call diagrams, though teaching staff introduced varying notations (e.g., data flow, execution order) during instruction.
• Task: In a 50-minute lab session, students were asked to draw a decomposition diagram for "Evil Word Guesser," a complex variant of Hangman where the computer dynamically updates the set of possible words based on user guesses to maximize difficulty. Students were instructed to use pencil and paper to represent functions, interactions, and relationships, without constraints on specific notation.
• Data Collection: 133 student diagrams were collected, scanned, and anonymized.
• Analysis: The researchers performed an inductive thematic analysis on a subset of 55 diagrams. Two coders independently analyzed the diagrams, identifying nodes (functions, data, fragments, annotations), relations (hierarchical calls, sequencing), and issues. They iteratively refined a codebook through negotiated agreement until saturation was reached. The analysis focused on representational strategies and the identification of specific planning and decomposition errors.

Key Findings

1. Representational Strategies (RQ1)

Students employed diverse representational strategies, often blending or conflicting with instructional norms:

• Sequencing Dominance: Contrary to the primary instruction of hierarchical function calls, the majority of diagrams (32 out of 55) utilized sequencing notations, representing the chronological order of execution.
• Hybrid and Clashing Notations: Many students mixed hierarchical function calls with sequencing relations within the same diagram. This "clashing" created ambiguity, as arrows were used inconsistently to denote both function calls and execution order.
• Ad-hoc Elements: Students introduced nodes and annotations not explicitly taught, such as explicit conditional branching nodes (e.g., "DONE" vs. "NOT DONE") and data nodes representing inputs, suggesting their mental models included control flow and data dependencies that were not captured by the standard hierarchical model.
• Fragmented Representations: Some students used code fragments or pseudocode outside of function nodes to describe assignments or logic, indicating a struggle to abstract these details into pure function definitions.

2. Decomposition and Planning Issues (RQ2)

The analysis revealed 16 distinct categories of issues, grouped into five main themes:

• Notation Issues: The most prominent issue was "clashing notations," where incompatible symbols (e.g., arrows for both calls and sequencing) were used, leading to contradictory interpretations (e.g., recursion vs. repetition). Ambiguous execution orders were also common, particularly in branching paths.
• Program Quality (Abstraction and Reuse): Students frequently failed to abstract functionality, resulting in:
  • Hard-coded values: Embedding constants (e.g., specific word lengths) directly in the diagram.
  • Repeated functionality: Creating multiple distinct nodes for the same task (e.g., prompting for a guess) rather than reusing a single function.
  • Poor modularity: Assigning too many unrelated responsibilities to a single function node.
• Encapsulation: Many diagrams exhibited poor encapsulation, such as loops that did not clearly include necessary steps (e.g., a game loop that excluded the "get user guess" step) or functions that generated their own inputs internally rather than accepting them as parameters.
• Clarity: Issues included vague language (e.g., "determine the pattern were revealed"), non-meaningful function names (e.g., "Function 1"), and implementation details without clear purpose.
• Problem-Specific Misunderstandings: A significant number of diagrams failed to capture the core "evil" mechanic of the task. Students often modeled the game as committing to a single hidden word rather than maintaining and updating a set of compatible words, or they selected word families based on incorrect criteria (e.g., minimizing options rather than maximizing them).

Significance and Implications

The paper argues that novice decomposition is shaped by multiple underlying models of program behavior, creating tension between structural reasoning (hierarchy) and sequence-focused reasoning (execution order).

• Pedagogical Implications: The findings suggest that current instruction may not sufficiently scaffold the transition from concrete code to abstract planning. The authors recommend:

  • Explicit Notation Instruction: Educators should explicitly teach the affordances and limitations of chosen notations (e.g., distinguishing between call graphs and flowcharts) to prevent clashing mental models.
  • Structured Design Processes: Implementing structured function design processes (defining contracts, inputs, and outputs before implementation) could improve encapsulation and abstraction.
  • Assessment of Planning: Decomposition should be assessed as a distinct skill from implementation. Diagrams provide a lightweight mechanism for frequent feedback on planning skills.
  • Plan Tracing: The authors propose "plan tracing" (simulating the execution of a diagram) as a future competency to help students identify encapsulation and logic errors before coding.

• Theoretical Contribution: The study extends the literature on novice planning by demonstrating that errors traditionally associated with code (e.g., poor modularity, boundary errors) manifest in free-form diagrams. It supports the use of drawings as artifacts to externalize and analyze students' mental models of notional machines, revealing that novices often hold contradictory models of program execution simultaneously.

The authors conclude that understanding these planning artifacts is crucial for developing instructional strategies that better support abstraction and program reasoning in the era of GenAI, where the ability to plan and decompose is becoming a primary competency.