Visible Adoption, Untracked Contribution: GitHub Evidence of the Accountability Gap Across Three Cohorts of an HCI Prototyping Course

This longitudinal study of three HCI course cohorts reveals that while student disclosure of AI tool usage has increased significantly, the persistent gap between general disclosure and specific attribution of AI contributions exposes the inadequacy of current accountability frameworks for capturing continuous, ambient co-creation in the "vibe-coding" era.

The Gist

Imagine a cooking class where students are tasked with building a complex dish. For three years in a row, the same group of teachers gave the same recipe and the same grading rules. The only thing that changed was the kitchen equipment available to the students.

• Year 1 (2022): The kitchen had no smart appliances. Students cooked everything from scratch.
• Year 2 (2023): A few smart ovens and mixers arrived. Students started using them, but mostly for small tasks like chopping onions.
• Year 3 (2025): The entire kitchen was retrofitted with AI. The smart ovens were built into the walls, the mixers were voice-activated, and the recipe book itself had AI instructions embedded in the margins.

This paper is a "forensic investigation" of what happened in that cooking class over those three years. The researchers didn't just ask the students, "Did you use AI?" (because people lie or forget). Instead, they looked at the digital fingerprints left behind in the students' code repositories (like a digital logbook of every change made to their projects).

Here is what they found, translated into everyday terms:

1. The "Name-Dropping" vs. "Credit-Giving" Gap

The class had a rule: "If you use a tool, write its name in your report."

• The Finding: In Year 3, 66% of students followed the rule and wrote down the names of the AI tools they used (like "ChatGPT" or "Copilot").
• The Problem: However, only 42% of them actually explained what the AI did.
• The Analogy: Imagine a student writes, "I used a blender." That's naming the tool. But they don't say, "The blender made the smoothie base, but I added the fruit." That's attributing the work.
• The Result: We know they used the blender, but we don't know if the student made the smoothie or if the blender made the whole thing. The "accountability gap" is that we see the tool, but we can't see the contribution.

2. AI Became the "Floor," Not the "Tool"

In Year 1 and 2, AI was like a special knife you had to pick up and use. In Year 3, AI was like the floor you were standing on.

• The course itself started shipping with AI tools pre-installed in the project templates.
• The Analogy: If you ask a student, "Did you use the floor?" they might say, "No, I just walked on it." But the floor was actually helping them move. Because the AI was built into the class structure, students felt they didn't need to "disclose" it, even though it was doing a huge amount of the work. The system made AI invisible by making it the default.

3. The "Ghost Commits" Mystery

The researchers counted how many times students saved their work (called "commits").

• The Finding: In Year 3, students saved their work much more often than in previous years.
• The Mystery: Did this mean the students were working harder and faster? Or did it mean they were using AI agents that automatically saved their work for them?
• The Analogy: It's like seeing a car drive 100 miles an hour. You don't know if the driver is a professional racer pushing the pedal, or if the car is on "autopilot" and the driver is just holding the wheel. The log shows the speed, but not who is actually steering.

4. The Style of the "Writing" Changed

The researchers looked at how the students wrote their code comments (notes inside the code).

• The Finding: In the AI era, the code had fewer "hand-written" notes (like "fix this later") but more "professional-looking" summaries at the top of functions.
• The Analogy: It's like the difference between a student's messy notebook with scribbles in the margins versus a perfectly typed, formatted report. The AI tends to write the "perfect" summaries, while the messy, human struggle of figuring things out (the scribbles) disappears. This makes it harder to tell if the student actually understood the code or just pasted a perfect answer.

The Big Conclusion: "Vibe Coding"

The paper introduces a term called "Vibe Coding."

• Old Way: "I used a calculator to solve this math problem." (Clear, discrete, easy to credit).
• Vibe Coding: The student is just "vibing" with the code. They talk to the AI, the AI suggests changes, the student accepts them, the AI suggests more, and the student accepts those too. The line between "what the student thought" and "what the AI wrote" is completely blurred.

The Takeaway:
The authors argue that our current rules for honesty (just "naming" the tools) are broken. They are designed for a world where AI is a rare, occasional helper. They don't work in a world where AI is the background noise of the entire project.

The paper concludes that we can't just ask students, "Did you use AI?" anymore. We need to figure out how to judge how they thought about the problem, not just what tools they listed. We need to know if they are the architects of the building, or just the people holding the blueprint while the robot builds it.

Technical Summary

Technical Summary: Visible Adoption, Untracked Contribution

Problem Statement
The paper addresses the "accountability gap" in Human-Computer Interaction (HCI) education as Generative AI (GenAI) shifts from an episodic tool to an ambient infrastructure. The authors argue that conventional accountability frameworks, which rely on students disclosing specific tools used, are insufficient for the "vibe-coding" era (a term coined by Andrej Karpathy). In this new paradigm, AI is embedded in workflows, making the boundary between student contribution and model output difficult to locate. The core problem is that while students increasingly acknowledge AI use, they rarely specify what the AI contributed, rendering disclosure-based norms inadequate for assessing genuine co-thinking and intellectual autonomy.

Methodology
The study is a longitudinal, observational case study of the "Interactive Device Design" (IDD) course at Cornell Tech, analyzing three cohorts:

• Fall 2022 (pre-AI): No major AI tools widely available.
• Fall 2023 (early-AI): Tools like ChatGPT newly available.
• Fall 2025 (vibe-AI): AI-integrated tools embedded in course infrastructure (e.g., GitHub Copilot chat modes and instruction files provided in templates).

Data Collection & Analysis:

• Dataset: 203 student GitHub repositories and 23,065 student commits (filtered from 181,555 raw commits).
• Operationalization: The authors distinguished two levels of accountability trace:
  1. Disclosure (F1): Naming an AI tool in the README.md.
  2. Attribution (F2): Explicitly crediting a specific artifact or task to an AI tool (e.g., "code generated by ChatGPT").
• Computational Techniques:
  • Text Analysis: Case-insensitive string matching for 27 named AI tools and 17 AI concept terms in README files and commit messages.
  • File Classification: Categorizing changed files into code, documentation, assets, and other.
  • Commit Metadata: Analyzing commit frequency, timing (night/weekend), and message style (auto/generic vs. intentional).
  • Code Metrics: Measuring inline comment density, docstring density, and LLM vocabulary markers.
• Statistical Approach: Mann–Whitney U tests with Bonferroni correction ($\alpha=0.017$) and rank-biserial correlation for effect sizes.

Key Results

1. Adoption vs. Attribution Gap: AI tool disclosure in READMEs grew monotonically from 0% (2022) to 36% (2023) to 66% (2025). However, explicit contribution attribution remained a minority practice, rising only to 42% in 2025. This reveals a persistent gap where students name tools but do not specify their contributions.
2. Tool Diversification: The median number of distinct tools named per repository increased from 0 to 8, shifting from a ChatGPT-dominant pattern to a diverse ecosystem including GitHub Copilot, Llama, and LangChain.
3. Increased Output Volume: Per-person commit frequency increased significantly in the 2025 cohort (49 commits/person vs. 26 in 2022). The proportion of code file changes also rose from 13% to 44%, while asset changes decreased.
4. Shift in Code Style: The 2025 cohort produced code with significantly fewer inline comments (12.8% vs. 15.6%) but significantly more structured docstrings (3.9% vs. 2.3%). This suggests a shift toward LLM-generated structural documentation rather than novice-style inline explanations.
5. Limitations of Traces: The study found no machine-authored attribution metadata (e.g., Co-Authored-By trailers) in any cohort. AI involvement was expressed in prose rather than structured metadata, making it impossible to distinguish between AI amplifying student capability and AI performing work under student supervision based solely on commit logs.

Significance and Claims
The paper claims that the failure of current accountability is not that students conceal AI use, but that norms designed for "episodic, identifiable acts" cannot capture "continuous, ambient co-creation."

• Infrastructure vs. Tool: By 2025, AI became part of the course infrastructure (shipped in templates), meaning even compliant students had no discrete "opt-in" act to disclose.
• Insufficiency of Disclosure: The authors argue that disclosure-based frameworks are necessary but insufficient. A statement like "Copilot was used for coding" fails to capture the degree of student agency or the nature of the co-thinking process.
• Proposed Shift: The paper advocates for "architecture-level accountability," suggesting that assessment should focus on whether a student can explain why an artifact does what it does and how AI was integrated into the design logic, rather than simply listing tools used.

The study serves as empirical grounding for the argument that accountability mechanisms must evolve to probe the reasoning behind design decisions, not just the final outputs or tool lists.