Adapting homework design principles in modern Computer Science education to enhance learning outcomes and reduce unintended consequences

Technical Summary: Adapting Homework Design Principles in Modern Computer Science Education

Problem Statement
The rapid emergence of Large Language Models (LLMs) and the proliferation of online freelancing services have compromised the authenticity and effectiveness of traditional Computer Science (CS) homework. Educators face increasing difficulty distinguishing between student-generated work and AI-assisted or outsourced solutions, particularly in technical fields where code and algorithms can be generated instantly. This challenge has been exacerbated by the shift to remote learning, which reduced opportunities for spontaneous, in-person conceptual discussions. Traditional assessment methods, such as "take-home exams" or static code submissions, are no longer sufficient to verify genuine understanding or prevent academic dishonesty. The core problem is how to redesign assignments to foster authentic engagement, critical thinking, and conceptual mastery in an era where AI tools can produce plausible technical solutions.

Methodology
The study, conducted within standard Information Technology and Engineering curricula at ADA University, proposes a series of case-based scenarios to restructure homework design. Rather than prohibiting AI tools, the methodology integrates them into the learning and evaluation process through three primary strategies:

1. Screen Recording with Verbal Explanation:

   • Implementation: Students are required to record their screen while solving technical tasks, providing verbal annotations of their thought process. To ensure authenticity, recordings must include visible timestamps, and students may be asked to perform specific actions (e.g., modifying a shared file) during the recording.
   • Evaluation: The study utilized a sample of 34 video submissions to compare AI-based grading against instructor grading. Quantitative analysis included Pearson correlation ($r=0.512, p=0.00199$) and a paired-samples t-test ($t=-3.266, p=0.00255$). Results indicated a statistically significant moderate positive correlation, though the AI system systematically assigned lower grades (mean difference of 0.88 points) and was more critical regarding maximum scores.
   • Student Feedback: Surveys indicated high perceived learning effectiveness (scores of 7.72–8.17) and a positive reception of the format as a transition to modern communication styles, despite initial concerns regarding time overhead.

2. Critical Comparison with LLMs (Self-Assessment):

   • Implementation: Instead of banning LLMs, assignments require students to generate a solution, use an LLM to generate an alternative solution, and then critically compare the two. Students must identify flaws, inconsistencies, or suboptimal approaches in the AI-generated code or diagrams and justify their own design choices.
   • Application: This approach was applied to functional programming tasks (comparing imperative vs. functional implementations) and database normalization (critiquing AI-generated E-R diagrams).
   • Goal: This shifts the focus from output correctness to the depth of conceptual understanding and the ability to critique technical solutions.

3. Incremental Work via Version Control:

   • Implementation: To combat procrastination and "last-minute" outsourcing, students must submit work incrementally to a GitHub repository. Requirements include a minimum number of commits (e.g., three per week or five on different days) and the use of branches and Pull Requests.
   • Goal: This fosters continuous engagement, allows instructors to track the evolution of the solution, and develops industry-standard version control skills.

Key Results

• Grade Distribution: The implementation of screen-recording assignments correlated with a reduction in low-scoring students and a shift toward average scores in both "Production and Operations Management" and "Database Systems" courses.
• AI vs. Human Grading: While AI grading showed a systematic bias toward lower scores compared to human instructors, it preserved the ranking structure of student performance, suggesting potential for partial automation in assessment.
• Student Perception: Students reported that the verbal explanation requirement forced a deeper understanding of the solution process rather than mere result generation. The methodology was perceived as increasing accountability and reducing reliance on unverified AI solutions.
• Skill Development: The incremental approach successfully encouraged daily engagement and the development of soft skills such as time management and technical documentation.

Significance and Claims
The paper claims that effective assignment design in the age of AI should not rely on prohibition but on adaptation. By leveraging iterative learning cycles (read-understand, ask-do-check) and continuous progress monitoring, educators can maintain academic integrity while fostering deeper conceptual learning.

The study posits that:

1. Process over Product: Shifting assessment focus from the final artifact to the problem-solving process (via video or commit history) better reflects student understanding.
2. AI as a Pedagogical Tool: Integrating LLMs as a comparative tool for self-assessment enhances critical thinking and prevents passive reliance on AI outputs.
3. Structural Design: The timing and structure of deadlines, combined with version control requirements, are critical in preventing procrastination and ensuring authentic student effort.

The authors conclude that these methodologies provide practical guidance for educators to design assessments that motivate students and accurately reflect their understanding, though they acknowledge limitations in applying these strategies to non-coding disciplines or tasks requiring complex technical reports. The paper advocates for a balanced approach that combines theoretical knowledge with practical application, ensuring students develop both technical proficiency and the critical thinking necessary for professional engineering roles.