🤖 AI

A Showdown of ChatGPT vs DeepSeek in Solving Programming Tasks

This study compares ChatGPT 03-mini and DeepSeek-R1 on Codeforces programming tasks, revealing that while both models perform similarly on easy problems and struggle with hard ones, ChatGPT significantly outperforms DeepSeek-R1 on medium-difficulty challenges.

Original authors: Ronas Shakya, Sam Urmian, Mohammad Khalil

Published 2026-05-21

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Ronas Shakya, Sam Urmian, Mohammad Khalil

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine two brilliant but very different students sitting down for a series of coding exams. One is ChatGPT (specifically the o3-mini version), a well-known, highly trained student who has read almost every book in the library. The other is DeepSeek-R1, a newer, open-source challenger from China known for its intense focus on logical reasoning and math puzzles.

The researchers from the University of Bergen decided to put them to the test using a "gym" for programmers called Codeforces. They gave both students 29 problems, ranging from simple warm-ups to grueling marathons, and asked them to write C++ code to solve them.

Here is how the showdown played out, broken down by difficulty:

1. The Warm-Up (Easy Tasks)

Think of these as simple arithmetic problems or basic logic puzzles.

The Result: Both students did a great job. They were like two top-tier athletes running a 100-meter sprint; they both finished the race successfully.
The Catch: While both got the answers right, DeepSeek sometimes ran a little slower and used a bit more "energy" (memory) than ChatGPT, but they both crossed the finish line.

2. The Middle Distance (Medium Tasks)

This is where the race got interesting. These problems required a bit more strategy and planning.

ChatGPT: This student was the clear winner here. They solved about 55% of these problems correctly. They were like a seasoned marathon runner who knows exactly how to pace themselves.
DeepSeek: This student struggled significantly, solving only about 18% of the medium problems. It was as if they got confused halfway through the race, tripped over their own feet, or forgot the rules.
Why? The paper suggests ChatGPT might have had more practice with this specific type of "competitive programming" data during its training, giving it a head start.

3. The Ultra-Marathon (Hard Tasks)

These were the toughest challenges, requiring complex, multi-step logic.

The Result: Both students hit a wall.
- ChatGPT managed to solve only 1 out of 9 hard problems.
- DeepSeek failed to solve any of the hard problems (0%).
The Metaphor: Imagine asking both students to build a skyscraper from scratch without a blueprint. They both got stuck. ChatGPT tried to build a few floors before the structure collapsed (runtime errors), while DeepSeek often couldn't even start the foundation (compilation errors) or ran out of energy (memory limits) before finishing.

The "Energy" Consumption (Memory and Time)

The researchers also looked at how much "fuel" (computer memory) and time each student used.

ChatGPT was generally faster and more efficient with its resources on the easier and medium tasks.
DeepSeek sometimes used a massive amount of memory or took a very long time to think, especially on the hard tasks. In one instance, DeepSeek took so long to think about a problem that it timed out completely.

The Bottom Line

The paper concludes that while DeepSeek-R1 is a powerful new contender that can handle simple tasks well, ChatGPT o3-mini is currently the better "coach" for medium-difficulty coding challenges. However, when the problems get truly difficult, both models need human help. They aren't ready to solve the hardest puzzles on their own yet.

Important Note from the Study:
The researchers only asked each student to try the problem once (a "single-shot" attempt). They didn't let them try again if they failed or ask for hints. The paper notes that if humans were allowed to guide them or if they used a different, more specialized version of DeepSeek (called DeepSeek-Coder), the results might have been different. But based on this specific test, ChatGPT came out ahead in the middle ground, while both struggled at the top level.

Technical Summary: A Showdown of ChatGPT vs DeepSeek in Solving Programming Tasks

Problem Statement

The rapid advancement of Large Language Models (LLMs) has created a competitive landscape for AI-assisted programming tools, particularly in educational and professional development contexts. While models like ChatGPT have established themselves as standard tools for code generation, new contenders such as DeepSeek-R1 have emerged, claiming comparable performance in logical reasoning and deductive tasks with significantly lower training costs. However, there remains a lack of comprehensive, empirical evaluations comparing these specific models—ChatGPT o3-mini and DeepSeek-R1—specifically within the domain of competitive programming. This study addresses the gap in understanding how these models differ in accuracy, execution efficiency, memory usage, and their ability to handle tasks of varying complexity (easy, medium, and hard) without human intervention.

Methodology

The study employed a controlled experimental design using 29 programming tasks sourced from Codeforces, a leading competitive programming platform. The tasks were categorized into three difficulty levels: easy, medium, and hard, based on user success rates and problem complexity.

Models Evaluated: ChatGPT o3-mini and DeepSeek-R1.
Programming Language: C++ was selected for all prompts due to its prevalence in competitive programming and fast compilation speed.
Execution Environment: To ensure uniformity, all generated code was compiled and tested using the GNU G++20 13.2 (64-bit, winlib) compiler on Codeforces.
Prompting Strategy: A consistent zero-shot prompt was used: "Solve the following question and return C++ code, which is compilable by g++ 20. Pay attention to memory and time limits," followed by the specific problem statement and input/output examples.
Evaluation Metrics:
- Acceptance Rate: Determined by Codeforces feedback (Accepted vs. Wrong Answer, Time Limit Exceeded, Memory Limit Exceeded, Compilation Error).
- Resource Efficiency: Measured via memory usage (KB) and execution time (ms).
- Weighted Scoring: A custom formula was applied to calculate a weighted score ( $S$ ) for each model, balancing acceptance ( $X$ ), difficulty ( $D$ ), and penalties for execution time ( $\alpha$ ) and memory usage ( $\beta$ ). Both penalty factors were set to 0.5 to ensure equal weighting.

Key Results

The study yielded distinct performance differences between the two models across difficulty levels:

1. Solution Accuracy

Easy Tasks: Both models performed similarly, with ChatGPT achieving a 100% acceptance rate and DeepSeek achieving 88.8%.
Medium Tasks: A significant divergence occurred. ChatGPT achieved a 54.5% success rate (6 out of 11 tasks), whereas DeepSeek struggled with only an 18.1% success rate (2 out of 11 tasks).
Hard Tasks: Both models faced substantial challenges. ChatGPT managed a 11.1% success rate (1 out of 9 tasks), while DeepSeek achieved a 0% success rate.

2. Resource Consumption (Memory and Time)

Execution Time: ChatGPT generally demonstrated shorter execution times compared to DeepSeek. DeepSeek exhibited a significant outlier in execution time for one hard task (Question 21), exceeding 4000 ms.
Memory Usage: ChatGPT showed higher memory consumption in medium-level tasks (mean ~3250 KB) compared to DeepSeek (mean ~1650 KB). However, DeepSeek's lower memory usage did not correlate with higher correctness; it frequently encountered "Wrong Answer" and "Compilation Error" states. Conversely, ChatGPT's higher resource usage in medium tasks coincided with its higher weighted scores.
Failure Modes: DeepSeek frequently failed due to compilation errors and memory limit exceedances, particularly in hard tasks. ChatGPT's failures were more often attributed to "Wrong Answer" or "Time Limit Exceeded."

3. Weighted Scores

Based on the weighted scoring formula:

ChatGPT: Achieved the highest weighted score on medium tasks (11.64), followed by easy (8.86) and hard (2.95).
DeepSeek: Achieved its highest score on easy tasks (7.87), dropped significantly on medium tasks (3.72), and scored 0 on hard tasks.

Key Contributions

This study provides a quantitative comparison of ChatGPT o3-mini and DeepSeek-R1 in a competitive programming setting, offering the following insights:

Performance Disparity: It identifies that while both models are capable of solving straightforward problems, ChatGPT o3-mini significantly outperforms DeepSeek-R1 on medium-complexity logical and algorithmic tasks.
Complexity Threshold: The research highlights a critical threshold where current LLMs, regardless of architecture, struggle with high-complexity (hard) programming problems, with success rates dropping to near zero for DeepSeek and single digits for ChatGPT.
Efficiency vs. Correctness Trade-off: The data suggests that lower memory consumption (observed in DeepSeek) does not necessarily equate to better code quality or correctness, as DeepSeek's solutions were often inefficient or logically flawed despite lower resource footprints.

Significance and Claims

The authors claim that these findings offer valuable insights for developers and researchers aiming to advance AI-driven programming tools. Specifically:

Tool Selection: For medium-difficulty coding tasks, ChatGPT o3-mini currently presents a more reliable option than DeepSeek-R1 for generating correct, executable code.
Educational Implications: The results suggest that while LLMs are effective for novice learning on simple tasks, they cannot yet replace human expertise for complex algorithmic problems. The study implies that zero-shot prompting is insufficient for medium and hard tasks, necessitating advanced prompting techniques or human intervention.
Future Directions: The paper notes that the underperformance on hard tasks indicates a need for further optimization of LLM architectures to handle high-complexity reasoning. It also suggests that future research should explore multi-shot prompting, different programming languages (e.g., Python), and specialized coding models (e.g., DeepSeek-Coder) to better understand the full potential of these AI tools.

The study concludes with a modest assessment: while LLMs are transforming programming education and development, significant challenges remain in achieving autonomous, high-accuracy solutions for complex competitive programming problems.