💻 computer science

MEMRES: A Memory-Augmented Resolver with Confidence Cascade for Agentic Python Dependency Resolution

MEMRES is an agentic system that significantly outperforms standard LLMs in Python dependency resolution by employing a multi-level confidence cascade and a self-evolving memory, reserving the LLM as a last resort only after leveraging curated knowledge bases, semantic analysis, and heuristic detectors.

Original authors: Dao Sy Duy Minh, Tran Chi Nguyen, Trung Kiet Huynh, Pham Phu Hoa, Nguyen Lam Phu Quy, Vu Nguyen

Published 2026-04-21

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Dao Sy Duy Minh, Tran Chi Nguyen, Trung Kiet Huynh, Pham Phu Hoa, Nguyen Lam Phu Quy, Vu Nguyen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to cook a complex recipe, but you've lost the ingredient list. You know the dish requires "flour," "eggs," and "a secret spice," but you don't know which brand of flour, how many eggs, or what the secret spice is called.

In the world of programming, this is called Dependency Resolution. When a programmer writes a piece of Python code, it often relies on other pre-made pieces of code (packages) to work. The problem is: there are over 500,000 of these "ingredients" (packages), and they change names, break, or stop working with new versions of Python.

Previously, the best way to solve this was to ask a super-smart AI (a Large Language Model or LLM) to guess the right ingredients. But this was like asking a genius chef to guess the recipe every single time: it was slow, expensive, and often wrong because the chef didn't have a reference book.

MemRes is a new, smarter system that acts like a super-organized kitchen assistant who knows exactly what to do before even waking up the genius chef.

Here is how MemRes works, using simple analogies:

1. The "Confidence Cascade" (The 6-Step Checklist)

Instead of immediately calling the expensive AI chef, MemRes runs through a 6-step checklist. It only calls the AI if all the other steps fail. Think of it like troubleshooting a car:

Did we fix this exact car before? (Intra-Session Memory)
Is there a manual for this specific model? (Static Compatibility Map)
Do other people with this car use the same parts? (Ecosystem Templates)
What parts do people usually buy together? (Co-occurrence Mining)
Does the mechanic have a rule of thumb? (Heuristic Rules)
Okay, call the AI Chef. (LLM Selection)

By doing this, MemRes solves 86.6% of problems without ever waking up the AI, whereas the old method (PLLM) only solved 54.7%.

2. The "Memory" (The Sticky Note System)

Intra-Session Memory: Imagine you are fixing 100 similar cars in a row. If you figure out that Car #1 needs a specific type of spark plug, MemRes writes a sticky note and uses that same plug for Car #2, #3, and #4 immediately. It doesn't re-solve the problem; it just reuses the answer.
Self-Evolving Memory: If MemRes encounters a tricky problem and figures it out, it writes a "Tip" (a rule) and a "Shortcut" (a quick fix) into its permanent notebook. Next time it sees a similar problem, it reads the notebook instead of guessing. It's like a student who learns from their mistakes and gets smarter every day.

3. The "Detective Work" (Spotting the Traps)

The old AI often got confused by two specific traps:

The "Old Language" Trap: Some code was written in "Python 2" (an old version) but the AI tried to run it in "Python 3" (the new version), causing a crash. MemRes has a special detector that spots the old language immediately and switches the engine to the right version.
The "Name Game" Trap: Sometimes code says import cv2, but the actual package is called opencv-python. The old AI got confused. MemRes has a Knowledge Base (a dictionary) with 200+ of these name mismatches, so it instantly knows, "Ah, cv2 means opencv-python!"

4. The "Reality Check" (The Sandbox)

MemRes doesn't just guess; it tests. It builds a tiny, isolated "sandbox" (a digital test kitchen) to see if the ingredients actually work together.

If the code needs a specific physical part (like a graphics card driver) that doesn't exist in the test kitchen, MemRes knows it can't fix it and stops wasting time.
If the code fails to compile, MemRes tries different versions of the ingredients (like trying a different brand of flour) until it finds one that works.

The Result

Speed: The old method took 2 to 3 minutes per code snippet. MemRes takes about 15 seconds.
Success Rate: The old method failed on nearly half the code. MemRes fixes 86.6% of them.
Cost: Because it rarely calls the expensive AI, it saves a massive amount of computing power and money.

In short: MemRes is a smart, memory-equipped assistant that uses logic, history, and quick rules to solve 9 out of 10 problems instantly. It only calls the "genius AI" for the 1 problem it truly can't solve, making the whole process faster, cheaper, and much more reliable.

1. Problem Statement

Python's ecosystem, comprising over 500,000 packages on PyPI, presents significant challenges for dependency resolution, particularly for legacy code. The core difficulties include:

Ambiguity: Import names often do not map directly to package names (e.g., cv2 maps to opencv-python).
Version Incompatibility: Conflicts between Python 2 and 3, deprecated packages, and specific version constraints.
System Dependencies: Many packages require non-Python system libraries (e.g., C-extensions, GTK, RPi.GPIO) that standard pip installers cannot resolve.
Current Limitations: The state-of-the-art approach, PLLM (a 5-stage RAG+LLM pipeline), achieves only a 54.7% success rate on the HG2.9K benchmark. It frequently fails on deterministic issues (like Python 2 syntax or known import mismatches) by wasting time calling Large Language Models (LLMs), which adds 60–120 seconds per snippet with no added value for these specific cases.

2. Methodology: The MemRes Architecture

MemRes is an agentic system designed to treat the LLM as a last resort. It employs a Confidence Cascade that prioritizes deterministic, fast-resolution methods before invoking an LLM. The system consists of four main components:

A. Intra-Session Memory

Function: Caches solutions proven successful within the current evaluation batch.
Mechanism: If a new code snippet shares high import similarity (Jaccard similarity $\ge$ 0.5) with a previously resolved snippet, the system reuses the solution (e.g., specific pip install commands) without re-inference.
Benefit: Rapidly resolves near-duplicate datasets (e.g., GitHub forks) and reduces redundant LLM calls.

B. Confidence Cascade (6-Level Resolution)

The system attempts resolution through a strict hierarchy of 6 levels. The cascade terminates immediately upon a successful resolution:

Session Memory: Reuses proven versions from the current batch.
Static Compatibility Map: Uses manually curated "known-good" pins for 40+ packages across 8 Python versions.
Ecosystem Templates: Applies pre-validated version sets for common groups (ML, Web, Data Science).
Co-occurrence Mining: Derives weighted co-installation scores from 50k open-source requirements.txt files (filtered to pre-2020 to avoid data leakage).
Heuristic Rules: Applies 45+ package-specific constraints.
LLM Selection: Only invoked if all deterministic levels (1–5) fail.

C. Self-Evolving Memory

Tips & Shortcuts: Following the Mobile-Agent-E paradigm, the system records natural language "tips" and reusable "shortcuts" (validated via quick Docker builds) during the session.
Anti-patterns: Records per-package failure statistics to avoid repeating known errors.
Safety: Memory is strictly isolated within Docker containers and reset between independent evaluation runs to prevent overfitting or cross-run bias.

D. Error Pattern Knowledge Base (KB) & Heuristics

Mappings: Contains 200+ curated import $\to$ pip package mappings and 35 name corrections (e.g., Image $\to$ Pillow).
Semantic Import Analyzer: Unlike simple string matching, this analyzes code usage patterns (e.g., import Image with Image.open() implies Pillow) and detects implicit Python versions (e.g., f-strings imply Python 3.6+).
Python 2 Detection: Identifies Python 2 code via 13 indicators (e.g., print x, raw_input). If detected, it forces a Python 2.7 environment using Docker-out-of-Docker (DooD) to handle legacy execution.
System Dependency Injection: Maps 35+ pip packages to their apt-get system dependencies (e.g., libgl1-mesa-glx) and injects them before pip install to resolve C-extension compilation failures.

3. Key Contributions

Confidence Cascade: A 6-level resolution strategy that reduces LLM calls by >60% while significantly improving accuracy.
Intra-Session Memory: Enables rapid knowledge transfer for similar codebases within a batch, eliminating repeated inference for near-duplicates.
Self-Evolving Memory: A dynamic system that learns tips and shortcuts during the session, validated via sandboxed Docker builds.
Enhanced Error Pattern KB: A knowledge base with 200+ mappings, runtime self-learning capabilities, and specific heuristics for Python 2 detection and system dependency injection.
Performance Benchmarking: Demonstrated a resolution rate of 86.6% on the HG2.9K benchmark, vastly outperforming the previous best (54.7%).

4. Results and Evaluation

The system was evaluated on the HG2.9K dataset (2,890 valid Python snippets) using Gemma-2 9B (10 GB VRAM) on a single machine.

Success Rate:
- MemRes: 86.6% (2,503 snippets resolved).
- PLLM Baseline: 54.7% (1,583 snippets resolved).
- MemRes successfully resolved over 950 snippets that PLLM failed to resolve (confidence = 0).
Efficiency:
- LLM Calls: Reduced to 0.34 calls per snippet (MemRes) vs. 1–5 calls (PLLM).
- No-LLM Success Rate: 68.0% of successful resolutions required zero LLM calls.
- Median Resolution Time: 15.2 seconds for MemRes (vs. ~120–180s for PLLM).
Failure Analysis:
- Remaining failures (13.4%) were primarily due to build/C-extension compilation issues (~~50%), unresolvable import errors for obscure/deprecated packages (~~33%), and timeouts (~15%).

5. Significance

Deterministic First Approach: MemRes proves that the majority of dependency resolution failures have deterministic root causes (syntax errors, import mismatches, version conflicts) that do not require generative AI. By prioritizing rules and heuristics, the system achieves higher accuracy and speed.
Cost and Latency Reduction: By bypassing the LLM for 68% of successful cases, MemRes drastically reduces token usage (~75% reduction) and inference time, making agentic dependency resolution scalable and cost-effective.
Robustness to Legacy Code: The specific handling of Python 2 code and system-level dependencies (via DooD and apt injection) addresses critical gaps in existing tools that assume modern, pure-Python environments.
Future Direction: The paper establishes a new paradigm for agentic systems where "search-heavy" methods (LLMs) are complemented or replaced by "knowledge-heavy" deterministic pipelines, suggesting a shift in how AI agents interact with complex software engineering tasks.

Conclusion: MemRes represents a significant leap in automated Python dependency resolution, moving away from pure LLM reliance toward a hybrid architecture that leverages memory, curated knowledge, and deterministic heuristics to achieve state-of-the-art performance.