Can AI be Easy? Lessons Learned from the EZR.py Toolkit

The paper argues that reading and refactoring code to create a minimal, unified Python toolkit (EZR.py) reveals that simple, lightweight algorithms can outperform complex state-of-the-art tools in tabular software engineering optimization tasks while requiring significantly less data and computational resources.

The Gist

The Big Idea: Do We Really Need Giant AI Machines?

Imagine the current trend in Artificial Intelligence is like building a massive, high-tech skyscraper to solve a simple problem, like finding a lost key in a garden. Everyone says, "You need a billion-dollar crane, a team of 50 engineers, and a supercomputer to find that key."

The authors of this paper say: "Wait a minute. You don't need a skyscraper. You just need a flashlight and a map."

⚠️ Important Scope Note:
This paper is not about all of Artificial Intelligence. It focuses specifically on one corner of the field: Tabular Software Engineering problems. This means tasks involving tables of numbers and specific goals, such as optimization, classification, prediction, regression, and basic text mining.

What this does NOT cover: It does not address Generative AI tasks (like ChatGPT or LLMs that generate new code, stories, or images). The authors have not tackled those generative tasks yet; applying these lessons to them is future work they hope to do. The claim here is that for tabular tasks, we are overcomplicating things.

They argue that for a huge chunk of software engineering problems (specifically those involving tables of numbers and goals), we are overcomplicating things. They built a tiny toolkit called EZR (only 400 lines of code) that does the job of massive, heavy software libraries, but it runs 500 times faster and needs almost no data to learn.

The Toolkit: A Swiss Army Knife vs. A Warehouse

Most modern AI tools are like a warehouse full of specialized tools: a giant saw for wood, a heavy drill for metal, a complex laser for glass. You have to buy the whole warehouse (installing huge libraries like pandas and sklearn) just to use one tool.

EZR is a Swiss Army Knife.
The authors realized that if you look closely at how these different tools work for tabular data, they are actually doing the same basic things. They stripped away the fancy packaging and found that:

• Classification (sorting things into groups)
• Clustering (finding natural groups)
• Optimization (finding the best solution)
• Text Mining (finding relevant documents)

...all rely on the same three simple building blocks:

1. Num: A bucket that counts numbers and averages them.
2. Sym: A bucket that counts symbols (like words or categories).
3. Data: A box that holds rows of information.

Instead of building a new engine for every task, EZR uses these same buckets to do everything. It's like realizing that a spoon, a fork, and a knife are all just handles with a specific shape at the end; you don't need three different factories to make them.

The Six Surprising Discoveries

The paper tested this tiny toolkit on 120+ real-world software problems. Here is what they found, using simple metaphors:

1. The "Heavy" Myth

The Belief: To do AI, you need a massive computer and huge libraries.
The Reality: For tabular tasks, you can do it with a tiny script.
Analogy: It's like thinking you need a full orchestra to play a lullaby. The authors showed that a single violin (EZR) can play the same tune just as well, without needing the 50 other musicians (the heavy dependencies).

2. The "Separate Subjects" Myth

The Belief: Sorting data, grouping data, and finding patterns are totally different subjects that need different code.
The Reality: For tabular data, they are nearly identical under the hood.
Analogy: It's like thinking driving a car, driving a truck, and driving a bus are completely different skills. The authors showed that once you strip away the size of the vehicle, the steering wheel and pedals are the same. They wrote 30 lines of code that handle all three tasks.

3. The "Tree" Myth

The Belief: Decision trees (like flowcharts for AI) for predicting numbers are totally different from those for predicting categories.
The Reality: They are the same tree; just the fruit is different.
Analogy: Imagine a tree that grows apples. If you want oranges, you don't need a new tree species; you just change the label on the branch. The authors showed that switching between predicting numbers and categories is a one-line change in the code.

4. The "Old vs. New" Myth

The Belief: Newer, complex search methods (Local Search with restarts) are always better than old, simple ones (Simulated Annealing from 1983).
The Reality: For optimization tasks, the old method is often just as good, or better.
Analogy: Imagine trying to find the lowest point in a foggy valley. The "new" method says, "If you get stuck, jump back to the start and try again!" The "old" method says, "If you get stuck, take a small, random step up to shake yourself loose." The authors found that the "shake loose" method (1983) worked just as well as the "jump back" method, but without the chaos of constantly restarting.

5. The "More Data" Myth

The Belief: You need thousands of labeled examples and thousands of features (variables) to build a good model.
The Reality: For tabular problems, you need very few labels and very few features.
Analogy: Imagine trying to guess the winner of a race. You might think you need to know the runner's height, weight, shoe size, diet, sleep schedule, and blood type (thousands of features). The authors found that knowing just two or three things (like "shoe size" and "sleep") was enough to predict the winner accurately. They also found that labeling just 50 examples was enough to train a model that usually requires thousands.

6. The "Text Mining" Myth

The Belief: To find relevant documents in a huge library, you need massive AI models (LLMs) with billions of parameters.
The Reality: For simple document retrieval, a simple math trick works better.
Analogy: Imagine looking for a specific needle in a haystack. The high-tech approach uses a giant magnet that weighs a ton. The authors used a simple "Complementary Bayes" trick (30 lines of code) that acts like a sharp needle. It found the relevant documents faster and with fewer mistakes than the giant magnet, and it exposed a flaw in how the giant magnet was being used.

The "Active Learning" Superpower

One of the coolest things EZR does is Active Learning.

• Passive Learning: Imagine a student who reads 1,000 pages of a textbook to learn a concept.
• Active Learning (EZR): Imagine a student who reads 10 pages, realizes what they don't understand, and asks the teacher only for those specific 10 pages.

EZR acts like that smart student. It looks at the data, figures out which few examples are the most confusing or important, and asks for labels on only those. This saves massive amounts of time and money because humans don't have to label thousands of boring, repetitive examples.

The Conclusion: Read the Code, Don't Just Trust the Hype

The paper's main message is a call to action for developers and researchers: Read the code.

The authors argue that we have stopped reading code and started blindly trusting "black box" AI tools. By actually reading the code of these tools, they realized that many of them are doing the same thing in different ways.

The Takeaway:
Before you buy a Ferrari to drive to the grocery store, try walking.

• If you can solve your problem with a tiny, simple toolkit (like EZR) for tabular software engineering tasks, you save time, money, and energy.
• If the simple toolkit doesn't work, then you know you genuinely need a complex solution.
• But if you just assume you need the complex solution because "everyone else is doing it," you might be carrying a heavy backpack when you only needed a pocket knife.

The authors conclude that in the world of software engineering optimization, less is often more, and the best way to find the "less" is to carefully read and simplify the code we already have.

Final Note on Scope: These lessons are demonstrated specifically for tabular SE tasks. Whether these simple methods extend to the complex world of Generative AI (LLMs) is an open question and a goal for future work. The authors are not claiming to have solved all of AI, but rather that we have overcomplicated a very large and important slice of it.

Technical Summary

Technical Summary: Can AI be Easy? Lessons Learned from the EZR.py Toolkit

Problem Statement

Recent discourse in software engineering and artificial intelligence suggests that human developers no longer need to read code, positing that AI (specifically Large Language Models) has become the new compiler. Concurrently, the field of software engineering (SE) optimization often relies on heavy, dependency-laden libraries (e.g., pandas, scikit-learn, SMAC3) and assumes that solving complex problems requires increasing data volume, feature counts, and algorithmic complexity.

This paper challenges two prevailing assumptions within the domain of tabular software-engineering optimization tasks (where rows represent configurations or projects, $x$ are independent attributes, and $y$ are expensive-to-obtain goals):

1. That AI infrastructure must be large and dependency-heavy.
2. That distinct algorithmic families (classification, clustering, optimization, active learning) require separate, complex implementations and massive datasets.

The authors argue that careful reading and refactoring of existing code can reveal that many "sophisticated" methods are structurally redundant, and that lightweight, unified toolkits can rival or exceed state-of-the-art (SOTA) performance with orders of magnitude less complexity.

Methodology

The core methodology is code refactoring through reading. The authors spent years reading, rewriting, and refactoring diverse AI tools to identify and eliminate redundancies. The result is EZR.py, a 400-line Python toolkit with no heavy third-party dependencies (relying only on the Python standard library).

The EZR Substrate

EZR is built on a minimal substrate consisting of four classes and one update primitive:

• Num: Summarizes numeric columns (tracking mean, second moment, standard deviation, and a "heaven" value for goal direction).
• Sym: Summarizes symbolic columns (tracking frequency counts).
• Cols: A factory that parses CSV headers to instantiate Num or Sym objects based on naming conventions (e.g., "!" for class, "+" for maximization, "-" for minimization).
• Data: Holds rows and the associated column summaries.
• add: A polymorphic update primitive. It incrementally updates Num statistics using Welford's algorithm and Sym frequency counts. Crucially, it supports both addition and subtraction ($w=1$ or $w=-1$), allowing rows to be moved between datasets in constant time without retraining.

Algorithmic Implementation

Using this substrate, the authors implemented six distinct AI capabilities, demonstrating that they share a common underlying machinery:

1. Classification & Clustering (70 lines): Implemented Naïve Bayes, k-means, and k-means++. The substrate eliminates the distinction between "fitting" and "using"; the Data object is inherently a fitted model.
2. Trees (43 lines): Unified implementation of classification and regression trees. The only difference is the scoring function (disty for regression, entropy for classification).
3. Optimization (56 lines): Implemented Simulated Annealing (SA) and Local Search (LS) as variations of a single (1+1) evolutionary algorithm. Both share the same oneplus1 loop, differing only in their mutation and acceptance strategies.
4. Active Learning (80 lines): An active learner that maintains two datasets: best (top $\sqrt{N}$ rows) and rest (remaining rows). New labels trigger a constant-time rebalance using the add/sub primitives, avoiding the full retraining required by ensemble methods like SMAC3.
5. Text Mining (30 lines): A relevance filter using Complementary Naïve Bayes (CNB). Instead of predicting the most likely class, CNB predicts the class a document is least likely to belong to, effectively filtering out irrelevant documents.

Experimental Setup

The toolkit was evaluated on 124 multi-objective optimization tasks from the MOOT repository, covering software configuration, performance tuning, defect prediction, and text mining.

• Comparators: EZR was compared against SOTA tools including SMAC3 (optimization), SHAP/LIME (explanation), and FASTREAD (text mining).
• Metrics: Performance was measured by "wins" (normalized regret), label efficiency (number of labels to reach optimum), feature efficiency (number of features used), and runtime.
• Statistical Rigor: Results were aggregated over 20+ repeats. Differences smaller than Sawilowsky's threshold (0.35$\sigma$) were clamped to zero to avoid over-interpreting trivial variations.

Key Results

1. Performance vs. Complexity

• Optimization: On 20 MOOT benchmarks, Simulated Annealing (in its 1983 default configuration, without restarts) matched or outperformed Local Search variants and SMAC3. SA achieved a mean win score of 98–99, while LS required restarts to approach similar performance.
• Speed: The EZR active learner ran 500× faster than SMAC3. This is because EZR updates models in constant time ($O(1)$) via row swapping, whereas SMAC3 requires rebuilding an ensemble of trees for every new label.
• Label Efficiency: EZR's active learner reached 85–95% of the reference optimum using fewer than 100 labels, whereas SOTA methods often require thousands.
• Feature Efficiency: Despite datasets containing hundreds or thousands of features, EZR's trees consistently built effective models using fewer than 10 variables. Performance did not degrade as the number of available features increased.

2. Text Mining

• Using Complementary Naïve Bayes, EZR achieved high recall on systematic literature review (SLR) tasks with fewer than 100 labels, compared to 300–800 labels required by FASTREAD (which uses linear SVMs).
• The study exposed a methodological gap in prior work: by measuring False Alarm rates (which previous studies ignored), the authors found that a recommended normalization step in CNB (by Rennie et al.) actually inflated false alarms, a flaw masked by the complexity of the original tools.

3. Code Size and Dependencies

• EZR: 400 lines of code, Python stdlib only, <1 MB install size.
• SOTA Comparators: Often >200k lines, requiring pandas, sklearn, numpy, and heavy compute clusters for reproducibility.

Significance and Claims

The paper does not claim that AI is universally simple or that LLMs are obsolete for all tasks. Instead, it makes a modest but specific claim regarding tabular SE optimization:

1. Reading Code is a Valid Research Method: The authors argue that "reading and refactoring code" is a useful method for generating insight. By stripping algorithms down to their core, they demonstrated that many seemingly distinct algorithms (Naïve Bayes, k-means, SA) collapse into a few lines of shared code.
2. Minimalism Rivals Complexity: Small, unified toolkits can rival large, specialized libraries. The "heavy" approach often introduces unnecessary complexity, maintenance burdens, and computational costs without proportional gains in performance.
3. Re-evaluating Assumptions: The results challenge the "No Free Lunch" assumption that more data and features always yield better models. In the tested domain, less is more: fewer labels, fewer features, and simpler models yielded superior or equivalent results.
4. Practical Implication: Practitioners should run simple baselines before deploying heavy pipelines. If a simple model matches a complex one, the complex one is "technical debt."

The authors conclude that while the "AI is the new compiler" narrative may hold for generation or perception tasks, in the domain of tabular optimization, careful reading and simplification remain powerful tools for generating insight and efficiency. The paper invites the community to apply similar scrutiny to other "sophisticated" methods, suggesting that many may be simplifiable.