🤖 machine learning

Cooperative Coevolution versus Monolithic Evolutionary Search for Semi-Supervised Tabular Classification

This paper proposes a cooperative coevolutionary method (CC-SSL) for semi-supervised tabular classification in extreme low-label regimes, demonstrating that while it achieves competitive test performance comparable to a monolithic evolutionary baseline (EA-SSL), the monolithic approach often exhibits superior search diversity and faster convergence to target fitness.

Original authors: Jamal Toutouh

Published 2026-04-21

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jamal Toutouh

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a teacher trying to teach a class of students (an AI model) how to recognize different animals. However, you only have a few flashcards with pictures and names (labeled data), but you have a huge pile of pictures without names (unlabeled data). This is the "low-label regime" the paper talks about.

The goal is to use those few flashcards to teach the students, and then let them guess the names of the unlabeled pictures. If they guess correctly, you add those new pictures to your flashcard pile and keep teaching. This is called Semi-Supervised Learning.

The problem? If the students guess wrong early on, they might teach themselves the wrong things, and the whole class gets confused.

This paper asks a big question: How should we design the "brain" of the teacher to avoid these mistakes? Specifically, should we build the teacher as one giant, complex brain, or should we build it as two smaller brains working together?

Here is the breakdown of the paper's story:

1. The Two Approaches: The Solo Artist vs. The Duo Team

The author, Jamal Toutouh, tested two ways to build this "teacher" using a method called Evolutionary Search (which is like natural selection: you create many versions, keep the best ones, and mix them to make better ones).

The Solo Artist (EA-SSL): Imagine one single, giant brain trying to do everything at once. It has to figure out which features of the animal to look at (e.g., "Is it furry? Does it have stripes?") AND how to decide if a student's guess is good enough to be added to the lesson plan. It's a "monolithic" approach—one big package.
The Duo Team (CC-SSL): Imagine splitting the job into two separate specialists who work together.
- Specialist A is the "Feature Detective." Their only job is to find the best two different ways to look at the animals (e.g., one view focuses on shape, the other on color).
- Specialist B is the "Rule Maker." Their only job is to decide the rules for accepting new guesses (e.g., "Only accept a guess if both views agree").
- They evolve separately but constantly check in with each other. If the Detective finds a great new way to look at animals, the Rule Maker adapts to use it.

2. The Experiment: A Tough Test

The author tested these two approaches on 25 different datasets (like 25 different classrooms with different types of students) and three different levels of difficulty:

1% Labeled: The teacher has almost no flashcards. (The "Extreme Low-Label" regime).
5% Labeled: A few flashcards.
10% Labeled: A decent amount of flashcards.

They compared their "Evolutionary Teachers" against three standard, simpler methods (like a basic self-taught student or a graph-based student) to see who learned best.

3. The Results: Who Won?

The Good News:
Both the Solo Artist (EA-SSL) and the Duo Team (CC-SSL) were much better than the standard, simpler methods, especially when the teacher had very few flashcards (1%). They both figured out how to learn from the unlabeled pictures without getting confused.

The Surprising Twist:
When comparing the Solo Artist vs. the Duo Team directly:

Final Grades: They ended up with almost the exact same test scores. The Duo Team didn't beat the Solo Artist, and the Solo Artist didn't beat the Duo Team. It was a statistical tie.
The Journey: However, how they got there was different.
- The Solo Artist was more "exploratory." It kept a wider variety of ideas in its head for longer (higher diversity) and found slightly better intermediate solutions during the learning process.
- The Duo Team settled down faster. It stopped exploring new ideas earlier and stuck to what it knew.
- Speed: The Solo Artist reached its peak performance in fewer "generations" (fewer steps of evolution), but because the Duo Team's steps were slightly more complex, they took about the same amount of real-world time to finish.

4. The "Ghost" in the Machine (Pseudo-Labels)

The researchers also checked if one method was "cheating" by accepting too many wrong guesses (bad pseudo-labels). They looked at:

Volume: How many new guesses did they accept?
Stability: Did the model get worse after adding new guesses?
Optimism: Did the model think it was doing better than it actually was?

The Verdict: Both methods behaved almost identically. Neither was more aggressive or more cautious than the other. They both managed the risk of learning from wrong guesses in the same way.

The Big Takeaway

Think of it like building a house.

The Solo Artist is a master builder who does the plumbing, electrical, and framing all by themselves.
The Duo Team is a plumber and an electrician working together, constantly talking to each other.

The paper found that for this specific job (teaching a computer with very few examples), both methods built a house of the same quality. The master builder (Solo) worked a bit more efficiently in terms of steps, while the team (Duo) worked just as well but with a different rhythm.

Why does this matter?
It tells us that we don't need to make our AI systems incredibly complex (splitting them into many cooperating parts) to get great results. Sometimes, a well-designed, single, focused approach works just as well as a complex team, saving us from unnecessary engineering headaches. However, the "Team" approach is still a valid and powerful tool if we want to try different strategies in the future.

In short: When you have very little data, using evolutionary search (letting the computer evolve its own learning rules) is a winning strategy. Whether you split that evolution into a team or keep it in one big brain doesn't change the final grade, but it does change the journey.

1. Problem Statement

The paper addresses Semi-Supervised Learning (SSL) for tabular classification in the extreme low-label regime (specifically 1%, 5%, and 10% labeled data).

The Challenge: In low-label settings, standard SSL methods (like self-training) are sensitive to hyperparameters, view construction, and pseudo-labeling policies. Incorrect pseudo-labels can dominate training, leading to performance degradation (confirmation bias).
The Gap: While evolutionary algorithms (EAs) can optimize these coupled decisions, it is unclear whether Cooperative Coevolution (CC)—which factorizes the search into interacting sub-populations—outperforms a Monolithic EA (which encodes all decisions in a single genotype) when computational budgets are matched.
Goal: To determine if factorizing the search space (separating feature view construction from pseudo-labeling policy) yields better predictive performance, stability, or convergence efficiency compared to a monolithic approach.

2. Methodology

A. Core Framework: Pseudo-Labeling SSL

Both proposed methods utilize a wrapper-style SSL loop (Algorithm 1) based on self-training with two views:

View Builder ( $a$ ): Selects two complementary feature subspaces (via binary masks, optional projections, and discretization).
Policy ( $b$ ): Controls the pseudo-labeling process, including:
- Base learner configuration (regularized linear classifier).
- Confidence thresholds (initial, decay, minimum).
- Acceptance rules (margin constraints, disagreement veto between views, per-class caps).
- Stopping criteria.

B. Evolutionary Approaches

The study compares two evolutionary strategies that optimize the pair $(a, b)$ :

CC-SSL (Cooperative Coevolution):
- Structure: Two separate populations: Pop-A (evolving view builders) and Pop-B (evolving policies).
- Collaboration: Fitness is assigned via a collaborative mechanism. An individual in Pop-A is evaluated against a set of partners from Pop-B (one elitist best + $R$ random samples). The fitness is the maximum performance achieved with any partner.
- Mechanism: This isolates the optimization of views and policies, allowing them to co-adapt.
EA-SSL (Monolithic Evolutionary Algorithm):
- Structure: A single population where each individual encodes the full genotype $(a, b)$ .
- Mechanism: Standard evolutionary operators (selection, crossover, mutation) are applied to the combined genotype.
- Control: The search budget (total evaluations), fitness function, and building blocks are strictly matched to CC-SSL to ensure a fair comparison.

C. Experimental Setup

Datasets: 25 heterogeneous OpenML datasets (11 binary, 14 multiclass).
Labels: Labeled fractions of 1%, 5%, and 10%.
Baselines: Compared against three lightweight SSL baselines (Self-Training, Heuristic Co-Training, Label Spreading) and strong supervised classifiers.
Metrics:
- Performance: Test MacroF1 and Accuracy.
- Dynamics: Population diversity, Best-so-far fitness.
- Efficiency: Time-to-Target (TTT) and Generations-to-Target (GTT).
- Diagnostics: Pseudo-label volume, ProbeDrop (stability), and Validation Optimism.

3. Key Contributions

Joint Optimization Formulation: Proposes a unified framework that couples two-view feature subset construction with pseudo-labeling policy design for tabular data.
Controlled CC vs. EA Comparison: Provides a rigorous, matched-budget comparison between cooperative coevolution and monolithic search in the context of SSL, isolating the effect of factorization.
Staged Tuning Protocol: Introduces a three-stage tuning protocol (population sizes, operator probabilities, fitness weights) restricted to a small development set to prevent overfitting and manage high-dimensional search spaces.
Comprehensive Benchmark: Evaluates methods across 25 datasets with multi-metric reporting (performance, convergence, and pseudo-label dynamics) and reproducible artifacts.

4. Key Results

A. Performance vs. Baselines (RQ1)

Superiority: Both CC-SSL and EA-SSL significantly outperform lightweight SSL baselines (Self-Training, Label Spreading, Co-Training) in the extreme low-label regime (1% labeled data).
Margin: The performance gap is largest at 1% labeled data. As the labeled fraction increases (5%, 10%), the advantage over baselines narrows, and strong supervised classifiers become competitive.

B. CC-SSL vs. EA-SSL (RQ2)

Final Performance: There is no systematic advantage for CC-SSL over EA-SSL in final test MacroF1 or accuracy. Most comparisons are statistical draws.
Search Dynamics:
- EA-SSL maintains higher population diversity and achieves higher best-so-far fitness during the search process.
- CC-SSL reduces diversity faster and shows lower within-subpopulation dispersion.
Conclusion: Factorization (CC) does not translate to better final predictive performance compared to monolithic encoding under the same budget.

C. Convergence and Efficiency (RQ4)

Time-to-Target (TTT): Wall-clock time to reach a target fitness is comparable between both methods.
Generations-to-Target (GTT): EA-SSL converges faster in terms of evolutionary generations, particularly in multiclass settings. This suggests the monolithic approach finds high-quality solutions in fewer iterations, even if the wall-clock time is similar due to evaluation overhead.

D. Pseudo-Label Dynamics (RQ3)

No Significant Differences: There are no statistically significant differences between CC-SSL and EA-SSL regarding:
- Pseudo-label volume: Both accept similar numbers of pseudo-labels.
- Stability (ProbeDrop): Both show negligible degradation on held-out probe sets (ProbeDrop $\approx$ 0 in >96% of runs).
- Validation Optimism: Both exhibit similar gaps between validation and test performance.
Implication: The differences in search behavior (diversity/fitness trajectories) do not correlate with systematic changes in pseudo-label quality or stability.

5. Significance and Conclusion

Practical Insight: For semi-supervised tabular classification in low-label regimes, monolithic evolutionary search is a robust and efficient alternative to cooperative coevolution. While CC is theoretically appealing for decomposing complex problems, it does not yield superior final performance or stability in this specific domain when budgets are matched.
Efficiency: The monolithic approach (EA-SSL) demonstrates higher algorithmic convergence efficiency (fewer generations to target) and maintains higher diversity, making it a preferable choice if computational resources are measured in evolutionary steps.
Future Directions: The authors suggest investigating alternative credit assignment strategies in CC (e.g., marginal contribution) or hybrid island models to potentially unlock the benefits of factorization in future work.

In summary, the paper demonstrates that while evolutionary optimization significantly boosts SSL performance over traditional baselines in low-data regimes, the specific choice between cooperative coevolution and monolithic search has little impact on the final model quality, with the monolithic approach showing slightly better convergence characteristics.