🤖 machine learning

SeBA: Semi-supervised few-shot learning via Separated-at-Birth Alignment for tabular data

The paper proposes SeBA, a novel semi-supervised few-shot learning framework for tabular data that eliminates the need for difficult-to-define data augmentations by aligning independent, complementary views based on nearest-neighbor correspondence, thereby achieving state-of-the-art performance across various benchmarks.

Original authors: Kacper Jurek, Wojciech Batko, Marek Śmieja, Marcin Przewięźlikowski

Published 2026-05-12

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Kacper Jurek, Wojciech Batko, Marek Śmieja, Marcin Przewięźlikowski

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a computer to recognize different types of cars (like sedans, SUVs, and trucks) based on a spreadsheet of data. The problem is, you only have five labeled examples (e.g., "This is a sedan") but you have thousands of unlabeled rows where the car type is hidden. This is called Semi-supervised Few-Shot Learning.

For images (like photos of cars), computers are great at this. They can take a photo, crop it, turn it sideways, or change the colors, and say, "Hey, this is still the same car!" These "twisted" versions help the computer learn.

But for tabular data (spreadsheets with numbers and categories), this doesn't work. You can't really "crop" a spreadsheet. If you randomly change a car's mileage or turn a "Red" color into "Blue," you might create a fake car that never existed (like a car with 0 miles but 100 years old). This confuses the computer.

Enter SeBA (Separated-at-Birth Alignment).

The authors of this paper say: "Let's stop trying to twist the data. Instead, let's split it."

The Core Idea: The "Split Personality" Analogy

Imagine you have a detailed biography of a person (the data row). Instead of trying to create a fake version of that person, SeBA takes the biography and splits it in half right at the beginning ("at birth").

The Feature View: You give the computer the first half of the story (e.g., "Age," "Job," "City").
The Target View: You give the computer the second half (e.g., "Salary," "Hobbies," "Car Type").

Now, here is the magic trick:

The computer looks at the Feature View of Person A and tries to guess who their "best match" is in the Target View.
It finds that Person A's "Feature View" looks very similar to Person B's "Feature View."
The computer then checks: "Do Person A and Person B have similar 'Target Views'?"
If they do, the computer learns: "Ah! These two people are related, even though I only saw half their story."

It's like a game of "Guess the Twin."

You show the computer a photo of Twin A's left side.
You show it a photo of Twin B's left side.
The computer says, "Those look like the same person!"
Then, it checks the right sides. If the right sides also match, the computer learns that "Left Side A" and "Left Side B" belong to the same "Right Side" family.

By doing this over and over with thousands of random splits, the computer learns a very smart way to organize the data without ever needing to invent fake data or twist the numbers.

Why is this better?

No More Fake Data: Previous methods tried to "augment" (twist) the data, which often broke the logic of the spreadsheet (e.g., making a car have negative mileage). SeBA doesn't do that. It just uses the real data, split in two.
The "Nearest Neighbor" Map: The computer builds a map based on who is closest to whom. It learns that if two rows look similar in the "Feature" half, they likely belong to the same group in the "Target" half.
Lightweight: The computer model used is small and simple (like a basic calculator), so it doesn't get confused or "overthink" when there are very few labeled examples.

The Results

The authors tested this on many different "spreadsheets" (datasets) involving things like:

Medical diagnoses
Credit risk
Car sales
DNA data

They found that SeBA was the best at guessing the correct labels in almost every test, especially when there were very few labeled examples (1-shot or 5-shot). It was particularly good at handling data that was messy, had many columns, or was mostly just categories (like "Yes/No" or "Red/Blue").

In a Nutshell

SeBA is a new way to teach computers to learn from spreadsheets when you have very few labeled examples. Instead of trying to "juggle" the data by creating fake versions, it simply splits the data in two and teaches the computer to match the halves. This avoids the confusion of making up fake data and results in a smarter, more accurate model for real-world table-based problems.

Technical Summary: SeBA - Semi-supervised Few-shot Learning via Separated-at-Birth Alignment for Tabular Data

1. Problem Statement

The paper addresses Semi-supervised Few-shot Learning (SS-FSL) for tabular data. In this setting, a model must learn to classify data using a very small number of labeled examples (the support set) while leveraging a large pool of unlabeled data.

While SS-FSL is well-established in computer vision (CV) and natural language processing (NLP), it remains underexplored for tabular data. The primary bottleneck for applying standard SS-FSL methods to tabular data is the reliance on Self-Supervised Learning (SSL) frameworks that require data augmentation.

The Augmentation Challenge: In CV, augmentations (e.g., cropping, rotation) preserve semantic meaning. In tabular data, defining meaningful augmentations is non-trivial. Common strategies like zero-masking, Gaussian noise, or sampling from marginal distributions often distort semantics, generate out-of-distribution samples, or violate feature constraints (e.g., making a categorical variable non-integer), thereby undermining the effectiveness of SSL.
Current Limitations: Existing tabular pretraining methods often sidestep SSL entirely, relying instead on cluster detection, diffusion models, or meta-learning on self-generated tasks, which may not fully exploit the potential of contrastive learning paradigms.

2. Methodology: Separated-at-Birth Alignment (SeBA)

The authors propose SeBA, a joint-embedding framework that eliminates the need for hand-crafted data augmentations. Instead of creating positive pairs via augmentation, SeBA constructs them by separating the data "at birth" into two independent, complementary views.

Core Mechanism

Data Separation: For every minibatch of unlabeled data, a binary mask $m$ is sampled. This mask splits the features of each data point $x$ into two views:
- Feature View ( $x_f$ ): Features where the mask is 0 ( $x \odot (1-m)$ ).
- Target View ( $x_t$ ): Features where the mask is 1 ( $x \odot m$ ).
  These views are complementary and independent.
Target Similarity Graph: Positive pairs are defined based on the Target View. For a sample $x$ , its nearest neighbor $x'$ is identified within the batch using only the features in the target view ( $x_t$ and $x'_t$ ). This nearest-neighbor relationship serves as the "ground truth" for semantic similarity.
Alignment Objective: The model is trained to align the representation of the Feature View ( $x_f$ ) with the representation of its nearest neighbor's feature view ( $x'_f$ ).
- Conditioned Projector: Since the feature view $x_f$ does not inherently contain information about the mask $m$ used to create it, SeBA employs a conditioned projector $\pi(h, m)$ . This projector takes the encoder output $h$ and the mask $m$ as inputs, allowing the model to learn a task-specific latent space that accounts for the specific separation scheme.
- Loss Function: The alignment is optimized using the InfoNCE loss, which pulls together the representations of positive pairs ( $z, z'$ ) and pushes apart unrelated samples.
Handling Tabular Specifics:
- Type-Aware Separation: To prevent splitting a single categorical variable (encoded via one-hot) across views, the mask is applied to the original representation before encoding.
- Missing Data Imputation: Separated features are replaced with zeros (zero imputation), which the authors found to be superior to sampling from marginal distributions for their lightweight MLP encoder.
- Ensembling: To handle the diversity of datasets and avoid selecting a suboptimal separation ratio, the authors train multiple models with different fixed separation ratios ( $r \in \{0.1, \dots, 0.5\}$ ) and ensemble their predictions during inference.

3. Key Contributions

Novel Pretraining Framework: Introduction of SeBA, the first SS-FSL method for tabular data that replaces augmentation-based positive pair construction with a nearest-neighbor graph derived from data separation.
Theoretical Justification: A theoretical analysis for Gaussian data demonstrating that the probability of a nearest neighbor in the target view belonging to a different class decays exponentially as the dimensionality of the target view and the separation between classes increase. This validates that nearest-neighbor matching in the target view yields meaningful positive pairs.
Empirical Validation: Extensive experiments showing that SeBA generates an output space that improves the feature-label relationship, outperforming state-of-the-art baselines in the majority of cases.

4. Experimental Results

The authors evaluated SeBA on 8 standard benchmark datasets (from OpenML-CC18) and 3 additional challenging datasets (high-dimensional and sparse: CNAE, Devnagari, FashionMNIST).

Performance: SeBA achieved state-of-the-art (SOTA) performance in 17 out of 24 instances across 1-shot, 5-shot, and 10-shot settings. It was among the top three methods in the remaining 7 instances.
Robustness: SeBA demonstrated superior stability compared to competitors like STUNT and D2R2-c, exhibiting lower variance in results across random seeds.
Specific Strengths:
- SeBA achieved the best performance on datasets with purely categorical values (DNA, PIX).
- The performance gap between SeBA and baselines was particularly pronounced on high-dimensional datasets (e.g., PIX, SEM).
Ablation Studies:
- Conditioned Projector: Essential for performance, especially in low-data regimes (1-shot).
- Encoder Architecture: A lightweight 2-layer MLP outperformed a more complex FT-Transformer architecture.
- Preprocessing: Normalization combined with zero imputation yielded the best results.
- Separation Ratio: Ensembling models trained with different ratios provided the most robust results compared to a single fixed ratio.

5. Significance and Claims

The paper claims that the inferior performance of SSL methods in tabular few-shot learning is not an inherent limitation of tabular data, but rather a consequence of relying on inappropriate augmentation strategies.

By rethinking SSL through Separated-at-Birth Alignment, the authors demonstrate that:

It is possible to construct meaningful positive pairs for tabular data without manual augmentation.
This approach yields representations that significantly improve the feature-label relationship, as evidenced by increased label consistency in the latent space compared to the input space.
SeBA opens a new avenue for the SS-FSL paradigm in tabular data, offering a practical, lightweight, and highly effective solution for domains like medicine, finance, and science where labeled data is scarce.

The work positions SeBA as a foundational step for data-constrained applications, suggesting that future research should explore similar augmentation-free alignment strategies for tabular representation learning.