Technical Summary: Understanding Domain-Aware Distribution Alignment in Budgeted Entity Matching

Problem Definition

Entity Matching (EM) is a fundamental task in data integration aimed at determining whether records from different sources refer to the same real-world entity. While state-of-the-art EM systems often rely on fine-tuning large Pre-trained Language Models (PLMs) or Large Language Models (LLMs), these approaches face practical limitations, including high training costs and the scarcity of labeled data for new tasks.

To address data scarcity, recent work has explored low-resource and budget-aware EM. Specifically, the Budget-Aware Entity Matching Across Domains (EMAD) setting, introduced by BEACON [12], aims to train an effective EM model for a target domain by selectively leveraging labeled data from other domains under a fixed annotation budget. BEACON utilizes Train–Validation Distribution Fitting (TVDF), a distribution-aware sampling strategy that selects out-of-domain samples to improve the alignment between the training distribution and a target (validation) distribution.

Despite BEACON's strong empirical performance, the underlying mechanisms driving its success remain unclear. It is not well understood how distribution alignment behaves under varying data constraints, the impact of different label availability scenarios, or whether the specific assumptions made by TVDF (e.g., centroid-based representation, unsupervised selection) are necessary for its effectiveness.

Methodology

This paper investigates the BEACON framework and the TVDF method through a series of targeted experiments designed to isolate the role of distribution alignment. The study focuses on three primary dimensions:

1. Label Availability: The authors analyze TVDF under varying levels of supervision, including settings with limited or missing in-domain and out-of-domain labels. They explore Label-Aware Sampling, where embeddings are partitioned into positive (matching) and negative (non-matching) subsets to align class-specific distributions rather than treating data as a homogeneous pool.
2. Alternative Domain Representations: The study evaluates whether richer representations of data distributions improve alignment beyond the standard centroid-based approach. Three alternatives are tested:
   • TVMed: Uses the medoid (most central sample) instead of the centroid.
   • TVDFVAR: Jointly considers centroid and variance alignment to capture the spread of the distribution.
   • TVCoverage: A coverage-based method that measures how well the training embedding space geometrically covers the proxy distribution, rather than relying on a single summary statistic.
3. Domain-Agnostic Distribution Alignment: The authors isolate the distribution-alignment component of TVDF by applying it to a domain-agnostic downsampling scenario. In this setting, the goal is to reduce the size of a single training dataset to match a target proxy distribution without explicit domain partitioning. They compare TVDF against random downsampling, Nearest to Centroid (NC), and Nearest Neighbor (NN) selection.

Experiments were conducted on the WDC Multi-Dimensional Entity Matching Benchmark, partitioned into 11 product categories, using a RoBERTa backbone.

Key Results

1. Label Availability

Contrary to the intuition that label information should always improve performance, the unsupervised TVDF model achieved the highest mean macro F1 score (0.716) and weighted F1 score (0.719) across most budget settings.

• Label-aware variants (TVDF with In-Domain, Out-of-Domain, or both labels) generally underperformed the unsupervised baseline.
• The authors suggest that partitioning limited data into smaller class-specific distributions may negatively impact smaller or underrepresented domains.
• However, offline greedy experiments indicated that label information can improve performance if applied selectively on a per-domain basis, suggesting its utility is context-dependent.

2. Domain Representations

The study found that simple centroid representations remain the most effective approach in practice.

• TVDF (centroid-based) consistently achieved the highest average performance (0.716 macro F1).
• More complex representations like TVMed, TVDFVAR, and TVCoverage did not consistently outperform the centroid approach. In some cases, they introduced noise or computational overhead without translating to better downstream performance.
• TVCoverage performed competitively at smaller budgets (e.g., 5k), but TVDF remained superior overall.

3. Domain-Agnostic Downsampling

In the domain-agnostic setting, TVDF significantly outperformed other downsampling strategies (Random, NC, NN) when reducing training data to 70% of its original size.

• Nearest to Centroid (NC) performed the worst (Avg F1 0.324) because it disproportionately retained negative (non-matching) samples, which dominate EM datasets and occupy the central regions of the embedding space, leading to a biased training set.
• Nearest Neighbor (NN) performed better than NC but worse than random sampling, indicating that proximity-based selection alone can still introduce bias.
• TVDF achieved the strongest performance among downsampling methods (Avg F1 0.736), demonstrating that distribution-aware selection can mitigate performance loss when data reduction is necessary.

Significance and Contributions

The paper provides deeper insight into the BEACON framework and the specific role of distribution alignment in budgeted EM. The authors claim the following contributions:

• Behavioral Analysis: They demonstrate that the unsupervised nature of TVDF is often beneficial, and that adding label information or complex distribution representations does not guarantee improved performance.
• Robustness of Centroids: The study validates that simple centroid-based representations are sufficient for capturing the necessary structure of embedding distributions for sample selection, challenging the need for more complex statistical summaries.
• Generalizability of Alignment: By showing that distribution-aware selection (TVDF) is effective even in a domain-agnostic downsampling setting, the paper suggests that the core mechanism of TVDF is a robust unsupervised strategy for reducing training data requirements while minimizing performance degradation.

The authors conclude that while BEACON serves as a strong foundation for low-resource EM, future work should explore the robustness of these findings across different PLMs and additional benchmarks beyond e-commerce applications.