💻 computer science

ESLA: Empirical and Semantic Label Alignment for Multi-Source NER Transfer in Unlabeled Target Domains

The paper introduces ESLA, a framework that combines empirical prediction consistency and semantic representations to align heterogeneous Named Entity Recognition labels across multiple source domains, enabling effective cross-domain transfer to unlabeled target domains without requiring target-side training supervision.

Original authors: Xiaobo Zhang, Congqing He, Ying He, Jian Peng, Dajie Fu, Tien-Ping Tan

Published 2026-09-04

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Xiaobo Zhang, Congqing He, Ying He, Jian Peng, Dajie Fu, Tien-Ping Tan

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of computers that read and understand human language, there is a persistent bottleneck: the need for massive amounts of labeled data. To teach a machine to recognize a named entity—such as a person's name, a company, or a specific date—researchers must provide it with thousands of examples where these items have been manually identified and tagged by humans. This process is expensive, slow, and requires deep expertise, making it particularly difficult in specialized fields like finance or medicine, where texts are abundant but annotated examples are scarce. While researchers have built large libraries of labeled text for general topics like news or social media, these resources often use different systems for tagging. One dataset might call a specific type of entity a "product," while another calls it a "creative work," and a third might ignore it entirely. These inconsistencies create a wall that prevents computers from learning from the vast pool of existing public data, leaving them unable to transfer their knowledge to new, unlabeled domains where it is needed most.

To break down this wall, a team of researchers led by Xiaobo Zhang has developed a new method called ESLA, which stands for Empirical and Semantic Label Alignment. Their work addresses the challenge of merging multiple, mismatched datasets into a single, unified training resource without needing any labeled examples from the target domain. The researchers focused on the Chinese language, combining three distinct public datasets: one containing general news and media, another with fine-grained details from online texts, and a third drawn from social media. Each of these sources used its own unique set of tags and definitions. The goal was to align these different systems so that a computer could learn from all of them simultaneously, creating a more robust model capable of understanding text in a completely new area: financial reports.

The core of the ESLA framework is a two-part strategy for deciding which tags from different datasets should be treated as the same thing. The first part looks at how the data actually behaves. The researchers trained a model on one dataset and then tested it on another to see how often the computer's predictions matched the human tags in the second dataset. If a model trained to find "companies" in one dataset consistently identified "organizations" in another, the system recognized a strong practical link between those two tags. This is the empirical signal, a measure of real-world consistency rather than just dictionary definitions. The second part looks at the meaning of the words themselves. Using advanced language models, the system analyzes the context in which these tags appear. It calculates how close the "idea" of a tag is to another tag in a mathematical space, ensuring that "movie" and "book" are understood as semantically similar, even if they are labeled differently in the source files. By blending these two signals—the way the data behaves and the meaning of the words—the system constructs a map that connects the disparate datasets.

However, simply merging data can be dangerous. If the researchers forced tags together that were only loosely related, they risked confusing the model and degrading its performance. To prevent this, the team introduced a stability check. They treated the merging process as an optimization problem, searching for the specific combination of rules that would allow the most tags to be merged while ensuring the model's accuracy on its original data did not drop significantly. They also addressed a common problem where one dataset might have a tag for a specific type of number that another dataset lacks. Instead of ignoring these missing categories, the system used the knowledge from the dataset that had them to generate "pseudo-labels" for the other, filling in the gaps so the model could learn from the complete picture.

The results of this approach were tested by training the unified model on the merged Chinese datasets and then evaluating it on a financial benchmark called FinReportNER, which contained text from semiconductor company annual reports. Crucially, the financial data was never used to train the model or tune the alignment; it served only as a final test of how well the system could generalize to a new domain. The study found that the ESLA method successfully aligned fifteen different label types across the three source datasets, creating a unified corpus that maintained high accuracy. When tested on the financial reports, the model trained on this merged data achieved a Macro-F1 score of 0.59, outperforming models trained on any single source dataset or on a direct combination of the data without alignment.

The improvement was particularly notable in specific categories. For instance, the model's ability to identify "products" in financial reports jumped significantly, reaching a score of 0.06, which was three times higher than the best result from a single-source model. Similarly, the identification of "time" entities improved to a score of 0.89, surpassing all other configurations. These gains suggest that by carefully aligning the different ways humans label information, the system learned a richer, more flexible understanding of entities. The study explicitly ruled out the idea that simply pasting datasets together works; a direct merge without alignment resulted in a poor score of 0.17, demonstrating that the alignment process is essential. Furthermore, the researchers showed that relying solely on human experts or large language models to define the rules was less effective than their data-driven approach, which could discover connections that were not immediately obvious to human observers.

The researchers conclude that their method offers a reliable, automated way to reuse existing data resources for cross-domain tasks, provided that the target domain has no labeled data available. While the current system works best when merging many fine-grained tags into broader categories, it acknowledges that it cannot yet fully handle the reverse situation where one broad tag needs to be split into many specific ones. Nevertheless, the study demonstrates that by combining empirical evidence with semantic understanding, it is possible to build a unified training resource that significantly boosts performance in specialized fields like finance, turning fragmented public data into a powerful tool for machine learning.

Technical Summary: ESLA for Multi-Source NER Transfer

Problem Statement

Named Entity Recognition (NER) relies heavily on high-quality, large-scale annotated corpora, which are expensive and time-consuming to construct, particularly for specific domains like finance and healthcare. While public NER datasets exist, they suffer from heterogeneous label schemas (different definitions, granularities, and naming conventions) and are often designed for general domains (e.g., news, social media) rather than specific target domains.

The core challenge addressed by this paper is cross-domain NER transfer in target-unlabeled settings. Researchers aim to reuse multiple public source datasets to train a model for a target domain where no labeled training data is available. However, simply merging these datasets introduces severe conflicts due to:

Schema Heterogeneity: Labels with the same name may have different semantic scopes (e.g., "TIME" in OntoNotes vs. BosonNER), and different names may refer to similar concepts.
Annotation Inconsistency: Direct merging without alignment creates conflicting supervision signals, degrading model performance.
Missing Labels: Some entity types present in one dataset may be absent in others, leading to implicit negative examples and supervision bias when datasets are fused.

Existing solutions, such as manual mapping, knowledge-graph-based alignment, or prompt-based LLM strategies, often lack scalability, rely on heavy expert intervention, or fail to resolve underlying semantic conflicts effectively.

Methodology: ESLA Framework

The authors propose ESLA (Empirical and Semantic Label Alignment), an automatic framework that aligns heterogeneous labels by combining two complementary signals: empirical prediction consistency and contextual semantic representations. The framework operates in three core stages:

1. Dual-Signal Matrix Computation

ESLA calculates alignment scores between label pairs $(L_s, L_t)$ from different source datasets using two signals:

Empirical Signal ( $S_{emp}$ ): Measures the consistency of annotation behavior. A model trained on dataset $D_s$ predicts entities on $D_t$ . The signal is defined as the proportion of entities truly labeled as $L_t$ in $D_t$ that are predicted as $L_s$ by the model. This captures how well the annotation norms of one dataset map to another.
Semantic Signal ( $S_{sem}$ ): Measures the similarity between label prototypes in a shared contextual embedding space (using Chinese-BERT-wwm-ext). For each label, a prototype vector is constructed by averaging normalized contextual representations of its entity spans. The signal is the cosine similarity between these prototypes, clipped to zero for negative values.

These signals are fused via linear interpolation:
$S_{merge}(L_s, L_t, \lambda) = (1 - \lambda) \cdot S_{sem}(L_s, L_t) + \lambda \cdot S_{emp}(L_s, L_t)$
where $\lambda$ balances the contribution of semantic vs. empirical evidence.

2. Sequential Merging and Stability-Constrained Optimization

Path Search: To merge $N$ datasets, ESLA uses a greedy strategy based on the asymmetric empirical signal to determine the optimal sequential merging order (e.g., $D_1 \to D_2 \to D_3$ ).
Deterministic Mapping: To avoid ambiguous supervision, ESLA enforces a single-target mapping constraint. A source label is mapped to the target label with the highest $S_{merge}$ score, provided it exceeds a threshold $\tau$ .
Stability-Constrained Optimization: The parameters $(\lambda, \tau)$ are not manually selected but optimized to maximize the number of merged labels ( $N_{merged}$ ) while strictly constraining the degradation of source-domain NER performance ( $\Delta F1 \ge -\delta$ , where $\delta=2\%$ ). This ensures the merged corpus does not harm the model's ability to recognize entities in the original source domains.

3. Context-Aware Label Completion

To address missing-label conflicts (where a label exists in one dataset but not another), ESLA employs a pseudo-labeling strategy. A model trained on a dataset containing a specific label (e.g., CARDINAL) is used to generate pseudo-labels for that entity type in datasets where it is absent. This provides positive supervision for otherwise missing categories.

Key Contributions

Dual-Signal Alignment Framework: ESLA formulates label alignment as an optimization problem jointly exploiting empirical prediction consistency and contextual semantic similarity, eliminating the need for external knowledge bases or heavy manual rules.
Scalable Merging Strategy: The framework introduces a stability-constrained optimization mechanism that balances label coverage with source-domain performance preservation, preventing aggressive merging that introduces noise.
Unified Corpus Construction: The authors successfully aligned and merged three heterogeneous Chinese NER datasets (OntoNotes 5.0, CLUENER2020, and BosonNER) into a unified corpus.
Target-Unlabeled Validation: The framework was validated in a cross-domain transfer setting to a financial benchmark (FinReportNER) where target annotations were used exclusively for evaluation, ensuring no data leakage during training or alignment.

Experimental Results

The study evaluated ESLA on three source datasets and a financial target domain (FinReportNER).

Source-Side Stability: ESLA successfully aligned 15 label pairs (the highest among baselines) while maintaining a Micro-F1 of 0.80, identical to independent training. In contrast, direct merging resulted in significant performance drops.
Cross-Domain Transfer: On the FinReportNER benchmark:
- ESLA-Merge achieved the highest Macro-F1 of 0.59 and matched the best Micro-F1 of 0.56.
- It outperformed single-source training and other alignment baselines (Rule-based, Expert-driven, and LLM-based).
- Notable gains were observed in categories with limited supervision in single sources, specifically PRODUCT (F1 0.06, three times the best single-source score of 0.02) and TIME (F1 0.89).
Ablation Studies:
- Removing the semantic signal reduced the number of aligned labels and hurt PRODUCT performance.
- Removing the empirical signal hurt ORG performance, highlighting the need for statistical evidence.
- Removing label completion caused a catastrophic drop in transfer performance (Micro-F1 from 0.56 to 0.21), particularly for NUM and RATIO categories, proving the necessity of pseudo-labeling for missing types.

Significance and Claims

The paper claims that empirical and semantic label alignment can effectively integrate heterogeneous public datasets for cross-domain NER without requiring target-domain training supervision.

Interpretability and Automation: Unlike manual or knowledge-graph-based methods, ESLA is data-driven and interpretable, relying on observable prediction behaviors and semantic distances rather than rigid heuristics.
Robustness: The stability-constrained optimization ensures that the benefits of data integration do not come at the cost of degrading the model's fundamental capabilities on the source data.
Generalizability: The framework demonstrates that combining multiple weakly supervised sources can yield better cross-domain generalization than relying on a single, high-quality source, particularly for categories that are under-represented in individual datasets.

The authors note that while ESLA supports many-to-one mappings effectively, it currently uses a deterministic single-target strategy and does not explicitly model one-to-many or hierarchical relations, which remains a direction for future work.