Technical Summary: Learning From How Humans Correct

Problem Statement

In industrial Natural Language Processing (NLP) applications, manually labeled datasets inevitably contain a certain amount of noisy data. While deep learning models, particularly those based on BERT, have achieved significant improvements across NLP tasks, existing methods generally fail to incorporate the specific information regarding how humans correct these errors. Current approaches often treat human labels as static ground truth or utilize semi-supervised learning to leverage unlabeled data, but they do not explicitly model the human correction process to improve the model's ability to rectify mistakes. The core problem addressed is how to inject human correction information into deep learning models to mimic the human learning process of correcting errors based on environmental feedback.

Methodology

The authors propose a five-step learning framework designed to identify noisy data, relabel it, and train a model to learn the correction process itself. The framework operates as follows:

1. Data Preparation: A dataset of 2,040,000 samples is manually labeled from a larger application database (500 million records). This is split into 2,000,000 training samples and 40,000 test samples.
2. Initial Model Training (Model-A): A BERT model is fine-tuned on the 2,000,000 training samples. This model serves as a baseline and is used to identify discrepancies.
3. Noisy Data Identification: Model-A is used to predict labels for the entire 2,040,000 dataset. Samples where the model's prediction differs from the original human label are flagged as noisy data. In this experiment, 300,000 such samples were identified (294,120 in the training set and 5,880 in the test set).
4. Human Relabeling: The 300,000 identified noisy samples are manually relabeled. During this process, human annotators are provided with both the original human label and Model-A's predicted label as references, though they make the final decision. This step generates the "correction information."
5. Correction Learning (Model-C): A new model (Model-C) is trained using the corrected dataset. Crucially, the input for Model-C includes the original text plus the one-hot encoded predicted label from Model-A.
   • Input: Text + Model-A's predicted one-hot label (representing "before-correction" information).
   • Target: The newly corrected human label (representing "after-correction" information).
   • Objective: Model-C learns to map the initial prediction (and text) to the corrected label, effectively learning the human correction logic alongside the primary classification task.

The authors distinguish this approach from Reinforcement Learning from Human Feedback (RLHF), noting that while RLHF uses reward models to guide policy, this method focuses on directly correcting the policy dataset to achieve high-quality data. It is also distinct from Confident Learning (CL), which focuses on estimating noise distributions; this method is described as more practical and simple, directly utilizing the correction act.

Key Contributions

The paper outlines three primary contributions:

1. Noisy Data Identification and Relabeling: A practical, step-by-step process to detect noisy data based on model-human disagreement and manually relabel them to create a high-quality dataset.
2. Injection of Correction Information: A novel method to inject human correction information into a BERT-based deep learning model by using the model's initial prediction as an additional input feature during fine-tuning.
3. General Applicability: A learning framework designed to be applicable to a broad set of deep learning industry applications involving manually labeled datasets, not limited to the specific text classification task tested.

Experimental Results

The method was evaluated on an industrial text classification dataset.

• Baseline (Model-A): Trained on the original dataset, achieving 83.3% accuracy on the test set.
• Corrected Dataset (Model-B): Trained on the dataset after relabeling the noisy data (without the correction input feature), achieving 91.7% accuracy. This demonstrates the value of cleaning the data.
• Correction Learning (Model-C): Trained with the additional Model-A prediction input, achieving 92.5% accuracy on the test set.
• Human Evaluation: When evaluated against a random sample of 5,000 data points from the full 500 million record database (representing the broader distribution), the models achieved:
  • Model-A: 88.0%
  • Model-B: 97.2%
  • Model-C: 97.7%

The results indicate that while relabeling alone significantly improves performance (from 83.3% to 91.7%), the proposed "learn-on-correction" method provides a further incremental gain (to 92.5% on the test set and 97.7% in human evaluation).

Significance and Claims

The paper claims to be the first study exploring the improvement of deep models for natural language understanding by injecting human correction information. The authors argue that because deep learning is statistic-based, correcting mislabeled data (e.g., changing a label from B to A when similar data points are labeled A) prevents the model from learning incorrect patterns for future, similar inputs.

The significance of the work lies in its practicality for industry applications where annotation costs are high. By identifying and correcting only the noisy subset of data and training the model to learn from these corrections, the framework aims to reduce the total volume of data requiring manual labeling. The authors suggest that if the labeling workload can be reduced to a level manageable by a single programmer, the need for large, dedicated labeling teams could be diminished.

The paper remains modest regarding its scope, stating that the method is designed for industry applications with manually labeled data and that future work will involve combining this relabeling method with other noisy data detection techniques to find the most efficient approach.