Low-resource Language Discrimination Towards Chinese Dialects with Transfer learning and Data Augmentation

Technical Summary: Low-Resource Language Discrimination Towards Chinese Dialects with Transfer Learning and Data Augmentation

Problem Statement
Chinese dialect discrimination is a challenging natural language processing (NLP) task primarily due to the scarcity of annotated resources. While general language identification for unrelated languages has achieved high accuracy, discriminating between related languages or variations of a specific language (such as Chinese dialects) remains difficult. Existing research on Chinese dialects, such as the Gan and Hakka dialects, is hindered by small datasets (e.g., approximately 2,000 sentences for Gan dialects), resulting in low discrimination accuracy (around 60%) with traditional methods. The lack of specialized research and sufficient data necessitates new approaches to handle low-resource scenarios effectively.

Methodology: CDDTLDA Framework
The authors propose a novel framework named CDDTLDA (Chinese Dialects Discrimination with Transfer Learning and Data Augmentation). The framework integrates three core modules:

1. Transfer Learning (Source-to-Target ASR):

   • Source Model Training: The authors utilize a relatively large-scale, 10-way Chinese dialect corpus (IFLYTEK) to train a source-side Automatic Speech Recognition (ASR) model (ASR1). This model employs a ResNet to extract local latent representations from speech features (FBank, MFCC, or log-Mels), followed by a multi-head self-attention mechanism to capture frame relationships, and a fully connected layer with softmax for phoneme sequence prediction. The model is trained using Connectionist Temporal Classification (CTC) loss.
   • Target Model Fine-tuning: A target-side ASR model (ASR2) is derived by fine-tuning the pre-trained ASR1. Specifically, the ResNet parameters are fixed, while the self-attention and linear modules are fine-tuned using the small, low-resource target dialect corpus (e.g., Gan or Hakka). This parameter-based transfer learning aims to leverage the common semantic features between the source and target domains.

2. Data Augmentation:

   • To address data scarcity, the authors employ a simple but effective augmentation strategy involving three perturbations:
     • Time Extension: Slowing down or speeding up audio sampling while keeping pitch constant (factors of 0.9 and 1.1).
     • Pitch Conversion: Shifting the pitch by specific semitone values (ranging from -3.5 to +3.5) while maintaining duration.
     • Noise Disturbance: Adding random Gaussian noise (mean 0, standard deviation 1).
   • These methods expand the target dataset size by a factor of 12. The paper notes that while complex methods like SpecAugment or GANs exist, they incur high computational costs; the proposed simple perturbations are designed to reduce cost while maintaining efficacy.

3. Dialect Discrimination Classifier:

   • The final discrimination task utilizes the hidden semantic representations (local hidden vectors) extracted from the fine-tuned target ASR model (ASR2).
   • These representations, combined with the initial speech features, are fed into a Bi-LSTM followed by two fully connected layers with a softmax activation function.
   • The model is optimized using cross-entropy loss with L2 regularization to predict the specific Chinese dialect type.

Key Contributions

1. Novel Framework: The paper introduces an ASR-based transfer learning framework coupled with simple data augmentation specifically designed for low-resource Chinese dialect discrimination. It demonstrates that a larger source-side ASR model can effectively assist in discriminating fine-tuned target-side dialects.
2. Semantic Feature Extraction: The authors show that the hidden semantic vector matrix (local hidden representation) from the fine-tuned ASR model represents the distribution of phonemes corresponding to speech frames. By using a self-attention mechanism, the model successfully captures potential common semantic features between the source and target models, aiding in dialect discrimination.
3. Empirical Validation: Extensive experiments are conducted on two benchmark corpora (Gan and Hakka dialects), verifying the model's efficiency against strong baseline methods.

Experimental Results
The proposed model was evaluated on two datasets: a 6-way Gan Chinese dialect corpus and a 5-way Hakka Chinese dialect corpus. The results were compared against state-of-the-art baselines, including LSTM, x-vector, ResNet&Bi-LSTM (the 2018 IFLYTEK champion model), and CNN_Att.

• Performance: The CDDTLDA model significantly outperformed all baseline methods on both datasets across four metrics: Accuracy, Precision, Recall, and F1-score.
  • On the Gan dataset, the proposed model achieved an accuracy of 63.10% and an F1-score of 58.54%, surpassing the x-vector baseline (55.48% accuracy) and the ResNet&Bi-LSTM baseline (49.05% accuracy).
  • On the Hakka dataset, the model achieved an accuracy of 79.91% and an F1-score of 76.14%, again outperforming the x-vector (69.23% accuracy) and other baselines.
• Ablation Studies:
  • Transfer Learning: Removing the transfer learning component resulted in a significant drop in performance (e.g., accuracy dropped from 63.10% to 58.57% on the Gan dataset), confirming the necessity of the source-side pre-training.
  • Data Augmentation: Removing data augmentation also degraded performance, highlighting its role in overcoming data scarcity.
  • Feature Fusion: Combining initial speech features with the extracted local hidden representations generally yielded better results than using features alone.
  • Transfer Learning Types: The proposed parameter-based transfer learning outperformed instance-based and feature representation-based transfer learning approaches.
• Speech Recognition Performance: While the primary focus is discrimination, the Phoneme Error Rate (PER) for the underlying ASR tasks was reported. The authors note that while PER on low-resource dialects remains a challenge requiring further improvement, the latent representations learned during ASR are sufficient to aid dialect discrimination.

Significance and Claims
The paper claims that the proposed CDDTLDA framework effectively addresses the challenge of low-resource language discrimination by leveraging transfer learning from a larger, related dialect corpus and employing computationally efficient data augmentation. The authors assert that the hidden semantic representations of phonemes, captured via self-attention within an ASR context, provide a robust mechanism for distinguishing between similar Chinese dialects. The study concludes that this approach significantly improves discrimination accuracy compared to existing state-of-the-art methods, offering a viable solution for protecting and analyzing endangered or low-resource Chinese dialects. The authors suggest future work will explore additional augmentation methods, more speech-driven features, and the application of dialect identification to other NLP tasks.