Acoustic scattering AI for non-invasive object classifications: A case study on hair assessment

Technical Summary: Acoustic Scattering AI for Non-Invasive Object Classifications

Problem Statement
Traditional object classification relies heavily on visual data, which faces limitations regarding lighting conditions, occlusion, privacy concerns, and the inability to assess internal structures or material properties. While acoustic scattering is a well-established phenomenon in fields like sonar and medical ultrasound, its application to general object classification using AI remains underexplored. This paper addresses the challenge of classifying complex objects non-invasively by analyzing the acoustic scattering field generated when incident waves interact with an object. The study uses hair assessment (classifying hair types and moisture levels) as a specific case study to demonstrate the feasibility of this approach.

Methodology
The research proposes a data-driven framework utilizing deep learning to classify objects based on scattered acoustic signals.

• Experimental Setup: The system employs a loudspeaker (Event ALP5) to emit an Exponential Sine Sweep (ESS) stimulus (100 Hz to 24 kHz, 5s duration) toward mannequin heads fitted with various wigs. A microphone (MX183) positioned at the mannequin's neck captures the total acoustic field, which is a superposition of the direct incident wave and the scattered wave containing structural and material information. Experiments were conducted in a soundproof room (RT60 = 0.5s) to minimize noise.
• Datasets: The study collected data across 26 recording rounds involving four hair types (A, B, MAMI, MINAYO) and three conditions (dry, shampoo-treated, cream-treated). The dataset comprises thousands of 5-second samples resampled to 48kHz.
• Classification Approaches: Four distinct deep learning strategies were benchmarked:
  1. Fully Supervised Training: A modified ResNet-50 architecture adapted for single-channel mel-scaled spectrograms.
  2. Embedding-Based Classification: Extraction of feature vectors from a pre-trained VGGish model (AudioSet) followed by classification using an XGBoost classifier.
  3. Supervised Foundation Model Fine-tuning: Fine-tuning the Audio Spectrogram Transformer (AST), a convolution-free model pre-trained on ImageNet and AudioSet.
  4. Self-Supervised Learning (SSL) Fine-tuning: Fine-tuning the Wav2Vec2-Conformer large model (pre-trained on Librispeech). Two strategies were tested: partial fine-tuning (updating only the Conformer Encoder) and complete fine-tuning (updating all model parameters).

Key Results
The models were evaluated on two tasks: classifying four distinct hair types (Task 1) and classifying three hair moisture conditions (Task 2).

• Task 1 (Hair Type Classification): The Wav2Vec2-Conformer model with complete fine-tuning achieved the highest performance, reaching an accuracy of 87.4% and an average AUC of 0.97. This outperformed the fully supervised ResNet-50 (73.2% accuracy) and the AST fine-tuning (60.0% accuracy).
• Task 2 (Hair Condition Classification): The Wav2Vec2-Conformer with complete fine-tuning again demonstrated superior performance, achieving 89.8% accuracy and an average AUC of 0.96.
• Model Comparison: Across both tasks, models leveraging Convolutional Neural Network (CNN) layers combined with attention mechanisms (specifically Wav2Vec2-Conformer and ResNet-50) consistently outperformed embedding-based approaches (VGGish-XGB) and the AST model. The ablation study confirmed that updating the parameters of the CNN feature extractors during fine-tuning significantly improved performance compared to partial fine-tuning.

Significance and Claims
The authors claim that this work represents the first attempt to study the feasibility of using AI-driven techniques for complex object classification via acoustic scattering. The primary significance of the study lies in demonstrating that acoustic scattering can serve as a privacy-preserving, non-contact alternative to visual classification. By successfully classifying hair types and moisture levels with nearly 90% accuracy, the paper suggests that this method can reveal biophysical properties (such as inner structure and material density) that are difficult to capture with optical methods. The authors posit that this approach holds potential for extension into material science, quality control, and various industrial applications, offering a robust solution where visual data is insufficient or privacy-sensitive. The code and analysis for the study have been open-sourced to facilitate further research.