A self-supervised learning approach to deep filter banks for texture recognition

Technical Summary: A Self-Supervised Learning Approach to Deep Filter Banks for Texture Recognition

Problem Statement
Texture recognition in computer vision faces significant challenges in real-world applications, primarily due to the scarcity of labeled training data, high interclass similarity, and intraclass dissimilarity. While deep learning has advanced the field, standard "off-the-shelf" Convolutional Neural Networks (CNNs) often fail to extract sufficient information from limited datasets. Current state-of-the-art solutions frequently rely on self-supervised pre-training using Masked Autoencoders (MAEs) combined with Vision Transformers (ViTs). However, these architectures are computationally intensive and rely on attention mechanisms to capture long-range dependencies. The authors argue that for texture images, where relevant information is compacted within local areas around each pixel, capturing long-range dependence may be unnecessary, suggesting that lighter, convolutional-based approaches could be more efficient.

Methodology
The proposed framework replaces the computationally heavy ViT-based pre-training with a deep convolutional autoencoder within a self-supervised learning pipeline. The methodology consists of four primary modules:

1. Self-Supervised Pre-training (Autoencoder):
   The model employs a U-Net-like encoder-decoder architecture. Two pre-training strategies are explored:

   • Identity Mapping: The input and target output are the same image.
   • Denoising: The input is a noisy version of the image, and the network learns to reconstruct the original.
     The architecture utilizes standard convolutional blocks with ReLU activation, max pooling for downsampling, and skip connections to preserve spatial information. The bottleneck layer output serves as the learned feature representation.

2. CNN Feature Extraction:
   The pre-trained encoder is used as a feature extractor. Local features are extracted from specific convolutional layers (specifically the last layers of blocks 5 and 6 of an EfficientNet-B5 backbone).

3. Fisher Vector (FV) Encoding:
   To aggregate the local features, the method employs Fisher Vector encoding coupled with a Gaussian Mixture Model (GMM). The GMM is trained on the aggregated local features to model the distribution of visual words. The Fisher Vector is computed as the gradient of the log-likelihood of the data with respect to the GMM parameters, normalized by the Fisher Information Matrix. This process captures higher-order statistics of the deep features.

4. Predictor:
   The final feature representation is a concatenation of the self-supervised bottleneck features (SSL) and the normalized Fisher Vectors (FV). The authors apply L2 and power normalization to the FV features. Classification is performed using a Support Vector Machine (SVM) with a Bhattacharyya coefficient kernel, which allows for efficient linear classification after a specific transformation of the feature vectors.

Key Contributions

• Architecture: Proposal of a deep convolutional encoder-decoder architecture for self-supervised feature extraction in texture recognition, avoiding the computational overhead of Vision Transformers.
• Hybrid Model: Introduction of a texture recognition model combining self-supervised CNN features with Fisher Vector pooling.
• Performance: Demonstration of state-of-the-art or competitive results on standard texture databases (Flickr Material Database, Describable Textures Dataset, and KTH-TIPS2-b) with reduced computational complexity.
• Practical Application: Successful application of the model to the identification of Brazilian plant species from leaf surface images, surpassing previously reported results.

Experimental Results
The method was evaluated on five texture datasets: KTH-TIPS2-b, FMD, DTD, UIUC, and UMD, as well as a practical 1200Tex dataset for plant species identification.

• Metric Performance: On the FMD dataset, the proposed FVAE model (Fisher Vector + Autoencoder) achieved an accuracy of 92.9 ± 0.4%, outperforming the baseline EfficientNet-B5 (87.4%) and other state-of-the-art methods like RADAM (88.7%) and DSRNet (86.0%).
• Feature Analysis: Experiments indicated that using Fisher features alone provided slightly better metrics than combining them with fully connected (FC) features, likely because the GMM-based statistical procedure better captures the distributional particularities of the deep features.
• Efficiency: The approach achieved high accuracy with a "straightforward and highly efficient" pre-training stage, validating the hypothesis that complex attention mechanisms are not strictly necessary for texture tasks.

Significance and Claims
The paper posits that while Vision Transformers are effective for tasks requiring the modeling of distant regions within an image, they are not always the optimal choice for texture recognition, where local characteristics dominate. The study demonstrates that "U-Net-like" convolutional architectures can provide rich deep representations of unlabeled images with relatively low computational costs.

The authors conclude that their work highlights the potential of efficient convolutional architectures in self-supervised frameworks, particularly in scenarios where pre-training data is limited. They suggest that future research should explore adapting other self-supervised paradigms (such as contrastive learning) to efficient CNN backbones and investigating how other state-of-the-art texture analysis models might benefit from similar pre-training stages. The work aims to address the limitations of current literature regarding computational efficiency in self-supervised learning without sacrificing classification accuracy.