Enhancing Cognitive Workload Classification Using Integrated LSTM Layers and CNNs for fNIRS Data Analysis

Technical Summary: Enhancing Cognitive Workload Classification Using Integrated LSTM Layers and CNNs for fNIRS Data Analysis

Problem Statement
Functional near-infrared spectroscopy (fNIRS) is a non-invasive neuroimaging technique used to monitor brain activation by measuring changes in oxygenated (HbO) and deoxygenated (HbR) hemoglobin. While machine learning (ML) and deep learning (DL) have been applied to fNIRS data for cognitive load assessment, significant limitations persist. Conventional ML methods often require complex, manual feature engineering and preprocessing, which can lead to reduced accuracy and scalability issues. Furthermore, existing deep learning research in this domain has predominantly focused on binary classification (distinguishing between only two levels of mental workload, e.g., low vs. high). Additionally, standard Convolutional Neural Networks (CNNs), while effective at spatial feature extraction, often struggle to capture temporal dependencies inherent in time-series fNIRS data, potentially leading to spatial feature overfitting and a lack of temporal context.

Methodology
The study proposes a hybrid deep learning architecture that integrates Convolutional Neural Networks (CNNs) with Long Short-Term Memory (LSTM) layers to address the limitations of spatial-only and temporal-only models.

• Dataset: The research utilizes the open-access Tufts fNIRS dataset, which includes data from 68 participants performing n-back tasks (0-back, 1-back, 2-back, and 3-back). This dataset allows for the classification of four distinct cognitive workload levels, moving beyond the binary scope of previous studies.
• Preprocessing: Raw fNIRS signals were bandpass filtered (0.001–0.2 Hz) to remove artifacts. Data was segmented into overlapping windows of 30 seconds with a 0.6-second stride, resulting in feature vectors of size 150 × 8 (150 time steps × 8 channels). The dataset was split into 80% training and 20% testing.
• Architecture:
  • CNN Component: The model begins with 1D-CNN layers to extract spatial features. It employs convolutional layers with Rectified Linear Unit (ReLU) activation and max pooling to reduce dimensionality and prevent overfitting.
  • LSTM Component: The extracted features are flattened and fed into two LSTM layers. These layers are designed to capture temporal dependencies and sequential patterns within the fNIRS data, addressing the inability of standard CNNs to model time-series dynamics effectively.
  • Classification: Fully connected (Linear) layers followed by a final Softmax layer classify the data into the four workload categories.
• Training: The model was trained using the Adam optimizer with a learning rate of 0.001 and a cross-entropy loss function over 1,000 epochs.

Key Contributions

1. Novel Framework for Multi-Level Classification: The paper introduces a structured approach to classify four distinct levels of cognitive load (0-back to 3-back) using fNIRS, expanding beyond the conventional binary classification found in most prior literature.
2. Hybrid CNN-LSTM Architecture: The study proposes a lightweight hybrid model that leverages the spatial feature extraction capabilities of CNNs and the temporal dependency modeling strength of LSTMs. This integration aims to mitigate spatial overfitting and capture the evolving nature of cognitive load.
3. Comprehensive Evaluation: The framework is evaluated against traditional ML algorithms (Naive Bayes, Nearest Centroid, Decision Trees, k-NN) and a baseline CNN model without LSTM layers. The study specifically investigates the impact of removing LSTM layers to isolate their contribution to performance.

Results

• Accuracy: The proposed CNN-LSTM model achieved a classification accuracy of 97.89% (reported as 97.92% in the abstract and 97.82% in the conclusion, with Table 2 showing 97.89%). This represents an improvement over the CNN-only model, which achieved 97.40%.
• Comparison with Traditional ML: The hybrid model significantly outperformed traditional ML classifiers. While k-NN was the strongest traditional performer (97.02% accuracy), the CNN-LSTM model surpassed it across all metrics, including F1-score, AUC, precision, and recall.
• Impact of LSTM Integration: Removing the LSTM layers resulted in a marginal decrease in accuracy and introduced fluctuations in the training curves, indicating that the CNN-only model struggled to capture sequential relationships and was more sensitive to noise.
• Computational Efficiency: The inclusion of LSTM layers increased the training time from approximately 7.79 minutes (CNN) to 9.76 minutes (CNN-LSTM). Inference times also increased slightly (e.g., from 0.0891 ms to 0.2153 ms on GPU), but the authors note that the model remains lightweight and suitable for real-time applications.

Significance and Claims
The paper claims that the integration of LSTM layers into CNNs provides a comprehensive solution for fNIRS data analysis by simultaneously addressing spatial and temporal feature extraction challenges. The authors assert that their lightweight hybrid model offers a robust alternative to complex, computationally intensive architectures while maintaining high accuracy. By successfully classifying four levels of cognitive load, the study demonstrates the potential of deep learning to handle more nuanced cognitive states than previously achieved with fNIRS. The authors conclude that their framework is scalable and adaptable, making it suitable for real-world applications such as adaptive learning systems and real-time workload monitoring, particularly given its ability to operate efficiently on resource-constrained devices.