Hybrid Optuna and Q Learning for Adaptive Hyperparameter Optimization in EEG Seizure Classification

Technical Summary: Hybrid Optuna and Q-Learning for Adaptive Hyperparameter Optimization in EEG Seizure Classification

Problem Statement
Epileptic seizure classification using electroencephalogram (EEG) signals is critical for automated neurological diagnosis. While deep learning models (such as CNNs, LSTMs, and Transformers) have improved performance, their efficacy is heavily dependent on the efficient selection of hyperparameters (e.g., learning rates, hidden units, attention heads). Existing frameworks, particularly those utilizing Bayesian optimization (Optuna) with GRU–HiBERT–MLP architectures, often face limitations regarding adaptive search behavior, convergence stability, and the balance between exploration and exploitation in high-dimensional search spaces. Conversely, standalone Reinforcement Learning (RL) approaches may exhibit slower convergence. This paper addresses the need for a more robust optimization strategy that combines the probabilistic efficiency of Bayesian methods with the adaptive decision-making capabilities of RL to enhance EEG seizure classification.

Methodology
The proposed framework integrates a Hybrid Optuna–Q Learning optimization strategy with a GRU–HiBERT–MLP classification architecture.

1. Data Preprocessing:

   • The study utilizes the Preprocessed CHB-MIT Scalp EEG Database, comprising recordings from 24 pediatric patients.
   • Signals are segmented into preictal and ictal states.
   • Discrete Wavelet Transform (DWT) using Daubechies wavelets (db4) is applied to extract time-frequency features, handling the non-stationary nature of EEG.
   • Features are standardized using StandardScaler and reshaped for sequential processing.

2. Classification Architecture (GRU–HiBERT–MLP):

   • GRU Layer: Captures temporal dependencies in multichannel EEG signals.
   • HiBERT (Transformer) Block: Applies multi-head self-attention mechanisms for contextual representation learning.
   • MLP Layer: Fully connected dense layers with dropout regularization for binary classification (preictal vs. ictal).

3. Hybrid Optimization Framework:

   • Optuna (Bayesian): Uses Tree-structured Parzen Estimators (TPE) to probabilistically sample hyperparameter configurations ($\theta$) based on historical observations.
   • Q-Learning (Reinforcement Learning): An agent dynamically guides the search process.
     • State ($s_t$): Current optimization condition.
     • Action ($a_t$): Adaptive hyperparameter refinement strategies.
     • Reward ($r_t$): Calculated as the improvement in validation accuracy ($Acc_t - Acc_{t-1}$).
   • Integration: The Q-learning agent updates its Q-table using temporal difference learning to refine the search guidance. It adaptively directs the Optuna search toward promising regions, mitigating premature convergence and improving stability.

Key Contributions

• Hybrid Optimization Framework: Proposes a novel integration of Bayesian optimization (Optuna) and Reinforcement Learning (Q-learning) to improve hyperparameter tuning efficiency and convergence stability for deep EEG models.
• Architecture Application: Implements a GRU–HiBERT–MLP architecture optimized by the hybrid framework for preictal and ictal classification using wavelet-transformed features.
• Comparative Validation: Demonstrates through ablation studies that the hybrid approach outperforms standalone Optuna and standalone Q-learning strategies in accuracy, ROC-AUC, and convergence speed.
• Explainability: Incorporates LIME techniques to identify influential EEG features, enhancing the transparency and clinical interpretability of the model's predictions.

Experimental Results
Experiments were conducted on the CHB-MIT dataset with an 80:20 train-test split. The proposed framework achieved the following results compared to baseline methods:

• Classification Accuracy: 93.10% (vs. 92.00% for Optuna-only and 91.51% for Q-learning only).
• ROC-AUC Score: 0.9774 (vs. 0.9710 for Optuna-only).
• Convergence Behavior: The hybrid model demonstrated faster convergence and greater stability across trials, with smoother training/validation loss curves and a reduced gap between them compared to standalone methods.
• Confusion Matrix: Showed a significant reduction in false negatives, a critical metric for medical safety in seizure detection.
• Optimal Hyperparameters: The framework identified specific optimal values, including 192 GRU units, 2 attention heads, a feed-forward dimension of 256, and a learning rate of $1.02 \times 10^{-4}$.

Significance and Claims
The paper claims that the integration of reinforcement learning-based adaptive exploration with Bayesian hyperparameter optimization provides a superior strategy for navigating complex, high-dimensional search spaces in deep EEG classification. The authors assert that this hybrid approach effectively balances exploration and exploitation, leading to:

1. Enhanced Stability: Reduced sensitivity to hyperparameter fluctuations and more stable convergence.
2. Improved Clinical Utility: Higher accuracy and lower false-negative rates make the system more reliable for automated seizure detection and monitoring.
3. Interpretability: The use of LIME ensures the model's decisions are explainable, which is vital for clinical decision support systems.

Limitations and Future Work
The authors acknowledge that the study is currently limited to the Preprocessed CHB-MIT database and binary classification (preictal vs. ictal). They note that the hybrid approach introduces increased computational complexity. Future work is proposed to validate the framework on larger, multi-center datasets, extend it to multi-class seizure analysis, and explore lightweight optimization techniques for real-time deployment.