The StackPip Framework for Hybrid Ensemble Learning in Stellar Classification

Technical Summary: The StackPip Framework for Hybrid Ensemble Learning in Stellar Classification

Problem Statement
The classification of celestial objects—specifically stars, galaxies, and quasars—is a fundamental challenge in astrophysics. Traditional methods, such as the Morgan-Keenan (MK) system and manual spectral analysis, are labor-intensive, time-consuming, and struggle to scale with the massive influx of data from modern astronomical surveys like the Sloan Digital Sky Survey (SDSS). The SDSS Data Release 17 (DR-17) presents high-dimensional datasets that require automated, scalable, and precise classification techniques. While supervised learning has been applied previously, this study addresses the need for a robust framework that mitigates overfitting risks and data labeling challenges by integrating unsupervised techniques with advanced ensemble learning.

Methodology
The authors propose the StackPip Framework, a hybrid approach combining unsupervised learning, feature engineering, and ensemble-based machine learning. The methodology proceeds through the following stages:

1. Data Source and Preprocessing: The study utilizes the SDSS DR-17 dataset, containing 100,000 instances with features including Right Ascension, Declination, photometric filters (u, g, r, i, z), and Redshift. The data undergoes Exploratory Data Analysis (EDA) involving univariate and multivariate analysis, box plots for outlier detection, and t-SNE for dimensionality visualization.
2. Feature Engineering and Selection:
   • Feature Construction: New features were derived by calculating differences between photometric bands (e.g., $g-r$, $i-z$, $u-r$, $i-r$, $z-r$).
   • Dimensionality Reduction: Principal Component Analysis (PCA) was applied to preserve 95% and 98% of the variance in the data.
   • Normalization and Regularization: Data was normalized to a [0, 1] range to prevent scale bias, and regularization techniques were employed to reduce model complexity and prevent overfitting.
3. Model Architecture:
   • Baseline and Unsupervised Models: The study evaluated baseline models (Dummy Classifier) and unsupervised techniques like K-Nearest Neighbors (KNN).
   • Ensemble Models: A comprehensive suite of ensemble algorithms was tested, including Decision Trees, Random Forest, AdaBoost, XGBoost, Gradient Boosting, and a Stacking Classifier.
   • Pipelining: A pipeline framework was implemented to automate the workflow from data preprocessing and feature engineering to model training and evaluation, ensuring consistency and efficiency.
4. Experimental Design: The dataset was split into two distinct training/testing/validation proportions to assess robustness:
   • Proportion (i): 60% training, 20% testing, 20% validation.
   • Proportion (ii): 70% training, 20% testing, 10% validation.

Key Contributions

• Hybrid Framework: The development of the StackPip framework, which integrates feature engineering with a Stacking Classifier meta-model to leverage the strengths of multiple base learners.
• Comparative Analysis: A rigorous comparison of ten different machine learning approaches (ranging from Dummy Classifiers to complex ensemble methods) across two different data split ratios.
• Feature Significance: Identification of "Redshift" as the most significant feature (contributing ~0.58–0.59 to feature importance), alongside photometric differences ($z-r$, $g-r$, $u-r$) as critical determinants for classification accuracy.
• Pipeline Implementation: Demonstration of how pipelining streamlines the classification process, reducing manual intervention and improving reproducibility.

Results
The study reports the following performance metrics:

• Baseline Performance: The Dummy Classifier served as a baseline with high loss and low accuracy, confirming the necessity of advanced algorithms.
• Individual vs. Ensemble: Ensemble methods consistently outperformed standalone models (e.g., Logistic Regression, SVM).
  • XGBoost and Random Forest achieved low test losses (0.09 and 0.085, respectively) in the respective proportions.
  • Proportion (ii) (70:20:10 split) generally yielded better performance than Proportion (i).
• Stacking Classifier: The Stacking Classifier achieved the highest accuracy among all tested methods, reaching 98% accuracy.
• Pipelining Performance: When the full pipeline (including feature engineering and Stacking) was applied to the test set, the framework achieved an overall accuracy of 97%, with F1-scores of 0.98 for Galaxies, 0.94 for QSOs, and 0.99 for Stars.

Significance and Claims
The paper claims that the proposed StackPip framework offers a scalable and efficient solution for stellar classification, significantly outperforming traditional manual methods and standard machine learning approaches. By achieving 97–98% accuracy, the framework demonstrates that combining feature engineering with hybrid ensemble learning effectively handles the high dimensionality and complexity of astronomical data. The authors position this work as a step toward automating the classification of celestial bodies, reducing the time and energy consumption associated with manual spectral analysis.

The study concludes that while ensemble models like XGBoost and Random Forest are highly effective, the Stacking Classifier provides superior robustness. The authors suggest that future work could explore deep learning architectures (CNNs, RNNs) and further hyperparameter optimization via grid or random search to potentially enhance performance further.