Comparison of Machine Learning Methods for Monthly Rainfall Forecasting Using k-Nearest Neighbors, Random Forest, and Artificial Neural Networks

Technical Summary: Comparison of Machine Learning Methods for Monthly Rainfall Forecasting

Problem Statement
Rainfall is a critical meteorological variable for sectors ranging from agriculture to disaster mitigation, yet it remains one of the most difficult atmospheric parameters to forecast due to its highly nonlinear and dynamic nature. This challenge is particularly acute in tropical archipelagic regions like Indonesia, where rainfall patterns are driven by complex interactions between atmospheric phenomena (e.g., ENSO, IOD, MJO) and local topography. Conventional statistical methods often rely on assumptions of linearity and stationarity that fail to capture these complex relationships. While machine learning (ML) offers a promising alternative, there is a lack of comparative studies within a unified framework that specifically evaluates k-Nearest Neighbors (kNN), Random Forest (RF), and Artificial Neural Networks (ANN) for monthly rainfall forecasting in the Bangka Belitung Islands using observational data from the Indonesian Meteorology, Climatology, and Geophysics Agency (BMKG).

Methodology
The study utilized observational data from the Depati Amir Meteorological Station in Pangkalpinang, covering the period from 2015 to 2024. The research workflow involved data validation, preprocessing (aggregating daily data to monthly), and feature engineering. The predictor variables included rainfall, mean air temperature, relative humidity, sunshine duration, and rainfall lag features.

The dataset was chronologically partitioned into a training set (2015–2022) and an independent testing set (2023–2024). To preserve temporal dependencies during hyperparameter optimization, Time Series Cross-Validation (TSCV) was employed. Three distinct ML algorithms were implemented:

1. k-Nearest Neighbors (kNN): An instance-based algorithm predicting target values based on the average of the k nearest observations in the feature space, utilizing Euclidean distance. Predictor variables were standardized to ensure equal contribution to distance calculations.
2. Random Forest (RF): An ensemble learning method constructing multiple decision trees via bootstrap sampling and random feature selection. Predictions were derived by averaging the outputs of individual trees to reduce variance and overfitting. Feature importance analysis was also conducted.
3. Artificial Neural Networks (ANN): A Multilayer Perceptron (MLP) model trained using forward propagation and backpropagation to minimize prediction error through iterative weight adjustments.

Model performance was evaluated using three statistical metrics: Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and the Coefficient of Determination (R²). Lower MAE and RMSE values, alongside higher R² values, indicated superior performance.

Key Results
The comparative analysis revealed distinct performance differences among the three algorithms:

• Random Forest (RF): Achieved the best predictive performance with an MAE of 75.73 mm, an RMSE of 89.50 mm, and an R² of 0.605. The model effectively captured rainfall variability across both wet and dry seasons, with residuals concentrated closely around zero.
• k-Nearest Neighbors (kNN): Ranked second with an optimal k value of 13. It achieved an MAE of 83.53 mm, an RMSE of 98.23 mm, and an R² of 0.492. While it captured the general seasonal pattern, the model exhibited an "over-smoothing" effect, leading to the underestimation of peak rainfall events.
• Artificial Neural Networks (ANN): Demonstrated the lowest performance with an MAE of 112.38 mm, an RMSE of 136.60 mm, and an R² of 0.303. The model struggled with stability, exhibiting larger deviations and less consistent error distribution compared to the other methods.

Significance and Claims
The paper claims that the Random Forest algorithm is the most effective approach for monthly rainfall forecasting in the study area, outperforming both kNN and ANN. The authors attribute this superiority to the ensemble learning framework of RF, which successfully captures complex nonlinear relationships among meteorological variables while minimizing model variance through the averaging of multiple decision trees.

In contrast, the study suggests that the limited size of the training dataset (approximately 96 monthly observations) likely hindered the ANN's ability to learn underlying nonlinear relationships effectively, as neural networks generally require larger datasets for optimal generalization. Similarly, the kNN model's reliance on averaging nearest neighbors limited its capacity to represent extreme rainfall variations.

The study positions its primary contribution as the integration of these three specific machine learning algorithms with long-term BMKG observational data from a tropical archipelagic region under a unified evaluation framework. It does not claim to have developed a new algorithm but rather provides empirical evidence that ensemble-based methods (specifically RF) are more robust than instance-based or neural network-based methods for this specific forecasting task when data availability is relatively limited. The findings are intended to guide the development of more accurate rainfall forecasting systems for tropical regions.