⚡ electrical engineering

When Does Feature Engineering Matter? A Multi-Algorithm Comparison of Sentinel-2 Classifiers for Coastal and Mangrove Mapping in the Persian Gulf

This study demonstrates that while machine learning classifiers (Random Forest, SVM, and XGBoost) achieve high accuracy for mangrove and coastal mapping in the Persian Gulf regardless of feature engineering, traditional Maximum Likelihood Classification relies heavily on the inclusion of spectral indices to significantly improve performance, though all methods struggle to distinguish between built-up, barren, and soil classes.

Original authors: Keivan Kabiri

Published 2026-09-24

📖 8 min read🧠 Deep dive

CC BY 4.0

Original authors: Keivan Kabiri

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The coast is a place of constant negotiation. It is where the land meets the sea, a shifting boundary that supports some of the most vital ecosystems on Earth, from mangrove forests that protect shorelines to tidal flats that feed countless marine species. Yet, these areas are also under pressure from human development, rising seas, and industrial activity. To protect them, scientists need to know exactly what is happening on the ground: where the trees are, where the water is shallow, and where the land is bare. For decades, the primary tool for seeing these vast, hard-to-reach places from above has been satellite imagery. These satellites capture the light reflected off the Earth's surface, creating a picture that computers can analyze to sort the landscape into different categories, such as forest, water, or city. However, turning those raw pictures into accurate maps is not a simple task. The light bouncing off a mangrove tree looks different from the light bouncing off a muddy flat, but the difference can be subtle, and the computer programs used to tell them apart must be chosen carefully. If the wrong program is used, or if the data fed into it is not prepared correctly, the resulting map could be misleading, potentially leading to poor decisions about how to manage these fragile environments.

In the Persian Gulf, a region where mangrove forests face intense pressure from oil and gas infrastructure, a researcher named Keivan Kabiri set out to find the most reliable way to map these coastal landscapes. He focused on five specific sites along the southern coast of Iran, ranging from relatively untouched mangrove stands to areas heavily influenced by industrial facilities. To test the best approach, he gathered high-resolution satellite images from the European Space Agency's Sentinel-2 mission. These images provide a clear view of the Earth with a resolution of ten meters, meaning each pixel in the image represents a ten-by-ten meter square on the ground. The challenge was to determine which computer algorithm could best sort these pixels into eight distinct categories: mangrove, mudflat, soil, shallow water, deep water, barren ground, built-up areas, and general vegetation. To do this, Kabiri compared four different methods. Two of these were modern, powerful machine-learning tools known as Random Forest and Extreme Gradient Boosting, which learn from examples to make complex decisions. A third was the Support Vector Machine, another sophisticated algorithm often used in remote sensing. The fourth was Maximum Likelihood Classification, a much older, traditional method that relies on statistical assumptions about how light behaves.

The study tested these algorithms under two different conditions to see how much the preparation of the data mattered. In the first scenario, the computers were fed only the six raw bands of light captured by the satellite, which correspond to different colors of light from the visible spectrum to the infrared. In the second scenario, the researchers added four extra layers of information to the data. These were not new images, but rather mathematical combinations of the existing light bands designed to highlight specific features, such as the health of vegetation or the clarity of the water. This process, known as feature engineering, is often thought to be a necessary step to improve accuracy. The researchers then ran the classification on all five sites and checked their work against a very high-resolution reference. They had previously flown drones over these same locations to create detailed aerial maps, which served as the ground truth to see which satellite-based maps were actually correct.

The results revealed a clear and surprising distinction between the old and new methods. The three modern algorithms—Random Forest, Support Vector Machine, and Extreme Gradient Boosting—performed exceptionally well in almost every case. They achieved an overall accuracy of roughly 95 to 97 percent, correctly identifying the land cover type for nearly every pixel. Remarkably, these high-performing tools did not need the extra, engineered information to do their job. Whether they were given just the raw satellite bands or the raw bands plus the extra indices, their accuracy remained consistently high. The addition of the extra data changed their performance by less than one percentage point in either direction, suggesting that these modern tools are robust enough to find the patterns they need directly from the raw images.

In stark contrast, the traditional method, Maximum Likelihood Classification, struggled significantly when it was given only the raw satellite bands. Without the extra information, it managed to correctly identify the land cover only about 55 percent of the time, a level of accuracy that would be useless for practical management. However, when the researchers added the extra spectral indices to the data, this old method improved dramatically, jumping to an average accuracy of 82 percent. While this was a massive improvement, it still fell far short of the modern methods, which remained in the high 90s. The study confirmed that the difference between the best modern method and the improved traditional method was statistically significant, meaning the gap was real and not just a fluke. The modern tools were simply better suited to the complex, messy reality of the coastal environment.

Even with the best tools, some challenges remained. The researchers found that all four algorithms, regardless of their sophistication, had trouble telling the difference between three specific types of land: built-up areas like buildings and roads, barren ground, and bare soil. These three categories often look very similar in satellite images because they are all bright and dry, reflecting light in nearly the same way. This confusion persisted even when the extra data was added, suggesting that the limitation lies not with the computer program, but with the satellite itself. The ten-meter resolution of the Sentinel-2 imagery is simply not fine enough to always distinguish between a patch of dry soil and a patch of concrete from that distance. This finding suggests that for tasks requiring precise monitoring of industrial expansion or specific ground types, satellite data alone may not be enough, and other data sources might be needed.

To ensure the results were trustworthy, the study employed a unique validation step. After the satellites were classified, the researchers compared the resulting maps against the high-resolution drone photos they had taken. This was an independent check, meaning the drone data was not used to teach the computers how to classify the images; it was used only to see if the final maps made sense. The results of this check were interesting. While the standard accuracy tests clearly showed that the modern algorithms were superior, the drone-based check gave a slightly different picture. It did not clearly separate the best performers from the worst, and in some cases, it even suggested the traditional method was doing as well as the modern ones. This happened because the drone check used broad rules to judge the maps, such as checking if a "soil" pixel looked bright enough to be soil. Since the traditional method often confused soil with buildings or barren ground, and all three of those look bright in the drone photos, the check gave credit to the traditional method for getting the general "look" right, even if it got the specific label wrong. This highlighted a crucial lesson: while independent checks are valuable for spotting obvious errors, they cannot replace detailed, ground-truthed accuracy tests when trying to rank which specific method is best.

Ultimately, this study offers clear guidance for anyone trying to map these coastal zones. If the goal is to create an accurate map of mangroves and tidal flats, the choice of computer algorithm matters less than one might think, provided a modern, machine-learning tool is used. Random Forest, Support Vector Machine, and Extreme Gradient Boosting are all reliable choices that work well without needing complex extra data preparation. The traditional method is not obsolete, but it requires a careful hand; it needs the extra spectral information to function at all, and even then, it cannot match the accuracy of the newer tools. The study also serves as a reminder that technology has limits. No matter how smart the computer is, it cannot always distinguish between a building and a patch of dirt if the satellite image is not sharp enough. For the scientists and managers working to protect the Persian Gulf, the path forward is to use the most robust modern tools available, but to remain aware that some details will always require a closer look from the ground or from a drone.

Technical Summary: When Does Feature Engineering Matter? A Multi-Algorithm Comparison of Sentinel-2 Classifiers for Coastal and Mangrove Mapping in the Persian Gulf

Problem Statement
Accurate monitoring of land use/land cover (LULC) along the Iranian coast of the Persian Gulf is critical for managing ecosystems under dual pressure from natural stressors and industrial development. While Sentinel-2 imagery offers high spatial resolution and frequent revisit times, few studies have systematically compared classification algorithms for this specific region. Existing literature often focuses on specific sites (e.g., Qeshm Island) or lacks a rigorous comparison between classical parametric methods and modern machine learning (ML) approaches. Furthermore, validation strategies frequently rely on training data sources (e.g., field surveys or Google Earth) rather than independent, high-resolution reference data, making it difficult to assess true generalization performance. This study addresses the gap in understanding how different classifiers—specifically Maximum Likelihood Classification (MLC), Support Vector Machine (SVM), Random Forest (RF), and Extreme Gradient Boosting (XGBoost)—perform on raw spectral bands versus index-augmented feature sets in complex coastal environments.

Methodology
The study evaluated four classifiers across five coastal sites in southern Iran (Melgonzeh, Dayyer, Asalouyeh, Khour-e Azini, and Tiyab), representing a gradient from natural mangroves to industrially influenced zones.

Data: Atmospherically corrected Sentinel-2 Level-1C imagery (acquired 2020–2023) was processed using the ACOLITE processor. Two feature sets were constructed: (1) six raw reflectance bands (Blue, Green, Red, NIR, SWIR1, SWIR2) and (2) the same six bands augmented with four spectral indices (NDVI, MNDWI, AWEI_sh, NDTI).
Classification: Each classifier was trained and tested using a 70/30 stratified split of reference data derived from field surveys and high-resolution Google Earth imagery.
- MLC: Implemented as a regularized Quadratic Discriminant Analysis.
- SVM: Used a radial basis function kernel.
- RF: An ensemble of 200 decision trees.
- XGBoost: Configured with 150 boosting rounds.
Validation:
1. Standard Accuracy: Overall Accuracy (OA) and Kappa coefficients were computed from the held-out test sets. Pairwise McNemar's tests were used to determine statistical significance between classifiers.
2. Independent Validation: To provide an external check, classified maps were compared against independent, high-resolution UAV orthomosaics (acquired 2–5 days after the satellite scenes). A rule-based RGB congruence score was calculated by evaluating if the classified pixel's label matched physical spectral rules derived from the UAV RGB data (e.g., Excess Green Index for vegetation, brightness thresholds for water).

Key Results

Classifier Performance: RF, SVM, and XGBoost demonstrated consistently high performance across all sites and feature sets, with mean Overall Accuracies ranging from 94.9% to 96.9%. The addition of spectral indices resulted in only marginal, classifier-specific changes for these three methods (ranging from -1.0% to +0.9%).
Feature Engineering Impact: MLC performed poorly using raw bands alone (mean OA: 55.1%). However, its performance improved dramatically with the inclusion of spectral indices (mean OA: 82.4%), representing an average gain of 27.3 percentage points. Despite this improvement, MLC remained significantly less accurate than RF (McNemar's $\chi^2$ = 165.885, $p < 0.001$ ).
Statistical Significance: McNemar's tests confirmed that RF, SVM, and XGBoost were statistically indistinguishable from one another ( $p > 0.05$ ) when using the index-augmented feature set. However, the inclusion of indices did produce a statistically significant, albeit small, improvement for RF ( $p = 0.048$ ).
Confusion Patterns: Across all classifiers, the "Built-up," "Barren," and "Soil" classes were the most persistently confused, suggesting a fundamental spectral limitation of 10m Sentinel-2 data for distinguishing these bright, dry terrestrial surfaces.
Independent Validation: The UAV-based RGB congruence scores (mean range: 44–49%) did not mirror the ranking established by ground-reference accuracy. Notably, MLC achieved a congruence score comparable to RF, despite its lower OA. The author attributes this to the coarse nature of the congruence rules, which validated general "terrestrial" plausibility rather than specific class discrimination.

Significance and Claims
The paper claims to offer practical guidance for classifier selection in data-scarce, ecologically important coastal settings. Its primary findings suggest that:

Classifier Choice: For modern, non-parametric methods (RF, SVM, XGBoost), the choice of algorithm matters less than the choice of feature set; these methods are robust and largely interchangeable for this application.
Feature Engineering Necessity: Feature engineering (adding spectral indices) is not uniformly necessary for all algorithms. It is disproportionately valuable for classical parametric methods like MLC, which rely heavily on engineered features to overcome their inability to model non-linear boundaries in raw spectral space.
Validation Practice: The study highlights the value of combining ground-reference accuracy with independent, training-free validation (UAV imagery). The divergence between the two validation methods underscores that independent plausibility checks serve a different purpose than labeled accuracy assessment and should not be used to contradict statistically significant accuracy differences.
Limitations: The author modestly notes that while their findings are transferable across the studied Persian Gulf sites, the persistent confusion between Built-up, Barren, and Soil classes indicates a ceiling on accuracy imposed by the sensor's spectral resolution, necessitating complementary data sources for specific monitoring tasks like tracking industrial expansion.

This work represents one of the first systematic, multi-site, multi-algorithm evaluations of Sentinel-2 classification performance for the Iranian Persian Gulf coast, providing a transferable framework for future monitoring efforts in similar environments.

More like this