🤖 machine learning

When Bigger is Worse: A Practitioner's Guide to Model Selection Under Data Scarcity

This paper challenges the prevailing scaling law assumption in Earth observation by demonstrating that under data-scarce conditions, smaller models like YOLO11N trained on high-resolution inputs significantly outperform larger models in both efficiency and accuracy, effectively inverting the traditional "bigger is better" paradigm.

Original authors: Kwame Mbobda-Kuate, Gabriel Kasmi

Published 2026-07-20

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Kwame Mbobda-Kuate, Gabriel Kasmi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to recognize tiny, specific things in a massive, messy photo album. In the world of artificial intelligence, there is a very popular rule of thumb that everyone follows: "Bigger is better." This idea, known as scaling, suggests that if you build a giant, super-complex brain for your robot and feed it a library full of millions of photos, it will always be smarter than a smaller brain with fewer photos. It's like assuming that a giant, 500-page encyclopedia will always help you find an answer faster and more accurately than a small, 10-page pamphlet, no matter what you are looking for. This rule works incredibly well when you have endless amounts of data, like in language processing where computers read the entire internet. But what happens when you are in a situation where data is rare, expensive to get, and the things you are looking for are tiny specks in a vast landscape? That is the question this paper asks. It explores whether the "bigger is better" rule still holds when you are working with limited resources and trying to spot small objects, like solar panels on rooftops, from satellite images.

The authors of this paper decided to test this "bigger is better" idea in a very specific, real-world scenario: finding rooftop solar panels in Madagascar. They knew that in many developing countries, getting high-quality, labeled data is hard and expensive, and the computers available to run these models are often small and weak. They set up a massive experiment using a family of AI models called YOLO11, which come in five different sizes, ranging from a tiny, lightweight version (YOLO11N) to a massive, heavy-duty version (YOLO11X). They didn't just train these models once; they created 180 different training runs by mixing and matching three things: the size of the model, the amount of data they fed it (from 10% of the total dataset up to 100%), and the resolution of the images (how sharp the picture is, ranging from 416 pixels to 1280 pixels).

The results were a complete surprise to the usual way of thinking. The paper found that in this data-scarce world, bigger is actually worse. The smallest model, the tiny YOLO11N, turned out to be the most efficient champion. It achieved nearly the exact same accuracy as the much larger models but used 22 times less storage space. In fact, the tiny model was so good that it reached the second-highest accuracy score possible (0.459), which was statistically indistinguishable from the best-performing larger model. The massive YOLO11X, which is 22 times bigger, performed just as well or worse while demanding a huge amount of computer power and memory. The authors explain that this happens because the large models are "overparameterized"—they have too many brain cells for the amount of information they are given, causing them to get confused and fail to generalize, much like a student trying to memorize a whole encyclopedia for a test that only has three questions.

Perhaps the most practical discovery was about where to spend your money and effort. The researchers found that if you have a limited budget, you should not buy more data or a bigger model; instead, you should buy sharper images. Increasing the image resolution from 416 pixels to 1280 pixels gave a huge boost in performance, far more than simply adding more low-resolution photos ever could. This is because the tiny solar panels are so small that at low resolution, they look like blurry smudges; no amount of extra data can fix a blurry picture, but a sharper picture makes the details visible immediately. The paper concludes that for this type of task, the best strategy is to pick the smallest possible model, use the highest resolution images you can get, and then use any remaining resources to label more of those sharp images. In this specific corner of Earth observation, the old rule is flipped: small, sharp, and efficient beats big, blurry, and expensive every time.

Technical Summary: When Bigger is Worse – A Practitioner's Guide to Model Selection Under Data Scarcity

Problem Statement
The prevailing paradigm in computer vision, driven by scaling laws, assumes that larger models trained on larger datasets consistently yield superior performance. This assumption, validated in data-rich domains like natural language processing (NLP), is often imported into Earth observation (EO) without verification. However, EO settings—particularly in developing countries—are structurally data-scarce due to expensive, expertise-dependent annotation and geographically uneven data distribution. Furthermore, operational deployments in these regions face hard compute constraints, rendering large models impractical regardless of their raw accuracy. In this regime, the relevant metric shifts from raw performance to efficiency (performance per unit of resource). A critical gap exists in the literature regarding the joint optimization of model size, dataset volume, and input resolution under these specific constraints. Existing studies address these dimensions in isolation or assume data abundance, leaving practitioners without principled guidance for resource allocation in data-scarce EO tasks.

Methodology
The authors conduct a systematic efficiency analysis using the YOLO11 model family on a rooftop photovoltaic (PV) detection task in Madagascar. The study utilizes the OpenStat Madagascar dataset, a crowdsourced collection of drone imagery (90% of the data) containing 8,977 images and 130,500 annotated solar objects. The dataset is characterized by small objects (64% of bounding boxes have a normalized area < 0.01) and heterogeneous acquisition conditions.

The experimental design is a factorial grid across three scaling dimensions:

Model Complexity: Five YOLO11 variants (N, S, M, L, X) ranging from 2.6M to 57.0M parameters (a 22× range in size).
Dataset Fraction: Training on 10%, 25%, 50%, and 100% of the available data (897 to 8,977 images).
Input Resolution: Three resolutions: 416 px, 640 px, and 1280 px.

This results in 60 unique configurations, each run with three random seeds for a total of 180 training runs. All models are initialized from COCO-pretrained weights and trained with identical hyperparameters.

Key Metrics
The primary metric is Model Efficiency, defined as $Eff = 10 \times \frac{mAP50}{ModelSize (MB)}$ . This metric prioritizes performance per unit of storage, which is the critical constraint for edge deployment. Secondary analysis includes inference throughput (FPS) to evaluate the accuracy-speed tradeoff and overparameterization ratios ( $\rho = \text{params} / N_{train}$ ) to analyze generalization behavior.

Key Results

Efficiency Inversion: Contrary to scaling laws, the smallest model, YOLO11N (2.6M parameters), achieves the highest efficiency (22× higher than the largest variant, YOLO11X). Crucially, YOLO11N reaches an absolute mAP50 of 0.459, which is statistically indistinguishable from the best-performing YOLO11S (0.464) and significantly outperforms all larger models.
Pareto Dominance: In the joint accuracy-throughput space, small models trained at high resolution (1280 px) occupy the apex of the Pareto frontier. They simultaneously maximize detection accuracy and inference speed. There is no tradeoff to resolve; selecting a larger model or lower resolution yields no benefit in either dimension.
Resolution vs. Data Volume: While increasing data volume from 10% to 100% at low resolution (416 px) yields a modest 10% efficiency gain, increasing resolution from 416 px to 1280 px at 10% data yields a comparable 8% gain. However, resolution is the dominant lever because it unlocks absolute performance ceilings that data volume alone cannot reach (an 8.7% mAP50 gap between 416 px and 1280 px at full data).
Mechanism of Inversion: The performance inversion is attributed to the bias-variance tradeoff in a data-scarce regime. Larger models are systematically overparameterized relative to the available data (high $\rho$ ), leading to underfitting and poor generalization. Additionally, COCO pretraining provides no exploitable prior for this specific domain (zero-shot mAP50 ≤ 0.004), meaning larger models cannot leverage pretraining to compensate for data scarcity.

Significance and Contributions
The paper claims to provide the first systematic analysis jointly characterizing model size, dataset fraction, and input resolution in a data-scarce EO setting. Its primary contributions are:

Empirical Evidence: Demonstrating that the "bigger is better" scaling prior inverts in data-scarce EO, with small models dominating both in efficiency and absolute accuracy.
Mechanistic Explanation: Grounding this inversion in overparameterization theory and the absence of domain transfer, showing that larger models are systematically undertrained relative to their capacity in this regime.
Practical Guidelines: Offering a clear resource allocation strategy for practitioners: select the smallest model consistent with task requirements, maximize input resolution, and only then invest remaining budget in additional annotations.
Robustness: Showing that these findings hold regardless of whether the deployment objective prioritizes static storage efficiency or real-time inference throughput.

The authors conclude that in data-scarce EO, where data abundance and domain transfer assumptions fail, the default reliance on large models is suboptimal. Instead, a principled efficiency analysis favoring small, high-resolution configurations is required.

More like this