📄 animal behavior and cognition

Resource depletion accelerates rate learning but not composition learning in patch foraging

This paper presents a normative Bayesian model demonstrating that while resource depletion accelerates the learning of local resource rates within patches, it does not speed up the slower, sample-dependent learning of global patch composition, leading to a divergence between reward-maximizing and information-seeking strategies that shapes optimal foraging departure rules.

Original authors: Kilpatrick, Z. P., El Hady, A.

Published 2026-08-05

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Kilpatrick, Z. P., El Hady, A.

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine you are a squirrel in a brand-new forest. You don't know where the best nut trees are, or even how many different kinds of trees exist. You have to do two things at once: find food to eat right now, and figure out the map of the forest so you can eat better later. This is a classic puzzle in animal behavior called "foraging." Scientists have long studied how animals decide when to stop eating in one spot and move to the next. The old rule of thumb, called the Marginal Value Theorem, assumes the animal already knows the forest perfectly. But in real life, animals are usually clueless when they arrive. They have to learn the rules while they are playing the game. This learning isn't just about remembering where the food is; it's about understanding the hidden structure of the world: how rich are the patches, how often do they appear, and how fast does the food run out?

This is where a new study by Zachary Kilpatrick and Ahmed El Hady steps in. They built a mathematical model to watch how a smart animal learns while it eats. They discovered something surprising: the act of eating itself is a secret teacher. As an animal eats from a patch, the food gets harder to find. This slowing down of the "food drip" tells the animal exactly how rich the patch was to begin with. It's like listening to a song that gets slower and slower; the pattern of the slowdown reveals the original tempo. However, this trick only works for learning about one patch at a time. It doesn't help the animal figure out what percentage of the whole forest is made of rich patches. To learn that, the animal has to keep moving, even if it means leaving a good meal early. The study shows that animals trying to maximize their belly-full (reward) and animals trying to learn the map (information) will actually behave differently, especially when they are new to the area.

The Great Forest Map and the Emptying Bowl

Let's dive into the forest. Imagine you are a forager, and your world is made of "patches"—think of them as giant, magical bowls of fruit scattered across a field. Some bowls are deep and full (high yield), and some are shallow and nearly empty (low yield). You don't know which is which when you arrive. You just know that once you start eating, the fruit doesn't stay at a steady pace. Every time you grab a piece, the bowl gets a little emptier, and the next piece takes a tiny bit longer to find. This is called depletion.

In the past, scientists thought depletion was just a cost. It meant you had to work harder for less food, so you should leave the bowl as soon as it got slow. But Kilpatrick and El Hady realized that this slowing down is actually a clue.

Think of it like a clock that is running out of battery. If you hear a clock ticking steadily, you can't tell if it's a brand-new battery or an old one that's been running for hours. But if you hear the clock ticking slower and slower, you can figure out exactly how much battery it started with. The pattern of the slowdown pins down the truth. In their simulations, they found that because of this pattern, an animal learns how rich a specific patch is much faster if the patch is depleting than if it were a magic bowl that never ran out. The "spacing" between the fruit tells the story.

The Two Different Goals: Stomach vs. Brain

Here is where things get tricky. The animal has two goals, and they sometimes fight each other.

The Stomach Goal (Reward-Seeking): "I want to eat as much as possible right now."
The Brain Goal (Information-Seeking): "I want to learn the map of the forest so I can be smarter later."

In a forest where every patch is the same (a "homogeneous" forest), the Stomach and the Brain usually agree: stay in the bowl until it's almost empty. But the study found a subtle disagreement. If the animal is purely trying to learn the rate of the food, it might stay a tiny bit longer than the Stomach wants, just to get that final, perfect piece of data.

But the real drama happens in a mixed forest with both rich and poor patches (a "binary" environment).

The Stomach wants to stay in the rich bowls as long as possible to get the most fruit.
The Brain realizes that staying in a rich bowl doesn't teach it anything about the poor bowls. To learn the map, it needs to visit different types of bowls.

So, the "Brain" animal starts leaving the rich bowls early. It sacrifices a few bites of fruit to go find a poor bowl and check if it's actually poor. It's like a student skipping the last few minutes of a fun movie to go check the library, because they need to know if the library has the book they need for tomorrow. The study shows that this "under-harvesting" (leaving food behind) is a rational strategy for learning, not a mistake.

The Magic of Replenishing Patches

The researchers also looked at what happens if the forest is special: the bowls refill between visits. Imagine a garden where you can come back to the same rose bush every day.

If the bushes refill slowly, the animal has to wander everywhere to find food. It learns the whole map.
If the bushes refill quickly, the animal finds a few rich bushes and just hops between them in a loop. It becomes an expert on those specific bushes but remains totally ignorant of the rest of the forest. The study suggests that in fast-replenishing worlds, the "Stomach" animal locks onto a few rich spots and stops exploring, while the "Brain" animal keeps wandering to make sure it hasn't missed a better spot.

Counting vs. Timing: The Secret Switch

Finally, the study asked: What is the best rule for when to leave? Should the animal count how many pieces of fruit it ate, or should it time how long it has been waiting since the last piece?

The answer depends on how different the patches are.

If all the patches are roughly the same, the animal should count. "I'll eat 5 pieces, then leave."
If the patches are very different (some are super rich, some are terrible), the animal should time. "If I haven't found food in a while, this must be a bad patch, so I'll leave."

The researchers found a specific tipping point: if the difference in richness between the best and worst patches is more than about 25% (a quarter), the animal should switch from counting to timing. It's a clever switch that helps the animal avoid wasting time in a terrible patch.

The Bottom Line

This paper uses math and computer simulations to show that learning is a layered process. Eating food teaches you about the current spot, but moving around teaches you about the whole world. Depletion is a helpful teacher for the current spot, but it doesn't help you learn the big picture.

The most important takeaway is that animals (and maybe even us!) aren't just mindless machines eating until full. They are balancing a trade-off between getting food now and learning the rules for later. Sometimes, the smartest thing to do is to leave a good meal early, just to make sure you aren't missing out on something even better elsewhere. And as long as the animal's guess about how fast food runs out is roughly right, this smart, model-based approach beats just guessing and reacting to rewards every time.

Technical Summary: Resource Depletion Accelerates Rate Learning but Not Composition Learning in Patch Foraging

Problem Statement
Foraging is a hierarchical inference problem where an animal must simultaneously exploit a resource patch and learn the structure of the environment that generates those patches. Classical foraging theory, including the Marginal Value Theorem, typically assumes the forager possesses full knowledge of environmental statistics (e.g., the distribution of patch types and their encounter rates) upon arrival. However, in novel environments, animals must infer these parameters through exploration. This paper addresses the gap in understanding how the physical process of resource depletion—where each food encounter lowers the subsequent encounter rate within a patch—shapes the learning of two distinct environmental features: the encounter rate within a specific patch type and the composition (the fraction of high-yield vs. low-yield patches) across the environment.

Methodology
The authors develop a normative Bayesian framework for an agent foraging in a patchy environment. The model treats learning as a second level of inference:

Within-Patch Inference: The agent infers the initial encounter rate ( $\lambda_k$ ) of the current patch. The model incorporates depletion, where the rate decreases linearly with each chunk found ( $\lambda(t) = \lambda_k - \rho K(t)$ ), contrasting this with non-depleting (constant rate) environments.
Across-Patch Inference: The agent infers the global distribution of patch types, specifically the fraction ( $p_H$ ) of high-yield patches and the rates associated with each type.
Strategies: The study compares two objective functions:
- Reward-Seeking: Maximizing the long-term reward rate (intake per unit time).
- Information-Seeking: Maximizing the rate of entropy reduction (learning) regarding environmental parameters.
Departure Rules: The authors analyze threshold policies where the agent departs a patch when its belief about the patch's richness or the opportunity cost crosses a specific level. They also compare these model-based agents against model-free reinforcement learners and optimal "oracle" agents.
Environments: The analysis covers homogeneous environments (one patch type), binary environments (two patch types), and replenishing environments where patches recover between visits.

Key Contributions and Results

1. Depletion as an Information Source for Rate Learning
The paper demonstrates that resource depletion is not merely a cost but a source of information that accelerates the learning of within-patch encounter rates.

Mechanism: In a depleting patch, successive inter-encounter intervals are drawn from a decelerating sequence of rates ( $\lambda, \lambda-\rho, \lambda-2\rho, \dots$ ). The spacing of these intervals uniquely identifies the initial rate $\lambda$ much more sharply than a constant-rate stream of the same duration.
Result: In homogeneous depleting environments, the Mean Squared Error (MSE) of the rate estimate decays significantly faster than in non-depleting environments. The time required to reach a fixed accuracy threshold can be up to three times shorter in depleting patches.
Optimal Behavior: For a reward-maximizer in a homogeneous depleting environment, the optimal strategy is to consume nearly the entire patch before departing. Under an entropy-rate measure, staying indefinitely is optimal, but under an MSE-threshold measure (minimizing time to a fixed accuracy), the best policy is to consume the patch and depart. This departure occurs after consuming essentially the whole patch, a behavior distinct from the "over-staying" relative to a reward-seeker who also stays long; rather, the information-seeker (under the MSE criterion) and reward-seeker both consume the patch, but the information-seeker's rule is driven by the need to resolve the rate via the falling intervals, while the reward-seeker balances the marginal value of the last chunks against travel costs.

2. Depletion is Neutral for Composition Learning
In contrast to rate learning, the learning of environmental composition (the fraction of high-yield patches, $p_H$ ) is unaffected by depletion.

Mechanism: Composition is inferred from categorical observations of patch types. Each patch visit contributes at most one categorical observation (the type of the patch), regardless of how many chunks are consumed within it.
Result: The learning rate for composition scales as $1/n$ (where $n$ is the number of patches visited), not with the number of encounters within patches. Depletion neither accelerates nor hinders this process, provided the agent uses the correct likelihood function.
Implication: An information-seeking agent aiming to resolve the global composition should leave patches early (even rich ones) to sample more distinct locations, whereas a reward-seeker might stay longer.

3. Divergence of Reward- and Information-Seeking Strategies
The optimal departure rules for reward maximization and information maximization diverge based on what is uncertain:

Homogeneous Environment (Uncertain Rate): Information-seekers and reward-seekers both tend to stay long in depleting patches, but their specific departure thresholds differ based on the learning metric. Under an entropy-rate measure, staying indefinitely is optimal for information gain, while under an MSE-threshold measure, the optimal policy is to consume the patch and depart. In both cases, the information-seeker does not necessarily "over-stay" relative to a reward-seeker in a simple sense; rather, the reward-seeker departs when the marginal intake rate drops to the opportunity cost, while the information-seeker (under MSE) departs once the posterior standard deviation falls below a threshold, which often coincides with consuming the patch.
Binary Environment (Uncertain Composition): Information-seekers under-stay rich patches. Since composition is resolved by visiting new patches, the information-seeker leaves high-yield patches early to sample the environment's diversity, whereas the reward-seeker stays until the marginal intake rate drops to the global average.
Replenishing Environments: When patches recover between visits, the reward-maximizing policy collapses onto a stable limit cycle (orbit) over a subset of high-yield patches. The replenishment rate determines whether the forager maps the whole environment or locks onto a rich subset. High replenishment rates suppress composition learning for reward-seekers because they rarely visit untyped patches.

4. Variability Dictates the Optimal Departure Rule
The paper identifies a critical transition in the optimal heuristic for leaving a patch based on environmental variability:

Uniform Patches: When patch richness is similar, a counting rule (leaving after a fixed number of chunks) is optimal.
Variable Patches: When richness varies significantly (by more than ~24%), a timing rule (leaving after a fixed gap since the last find) becomes optimal.
Model-Based vs. Model-Free: Representing the environment (using a Bayesian model) yields a significant intake advantage (~20 percentage points) over learning a rule purely from reward (model-free), but only if the agent's model of the depletion rate is reasonably accurate. If the depletion rate is misspecified by more than ~25%, the model-based agent's performance collapses, and a model-free agent may outperform it.

Significance and Claims
The authors claim that their work provides an analytic account of why the early phase of foraging learning is accelerated by depletion and how movement strategies depend on the specific uncertainty being reduced.

Hierarchical Inference: The study clarifies that depletion operates at the rate-estimation level (within-patch) but is neutral at the composition level (across-patch).
Information vs. Reward: The paper highlights that information-seeking is not a monolithic drive; different learning objectives (reducing rate uncertainty vs. composition uncertainty) prescribe opposite movement behaviors (staying vs. leaving).
Robustness of Models: The advantage of maintaining an explicit environmental model is bounded by model accuracy. The paper argues that the "flatness" of the reward surface near the optimum means that precise parameter estimation is less critical for intake than the ability to adapt to changing costs (like travel time) and the correctness of the structural model (depletion dynamics).
Experimental Predictions: The framework generates testable predictions, such as the shift from counting to timing rules as environmental variance increases, and the specific divergence in residence times between reward- and information-seeking foragers in homogeneous versus binary environments.

The paper concludes that learning the environment buys significant intake over learning a rule from reward alone, provided the agent's assumptions about depletion leave room for the truth.