Towards the definition and measurement of routines and the cognitive processes that underpin their maintenance

This paper proposes a novel metric called transition entropy to define and quantify routines, demonstrating through two experiments and a normative model that routine maintenance is robustly disrupted by frequent task switches due to reduced sensitivity to the odds of success.

The Gist

The Big Picture: Why Do We Get Out of Routines?

We all know the value of a good routine. Waking up, brushing your teeth, grabbing coffee, and heading to work in the same order every day saves mental energy. It's like driving a familiar route where you don't have to think about every turn.

But what happens when your routine gets disrupted? What if you have to switch between two different "jobs" constantly? Do you get better at switching, or does your routine fall apart?

This paper asks: How do we measure a routine, and why do they break down when the environment gets chaotic?

---

The Experiment: The "Hidden Animal" Game

Imagine you are playing a game on a computer screen. There is a grid of 16 squares. Somewhere behind one of those squares is a hidden animal (the "target").

• The Rules: You have to click squares to find the animal.
• The Trick: There are two different "houses" (Task A and Task B).
  • House A has the animal hidden in 4 specific squares.
  • House B has the animal hidden in 4 different specific squares.
• The Clue: In the beginning, the grid has a colored border (Red for House A, Blue for House B) to tell you which house you are in.

Phase 1: Learning the Routine
First, you learn the game. You realize, "Oh, if the border is Red, I only need to check these four Red squares." You develop a routine: Click Square 1, then 2, then 3, then 4. You do this over and over until it becomes automatic.

Phase 2: The Chaos (The Rehearsal)
Now, the researchers take away the colored borders. The grid is just grey. You don't know if you are in House A or House B anymore. You have to guess based on where you found the animal last time.

Here is where the two groups of people get different experiences:

1. The "Stable" Group: They are told, "Don't worry, you'll probably stay in the same house for a long time." (Only 5% chance of switching).
2. The "Variable" Group: They are told, "Hey, the house might change often!" (30% chance of switching).

The New Tool: "Transition Entropy" (The "Wiggle Factor")

The researchers needed a way to measure how "stuck" in a routine someone was. They invented a metric called Transition Entropy.

Think of it like a dance routine:

• Low Entropy (Good Routine): You do the exact same dance moves in the exact same order every time. Step, step, turn, jump. It's predictable.
• High Entropy (Broken Routine): You are dancing, but sometimes you step left, sometimes right, sometimes you spin, sometimes you jump randomly. It's chaotic.

The researchers calculated this "Wiggle Factor" for everyone.

• Result: The "Stable" group had a low Wiggle Factor (they stuck to their dance). The "Variable" group had a high Wiggle Factor (they were all over the place).

The Surprise: The "Variable" group didn't just switch houses; they started checking squares from the wrong house before they even finished checking the right one. They were so anxious about the house changing that they started "pre-emptively" checking the other house's squares, breaking their own routine.

Why Did the Routine Break? (The "Odds" of Success)

The researchers used a computer model to figure out why the Variable group messed up. They looked at two things the players were calculating in their heads:

1. The "Task Odds": "What are the chances I'm still in House A?"
2. The "Success Odds": "If I keep clicking House A squares, what are the chances I'll find the animal right now?"

The Finding:
Both groups were pretty good at guessing which house they were in (Task Odds). But the Variable group was terrible at judging the Success Odds.

Because they were switching so often, they became oversensitive to failure.

• Analogy: Imagine you are looking for your keys in your usual spot (the kitchen counter). You look, and they aren't there.
  • Stable Person: "Okay, I'll check the next usual spot (the table)."
  • Variable Person: "They aren't on the counter! The house must have changed! Maybe they are in the car! Maybe they are in the bathroom!" They immediately start checking random other places, even though the keys are probably just on the table.

The frequent switching made them feel like "failure" (not finding the target immediately) meant the whole situation had changed, so they abandoned their routine too early.

The Trade-Off: Routines vs. Flexibility

Here is the most interesting part. The researchers tested these people on a new, mixed-up game where the animal could be in a combination of House A and House B squares.

• The Result: The people who had the strongest routines (Low Wiggle Factor) were actually slower to learn this new mixed game.
• The People with "Broken" Routines (High Wiggle Factor) were faster to adapt to the new mixed game.

The Metaphor:

• The Rigid Dancer: They are amazing at their specific routine. But if you ask them to improvise a new dance, they freeze because they are so used to doing it exactly the same way.
• The Chaotic Dancer: They were messy and inconsistent during the practice. But because they were already checking different moves and mixing things up, they were actually better at learning a brand new, weird dance style.

The Takeaway

1. Routines are fragile: If you are in a volatile environment (where things change often), your brain gets anxious. It starts "testing the waters" of other options too early, breaking your routine.
2. Failure sensitivity: We break routines not just because the rules change, but because we get scared of not succeeding immediately. We think, "This isn't working, so the whole world must have changed," and we jump ship.
3. The Double-Edged Sword: There is a trade-off.
   • Strong Routines = Great for efficiency in stable times, but bad at adapting to new, mixed-up problems.
   • Weak Routines = Messy and inefficient, but great at adapting when the rules suddenly change.

In short: If you want to be efficient, stick to your routine. But if you want to be ready for anything, maybe you need to be a little bit "chaotic" and willing to try things that don't fit your usual pattern.

Technical Summary

1. Problem Statement

While the benefits of routines for daily functioning are well-recognized, the field lacks a rigorous, quantitative framework for defining routines and measuring their maintenance. Existing literature often focuses on deterministic sequences (fixed orders) or task-switching costs but fails to address:

• Self-ordering: How individuals spontaneously establish and maintain their own specific order of behaviors without explicit external constraints.
• Quantification: How to measure the regularity of a routine independent of the total number of steps taken (a limitation of previous variance-based metrics).
• Disruption Mechanisms: The specific cognitive processes that cause routine maintenance to fail in volatile environments, particularly regarding cross-task interference and the trade-off between routine stability and learning transfer.

2. Methodology

Experimental Design

The study re-analyzed data from two experiments (N=100 each) originally conducted by Barnes et al. (2026) using a Grid-Search Task.

• Task Acquisition: Participants learned to find a hidden target (an animal image) in a 16-square grid. Two distinct "Tasks" (A and B) were signaled by colored borders, each corresponding to a unique subset of four target locations.
• Rehearsal Phase: The colored borders were removed. Participants were told the task could switch on any trial but was more likely to stay the same.
  • Stable Group: Low probability of task switch ($p = 0.05$).
  • Variable Group: High probability of task switch ($p = 0.30$).
  • Optimal Strategy: To minimize moves, participants should exhaust all locations of the current task (where the last target was found) before switching to the other task.
• Learning Transfer Test (Experiment 1 only): Participants faced new tasks with new borders.
  • Identity Transfer: New border mapped to a previously learned full set (A or B).
  • Mixed Transfer: New border mapped to a combination of locations from both A and B.

Novel Metric: Transition Entropy (TE)

The authors introduced Transition Entropy (TE) to quantify routine maintenance.

• Calculation:
  1. Construct a transition count matrix where rows represent the current state (location $s$) and columns represent the next state ($s'$).
  2. Normalize rows to obtain transition probabilities $p(s'|s)$.
  3. Calculate Shannon entropy for each state $s$: $H_s = -\sum p(s'|s) \log p(s'|s)$.
  4. Sum entropies across all states to get the total TE score: $TE = \sum H_s$.
• Interpretation: Lower TE indicates high regularity (a strong routine); higher TE indicates disorder and weak routine maintenance. Crucially, TE is agnostic to the total number of steps, allowing comparison between efficient and inefficient routines.

Normative Modeling

To explain why routines fail, the authors employed a Bayesian normative model to estimate trial-by-trial:

1. Task Odds ($OR_{LT}$): The probability that the current task is the same as the last task.
2. Odds of Success ($OR_S$): The probability of finding the target if sticking with the current task vs. switching.
   These estimates were used as predictors in a logistic regression to model the likelihood of making a "cross-task" selection (selecting a location from the non-current task).

3. Key Results

A. Elicitation of Routines

• Participants robustly developed routines (low TE scores) significantly below chance levels ($p < .001$) and significantly lower than a randomly ordered agent.
• This confirms that humans spontaneously self-order behaviors to optimize search efficiency even without explicit cues.

B. Impact of Volatility on Routine Maintenance

• Variable Group Disruption: The Variable group (high switch probability) exhibited significantly higher Transition Entropy than the Stable group in both experiments ($p < .001$).
• Interpretation: Frequent task switching disrupts the maintenance of a consistent behavioral order. The Variable group did not simply form a different routine; they exhibited greater disorder in their selection order.

C. Cognitive Mechanisms of Disruption

• Odds of Success Sensitivity: Logistic regression revealed that the Variable group was less sensitive to the odds of success ($OR_S$) compared to the Stable group.
  • The Stable group correctly reduced cross-task selections when the odds of success for the current task were high.
  • The Variable group continued to make cross-task selections even when sticking to the current task was statistically more likely to yield a reward.
• Task Odds: Both groups estimated the probability of the task switching ($OR_{LT}$) similarly; the disruption was not due to misestimating when a switch occurred, but rather oversensitivity to early failures (lack of reward) within the current task, leading to premature exploration of other tasks.

D. Routine Maintenance vs. Learning Transfer

• Trade-off: There was a significant negative correlation ($r = -0.23, p = .039$) between Transition Entropy and Transfer Bias.
• Finding: Participants with lower TE (stronger routine maintenance) performed worse on Mixed Transfer tasks (requiring novel combinations of A and B elements) compared to Identity Transfer tasks.
• Implication: Strongly maintaining a routine (high predictability) comes at the cost of flexibility. Those who maintained rigid routines struggled to reconfigure their knowledge for novel, mixed tasks, whereas those with "looser" routines (higher TE) adapted faster.

4. Key Contributions

1. Theoretical Definition: Proposes a definition of routine maintenance based on the reliability of transitions between states rather than a fixed sequence of events.
2. Measurement Innovation: Introduces Transition Entropy (TE) as a metric that quantifies behavioral regularity independent of path length or efficiency, overcoming confounds in previous variance-based measures.
3. Mechanistic Insight: Identifies that routine disruption in volatile environments is driven by altered expectations of success (oversensitivity to early lack of reward) rather than a failure to track task-switch probabilities.
4. Cognitive Trade-off: Demonstrates a fundamental trade-off: Routine stability vs. Generalization. High routine maintenance (low entropy) facilitates efficiency but impairs the ability to generalize and combine learned components for novel problems.

5. Significance

This paper bridges the gap between habit formation theories and cognitive control in uncertain environments. It suggests that the failure to maintain daily routines is not merely a lack of discipline but a rational (though suboptimal) response to environmental volatility, where individuals prematurely abandon current strategies due to an overestimation of the utility of alternative tasks.

Furthermore, the findings challenge the view that routines are purely beneficial. The study posits that the cognitive mechanisms that enforce strict routine maintenance (exclusion of cross-task cues) inherently limit the flexibility required for transfer learning. This has implications for understanding how to design environments that balance the need for stable habits with the need for adaptive learning in complex, changing systems.