Habit and the hippocampus: Model-based spatial representations without outcome-sensitive control

This study demonstrates that the hippocampus can support habitual, outcome-insensitive action selection during spatial navigation, thereby dissociating the use of cognitive maps from value-sensitive, goal-directed control.

The Gist

The Big Question: Does the Brain's "GPS" Always Know What You Want?

Imagine your brain has two main modes for making decisions:

1. The Goal-Directed Mode (The Chef): You know exactly what you want (e.g., a spicy taco). If the restaurant runs out of spicy tacos and only has plain ones, you stop ordering. You update your plan based on your current hunger and desires.
2. The Habit Mode (The Robot): You have a programmed routine. "When I see the red light, I turn left." Even if you suddenly hate red lights or the left turn leads to a wall, the robot keeps turning left because that's what it was programmed to do. It doesn't care about the outcome.

Scientists have long believed that the hippocampus (a part of the brain famous for navigation and memory) is the "Chef." They thought that because the hippocampus builds a mental map of the world (a "cognitive map"), it must automatically help us make flexible, goal-directed choices.

This paper asks a surprising question: Can the hippocampus build a perfect map, but still let the "Robot" take the wheel?

The Experiment: The Maze of Milk and Water

The researchers set up a tricky game for rats to test this.

The Setup:
Imagine a giant square track with a cross in the middle. There are four corners.

• Two corners have Milk (which rats love).
• Two corners have Water (which rats usually ignore unless they are very thirsty).

The Twist:
The rule changes depending on where you are in the maze.

• If you are at the North intersection: Turn Left for Milk, Turn Right for Water.
• If you are at the South intersection: Turn Left for Water, Turn Right for Milk.

The rats had to learn that the same action (turning left) leads to different rewards depending on where they are. This requires a mental map, not just a simple "turn left at the red sign" habit.

The Three Groups of Rats

The scientists split the rats into three groups to see how they learned:

1. The Beacon Group: They had bright LED lights shining on the milk corners. They could just follow the light like a moth to a flame. They didn't need a map; they just needed eyes.
2. The Texture Group: The floor felt different at the North and South intersections (like sandpaper vs. smooth tile). They had to feel the floor to know where they were.
3. The Path Integration Group: It was pitch black, and the floor felt the same everywhere. They had to rely entirely on their internal sense of movement (like counting steps) to know where they were.

The Test:
Once the rats were experts, the scientists made them thirsty. Suddenly, water became the most valuable thing in the world.

• The Goal-Directed Test: If the rats were "Chefs," they should realize, "Hey, I'm thirsty! I need water, not milk!" and start turning toward the water corners.
• The Habit Test: If the rats were "Robots," they would keep turning toward the milk corners because that's what they practiced for weeks, even though they are now thirsty.

The Shocking Results

1. The "Robot" Took Over
Even though the rats had spent weeks learning a complex map (especially the Texture and Path Integration groups), when they were thirsty, they kept choosing milk. They ignored their new desire for water. They were stuck in "Habit Mode."

2. The GPS Was Still Working
To prove the rats were actually using their hippocampus (the map), the scientists temporarily "turned off" the hippocampus using a drug (muscimol).

• Result: The rats who needed the map (Texture and Path groups) immediately got lost and failed the maze. The rats who just followed the lights (Beacon group) were fine.
• Conclusion: The rats were using their hippocampus to navigate, but they were using it to run a habit, not to make a smart, flexible choice.

3. The "Chef" Only Wakes Up with Practice
The scientists tried a different approach with a new group of rats. Before testing them, they made the rats practice the maze while they were thirsty. They forced the rats to learn the new rules while in that state.

• Result: These rats did switch to water. They became "Chefs."
• Conclusion: The hippocampus can support goal-directed behavior, but only if you explicitly train it to link the map to the new value. Just having a map isn't enough; you have to teach the map how to care about what you want right now.

The Takeaway: A Map is Not a Plan

Think of the hippocampus as a GPS navigation system.

• In this study, the GPS was working perfectly. It knew exactly where the rat was and how to get to the corners.
• However, the GPS was set to "Follow the Old Route" (Habit) rather than "Recalculate for Current Traffic" (Goal-Directed).
• The fact that the GPS was on didn't mean the driver was making a smart, flexible decision. The driver was just blindly following the blue line on the screen, even if the destination was no longer desirable.

In simple terms:
Having a great mental map of the world (the hippocampus) does not automatically mean you are making smart, flexible decisions. You can use a perfect map to follow a rigid, mindless habit. To be truly "goal-directed," your brain needs to do more than just know where you are; it needs to actively link that location to what you want at this exact moment.

Technical Summary

1. Problem Statement

The prevailing view in neuroscience posits that the hippocampus supports goal-directed navigation by constructing internal "cognitive maps" (model-based representations) that allow for flexible decision-making. Goal-directed behavior is operationally defined by sensitivity to outcome revaluation (e.g., changing the value of a reward based on motivational state, such as thirst). Conversely, habitual behavior is defined by insensitivity to outcome revaluation and is typically associated with stimulus-response (S-R) associations mediated by the dorsolateral striatum.

While previous studies suggest the hippocampus is critical for spatial strategies and goal-directed actions, it remains unclear whether hippocampal-dependent spatial navigation can ever be governed by habitual, outcome-insensitive control. Specifically, does the mere reliance on a hippocampal cognitive map guarantee that an animal's decisions will be sensitive to changes in reward value? This study investigates whether the hippocampus can support habitual, value-insensitive action selection in a spatial task.

2. Methodology

Subjects and Groups

• Subjects: 38 Long-Evans rats (16 males, 22 females) with bilateral dorsal hippocampal cannulae implants.
• Experimental Design: Rats were assigned to three task conditions based on how they discriminated between choice points:
  1. Path Integration (PathInt): Trained in darkness with no local cues; required self-motion integration (path integration) to distinguish North vs. South choice points.
  2. Texture: Trained in darkness with distinct tactile floor textures at choice points.
  3. Beacon: Trained with visual LED beacons near reward wells; allowed navigation without distinguishing specific choice points (hippocampal-independent).
• Assessment Regimens: Within each task condition, rats were split into two subgroups:
  • Incentive Learning (IL): Received water restriction followed by a two-bottle preference test (learning the new value of water) without re-experiencing the task contingencies.
  • Revaluation Training (RT): Received water restriction followed by a full rewarded training session on the maze under deprivation (learning the new value of the action/location leading to water).

Task Apparatus

• A custom 2m × 2m square track surrounding a central plus-maze.
• Contingency: Identical turning actions (Left/Right) led to opposite rewards (Milk vs. Water) depending on the spatial location (North vs. South choice point).
  • North: Left → Milk, Right → Water.
  • South: Left → Water, Right → Milk.
• The environment was visually symmetrical, devoid of distal cues, and rotated daily to prevent reliance on external landmarks.

Procedures

1. Acquisition: Rats trained until they met strict accuracy criteria (≥80% in first 10 trials, ≥90% overall).
2. Revaluation Manipulation:
   • IL: Rats experienced the new value of water (thirst) in isolation (two-bottle test) but did not perform the maze task while thirsty.
   • RT: Rats performed the maze task while thirsty, directly associating the action/location with the new high value of water.
3. Testing:
   • Extinction/Revaluation Tests: Non-rewarded probe sessions conducted under water restriction (Revaluation) and without restriction (Extinction).
   • Hippocampal Inactivation: Intracranial infusions of Muscimol (GABA-A agonist) or aCSF (vehicle) were administered to test the dependence of performance on the hippocampus.
4. Outcome Sensitivity Measure: The primary metric was whether rats shifted their choice from the previously preferred "Milk" path to the "Water" path during the Revaluation Test under thirst, despite no reward being delivered.

3. Key Results

Acquisition and Hippocampal Dependence

• PathInt rats required the most training sessions, followed by Texture, and then Beacon rats (fastest).
• Muscimol Inactivation:
  • In the Discrimination Groups (PathInt and Texture combined), muscimol infusion significantly impaired choice accuracy, confirming these tasks were hippocampal-dependent.
  • In the Beacon Group, muscimol had no effect on accuracy, confirming these tasks were hippocampal-independent.

Outcome Sensitivity in the Incentive Learning (IL) Regimen

• Behavioral Shift: Water restriction successfully shifted fluid preference from milk to water in all groups (confirmed by two-bottle tests).
• Maze Performance: Despite the shift in preference, both the Beacon-IL and Discrimination-IL groups continued to select the milk-associated response during the extinction/revaluation test.
• Conclusion: Even when accurate performance required the hippocampus (Discrimination-IL), the behavior was habitual and outcome-insensitive. The rats did not update their navigational choices based on the new value of water.

Outcome Sensitivity in the Revaluation Training (RT) Regimen

• Behavioral Shift: When rats received explicit revaluation training (performing the maze while thirsty), both Discrimination-RT and Beacon-RT groups successfully shifted their choices to the water-associated path.
• Conclusion: Outcome sensitivity emerged only when the new value was learned in the context of the action, regardless of whether the task was hippocampal-dependent or independent.

Hippocampal Role in RT

• In the Discrimination-RT group, muscimol inactivation still impaired spatial accuracy, confirming the task remained hippocampal-dependent even after the rats became outcome-sensitive.

4. Key Contributions

1. Dissociation of Cognitive Maps and Value Control: The study provides direct evidence that reliance on hippocampal representations (cognitive maps) is not sufficient to confer outcome-sensitive (goal-directed) control. The hippocampus can support spatial navigation that is entirely habitual and insensitive to reward devaluation.
2. Habitual Navigation: It demonstrates that "model-based" spatial strategies (requiring internal maps) can be governed by "model-free" value policies (cached action values) if the animal is not explicitly trained to update those values in the context of the action.
3. Mechanistic Interpretations: The authors propose two potential mechanisms for this dissociation:
   • "Memory without Prediction": The hippocampus infers the current state (mnemonic function) to trigger a cached habit, without simulating future outcomes.
   • "Prediction without Value": The hippocampus predicts future states but fails to couple these predictions with a value system that updates dynamically based on motivational state.
4. Role of Revaluation Training: The study clarifies that outcome sensitivity is not an inherent property of the hippocampus but depends on the learning history (specifically, whether the animal learns the new value while performing the task).

5. Significance

• Theoretical Impact: This work challenges the binary view that hippocampal strategies are inherently "goal-directed" and striatal strategies are "habitual." It suggests a more complex interaction where the same neural substrate (hippocampus) can support both flexible and rigid behaviors depending on how value information is integrated.
• Clinical/Behavioral Relevance: It refines the understanding of how animals (and potentially humans) navigate complex environments. It suggests that extensive training on a spatial task can lead to rigid, habitual navigation even when the brain region responsible for the map is intact.
• Future Directions: The findings suggest that to achieve flexible, goal-directed behavior in spatial tasks, the brain must not only construct a map but also explicitly couple that map to a value system capable of rapid updating. This has implications for understanding disorders involving maladaptive habits (e.g., addiction, OCD) where spatial or contextual cues trigger rigid behaviors despite negative outcomes.

In summary, the paper demonstrates that the use of a cognitive map is necessary for complex spatial navigation but insufficient for goal-directed flexibility; the coupling of spatial representations to dynamic value systems is a distinct process that requires specific learning conditions.