Scalable Multi-Task Learning through Spiking Neural Networks with Adaptive Task-Switching Policy for Intelligent Autonomous Agents

The paper proposes SwitchMT, a novel methodology for scalable multi-task learning in resource-constrained autonomous agents that combines a Deep Spiking Q-Network with active dendrites and an adaptive task-switching policy to effectively mitigate task interference and outperform state-of-the-art methods in Atari games.

The Gist

Imagine you are training a robot to be a master of many different video games at the same time. You want it to play Pong, Breakout, and Enduro all in one go, without needing a massive supercomputer or a huge battery.

This paper introduces a new training method called SwitchMT. Think of it as a "Smart Coach" for your robot that knows exactly when to switch games to get the best results.

Here is the story of how it works, broken down into simple concepts:

1. The Problem: The "Stuck" Student

Imagine a student trying to learn three different sports: Tennis, Chess, and Swimming.

• The Old Way (Fixed Schedule): The coach says, "You will play Tennis for exactly 25 minutes, then switch to Chess for 25 minutes, then Swimming for 25 minutes, and repeat."
• The Flaw: What if the student is a genius at Tennis and masters it in 5 minutes? They waste 20 minutes just spinning their wheels. Conversely, what if Swimming is super hard for them, and they need 2 hours to get good, but the coach forces them to stop after 25 minutes? They never get better.
• The Result: The student gets frustrated, mixes up the rules (learning Tennis while trying to swim), and ends up being mediocre at everything. In the paper, this is called "Task Interference."

2. The Solution: The "Smart Coach" (SwitchMT)

The authors created SwitchMT, which changes the rules. Instead of a timer, the coach watches the student's brain.

• How it knows when to switch: The coach monitors the student's "brain waves" (the internal math of the neural network).
  • If the student is still improving rapidly, the coach says, "Keep going! You're getting better!"
  • If the student hits a wall and isn't improving anymore (a "plateau"), the coach says, "Okay, you've got this for now. Let's switch to a different game to keep your brain fresh."
• The Benefit: The student spends just the right amount of time on each game—no more, no less. They learn faster and don't get confused.

3. The Special Brain: "Spiking Neurons" with "Active Dendrites"

To make this robot efficient enough to run on a small device (like a drone or a self-driving car), the authors didn't use a standard computer brain. They used a Spiking Neural Network (SNN).

• The Analogy: A standard computer brain is like a lightbulb that is always on, burning energy even when it's just thinking about nothing. A Spiking Neural Network is like a firefly. It only flashes (spikes) when it has something important to say. This saves a ton of energy.
• The "Active Dendrites": Imagine the brain has special "switches" (dendrites) that can turn specific parts of the brain on or off depending on the game.
  • When playing Pong, the "Tennis Mode" switches on.
  • When playing Breakout, the "Chess Mode" switches on.
  • This prevents the robot from mixing up the rules of the games. It creates a specialized "mini-brain" for each task inside the same big brain.

4. The Results: Beating the Best

The researchers tested this "Smart Coach" on three classic Atari video games:

• Pong: The robot played almost as well as a human.
• Breakout: It did significantly better than all previous robots, learning a strategy to hit the ball to the edges.
• Enduro: It drove the car for a very long time, almost matching human skill.

The Big Win:
Usually, to make a robot smarter, you have to make its brain bigger and heavier (more complex). SwitchMT managed to beat the current "champion" robots (like MTSpark) without making the brain any bigger. It just got smarter about when to switch tasks.

Summary

SwitchMT is like a personal trainer who doesn't use a stopwatch. Instead, they watch your sweat and heart rate. If you're crushing the workout, they keep you going. If you're hitting a wall, they switch you to a different exercise.

By using a super-efficient "firefly brain" (Spiking Neural Networks) and this smart switching strategy, they created an AI that can learn many tasks at once, faster, cheaper, and without getting confused. This brings us one step closer to robots that can truly adapt to our messy, real-world lives.

Technical Summary

Here is a detailed technical summary of the paper "Scalable Multi-Task Learning through Spiking Neural Networks with Adaptive Task-Switching Policy for Intelligent Autonomous Agents," accepted at DAC 2026.

1. Problem Statement

The paper addresses the challenge of training resource-constrained autonomous agents to learn multiple tasks simultaneously (Multi-Task Learning or MTL) in dynamic real-world environments.

• Core Issue: Existing Reinforcement Learning (RL) methods, particularly those based on Artificial Neural Networks (ANNs) and Spiking Neural Networks (SNNs), suffer from task interference. Conflicting objectives between tasks degrade learning performance.
• Limitations of State-of-the-Art (SOTA): While recent SNN-based methods (e.g., MTSpark) improve temporal processing and energy efficiency, they rely on fixed task-switching intervals (e.g., training for a set number of episodes per task before switching).
  • Inefficiency: Fixed intervals ignore the actual learning progress. If a task is mastered early, resources are wasted; if a task is difficult, the agent switches too soon, leading to under-training.
  • Scalability: This rigidity hinders the deployment of scalable, on-device learning agents that must adapt to diverse environments without manual hyperparameter tuning.

2. Methodology: SwitchMT

The authors propose SwitchMT, a novel framework that combines a specialized SNN architecture with an Adaptive Task-Switching Policy.

A. Network Architecture Selection

SwitchMT utilizes a Deep Spiking Q-Network (DSQN) enhanced with two key components to handle multi-task interference:

1. Active Dendrites: The integrate-and-fire (IF) neurons are equipped with active dendrites that receive task-specific context signals. These signals dynamically modulate neuronal activations, effectively creating specialized sub-networks within a single model for different tasks. This reduces interference by isolating task-relevant features.
2. Dueling Structure: The network separates the estimation of the State Value ($V(s)$) and the Action Advantage ($A(s, a)$). This improves generalization across actions, allowing the agent to better differentiate the impact of individual actions in varying states.

B. Adaptive Task-Switching Policy

The core innovation is a dynamic policy that replaces fixed intervals with a learning-progress-based trigger.

• Mechanism: The system monitors the relative change in model parameters ($\Delta \theta$) over a sliding window of $K$ episodes.
• Decision Logic:
  • Calculate the L2 norm of parameter changes: $\Delta \theta = \frac{||\theta_t - \theta_{t-K}||_2}{||\theta_{t-K}||_2} \times 100$.
  • Switch Condition: If $\Delta \theta$ falls below a predefined threshold (e.g., 10%), it indicates the agent has reached a learning plateau on the current task. The agent then automatically switches to a new task.
• Benefits:
  • Task-Agnostic Adaptation: Automatically allocates more training time to complex tasks (sparse rewards) and less to simple ones.
  • Prevents Catastrophic Interference: Avoids switching before a task is sufficiently learned.
  • Prevents Overfitting: Stops training on mastered tasks, saving computational resources.

3. Key Contributions

1. Novel Adaptive Policy: Introduction of a task-switching mechanism driven by internal network dynamics (parameter stability) rather than fixed time steps, eliminating the need for manual tuning of switch intervals.
2. Architectural Integration: Successful integration of active dendrites and dueling structures into a spiking Q-network to create specialized sub-networks for simultaneous multi-task learning without increasing model complexity significantly.
3. Scalability: The methodology enables a single model to solve multiple tasks efficiently, maintaining a fixed network size while improving performance, which is critical for resource-constrained edge devices.

4. Experimental Results

The authors evaluated SwitchMT on three Atari games (Pong, Breakout, Enduro) using an Nvidia GeForce RTX 4090 GPU, comparing it against DQN, DSQN, and the SOTA MTSpark_ADD.

Metric	Pong (Score)	Breakout (Score)	Enduro (Score)
Human Performance	-3	31	368
MTSpark_ADD (SOTA)	-5.4	0.6	371.2
SwitchMT (Proposed)	-8.8	5.6	355.2

• Performance: SwitchMT achieved competitive or superior scores compared to MTSpark_ADD. Notably, it achieved the highest game points (visual progress) in Breakout (7 points vs. 2 for MTSpark) and Enduro (2250 vs. 2180), demonstrating longer, more stable game episodes.
• Model Size: SwitchMT has 3,300,357 trainable parameters, identical to MTSpark_ADD. This proves that the performance gains come from the policy, not increased network complexity.
• Efficiency: By avoiding fixed schedules, SwitchMT reduces training time and mitigates overfitting, as it stops training on tasks once learning plateaus.

5. Significance

• Real-World Applicability: The ability to learn multiple tasks simultaneously without manual intervention or excessive memory/compute overhead makes SwitchMT ideal for intelligent autonomous agents (e.g., robotics, autonomous vehicles) operating in unpredictable environments.
• Energy Efficiency: By leveraging SNNs and adaptive switching, the approach minimizes unnecessary computation on mastered tasks, aligning with the low-power requirements of edge AI.
• Paradigm Shift: The work moves the field away from rigid, pre-defined training schedules toward dynamic, self-regulating learning curricula, solving the "task interference" problem more effectively than previous architectural or replay-based methods.

In conclusion, SwitchMT demonstrates that combining biologically-inspired SNN architectures (active dendrites) with data-driven adaptive policies creates a scalable, efficient, and high-performing solution for multi-task reinforcement learning.