Optimized control protocols for stable skyrmion creation using deep reinforcement learning

This paper demonstrates that a deep reinforcement learning approach can identify optimized dynamic magnetic-field and temperature protocols to achieve deterministic, stable skyrmion creation in Fe3GeTe2 monolayers by minimizing dissipated work and suppressing thermal annihilation.

The Gist

The Big Picture: Building a Magnetic "Bubble" that Won't Pop

Imagine you are trying to build a tiny, perfect soap bubble out of magnetic energy. In the world of future computers (spintronics), these magnetic bubbles are called Skyrmions. They are incredibly useful because they can store data like tiny, durable bits of information.

However, building them is hard. They are fragile. If you try to blow them up too fast, they pop. If the room is too hot (thermal noise), they wobble and disappear. And if you build them in a hurry, they might end up shaped like a weird, stretched-out potato instead of a perfect sphere, which makes them unstable.

This paper is about a team of scientists who taught a computer (an AI) how to blow these magnetic bubbles perfectly, every single time, even in a hot, chaotic environment.

---

The Problem: The "Hammer and Nail" Approach

Before this study, scientists tried to create these skyrmions using a fixed-temperature field sweep.

The Analogy:
Imagine you are trying to shape a lump of clay into a perfect sphere. The old method was like taking a heavy hammer and hitting the clay at a constant speed while the room temperature stayed the same.

• The Result: Sometimes you get a sphere. But often, you smash the clay into a flat pancake, or you create a weird, lopsided blob that falls apart the moment you stop hitting it.
• The Success Rate: In this specific material (a magnetic crystal called $Fe_3GeTe_2$), the old method only worked about 16% of the time. That's like trying to throw a ball into a basket and missing 84 times out of 100.

The Solution: The "Smart Chef" (Deep Reinforcement Learning)

The researchers used Deep Reinforcement Learning (DRL). Think of this as a "Smart Chef" or a "Video Game AI" that learns by trial and error.

How it works:

1. The Goal: The AI wants to turn a messy magnetic pattern (a "stripe domain") into a perfect, stable skyrmion.
2. The Controls: The AI can control two things: the Magnetic Field (the force shaping the clay) and the Temperature (the heat that makes the clay soft and moldable).
3. The Learning: The AI tries thousands of different recipes.
   • Recipe A: Heat it up a little, push the magnet this way. (Result: Fail).
   • Recipe B: Heat it up a lot, push the magnet that way. (Result: Success!).
   • Recipe C: Heat it up, then cool it down very slowly. (Result: Perfect!).

The AI gets a "score" based on two things:

1. Did I make a skyrmion? (Yes/No).
2. Did I waste energy? (This is the secret sauce).

The Secret Sauce: Minimizing "Wasted Effort"

This is the most important part of the paper. The AI wasn't just told to "make a skyrmion." It was also told to minimize "dissipated work."

The Analogy:
Imagine you are walking through a crowded room to get to a door.

• The Old Way (High Dissipation): You run, shove people aside, trip over chairs, and sweat buckets. You get to the door, but you are exhausted, messy, and your clothes are torn. You are unstable and might collapse.
• The AI Way (Low Dissipation): You move smoothly, glide through the crowd, and arrive at the door calm and collected. You are perfectly balanced.

In physics terms, "dissipated work" is the energy wasted as heat and chaos during the process.

• If you create a skyrmion with high dissipation, it's like a stretched, wobbly balloon full of internal tension. It wants to snap back or pop immediately.
• If you create it with low dissipation, it's like a perfectly relaxed, round bubble sitting comfortably in its natural shape. It is stable and lasts a long time.

The Results: From 16% to 77%

The AI learned a very specific, complex dance of heating and cooling the material while applying magnetic fields. Here is what happened:

1. The "Borrowing" Trick: The AI realized that to shape the magnetic clay, it needed to "borrow" energy from the heat. It would briefly heat the material up to make the magnetic spins loose and easy to move, then cool it down to lock the shape in place.
2. The Shape: The old method made "potato-shaped" skyrmions that were unstable. The AI made "perfectly round" skyrmions.
3. The Score:
   • Old Method: 16% success rate.
   • AI Method (after training): 77.5% success rate.

Why Does This Matter?

Think of this like teaching a robot to fold laundry.

• Before: You told the robot, "Fold the shirt." It grabbed the shirt, crumpled it, and threw it on the bed. It worked 1 out of 6 times.
• Now: The robot learned the perfect sequence of movements. It doesn't just fold the shirt; it folds it in a way that the shirt stays neat for days.

The Takeaway:
This paper proves that by using AI to find the smoothest, least wasteful path to create these magnetic bubbles, we can make them stable enough to actually be used in real computers. It turns a chaotic, hit-or-miss process into a reliable, industrial-grade manufacturing step.

Summary in One Sentence

The researchers taught an AI to gently and efficiently "mold" magnetic bubbles using a perfect mix of heat and magnetism, turning a clumsy, 16% success rate into a reliable 77% success rate by ensuring the bubbles are created without any internal stress or wasted energy.

Technical Summary

1. Problem Statement

The practical application of magnetic skyrmions in spintronic devices is hindered by two critical challenges:

• Deterministic Creation: Existing methods for creating isolated skyrmions (e.g., controlling stripe domain instability via magnetic field sweeps) are often stochastic. They suffer from low success rates and frequently produce unwanted topological defects (like antiskyrmions) or unstable, elongated skyrmions that collapse.
• Thermal Stability: Even when created, skyrmions often lack long-term thermal stability. High dissipation during the creation process leaves the skyrmion in a high-energy, non-equilibrium state with significant internal excitations (e.g., elliptical distortion). This makes them susceptible to annihilation by thermal fluctuations, especially in materials like $Fe_3GeTe_2$ (FGT) where magnetic parameters are strongly temperature-dependent.

2. Methodology

The authors propose a Deep Reinforcement Learning (DRL) framework to discover optimized, time-dependent control protocols for external magnetic fields ($H_x, H_z$) and temperature ($T$).

• Simulation Environment:

  • Material: Monolayer $Fe_3GeTe_2$ (FGT), chosen for its perpendicular magnetic anisotropy and interfacial Dzyaloshinskii-Moriya interaction (DMI).
  • Physics Model: Micromagnetic simulations using the stochastic Landau-Lifshitz-Gilbert (LLG) equation via the MuMax3 package.
  • Parameters: Temperature-dependent saturation magnetization ($M_s$) and anisotropy ($K$) are incorporated to reflect real material behavior.
  • Initial State: A stripe domain configuration.
  • Target State: An isolated skyrmion with topological charge $Q = -1$.

• DRL Agent & Cost Function:

  • The agent generates time-dependent trajectories for $H(t)$ and $T(t)$ over a 100 ps window.
  • Cost Function ($\phi$): The agent minimizes a composite cost function:
    $$\phi[H_x(t), H_z(t), T(t)] = ||Q_f| - 1| + k_s \frac{W_f}{k_B T_f}$$
    • Term 1 ($||Q_f| - 1|$): Ensures the final topological charge is exactly 1 (successful nucleation).
    • Term 2 ($W_f$): Represents the dissipated work during the control trajectory. $W_f = \Delta E + T_f \Delta S_{res}$.
    • Significance of $W_f$: Minimizing dissipated work is crucial because, according to thermodynamic principles, the Kullback-Leibler (KL) divergence between the final state distribution and the thermal equilibrium distribution is upper-bounded by the dissipated work ($D_{KL} \leq W_f / k_B T_f$). Minimizing $W_f$ forces the system to remain close to equilibrium, enhancing thermal stability.
  • Training: A genetic algorithm optimizes the agent over 200 generations, averaging the cost over 10 stochastic thermal trajectories to ensure robustness.

3. Key Contributions

• Physics-Informed DRL: Unlike previous RL approaches that focused solely on the final spin configuration, this work integrates thermodynamic constraints (dissipated work) directly into the reward function. This bridges the gap between control theory and non-equilibrium thermodynamics.
• Dynamic Temperature Control: The DRL agent discovers that transiently increasing the temperature (absorbing heat from the environment) is a superior strategy compared to fixed-temperature sweeps. This "borrowing" of thermal energy helps the system overcome energy barriers to nucleate skyrmions more efficiently.
• Mechanism of Stability: The paper establishes a direct causal link between minimizing dissipated work, reducing the KL divergence, and achieving isotropic (circular) skyrmion shapes that resist thermal collapse.

4. Key Results

• Success Rate Improvement:

  • Conventional Method: Fixed-temperature field sweeps achieved only a 16.4% success rate for creating stable isolated skyrmions.
  • DRL Method (200th Generation): The optimized protocol achieved a 77.5% success rate.
  • The DRL agent successfully avoids the formation of unstable antiskyrmions and elongated skyrmions that plague conventional methods.

• Thermal Stability & Morphology:

  • Shape: Skyrmions generated by the 200th-generation agent are nearly circular (isotropic), whereas conventional methods or early-generation DRL (50th gen) produce elliptical or oversized skyrmions.
  • Relaxation Dynamics: In 10 ns relaxation simulations:
    • Conventional/Early DRL skyrmions collapsed rapidly due to internal elliptical excitation modes.
    • 200th-generation skyrmions remained topologically intact with bounded oscillations in area and ellipticity.
  • Effective Success Rate: When accounting for both nucleation success and subsequent thermal survival, the DRL protocol reached 77.5%, compared to 16.4% for the conventional method.

• Thermodynamic Analysis:

  • Entropy Production: The DRL protocol exhibits a unique "negative entropy production" phase initially, indicating the system absorbs thermal energy to facilitate nucleation.
  • Energy Landscape: As training progresses, the average magnetic energy density of the generated skyrmions converges to the equilibrium value ($\approx 1.189$ MJ/m³).
  • KL Divergence: The KL divergence between the generated state and the equilibrium state drops from $\approx 0.52$ (50th gen) to $\approx 0.03$ (200th gen), confirming the system is driven close to its local energy minimum.

5. Significance

• Robust Skyrmionics: This work provides a viable pathway to create skyrmions that are not only formed deterministically but are also robust enough to survive in thermally agitating environments, a prerequisite for practical spintronic memory and logic devices.
• Generalizable Framework: The DRL strategy is not limited to $Fe_3GeTe_2$. The framework can be adapted to other magnetic materials (e.g., $Fe_3GaTe_2$) and other generation methods (laser pulses, spin-transfer torque) by adjusting the physical model parameters.
• Beyond Microscopic Understanding: The study demonstrates that DRL can autonomously discover complex, non-intuitive control protocols (like transient heating) that optimize system performance without requiring a complete prior theoretical understanding of every microscopic interaction, offering a solution for controlling complex non-linear dynamics in topological spin textures.