Pretrained battery transformer (PBT): A foundation model for universal battery life prediction

This paper introduces the Pretrained Battery Transformer (PBT), a foundation model that leverages battery-knowledge-encoded mixture-of-experts layers to overcome data scarcity and heterogeneity, achieving state-of-the-art universal battery life prediction across diverse chemistries and conditions.

The Gist

The Big Problem: The "Black Box" of Battery Life

Imagine you buy a new electric car. You want to know: "How many years will this battery last before it dies?"

Right now, figuring this out is like trying to guess the lifespan of a human by watching them run a single lap. To get a real answer, scientists have to run batteries through thousands of charge-and-discharge cycles. This takes months or even years. It's expensive, slow, and creates a bottleneck for making better batteries.

Worse, every battery is different. Some are made in Japan, some in China. Some use different chemicals (like Lithium-ion vs. Sodium-ion). Some are charged fast, some slow. Some are used in hot deserts, others in cold snow.

Because of these differences, a computer program that learns from one type of battery often fails when you show it a different type. It's like teaching a student to drive a Toyota and then asking them to drive a tractor immediately after. They might know how to steer, but they won't know how to handle the tractor's unique engine.

The Solution: The "Battery Super-Student" (PBT)

The researchers created a new AI model called PBT (Pretrained Battery Transformer). Think of PBT not just as a calculator, but as a super-student who has read every battery manual ever written.

Here is how it works, broken down into three simple concepts:

1. The "Mixture of Experts" (The Team of Specialists)

Usually, AI models try to be a "jack of all trades," learning one big rule for everything. But batteries are too complex for one rule.

PBT uses a strategy called Mixture of Experts (MoE). Imagine a hospital emergency room.

• If a patient comes in with a broken leg, the Orthopedic Doctor takes over.
• If they have a heart issue, the Cardiologist steps in.
• If it's a skin rash, the Dermatologist handles it.

PBT does the same thing. It has a "Gatekeeper" (a smart router) that looks at the battery.

• Is it a Lithium battery? The Lithium expert wakes up.
• Is it a Sodium battery? The Sodium expert wakes up.
• Is it being used in extreme heat? The Heat expert wakes up.

By only using the right "expert" for the specific battery, the model learns much faster and more accurately, even when it doesn't have much data to work with.

2. The "Knowledge Injection" (The Textbook vs. The Trial-and-Error)

Most AI models learn by trial and error. They look at data and guess. But in battery science, we already know a lot of physics (e.g., "Heat makes batteries degrade faster").

PBT is special because it injects human knowledge directly into its brain.

• The Soft Encoder: Imagine a librarian who reads a book description and summarizes the key points for you. PBT reads the battery's specs (like "18650 size," "Graphite anode") and turns them into a smart summary.
• The Hard Encoder: Imagine a strict bouncer at a club. If the battery is a "Lithium" type, the bouncer physically blocks the "Sodium" expert from entering the room. This forces the AI to focus only on the rules that actually apply to that specific battery.

This combination allows PBT to learn from a small amount of data because it already "knows" the physics behind the numbers.

3. The "Universal Translator" (Transfer Learning)

The researchers trained PBT on 13 different datasets containing thousands of batteries. This is like the student reading 13 different textbooks on driving.

Once trained, they tested PBT on batteries it had never seen before (like new Sodium-ion batteries or huge industrial batteries).

• Old AI: "I've never seen a Sodium battery. I give up." (High error).
• PBT: "I've seen Lithium, and I know the physics of how ions move. Even though this is Sodium, the rules are similar. I can predict this." (Low error).

The Results: Why It Matters

The paper shows that PBT is a massive leap forward:

• It's Faster: It can predict battery life using just the first few cycles of data, saving months of testing time.
• It's Smarter: It outperformed the best existing methods by 21.8% on average. In some difficult cases, it was 86.9% better.
• It's Universal: It works on Lithium, Sodium, and Zinc batteries, and even on huge industrial batteries used in factories.

The Bottom Line

Think of PBT as the first "Foundation Model" for batteries. Just as large language models (like the one you are talking to now) learned to speak any language by reading the whole internet, PBT learned to "speak" battery life by reading every battery dataset it could find.

This means we can now design better batteries, manufacture them faster, and predict when they will fail with much higher accuracy. It turns the slow, expensive process of battery testing into a quick, smart prediction, helping us move faster toward a world powered by clean energy.

Technical Summary

1. Problem Statement

Accurate early prediction of battery cycle life is critical for optimizing battery design, manufacturing, and deployment. However, current data-driven approaches face two fundamental challenges:

• Data Scarcity: High-quality, full-life cycling data is expensive and time-consuming to generate (taking months or years), resulting in small datasets (often <1,000 cells).
• Data Heterogeneity: Battery data varies significantly across chemistries (Li-ion, Na-ion, Zn-ion), physical formats (cylindrical, pouch), manufacturing processes, formation protocols, and operating conditions. This leads to severe domain shifts, where models trained on one dataset fail to generalize to others.

Existing transfer learning methods (fine-tuning or domain adaptation) often struggle because they lack a foundation model capable of capturing broadly transferable knowledge from such diverse and scarce data sources.

2. Methodology: The Pretrained Battery Transformer (PBT)

The authors propose PBT, the first foundation model for battery life prediction that operates directly on raw voltage-current signals. The core innovation is a novel architecture called BatteryMoE (Battery-knowledge-encoded Mixture-of-Experts).

A. BatteryMoE Architecture

BatteryMoE integrates domain-specific battery knowledge into the learning process to guide the model under data-scarce conditions. It consists of two main components:

1. Soft Encoder (Knowledge-Guided Pattern Decomposition):
   • Instead of treating aging factors (cathode, anode, temperature, protocols) as rigid categorical inputs, the model uses a Large Language Model (LLM) to convert textual descriptions of aging conditions into vector embeddings.
   • These embeddings capture the semantic relationships between different aging factors and guide the "gate network" to route inputs to appropriate expert networks.
2. Hard Encoder (Physically Constrained Routing):
   • This component imposes hard-coded physical constraints based on established battery knowledge.
   • It filters expert networks based on specific categories (e.g., only activating experts for "LFP" cathodes if the input is an LFP battery) or continuous ranges (e.g., selecting temperature experts within ±5°C of the input).
   • This ensures the model only considers physically relevant degradation mechanisms, reducing the search space and improving learning efficiency.

B. Model Pipeline

1. CyclePatch: Raw voltage-current-capacity time series for each cycle are tokenized into "cycle tokens."
2. Intra-cycle Encoder: Processes individual cycle tokens to extract high-level representations.
3. Inter-cycle Encoder: Uses a Transformer architecture with BatteryMoE layers replacing standard feedforward layers to model relationships across cycles.
4. Prediction: The final output projects the learned representations to predict the cycle life (number of cycles until capacity drops to 80% or 90%).

3. Key Contributions

• First Battery Foundation Model: PBT is the first model designed specifically as a foundation model for battery life prediction, capable of universal transfer across diverse chemistries and conditions.
• BatteryMoE Module: A novel Mixture-of-Experts layer that uniquely combines soft knowledge injection (via LLM embeddings) and hard physical constraints to handle heterogeneity and scarcity.
• Comprehensive Dataset: The study utilizes the largest and most diverse battery life database to date, comprising 16 public datasets, 977 batteries, and 533 distinct aging conditions, covering Li-ion, Na-ion, and Zn-ion chemistries.

4. Results

The model was evaluated across 15 datasets (including 13 for pretraining and 3 for out-of-distribution testing) and compared against state-of-the-art baselines (CPMLP, CPTransformer, and BatLiNet).

• Pretraining Performance: On 13 Li-ion datasets, PBT achieved a Mean Absolute Percentage Error (MAPE) of 0.143, outperforming the second-best model (CPMLP) by 19.8%. It showed superior performance in both "seen" (in-distribution) and "unseen" (out-of-distribution) aging conditions.
• Transfer Learning: When adapted to 12 downstream tasks via fine-tuning or adapter tuning, PBT-TL achieved the lowest MAPE on all 12 datasets, surpassing the best baseline by an average of 21.8% (with gains up to 86.9% on highly challenging, data-scarce datasets like MICH_EXP).
• Zero-Shot/Out-of-Distribution Generalization:
  • PBT successfully transferred to Sodium-ion and Zinc-ion batteries (chemistries not seen during pretraining), outperforming baselines by 11.5% and 40.1% respectively.
  • It also generalized to Industrial large-format Li-ion cells, demonstrating robustness beyond laboratory-scale data.
• Data Efficiency: PBT trained on data from just the first cycle outperformed baselines trained on 100 cycles, significantly reducing the time required for early-life prediction.

5. Significance

• Scalability: PBT provides a scalable route toward universal battery lifetime prediction systems, moving away from dataset-specific models.
• Domain Knowledge Integration: The study demonstrates that for scientific domains with scarce and heterogeneous data, embedding physical and chemical knowledge into the model architecture (via BatteryMoE) is more effective than simply scaling up data or using generic neural networks.
• Broader Impact: The "knowledge-guided expert selection" paradigm (Soft + Hard encoders) offers a template for other scientific prediction problems involving scarce data, such as fatigue failure, corrosion, and solar cell lifetime prediction.
• Practical Application: The model enables rapid battery screening, optimization of fast-charging protocols, and improved manufacturing quality control by predicting lifetime from minimal early-cycle data.