💬 NLP

RHYTHM: Reasoning with Hierarchical Temporal Tokenization for Human Mobility

RHYTHM is a unified framework that leverages a frozen large language model with hierarchical temporal tokenization to efficiently predict human mobility by capturing multi-scale periodic dependencies, achieving significant improvements in accuracy and training speed compared to state-of-the-art methods.

Original authors: Haoyu He, Haozheng Luo, Yan Chen, Qi R. Wang

Published 2026-02-25

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Haoyu He, Haozheng Luo, Yan Chen, Qi R. Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine trying to predict where a person will be tomorrow. It's like trying to guess the next word in a sentence, but instead of words, you're guessing locations.

For a long time, computers have tried to do this by looking at a person's entire history of movements as one giant, messy list of data points. But human movement isn't random; it has a rhythm. We have daily routines (wake up, go to work, go home) and weekly rhythms (workdays vs. weekends).

The paper introduces a new system called RHYTHM (Reasoning with Hierarchical Temporal Tokenization for Human Mobility). Think of it as a "Super-Predictor" that finally understands the beat of human life.

Here is how it works, broken down into simple concepts:

1. The Problem: Reading a Novel One Letter at a Time

Imagine you are trying to understand a story, but instead of reading whole sentences or paragraphs, the computer is forced to read every single letter one by one.

Old Methods: They look at every single location check-in (every 30 minutes) as a separate item. If you have a week of data, that's hundreds of items. It's slow, confusing, and the computer gets tired (it loses track of the "big picture" patterns like "Oh, this person goes to the gym every Tuesday").
The Result: The computer gets overwhelmed by the sheer volume of data and misses the obvious patterns.

2. The Solution: RHYTHM's "Chapter" Approach

RHYTHM changes the game by grouping the data. Instead of reading letter-by-letter, it reads chapter-by-chapter.

The Analogy: Imagine your week of movement is a book.
- Old Way: The computer reads every single word (every 30-minute check-in).
- RHYTHM Way: RHYTHM groups the words into Daily Chapters. It says, "Okay, Monday is one chapter, Tuesday is another."
- Why this helps: It turns a 1,000-page book into a 7-chapter novella. The computer can now see the story of the week much faster. It captures the "Daily Routine" (the chapter) and the "Weekly Pattern" (the whole book) simultaneously.

3. The "Brain": Using a Giant Language Model (LLM)

RHYTHM uses a Large Language Model (LLM)—the same kind of AI that writes poems or answers questions—as its brain.

The Magic: LLMs are experts at understanding context. They know that "I'm hungry" usually leads to "I go to a restaurant."
RHYTHM's Trick: It teaches the LLM to treat locations like words.
- Instead of saying "The user is at coordinates X,Y," it says, "The user is at 'Home'."
- It then asks the LLM: "Based on the story of this week, where is the user likely to go next?"
- Because the LLM is so good at spotting patterns in language, it becomes surprisingly good at spotting patterns in movement.

4. The "Cheat Sheet": Pre-computed Prompts

One of the smartest parts of RHYTHM is how it saves time.

The Problem: Asking a giant AI to read a whole book every time you want a prediction is slow and expensive.
The Fix: RHYTHM prepares a "Cheat Sheet" (called Semantic Embeddings) before the training even starts.
- It takes the description of a day (e.g., "A rainy Tuesday where the user went to work and then the gym") and turns it into a summary note.
- When the model needs to make a prediction, it just reads the summary note. It doesn't have to re-read the whole book. This makes it incredibly fast and cheap to run.

5. The Results: Faster, Smarter, and Better

The paper tested RHYTHM on real data from three Japanese cities (Kumamoto, Sapporo, Hiroshima).

Accuracy: It predicted locations better than any previous method, especially on weekends. Why? Because weekends are chaotic and less routine. Old models fail here because they just memorize patterns. RHYTHM reasons through the chaos, just like a human would.
Speed: Because it groups data into "chapters" and uses the "Cheat Sheet," it trains 24% faster and uses less memory than its competitors.

Summary Analogy

If predicting human movement was a race:

Old Models were like a runner trying to count every single grain of sand on the beach to find the finish line. They were slow and got lost in the details.
RHYTHM is like a runner who looks at the map, sees the "Daily Routes" and "Weekly Habits," and uses a smart guide (the LLM) to intuitively know where the finish line is.

In short: RHYTHM turns the messy, chaotic data of human movement into a structured story that a super-smart AI can read, understand, and predict with high accuracy and low cost.

1. Problem Definition

Human mobility prediction is a critical task for urban planning, epidemic control, and transportation optimization. However, it faces significant challenges:

Complex Dependencies: Human trajectories exhibit long-range dependencies and multi-scale periodic behaviors (e.g., daily commutes vs. weekly routines).
Non-Stationarity: Movements are influenced by dynamic factors like weather and special events, creating non-stationary spatio-temporal patterns.
Limitations of Existing Models:
- Markov/RNN-based methods: Often fail to capture long-term periodicity or suffer from vanishing gradients over long sequences.
- Transformer-based methods: While good at long-range dependencies, they often treat time as static, failing to disentangle hierarchical temporal patterns (daily vs. weekly). They also suffer from quadratic computational complexity ( $O(T^2)$ ) when processing long trajectories.
- LLM-based methods: Recent attempts often treat mobility as generic sequences, ignoring inherent periodicity, or require extensive fine-tuning, leading to high computational costs.

The goal is to develop a framework that captures multi-scale temporal dependencies, leverages the reasoning capabilities of Large Language Models (LLMs), and remains computationally efficient.

2. Methodology: RHYTHM Framework

RHYTHM (Reasoning with Hierarchical Temporal Tokenization for Human Mobility) is a unified framework that rethinks mobility modeling by treating human movement patterns as compositional structures similar to language (daily routines as "tokens," weekly rhythms as "syntax").

A. Spatio-Temporal Feature Encoding

The model first encodes raw trajectory data ( $x_i = (t_i, l_i)$ ) into embeddings:

Temporal Embeddings: Captures cyclical patterns via Time-of-Day (ToD) and Day-of-Week (DoW) embeddings.
Spatial Embeddings: Combines categorical location IDs with projected geographic coordinates (latitude/longitude).
Combined Embedding: Element-wise addition of temporal and spatial features.

B. Hierarchical Temporal Tokenization

This is the core innovation designed to reduce sequence length and capture multi-scale patterns:

Segmentation: The trajectory sequence is partitioned into $N$ non-overlapping segments (e.g., daily segments of 48 time slots).
Intra-Segment Attention: Within each segment, an attention mechanism models local temporal dependencies (e.g., morning commute flow).
Pooling: Each segment is condensed into a single discrete "segment token" via a learnable pooling operation.
Inter-Segment Attention: These segment tokens undergo a second attention layer to model long-range dependencies across days (e.g., weekly routines).

Benefit: This reduces the sequence length from $T$ (total time steps) to $N$ (number of segments), lowering attention complexity from $O(T^2)$ to $O(N^2)$ (quadratic reduction).

C. Semantic Context Integration (Prompt-Guided)

To enrich the tokens with semantic meaning without extending the context length:

Pre-computed Embeddings: A frozen LLM is used offline to generate semantic embeddings for trajectory segments (describing movement patterns) and task descriptions (predicting future locations).
Alignment: These semantic embeddings are added element-wise to the temporal segment tokens. This acts as a positional/semantic alignment mechanism, allowing the model to interpret complex mobility patterns without the overhead of concatenating long text prompts during training.

D. Frozen LLM Backbone & Reasoning

Architecture: The enriched combined embeddings are fed into a frozen pre-trained LLM backbone.
Reasoning: The LLM performs in-context reasoning over the aligned temporal and semantic information to capture deep interdependencies.
Output: A projection layer maps the final hidden states to logits for candidate locations.
Efficiency: By freezing the LLM, the model avoids extensive fine-tuning, significantly reducing trainable parameters and memory usage.

3. Key Contributions

Hierarchical Temporal Tokenization: A novel mechanism that encodes daily mobility patterns as discrete tokens, reducing sequence length while capturing cyclical and multi-scale dependencies through a two-level attention mechanism.
Prompt-Guided Semantic Integration: An efficient approach that integrates semantic trajectory information and task descriptions via pre-computed embeddings, enhancing the model's ability to interpret complex patterns without runtime LLM inference.
Parameter-Efficient Adaptation: A strategy using frozen LLMs that reduces trainable parameters to 12.37% of the full model size, achieving a 24.6% reduction in training time compared to baselines.
State-of-the-Art Performance: Empirical validation showing superior accuracy and efficiency across multiple datasets.

4. Experimental Results

The model was evaluated on three real-world datasets (Kumamoto, Sapporo, Hiroshima) from the YJMob100K dataset, comparing against LSTM, Transformer (PatchTST, iTransformer), and LLM-based baselines (Time-LLM, Mobility-LLM).

Accuracy: RHYTHM achieved a 2.4% improvement in overall Accuracy@1 compared to the best baseline. Notably, it showed a 5.0% increase in accuracy on weekends, where human movement is less deterministic and requires more complex reasoning.
Efficiency:
- Training Time: 24.6% faster than the best-performing baseline.
- Memory: Significantly lower memory footprint due to the frozen backbone and reduced sequence length.
- Scalability: Performance scales with LLM size (e.g., Llama-3B outperforms Llama-1B), but the 1B/2B variants offer the best balance of cost and performance.
Geographical Metrics: RHYTHM demonstrated superior spatial alignment (lowest DTW error) and strong ranking capabilities (highest MRR).
Ablation Studies: Removing temporal tokenization caused a 5.39% performance drop, confirming it as the most critical component. Removing hierarchical attention caused a 0.90% drop.

5. Significance and Impact

Bridging Modalities: RHYTHM successfully bridges the gap between spatio-temporal data and LLM reasoning capabilities, treating mobility patterns as structured "language" rather than raw sequences.
Resource Efficiency: By freezing the LLM and utilizing hierarchical tokenization, RHYTHM makes high-performance mobility prediction feasible in resource-constrained environments, addressing a major bottleneck in deploying foundation models for time-series tasks.
Handling Irregularity: The model excels in non-routine scenarios (weekends, evenings) where traditional pattern-matching models fail, suggesting that LLM-based reasoning is crucial for capturing the nuance of human decision-making in mobility.
Generalizability: The framework is "plug-and-play," capable of integrating various pre-trained LLMs (OPT, Llama, Gemma, etc.) without retraining the backbone, offering a flexible foundation for future mobility research.

In summary, RHYTHM represents a paradigm shift in human mobility prediction, moving from purely statistical or deep learning sequence models to a reasoning-based foundation model that is both highly accurate and computationally efficient.