💻 bioinformatics

A Benchmark of Unsupervised Circadian Phase Inference Methods for Bulk Transcriptomic Data

This study systematically benchmarks ten unsupervised circadian phase inference methods across 25 datasets, identifying CHIRAL as the top performer with the lowest mean error, while demonstrating that method accuracy varies by dataset characteristics and that top methods effectively recover rhythmic gene patterns even when sample collection times are unknown.

Original authors: Purohit, S., Grant, G. R., Brooks, T. G.

Published 2026-10-15

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Purohit, S., Grant, G. R., Brooks, T. G.

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Inside nearly every cell of the human body, a silent, internal clock ticks away, orchestrating a daily cycle of activity that repeats roughly every twenty-four hours. This rhythm, known as the circadian cycle, dictates when genes turn on and off, influencing everything from metabolism to the timing of drug treatments. To understand how this clock works in different tissues or how it might go wrong in disease, scientists need to know exactly what time of day a sample of tissue was collected. However, in many existing medical records and public databases, this crucial piece of information is missing. Researchers have developed computer programs to guess the time of day based solely on the pattern of genes present in a sample, but until now, no one had systematically tested which of these guessing tools actually works best.

A team of researchers at the University of Pennsylvania set out to solve this problem by putting ten different computer programs through a rigorous test. They gathered twenty-five distinct sets of data, including real tissue samples from humans and mice where the collection time was already known, as well as carefully constructed computer simulations. The goal was to see how accurately each program could look at a list of genes and correctly identify the time of day the sample was taken. The researchers measured success by calculating the average difference in hours between the program's guess and the actual time. They also checked whether the programs could correctly reconstruct the daily rhythms of individual genes, a task that is essential for understanding how the body's clock functions.

The results revealed a clear hierarchy among the tools. One program, named CHIRAL, emerged as the most accurate, guessing the time with an average error of just 1.80 hours. This means that if a sample was taken at noon, CHIRAL would typically estimate the time to be somewhere between 10:15 AM and 1:45 PM. The next best performers, including programs called TEMPO, CYCLOPS, and CYCLOPS 2.0, had average errors ranging from roughly 2.5 to 2.8 hours. While these were still useful, they were less precise than CHIRAL. A simple baseline method, which relied only on the known behavior of twelve specific "clock genes" without using complex algorithms, performed nearly as well as the top computer programs, with an error of about 2.15 hours. This finding suggests that a deep knowledge of these core clock genes is a powerful tool in itself. The remaining six programs tested performed significantly worse, with errors ranging from nearly four to almost five hours, a level of inaccuracy that is barely better than random chance.

The study also uncovered that the success of these programs depends heavily on the quality of the data they are given. The programs worked best when the genes in the tissue showed strong, clear daily rhythms. In datasets where these rhythms were weak or noisy, such as in certain human brain tissues, even the best programs struggled, with errors rising to nearly three hours. The researchers found that the accuracy of the top tools remained consistent across different types of tissue and species, but they also discovered that some programs were sensitive to how the data was prepared. For instance, correcting for technical differences between batches of samples helped some programs improve their guesses, while it actually made others less accurate. Furthermore, the researchers tested how stable the results were by running the programs multiple times with different random starting points. They found that while some tools produced the same result every time, others varied significantly depending on the random seed used, meaning a single run of those tools might not be reliable.

To see if these findings held up in the real world, the team applied the programs to two large datasets from mice where the collection times were completely unknown. They compared the programs' guesses against established references for when genes typically peak in the liver and fat tissue. CHIRAL was the only program that consistently matched the known patterns for the liver tissue, while no program could reliably match the patterns for fat tissue, suggesting that the clock in fat might be harder to read or that the reference data was insufficient. The study concludes that while no single tool is perfect, CHIRAL is currently the most reliable choice for estimating time from bulk tissue samples. However, the researchers caution that the accuracy of any estimate depends on the strength of the biological rhythms in the specific tissue being studied, and that for the most critical applications, using multiple methods or simple clock-gene baselines may provide a necessary check against error.

Technical Summary: A Benchmark of Unsupervised Circadian Phase Inference Methods for Bulk Transcriptomic Data

Problem Statement
Circadian rhythms in gene expression are critical for understanding biological timing, drug metabolism, and disease mechanisms. However, many public transcriptomic datasets lack sample collection times, preventing direct analysis of temporal patterns. While unsupervised methods exist to infer circadian phase from gene expression matrices without collection times, they have not been systematically compared. Previous evaluations were often conducted by method developers on their own datasets with varying preprocessing and metrics, leading to potential bias. Furthermore, it remains unclear how accurately these methods reconstruct sample phases or downstream gene-level rhythms (acrophase, amplitude, and rhythmicity) across diverse biological contexts.

Methodology
The authors conducted a systematic benchmark of ten unsupervised phase inference methods and two clock-gene baseline approaches.

Datasets: The evaluation utilized 25 datasets with known reference times (16 real datasets from five studies across human and mouse tissues, and 9 simulated datasets) and two mouse datasets (Diversity Outbred liver and adipose) without collection times. The datasets were grouped into six "families" to account for inter-study dependencies.
Methods Evaluated:
- Unsupervised Methods: CHIRAL, TEMPO, CYCLOPS, CYCLOPS 2.0, COFE, DCPR, PENN, PROTECT, Cyclum, and reCAT. These employ various strategies including probabilistic modeling, neural networks/autoencoders, matrix factorization, and geometric ordering.
- Baselines: Two simple baselines assigning phases based on the principal component analysis (PCA) of either 12 canonical clock genes or just two genes (ARNTL and NR1D1).
Evaluation Metrics:
- Primary: Aligned Circular Mean Absolute Error (MAE), measuring the shortest circular distance between inferred and reference phases after optimizing global rotation and reflection.
- Secondary: Absolute Fisher–Lee circular correlation, fraction of samples within 1 hour of reference, gene-level reconstruction of acrophase and amplitude, and rhythmicity detection (ROC AUC).
- Robustness Checks: Permutation tests to establish chance-level performance, sensitivity analyses regarding random seeds, batch/covariate adjustments (ComBat-seq, regression), and simulation variations (sampling design, sample size, waveform shape).
Reproducibility: The authors attempted to replicate published results for all methods; they successfully reproduced results for eight methods but could not reproduce PENN and PROTECT with the released code.

Key Results

Performance Ranking: CHIRAL achieved the lowest mean aligned MAE (1.80 h), outperforming the two clock-gene baselines (2.15 h and 2.20 h). TEMPO, CYCLOPS, and CYCLOPS 2.0 followed with mean errors of 2.49–2.80 h. The remaining six methods (COFE, DCPR, PENN, PROTECT, Cyclum, reCAT) had mean errors of 3.91–4.86 h, which were only marginally better than the chance level (5.08–5.23 h).
Statistical Significance: While CHIRAL had the lowest mean error, the confidence intervals for the top methods overlapped substantially. Post-hoc comparisons confirmed CHIRAL was statistically superior to the lower-performing group (COFE, PENN, PROTECT) but could not be statistically separated from TEMPO, CYCLOPS, or CYCLOPS 2.0.
Gene-Level Reconstruction: The top four methods (CHIRAL, TEMPO, CYCLOPS, CYCLOPS 2.0) and the baselines successfully reconstructed gene acrophases with median errors of 0.18–0.64 h and identified rhythmic genes with median ROC AUCs of 0.64–0.70.
Factors Influencing Accuracy:
- Rhythm Strength: Accuracy was strongly correlated with the strength of clock-gene rhythms in the dataset (Spearman $\rho = -0.58$ ). Datasets with weak rhythms (e.g., human striatum) yielded higher errors for all methods.
- Batch/Covariate Adjustment: The impact of adjustment varied by method. ComBat-seq correction significantly improved CYCLOPS and CYCLOPS 2.0 but degraded TEMPO's performance. Notably, regressing out covariates (like post-mortem interval) that are correlated with time removed circadian signals, increasing errors for CHIRAL and baselines beyond chance levels.
- Simulation Sensitivity: In simulations, CYCLOPS and COFE performed exceptionally well (0.3–0.4 h error), but this did not translate to real data, suggesting simulated models may favor specific method assumptions.
- Random Seeds: Results for DCPR, TEMPO, PENN, PROTECT, and Cyclum showed high sensitivity to random seeds (SD of 0.27–0.49 h), whereas CHIRAL and CYCLOPS were highly stable.
Datasets Without Collection Times: In the Diversity Outbred mouse datasets, CHIRAL was the only method consistent with external reference acrophases for liver tissue. However, no method agreed with adipose references, and cross-tissue phase agreement was low even for the best method (correlation 0.47).

Significance and Claims
The paper provides the first independent, systematic comparison of unsupervised circadian phase inference methods using a unified protocol. The authors claim that:

CHIRAL is the current leading method for bulk RNA-seq data, offering the lowest mean error and consistent performance across datasets, though TEMPO, CYCLOPS, and CYCLOPS 2.0 remain reasonable alternatives.
Prior knowledge of clock genes is beneficial but not sufficient. While baselines using clock genes performed well, they were outperformed by CHIRAL. However, methods relying solely on circadian priors (like DCPR) did not necessarily outperform those without, indicating that model architecture and data handling are equally critical.
Context matters. The accuracy of these methods is heavily dependent on the dataset's rhythm strength and the specific preprocessing steps (e.g., batch correction) applied.
Caution in interpretation. The authors emphasize that the ranking reflects the performance of these tools under the specific configurations tested and may not generalize to all data types (e.g., single-cell or proteomic data for which some tools were originally designed). They also note that the inability to reproduce PENN and PROTECT limits conclusions regarding those specific tools.

The study concludes that while unsupervised phase inference is a viable tool for analyzing unannotated transcriptomic data, users must carefully consider the specific characteristics of their dataset and the stability of the chosen method.

More like this