Measuring the Vertical Structure of Active Galactic Nuclei Disks with Transformer Models and the Vera C. Rubin Observatory

Technical Summary: Measuring the Vertical Structure of Active Galactic Nuclei Disks with Transformer Models and the Vera C. Rubin Observatory

Problem Statement
Active Galactic Nuclei (AGN) accretion disks are typically studied via continuum reverberation mapping, which measures "short lags" (light crossing timescales) to determine radial structure. However, detecting "long negative lags"—caused by the inward propagation of fluctuations on the viscous or inflow timescale—offers a unique window into the vertical structure (scale height) of AGN disks. Current observational campaigns face significant hurdles in detecting these long lags:

1. Seasonal Gaps: The Vera C. Rubin Observatory's light curves will contain roughly 6-month seasonal gaps, leading to aliasing and false signal detections on timescales comparable to long lags.
2. Signal Strength: The long lag signal is often significantly weaker than the short lag signal, confounding traditional detection methods.
3. Data Volume: The sheer influx of data from millions of AGN requires efficient processing techniques.
   Traditional methods, such as the Interpolated Cross-Correlation Function (ICCF) and javelin (a Monte Carlo Markov Chain approach), have been shown to struggle with these specific challenges, particularly in the presence of seasonal gaps and weak long-lag signals.

Methodology
The authors develop and train a transformer-based machine learning model to predict the posterior distribution of both short and long lags in mock Rubin AGN light curves.

• Data Simulation: The team generated $2 \times 10^6$ mock light curves using EzTao to model AGN variability as a Damped Random Walk (DRW). They introduced short lags ($\tau_{short} \in [0.05, 10]$ days) and long negative lags ($\tau_{long} \in [-3650, -10]$ days) via Gaussian response functions. Approximately 50% of the curves were assigned $\tau_{long} = 0$ to ensure the model learns to distinguish between the presence and absence of long lags. The amplitude ratio ($r$) of the long lag to the short lag was drawn from a log-uniform distribution ($10^{-3}$ to $1$). These curves were then "observed" using the Rubin Operations Simulator (OpSim) for deep drilling fields, incorporating realistic cadences, noise, and seasonal gaps.
• Model Architecture: The transformer model processes time and brightness data (normalized to 0–1, aggregated to daily intervals, with unobserved steps marked as -2). The architecture includes:
  • An embedding layer with time-based positional encoding.
  • 8 transformer blocks (256 dimensions, 16 heads).
  • Global average pooling and a dense layer.
  • The output parameterizes a mixture of two Gaussians: one for the case with no long lag ($\phi=1$) and one for the case with a long lag ($\phi=0$).
• Training: The model was trained using the Adam optimizer with cosine decay on a single NVIDIA A100 GPU for 13 hours. Hyperparameters were optimized via Bayesian optimization (75 trials) using the Tree-structured Parzen Estimator algorithm. The loss function minimizes the KL divergence to learn the probability of a long lag ($p$) and the conditional distribution of lag durations.
• Baselines: Performance was compared against the ICCF and javelin, applied to the same mock datasets.

Key Results
The transformer model demonstrates superior performance in detecting and characterizing long negative lags compared to traditional methods:

• Detection Accuracy: On a test set of $2 \times 10^4$ mock light curves, the model identifies the presence of a long negative lag with 96% recall (completeness) and only 0.04% contamination (false positives) when using a probability threshold of $(1-p_\phi) > 0.906$.
• Lag Prediction: For light curves identified as having a long lag, the model predicts the duration with 98% accuracy within 20% of the true value and 92% within 10%.
• Robustness to Weak Signals: The model successfully detects long lags even when the amplitude ratio ($r$) is low ($r < 0.01$), missing only 13% of such cases. In contrast, the ICCF fails to detect 87% of long lags in this low-ratio regime.
• Comparison to Baselines:
  • ICCF: Accurate to within 20% of the true lag for only 54% of cases. It struggles significantly when the short lag signal dominates.
  • javelin: Accurate to within 20% for only 21% of cases. It suffers severe aliasing, frequently predicting lags at odd integer multiples of the seasonal gap (180 days). It also produces a higher false-positive rate (4.0%) compared to the transformer.
• Efficiency: The trained transformer processes 1,000 light curves in 0.9 seconds on a single GPU. In comparison, ICCF requires 12 hours and javelin requires 5 days on a single CPU for the same volume.

Significance and Claims
The paper claims that the transformer model represents a "revolutionary" step in preparing for the Vera C. Rubin Observatory's data deluge. By implicitly learning to marginalize over missing data (seasonal gaps) and confounding signals (short lags), the model overcomes the specific limitations of traditional reverberation mapping techniques.

The authors emphasize that this capability is crucial for:

1. Measuring Vertical Structure: Long negative lags provide a direct probe of the disk aspect ratio ($h/r$), allowing for tests of the standard thin disk model versus magnetically elevated disk models.
2. Scalability: The model's speed makes it feasible to analyze the millions of AGN light curves expected from Rubin, a task impossible with current MCMC-based methods.

The authors maintain a modest tone regarding real-world application, noting that the model was trained and tested exclusively on mock data generated with specific assumptions. They acknowledge that real Rubin light curves may present unforeseen challenges and that the model does not currently account for potential correlations between physical parameters (e.g., black hole mass and lag duration) due to a lack of empirical constraints. Future work is proposed to validate the model on Zwicky Transient Facility (ZTF) data and to refine the model as more real long-lag detections become available.