Technical Summary: LMT – A Bayesian Framework for Causal Discovery from Textual Alarm Records

1. Problem Statement

Manufacturing and engineering systems increasingly generate event data in the form of textual alarm logs, maintenance records, and process abnormalities. While engineers can identify statistical correlations or recurring patterns in these logs, understanding the causal mechanisms—specifically, which types of events trigger or influence others—remains a critical challenge.

Existing approaches face two primary limitations:

1. Text-Only Approaches: Large Language Models (LLMs) can extract semantic clues about causality from event descriptions. However, LLMs are trained on statistical associations within corpora, not physical or temporal mechanisms. Consequently, they often confuse causation with correlation or frequent sequential patterns (e.g., events described in a fixed narrative order) and lack the ability to distinguish cause from effect based solely on text.
2. Time-Only Approaches: Traditional causal discovery methods utilizing temporal data (e.g., Granger causality, Hawkes processes) rely on timestamps but ignore the rich semantic context contained in event descriptions. This can lead to ambiguity when different event types have similar temporal patterns but distinct physical meanings.

The paper addresses the need for a framework that jointly leverages textual descriptions and timestamps to infer causal graphs in engineering systems, particularly in scenarios where event records are limited (small-sample settings) because systems are often stopped immediately after an alarm triggers.

2. Methodology: The LMT Framework

The authors propose LMT, a Bayesian causal discovery framework that integrates LLM-derived semantic priors with a Poisson-process-based temporal likelihood. The framework operates in three main stages:

A. Data Preprocessing and Clustering

Raw event data consists of timestamps ($t_i$) and textual descriptions ($s_i$).

• Embedding: Text descriptions are embedded using a Sentence-Transformer model.
• Clustering: Events are grouped into $K$ semantic clusters based on embedding similarity. Each event $i$ is represented as $(t_i, k_i, x_i)$, where $k_i$ is the cluster label and $x_i$ is the embedding vector. The goal is to infer a Directed Acyclic Graph (DAG) among these $K$ clusters.

B. LLM-Informed DAG Prior

To utilize textual information without relying on it exclusively, LMT constructs a prior distribution over causal graphs:

1. Role Scoring: An LLM is prompted to evaluate each event description independently (without temporal context) to assign two continuous scores: a "cause-like" score ($c^{LLM}$) and an "effect-like" score ($r^{LLM}$).
2. Role Distillation: A lightweight neural network learns to project text embeddings into cause and effect representation spaces ($h^{(c)}$ and $h^{(e)}$) by minimizing a loss function against the LLM's scores.
3. Semantic Matching: A directional compatibility score $a_{ij}$ is computed between event $i$ (as cause) and event $j$ (as effect) using the dot product of their role-specific embeddings.
4. Cluster Aggregation: Event-level scores are aggregated to form a cluster-level prior belief $q^{LLM}_{ab}$, representing the probability that cluster $a$ causes cluster $b$.
5. Prior Definition: A continuous prior density $p(G | E^{LLM})$ is defined over the soft adjacency matrix $G$, encouraging edges consistent with the LLM's semantic signals while enforcing a differentiable acyclicity constraint (using the trace of the matrix exponential) to ensure the result is a DAG.

C. Temporal Likelihood and Posterior Inference

To refine the semantic prior, the framework incorporates temporal evidence using a multivariate Hawkes process:

1. Intensity Modeling: The conditional intensity function $\lambda_b(t)$ for cluster $b$ is modeled as a background intensity plus a triggering term from past events. The triggering strength is governed by the soft edge weights $G_{ab}$ in the causal graph.
2. Likelihood Construction: A log-likelihood function $L_{time}(G, \Theta)$ is derived based on the observed event sequence, where $\Theta$ includes background intensities and decay rates.
3. Bayesian Integration: The final objective is to find the Maximum A Posteriori (MAP) estimate of the graph $G$ and parameters $\Theta$:
   $$(G^*, \Theta^*) = \arg \max_{G, \Theta} \left[ L_{time}(G, \Theta) + \log p(G | E^{LLM}) \right]$$
   This optimization is performed using gradient descent on unconstrained variables (logits for edge weights and softplus for positive parameters), allowing the temporal data to correct or reinforce the LLM's initial semantic biases.

3. Key Contributions

The paper outlines three primary contributions:

1. Hybrid Bayesian Framework: A novel framework that assigns distinct probabilistic roles to textual and temporal data. Text provides an informative but imperfect prior, while temporal data provides the likelihood for refinement. This design improves the reliability of LLM-derived information in manufacturing contexts.
2. Interpretable Causal Graphs: The method infers interpretable DAGs among event clusters, facilitating root-cause analysis and decision-making in complex engineering systems.
3. Small-Sample Suitability: The framework is specifically designed for scenarios with limited event records (common in alarm-driven systems where operations halt after a fault), demonstrating robustness where data-driven-only methods typically fail.

4. Experimental Results

The authors evaluated LMT on simulated chemical manufacturing datasets and a real-world semiconductor test handler dataset.

Simulation Studies

• Baselines: LMT was compared against:
  • CASCADE: A temporal-only method (Poisson-based).
  • VOC-Emb: A text-only method (embedding-based).
  • LLM Prompting: A direct LLM inference approach.
• Metrics: Structural Hamming Distance (SHD), True Positive Rate (TPR), and False Discovery Rate (FDR).
• Findings:
  • LMT consistently outperformed all baselines across varying numbers of clusters ($p=5, 8, 16$) and sample sizes ($N=500, 1000$).
  • In small-sample settings ($N=50$), LMT significantly reduced SHD and FDR compared to a uniform-prior variant, demonstrating that the text-informed prior effectively filters spurious edges when temporal evidence is scarce.
  • Pure LLM prompting and text-only methods showed high FDR and low TPR, confirming that semantic patterns alone are insufficient for accurate causal discovery.
  • Temporal-only methods performed better than text-only but still lagged behind LMT, highlighting the value of semantic priors.

Real-World Application

• Dataset: Alarm records from a pick-and-place integrated circuit test handler (97 events over one week).
• Clustering: Events were grouped into 5 functional module clusters (e.g., Test-site, Transfer PnP, Input-side).
• Results: The learned DAG identified that Test-site/contactor alarms act as upstream triggers for Transfer, Input, and Output modules. This aligns with engineering intuition: a "stuck device" at the test site causes system stops and subsequent recovery actions that manifest as alarms in other modules. The graph provided actionable insights for prioritizing maintenance on the test-site module to prevent downstream cascades.

5. Significance and Claims

The paper claims that LMT offers a practical solution for causal discovery in data-scarce engineering environments. Its significance lies in:

• Bridging Semantics and Time: It demonstrates that LLMs can serve as "imperfect experts" to provide initial causal hypotheses, which are then rigorously validated and refined by physical temporal data.
• Addressing Small-Data Regimes: By leveraging semantic priors, the framework achieves high accuracy even when only a short stream of alarm events is available, a common constraint in real-world industrial operations where systems are stopped for diagnosis.
• Interpretability: The resulting causal graphs are not just statistical artifacts but are grounded in both the semantic meaning of the alarms and the temporal sequence of their occurrence, making them suitable for engineering decision-making and root-cause analysis.

The authors conclude that this approach represents a promising direction for integrating language-model-derived semantic knowledge into the causal analysis of advanced manufacturing systems, enabling better predictive maintenance and system management.