Technical Summary: Feature Attribution-Based Explainability Analysis in Predictive Process Monitoring

1. Problem Statement

Predictive Process Monitoring (PPM) utilizes deep learning models, such as Recurrent Neural Networks (RNNs) and Transformers, to forecast the outcomes, remaining time, or future activity sequences of ongoing business processes. While these models achieve high predictive performance by capturing sequential dependencies in event logs, their "black-box" nature hinders trust and adoption in high-stakes operational environments.

Existing explainability methods, particularly feature attribution techniques like SHAP (Shapley Additive Explanations) and LIME, face a dilemma when applied to PPM:

• Event-level explanations: Assigning importance to individual events often results in fragmented insights that fail to reveal meaningful process behavior or the broader context of execution.
• Trace-level explanations: Aggregating the entire trace into a single unit introduces information loss and obscures which specific parts of the execution drive the prediction.
• Time-series limitations: While time-series literature offers segmentation methods to group adjacent observations, these generic approaches often fail to respect the specific control-flow constraints, activity semantics, and irregular timing inherent in business process event logs. They may split coherent process stages or merge unrelated behaviors.

Consequently, there is a need for an explainability approach that balances computational efficiency with the preservation of local control-flow context, allowing stakeholders to understand which process phases influence a prediction.

2. Methodology

The authors propose a local post-hoc explainability method that integrates a control-flow-aware segmentation algorithm with segment-level SHAP value computation.

2.1 Control-Flow-Aware Trace Segmentation

The core of the methodology is an algorithm that partitions a trace into coherent subsequences (segments) based on shifts in transition probabilities.

• Directly-Follows Relation (DFR): The algorithm first constructs a Directly-Follows Graph (DFG) from the event log to derive empirical transition probabilities between activities.
• Cost Function: For a given trace, the algorithm assigns a cost to potential segments. Transitions that are frequent in the log contribute low cost, while rare or unseen transitions incur a penalty ($\gamma$). The cost of a segment $s$ is defined as the sum of negative log-probabilities of its transitions:
  $$C(s) = \sum_{i=1}^{m-1} c(a_i, a_{i+1})$$
  where $c(a_i, a_{i+1}) = -\log T[a_i, a_{i+1}]$ if the transition exists, and $\gamma$ otherwise.
• Change Point Detection: The algorithm employs the PELT (Pruned Exact Linear Time) algorithm to detect change points that minimize a penalized objective function. This function balances the total segmentation cost against a penalty term ($\beta k$) for the number of segments, preventing over-segmentation. The hyperparameter $\beta$ is selected via grid search to minimize the Akaike Information Criterion (AIC).
• Result: The trace is partitioned into segments where each segment represents a phase of consistent local control-flow behavior.

2.2 Segment-Level SHAP Computation

Once traces are segmented, the method treats each segment as a single interpretable unit for SHAP value calculation, rather than individual events.

• Coalition Definition: A coalition vector $z \in \{0, 1\}^k$ is defined for $k$ segments. $z_l=1$ indicates the segment retains its original values, while $z_l=0$ indicates it is replaced by a baseline value (derived from the mode of categorical attributes and mean of numerical attributes in the training data).
• Perturbation: The model is evaluated on perturbed traces where specific segments are masked and replaced by the baseline.
• SHAP Values: KernelSHAP is applied to compute the Shapley values for each segment, satisfying the efficiency property where the sum of segment attributions equals the difference between the model's prediction and the baseline prediction.

3. Experimental Setup

The study evaluates the proposed method using:

• Datasets:
  • A synthetic dataset generated by the SimBank simulator, featuring known ground-truth change points (cancellation vs. acceptance outcomes) to validate segmentation accuracy.
  • Two real-world benchmarks: BPIC15 (Dutch municipality administrative processes) and BPIC17 (financial institution loan applications).
• Predictive Model: LSTM-based sequence classifiers trained on fixed-length prefixes (30 events) to predict binary outcomes (e.g., accepted vs. canceled).
• Baselines:
  • Per-event attribution: Standard event-level SHAP.
  • Distribution-based segmentation: A method grouping events based on time windows and session definitions (adapted from recommendation systems).
• Evaluation Metrics:
  • Segmentation Quality: RandIndex, Precision, Recall, and F1-score (synthetic); Entropic Relevance (ER) and pattern coverage (real-world).
  • Explanation Faithfulness: Relative Prediction Change (RPC) via perturbation. This measures how much the prediction changes when important segments (high SHAP) are perturbed versus unimportant segments.

4. Key Results

4.1 Segmentation Quality

• Synthetic Data: The proposed algorithm achieved high agreement with ground-truth change points, with RandIndex scores exceeding 0.9 and F1-scores of 89% (canceled cases) and 87.4% (accepted cases).
• Real-World Data (BPIC17): The method identified coherent process phases. For accepted cases, the top 10 segmentation patterns covered 81.4% of traces, indicating stable control-flow structures. The Entropic Relevance (ER) dropped significantly from 37.81 (full log) to 6.68 (segmented), indicating the segments capture highly deterministic, compressible patterns.
• Real-World Data (BPIC15): Due to high variability and low transition probabilities between many activity pairs, the algorithm produced fine-grained segments (average 2 events per segment). This suggests that logs with high control-flow variability limit the ability to form generalizable segment patterns.

4.2 Explanation Faithfulness

• Perturbation Analysis: The segment-level SHAP values demonstrated high faithfulness. Perturbing the top-k most important segments (covering ~40% of the trace) resulted in significant prediction changes (high RPCI), while perturbing unimportant segments had minimal effect (low RPCU).
• Comparison: While per-event attribution showed slightly higher granularity, segment-level methods (both distribution-based and control-flow-aware) provided cleaner separation between important and unimportant parts of the trace. Per-event methods sometimes disrupted local dependencies by perturbing isolated events, leading to unexpected prediction shifts.

4.3 Qualitative Insights

• Accepted Cases (BPIC17): Segments containing smooth transitions (e.g., Concept $\to$ Complete application) consistently pushed predictions toward acceptance (negative SHAP values).
• Canceled Cases (BPIC17): Segments involving repeated offer creation received high positive SHAP values, strongly pushing predictions toward cancellation. This identified a specific bottleneck (repeated offer generation) associated with failure.
• Interpretability: The method successfully linked model predictions to concrete process phases (e.g., "application completion" vs. "offer creation") rather than isolated events.

5. Significance and Claims

The paper claims that the proposed method offers a useful middle ground between fine-grained event-level and coarse trace-level explanations.

• Context Preservation: By grouping activities based on control-flow dynamics, the method preserves the local context necessary to interpret why an activity influences an outcome.
• Comprehensibility: It mitigates the "interleaving" of important and unimportant events, making explanations more comprehensible for process stakeholders.
• Dependency on Log Structure: The authors modestly note that the method's effectiveness is contingent on the event log's structure. Logs with moderate control-flow variability (like BPIC17) yield meaningful, comparable segments, whereas highly variable logs (like BPIC15) result in case-specific, fine-grained segmentation that limits generalization.
• Practical Utility: The approach enables the identification of specific process phases (change points) that steer cases toward predicted outcomes, supporting targeted decision-making and bottleneck detection without requiring full model transparency.

The study concludes that while event-level attributions remain slightly more faithful due to their granularity, segment-level explanations significantly enhance the interpretability of deep learning models in PPM by aligning with the structural constraints of business processes.