Technical Summary: Toward Semantically-Seeded, Graph-Propagated Impact Analysis Across Software Artifacts

Problem Statement

Software engineers face significant challenges in Change Impact Analysis (CIA) when a single artifact (e.g., a requirement, configuration, or function) is modified. Existing tooling typically relies on one of two isolated signals, each possessing a characteristic "blind spot":

1. Semantic/Information Retrieval (IR) Approaches: These rely on text similarity (e.g., embeddings, vector space models) to link artifacts. While they cross artifact types freely, they fail to detect impacts where the text shares no vocabulary with the change (e.g., a requirement change to a "timeout" that impacts a function named completeOrder without mentioning timeouts).
2. Structural/Dependency Approaches: These follow explicit edges (call graphs, data flow, test invocation). While precise where edges exist, they miss meaning-level relationships not encoded as edges and often fail to cover the full chain from Requirements → Config → Service → Test, particularly for operational artifacts like container images, metrics, and database schemas.

The paper argues that purely structural analysis misses impacts on operational artifacts (e.g., base-image updates breaking services, metric renames disabling alerts) because no code-level edge connects them to the change. Conversely, purely semantic analysis misses impacts that are structurally dependent but lexically distinct.

Methodology

The authors propose a training-free, interpretable analyzer that fuses semantic and structural signals over a heterogeneous artifact graph. The core innovation is using the "same embeddings, different reasoning layer."

1. Heterogeneous Artifact Graph

The system models the software ecosystem as a typed graph with node types including Requirements (R), Configs (C), Services/Functions (S), Tests (T), and operational artifacts (Images, Metrics, Alerts, Data Schemas).

• Edges: Recovered automatically via static analysis (e.g., Python AST) or manifest parsing.
  • Source edges: R→C, C→S, S→S, T→S.
  • Operational edges: Image→Service, Service→Metric, Metric→Alert, Service→Data.
• Propagation Direction: Edges are reversed for impact propagation (e.g., a change in a callee impacts its callers; a service change impacts its tests).

2. Two-Layer Scoring

The impact score for any artifact is a weighted blend of two layers:

• Layer 1: Semantic Prior ($s$)

  • Computes cosine similarity between the text of every artifact and the changed artifact.
  • Supports interchangeable backends: TF-IDF or dense embeddings (e.g., Sentence-BERT).
  • Captures meaning-level relationships even without structural edges.

• Layer 2: Structural Propagation ($p$)

  • Computes impact spread over the typed graph using a row-normalized propagation matrix $M$.
  • Uses a multi-hop decay mechanism (inspired by Katz centrality and PageRank):
    $$p = \sum_{k=1}^{K} \gamma^k (e_c M^k)$$
    Where $e_c$ is the changed artifact, $\gamma$ is the decay factor, and $K$ is the horizon.
  • Captures explicit dependency chains.

3. Fusion

The final impact score is a linear combination controlled by a single tunable weight $\lambda$:
$$\text{impact} = \lambda s + (1 - \lambda) p$$

• $\lambda=1$: Pure semantic baseline.
• $\lambda=0$: Pure structural analyzer.
• $\lambda \in (0,1)$: A blend that explicitly controls the precision/recall trade-off.
• Interpretability: Every flagged artifact includes an explicit explanation: its semantic score, structural score, and the specific propagation path taken.

Proof-of-Concept Evaluation

The authors evaluated the approach on a small, complete payment subsystem consisting of 13 artifacts (3 requirements, 3 configs, 4 services, 3 tests) and 14 typed edges, across 5 labeled change scenarios.

Key Findings:

1. Blind Spot Coverage: The fusion is the only configuration that covers both blind spots.
   • Semantic Blind Spot: Artifacts with zero textual overlap (e.g., completeOrder vs. a "timeout" requirement) were recovered purely through structural propagation.
   • Structural Blind Spot: Helper functions (e.g., loadConfig) that are not downstream in the call graph but are semantically related were recovered purely through the semantic layer.
2. Performance:
   • Pure Structural ($\lambda=0$): Achieved high precision (0.927) but missed ~21% of true impacts (Recall 0.786) due to missing helper functions.
   • Pure Semantic ($\lambda=1$): Achieved high recall (0.873) but low precision (0.417) due to false positives.
   • Proposed Blend ($\lambda=0.5$): Achieved perfect recall (1.000) across all scenarios, capturing all ground-truth impacted artifacts, with a slightly improved F1 score (0.589) over the semantic baseline.
3. Limitations of the Benchmark: The authors note that the high F1 of the structural-only approach is an artifact of the tiny synthetic graph where most true impacts were reachable by edges. The primary contribution is mechanism-level coverage, not a generalization claim of accuracy.

Significance and Claims

The paper positions this work as a specific, under-occupied point in the design space of impact analysis:

• Training-Free & Deterministic: Unlike Graph Neural Networks (GNNs) or GraphRAG approaches that require training data or expensive, non-deterministic LLM calls, this approach uses fixed propagation rules and explicit embeddings.
• Explainable: It emits a verifiable propagation path for every flagged artifact, making it suitable for audit and safety-critical settings.
• Operational Scope: It extends impact analysis beyond source code to operational artifacts (images, metrics, schemas) where code-only analysis fails, addressing real-world failure modes like base-image updates and metric renames.

The authors explicitly state that the prototype numbers are illustrative of the mechanism, not a claim of generalization. The contribution is the design of a fusion architecture that recovers artifacts invisible to either signal alone, providing a foundation for future work on learned propagation, LLM verification, and scalable cross-service deployment.