💻 computer science

ATGBuilder: Feature-Assisted Graph Learning for Activity Transition Graph Construction with Seed Supervision

ATGBuilder is a feature-assisted graph learning framework that leverages Large Language Models to summarize UI metadata and explicitly models widget-trigger information to significantly improve the accuracy of Activity Transition Graph construction for Android applications compared to existing state-of-the-art methods.

Original authors: Chenhui Cui, Zixiang Xian, Danyu Li, Tao Li, Rubing Huang, Dave Towey, Shikai Guo, Jiakun Liu

Published 2026-06-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Chenhui Cui, Zixiang Xian, Danyu Li, Tao Li, Rubing Huang, Dave Towey, Shikai Guo, Jiakun Liu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a massive, complex maze made of thousands of rooms. Each room is a screen in an Android app, and the doors between them are the ways you can move from one screen to another. In the world of software testing, we call this map an Activity Transition Graph (ATG). It's a blueprint that shows how a user can navigate through an app.

The problem is that drawing this map is incredibly hard.

Static analysis (looking at the app's code without running it) is like trying to draw the map by looking at the blueprints. It's fast, but it often misses doors that only open when you turn on the lights, or it draws doors that lead to brick walls.
Dynamic exploration (actually running the app and clicking around) is like sending a robot to walk through the maze. It finds real doors, but the robot gets tired, runs out of battery, and only maps a tiny fraction of the maze before giving up.

AtgBuilder is a new tool designed to fix this. It acts like a super-smart detective that combines the speed of reading blueprints with the intuition of a human explorer to draw a complete and accurate map.

Here is how it works, using simple analogies:

1. The "Summary" Trick (Solving the "Look-Alike" Problem)

Imagine two different coffee shops. One has a sleek, modern design with glass walls. The other is a cozy, rustic cabin with wooden beams. They look totally different, but they both serve the same thing: Coffee.

Old methods tried to map the app by looking at the "wood" or the "glass" (the specific visual layout). This failed because the same function (like "Login") could look completely different in every app.

AtgBuilder's Solution: It uses a Large Language Model (LLM)—think of it as a very smart librarian—to read the blueprint of a screen and write a one-sentence summary of what that screen does.

Instead of saying "This screen has a blue button and a white box," it says, "This is the Login Screen."
Now, even if two apps look different, the system knows they are both "Login Screens" and can predict how they connect to other parts of the app. It focuses on the function, not the furniture.

2. The "Who Pushed the Button?" Trick (Solving the "Which Door?" Problem)

In a room, you might have a red door, a blue door, and a green door. If you just say "Go from Room A to Room B," you don't know which door you used. But in an app, the specific button you press (the widget) matters.

AtgBuilder's Solution: Instead of just drawing a line between two rooms, AtgBuilder attaches a "tag" to the line that says exactly which button was pressed to open that door.

It treats the button information as a separate label on the path, not just part of the room's description.
To make sure it remembers these tags, it plays a little game during training: it tries to guess the button name based on the path it just drew. If it gets it right, it learns better. If it gets it wrong, it adjusts. This ensures the "who pushed the button" detail isn't lost.

3. The "Smart Guessing" Engine

Once AtgBuilder has the summaries and the button tags, it uses a Graph Neural Network (GNN). Think of this as a super-powered pattern matcher.

It looks at the "Login Screen" summary and the "Home Screen" summary.
It looks at the history of how other apps connect these two screens.
It looks at the specific button tags.
Then, it predicts: "There is a 90% chance a door exists between these two screens."

Does it actually work?

The researchers tested this on 98 different apps where they already had the "correct" map (ground truth) drawn by humans.

The Result: AtgBuilder was significantly better than the best existing tools. It improved the accuracy of the map by about 15% to 77% compared to previous methods.
The "Robustness" Test: They even tried to trick the system by feeding it bad data (like telling it a door exists when it doesn't). AtgBuilder was surprisingly tough; it didn't get confused easily and still drew a good map.

The "GPS" Bonus

Finally, the researchers asked: "If we give this map to automated robots that test apps, does it help them?"

They took three popular testing robots (Monkey, APE, and FastBot2) and gave them the AtgBuilder map as a "GPS guide."
The Result: The robots explored more rooms and found more doors than they did on their own. The map didn't replace the robots; it just gave them a better sense of direction so they didn't waste time walking in circles.

In a Nutshell

AtgBuilder is a tool that helps us understand how Android apps work by:

Ignoring the messy visual details and focusing on what the screen actually does (using AI summaries).
Remembering which specific button opens the door.
Using a smart pattern-matching engine to fill in the missing parts of the app's map.

This results in a much more complete and accurate map of an app, which helps developers and testers find bugs and understand the software much faster.

Technical Summary: AtgBuilder

Problem Statement

Android applications rely on Activity Transition Graphs (ATGs) to model GUI navigation, serving as a fundamental layer for automated testing and behavior analysis. However, constructing high-quality ATGs remains challenging due to the limitations of existing approaches:

Static Analysis: Methods like Frontmatter can extract transitions without execution but often suffer from false negatives (missing feasible transitions) and false positives (including infeasible transitions).
Dynamic Exploration: Tools like Monkey or APE generate trustworthy transitions but often yield incomplete graphs due to limited time budgets.
Learning-Based Approaches: Recent methods treat ATG construction as a seed-supervised link-prediction task. However, they typically underutilize activity-layout information and widget-trigger information. Specifically, they struggle to represent functional similarity across different UI implementations and often fail to distinguish between different widget-triggered navigations within the same activity pair.

Methodology: AtgBuilder

The authors propose AtgBuilder, a feature-assisted graph-learning approach for seed-supervised ATG construction. The framework addresses the identified challenges through three core design choices:

1. Functionality-Oriented Activity Representation

To overcome the variability of UI layouts for the same functionality (e.g., a subscription list implemented as a full-screen view in one app and a bottom-sheet in another), AtgBuilder avoids relying on raw layout structures.

LLM Summarization: It uses a Large Language Model (GPT-4o) to summarize per-activity UI metadata (layout trees, widget types, texts) into compact, one-sentence functional descriptions.
Embedding: These summaries are encoded using a sentence-embedding model (MiniLM), while activity identifiers are encoded using a word-level encoder (BERT). These are fused to create node features that emphasize semantic functionality over implementation details.

2. Explicit Widget-Trigger Modeling

To address the loss of trigger context when modeling transitions merely as activity pairs:

Edge Attributes: Widget-trigger information (e.g., specific buttons or listeners) is explicitly modeled as edge attributes rather than being mixed into activity node representations. This preserves the distinction between different transitions sharing the same source and destination activities.
Auxiliary Reconstruction Objective: During training, an auxiliary Multi-Layer Perceptron (MLP) attempts to reconstruct the widget attribute vector from the latent edge representation. This acts as a regularizer, forcing the model to retain widget-trigger information in its learned representations without making it the sole prediction signal.

3. Graph Learning Architecture

AtgBuilder frames ATG construction as a link-prediction problem on a seed graph derived from static analysis (Frontmatter).

Encoder: A two-layer Graph Isomorphism Network (GIN) computes latent activity representations by integrating node features and graph connectivity.
Decoder: A gravity-based decoder predicts transition existence. It calculates a score based on the "attractiveness" of the destination activity and the Euclidean distance between source and destination embeddings in the latent space.
Training Objectives: The model optimizes a weighted sum of two losses:
1. Link Prediction Loss: Binary cross-entropy for predicting transition existence.
2. Widget Reconstruction Loss: Mean Squared Error (MSE) for reconstructing widget attributes on positive edges.
Robustness Mechanisms: To handle noisy seed labels, the model employs seed-consistency regularization (ensuring consistent predictions under stochastic perturbations) and bootstrapping (mixing seed labels with model predictions).

Key Contributions

AtgBuilder Framework: A novel approach that augments analysis-derived seed ATGs with static UI metadata to predict missing transitions using graph learning.
LLM-Driven Activity Representation: A method to convert complex UI layouts into normalized functional summaries, improving cross-app generalization.
Widget-Aware Edge Modeling: The explicit separation of widget-trigger information as edge attributes, supported by an auxiliary reconstruction objective to preserve trigger context.
Comprehensive Evaluation: Extensive ablation studies, comparisons with seven state-of-the-art (SOTA) baselines on a 98-app ground-truth benchmark, and an evaluation of the model's utility in guiding automated GUI exploration tools.

Experimental Results

The authors evaluated AtgBuilder on the Frontmatter corpus (for ablation studies) and a 98-app benchmark with manually verified ground-truth ATGs.

Performance vs. SOTA: On the 98-app benchmark, AtgBuilder outperformed seven SOTA baselines (including Monkey, Stoat, APE, and FastBot2). Specifically, it improved the F1-score by 15.41% to 77.34% over the best baselines. The model achieved a recall of 93.63% and an F1-score of 81.01% when fine-tuned and using a validation-selected threshold.
Ablation Insights:
- Feature Fusion: The inclusion of LLM-generated activity summaries provided the most significant performance gain, confirming that semantic functionality is more robust than raw identifiers or layout structures.
- Widget Information: While widget features provided auxiliary improvements, the auxiliary reconstruction objective was crucial for maintaining performance stability.
- Thresholding: Using a validation-selected threshold ( $t^*$ ) rather than a fixed 0.5 significantly improved recall and F1-scores.
- Noise Robustness: AtgBuilder remained robust under synthetic label noise (up to 100% noise rate in some settings), outperforming seed-only baselines.
Practical Utility: When used as a lightweight navigation guide for Monkey, APE, and FastBot2, AtgBuilder significantly improved activity coverage (up to +25.0%) and transition coverage (up to +26.5%) without altering the underlying event-generation logic of the tools.

Significance and Claims

The paper claims that AtgBuilder offers a practical solution to the incompleteness of dynamic exploration and the infeasibility of static analysis by leveraging feature-assisted graph learning.

Generalization: By prioritizing functionality summaries over UI implementation details, the approach generalizes better across diverse app designs.
Complementarity: The results suggest that AtgBuilder does not replace existing automated exploration tools but complements them. By providing a structural guide (the predicted ATG), it directs exploration engines toward under-explored activities and transitions, thereby increasing coverage efficiency.
Robustness: The model's ability to handle noisy seed supervision and varying noise injection modes indicates its viability for real-world scenarios where ground-truth data is scarce or imperfect.

The authors conclude that while activity-level ATGs cannot capture fine-grained UI states (like fragments or WebView content), they provide a necessary and fundamental navigation layer. AtgBuilder demonstrates that combining LLM-derived semantic features with graph learning can effectively reconstruct high-quality activity transition graphs, enhancing the efficacy of automated GUI testing.