💻 computer science

LLM-Assisted Model-Based GUI Testing for Vue.js Web Applications

This paper presents LLMVue, a novel framework that leverages large language models to automatically extract and unify Vue.js source code into executable page transition graphs, thereby significantly enhancing the coverage and efficiency of model-based GUI testing compared to random exploration.

Original authors: Tao Li, Chenhui Cui, Rubing Huang, Dave Towey, Shikai Guo, Lei Ma

Published 2026-06-29

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Tao Li, Chenhui Cui, Rubing Huang, Dave Towey, Shikai Guo, Lei Ma

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a massive, complex digital building (a Vue.js web application) with thousands of rooms, secret passages, and doors that only open if you wear a specific badge. Your job is to find every single room and try every single door to make sure the building is safe and doesn't have any broken locks.

Doing this manually is slow and exhausting. Doing it randomly (just walking around and guessing which door to try next) is even worse because you might get stuck in the same hallway for hours, missing entire wings of the building.

This paper introduces a new tool called LLMVue that acts like a super-smart architect who can instantly draw a perfect map of this digital building before you even start walking.

The Problem: The "Hidden Door" Puzzle

In modern web apps, the rules for how you move from one page to another are scattered everywhere. Some rules are in the main blueprint (router config), some are written inside the walls of individual rooms (components), and some doors only appear if you are a "Manager" and not a "Guest."

Traditional tools try to guess the map by looking at the code, but they often get confused. They might miss a door that only opens for managers, or they might think two different doors are the same one, leading to a broken map.

The Solution: The Three-Stage "Map Maker"

The authors built a system called LLMVue that uses a Large Language Model (an AI that is very good at reading and understanding code) to build this map. It works in three simple steps:

The "Safety Net" Scan (Stage 1):
Imagine a robot sweeping the entire building with a wide net. It doesn't care if it catches a leaf or a rock; it just wants to make sure it catches everything. It scans all the code and lists every possible door and hallway it can find, even if it's not 100% sure yet. This ensures they don't miss any hidden rooms.
The "Smart Architect" Refinement (Stage 2):
Now, the AI (specifically GPT-4o) looks at that messy list. It acts like a detective. It reads the code to figure out:
- "Ah, this door is actually labeled 'Book Management' and only opens if you are a Manager."
- "This other door is just a generic button, but the text says 'Book Search'."
  The AI cleans up the list, adds specific instructions on how to open each door, and notes the conditions (like "only open for Managers"). It doesn't throw anything away; it just makes the instructions clearer.
The "Tidy-Up" (Stage 3):
Finally, the system removes duplicates. If the map says "Go to the kitchen" three different ways, it keeps the best instruction and deletes the rest. It also removes "dead ends" (like doors that just lead back to the same room). The result is a clean, executable Page Transition Graph (PTG)—a perfect, step-by-step guide for a robot to follow.

The Results: A Better Tour

The researchers tested this on 10 different real-world web applications. They compared their AI-built map against a map made by human experts (the "Gold Standard").

Accuracy: The AI map was incredibly accurate. It got about 91% of the map right, matching the human experts very closely. It rarely invented fake doors, though it did miss a few tricky ones that depended on complex, hidden conditions.
Efficiency: When they used this map to guide a testing robot (instead of letting the robot wander randomly), the robot was much faster.
- It found more "rooms" (pages) in less time.
- It clicked more buttons and tried more features.
- It covered more of the actual code inside the building.

The Bottom Line

Think of LLMVue as a GPS for testing web apps. Instead of letting a tester wander around blindly, hoping to find bugs, this tool reads the source code, understands the complex rules of the building, and draws a precise map. This map allows the testing robot to explore the entire building much faster and more thoroughly than it could ever do on its own.

The paper concludes that this approach works very well for Vue.js applications, making automated testing more reliable and efficient, especially for complex apps where navigation rules are hidden or conditional.

Technical Summary: LLM-Assisted Model-Based GUI Testing for Vue.js Web Applications

1. Problem Statement

Vue.js has become a dominant framework for modern web development, characterized by its reactive component architecture and client-side routing. However, ensuring the reliability of large-scale Vue.js Single-Page Applications (SPAs) through automated testing presents unique challenges. While Model-Based Testing (MBT) has successfully automated Graphical User Interface (GUI) testing on other platforms by utilizing Page Transition Graphs (PTGs), applying this to Vue.js remains difficult due to three specific framework complexities:

Distributed Navigation Logic: Navigation intent is fragmented across router configurations, Single-File Components (SFCs), shared layout components, and nested routing structures, making it hard to construct a unified model.
Encapsulation by UI Libraries: Navigation actions are often encapsulated within third-party UI components (e.g., Element UI/Plus), which obscure the underlying clickable elements and routing APIs.
Runtime Conditional Navigation: Menus and transitions are frequently protected by mutually exclusive runtime conditions (e.g., user roles, authentication states). Static analysis often fails to distinguish between these conditional branches, leading to conflated transitions that are not executable in specific states (e.g., attempting to click a menu item visible only to administrators while logged in as a standard user).

Existing approaches struggle to recover an executable, cross-component navigation model from Vue.js source code without missing valid transitions (low recall) or generating non-executable paths (low precision).

2. Methodology: The LLMVue Framework

To address these challenges, the authors propose LLMVue, a three-stage framework that generates a source-level, cross-component PTG from Vue.js source code to guide MBT. The framework utilizes a Large Language Model (LLM), specifically GPT-4o, to refine static analysis results.

Stage I: Static Graph Extraction (High Recall)

The process begins with a conservative static analysis of the Vue.js source code (router configurations and SFCs) using Abstract Syntax Trees (ASTs).

Goal: Maximize recall by capturing all plausible route-to-route transition candidates, even if they are imprecise or ambiguous.
Mechanism: The system parses router tables for explicit redirects and extracts navigation widgets (e.g., router-link, menu items) and script-based navigation calls (e.g., $router.push).
Output: An initial, potentially noisy candidate PTG ( $G_1$ ) where selectors are generic placeholders (e.g., tag-based or class-based) and conditions are not yet fully resolved.

Stage II: LLM-based Semantic Refinement

This stage leverages an LLM to enrich the candidate edges without deleting any, preserving the high recall established in Stage I.

Selector Enhancement: The LLM converts generic placeholders into executable Playwright locators by appending text constraints (e.g., :has-text(...)) and normalizing framework-specific class names (e.g., handling differences between Element UI and Element Plus).
Condition Extraction: The LLM extracts visibility and enabling conditions (e.g., v-if directives, script-level logic) and annotates edges with metadata (e.g., CROSS_ROLE for mutually exclusive branches).
Validation: The process includes iterative retry mechanisms with reflection prompting to ensure the output adheres to the edge schema and route table validity. If validation fails after retries, the original Stage I edges are retained to prevent data loss.

Stage III: Graph Normalization

The final stage applies deterministic filtering and deduplication to produce the executable PTG ( $G_3$ ).

Redundancy Removal: Structural edges (parent-child relations in the router table) and global redirect artifacts are removed.
Deduplication: Redundant selectors are consolidated. The system prioritizes specific selectors over generic ones and handles submenu prefixes to ensure distinct actionable paths.
Filtering: Cross-group edges derived from mutually exclusive conditional branches are filtered based on branch labels, and spurious self-loops are removed except for necessary UI expansion steps.

The resulting PTG is used by a testing engine (using Playwright) to systematically select and execute outgoing edges from the current route node until a time/action budget is exhausted.

3. Key Contributions

The paper makes the following specific contributions:

LLMVue Framework: An end-to-end framework for constructing source-level PTGs for Vue.js applications, directly enabling model-based GUI testing.
LLM-Assisted Edge Refinement: A novel technique that transforms recall-focused static candidates into executable edges by synthesizing precise locators and attaching condition annotations, all while maintaining high recall.
Deterministic Normalization: A process to clean and normalize the graph for testing, followed by an empirical evaluation of both PTG modeling quality and end-to-end testing effectiveness.
Empirical Evaluation: A comprehensive study on ten open-source Vue.js projects, comparing LLMVue against random exploration baselines.

4. Experimental Results

The framework was evaluated on ten open-source Vue.js projects from GitHub, using GPT-4o as the LLM backbone.

PTG Quality (RQ1)

When compared against expert-constructed ground truth:

Precision: High across all projects (average 94.57%), with four projects achieving 100% precision. This indicates that once a transition is identified, the refinement process effectively suppresses spurious edges.
Recall: Strong but variable (average 88.79%). Some projects showed lower recall due to missed edges in complex conditional or callback-driven routing.
F1-Score: Average of 91.25%, with five projects exceeding 90%.
Graph Edit Distance (GED): The average GED was 16, indicating that most constructed graphs are structurally close to the ground truth, though a few projects with complex navigation patterns showed higher distances.

Testing Effectiveness (RQ2)

LLMVue-guided testing was compared against random exploration under identical time budgets (60s to 600s) on two representative projects (P1: high-quality model; P3: structurally complex).

Action Efficiency: LLMVue consistently executed significantly more actions than random exploration (e.g., +59.4% to +66.8% more actions at 300s), demonstrating higher exploration throughput and stability.
Page Coverage: LLMVue achieved higher page coverage, particularly in the early stages of testing (e.g., +13 percentage points at 60s for P1), accelerating route discovery.
Statement Coverage: Improvements in code coverage were more pronounced with longer time budgets. For the high-quality model (P1), statement coverage increased steadily. For the complex model (P3), benefits were delayed but became significant at larger budgets, suggesting that structural guidance eventually translates to deeper code execution.

5. Significance and Claims

The paper claims to be the first use of Large Language Models for model-based GUI testing of Vue.js applications using source-level PTG extraction.

The authors position LLMVue as a practical solution that bridges the gap between static source code analysis and executable testing models for modern SPAs. By combining high-recall static extraction with LLM-based semantic refinement, the framework successfully navigates the complexities of Vue.js routing and UI library abstractions. The results suggest that LLM-assisted PTG construction provides a robust structural abstraction that significantly improves the efficiency and coverage of automated GUI testing compared to random exploration, particularly in time-constrained scenarios.

The authors remain modest regarding the scope, noting that the study focuses on systematic navigation and coverage measurement, while domain-specific functional assertions are out of scope. They acknowledge that while the approach is effective, challenges remain in modeling complex conditional navigation and callback-driven routing, suggesting future work in closing the feedback loop between extraction and testing execution.