Automated Web Application Testing: End-to-End Test Case Generation with Large Language Models and Screen Transition Graphs

1. Problem Statement

Web applications are critical to modern ecosystems, but ensuring their reliability is hindered by the complexity and dynamic nature of web interfaces. Traditional testing methods face several bottlenecks:

• Scalability Issues: Manual testing is time-consuming and error-prone, while existing automated tools (e.g., Selenium) typically require manually written test cases, which do not scale well to large, interactive applications.
• Dynamic Navigation Gaps: Current automated approaches struggle to model complex, dynamic navigation flows, leading to incomplete coverage of critical user pathways.
• Complex Form Interactions: Handling conditional forms (where fields appear based on prior inputs) and specific validation logic remains a significant challenge.
• Lack of Unified Workflow: Existing systems often treat navigation modeling and form validation as separate processes, lacking a cohesive framework to address both simultaneously.

2. Methodology

The authors propose an automated system that integrates Graph Structures (Screen Transition Graphs and State Graphs) with Large Language Models (LLMs) to generate end-to-end test cases. The process is divided into two primary modules:

A. Site-Navigation Test Generation

This module focuses on verifying that specific web pages are reachable from a starting point.

1. Graph Construction: The system performs Selenium-based web crawling to construct a Screen Transition Graph.
   • Nodes: Represent web pages (encapsulating URL, HTML content, metadata).
   • Edges: Represent navigational paths (links, buttons) between pages.
2. Path Identification: Given a start page and a destination, the system uses Dijkstra's algorithm to find the shortest path in the transition graph.
3. Action Generation: For each edge in the identified path, an LLM generates detailed, natural language descriptions of the user actions required (e.g., "Click the 'Blog' button").
4. Scenario Construction: These actions are aggregated into a structured, executable test scenario with expected outcomes.

B. Form-Filling Test Generation

This module addresses the complexity of static and dynamic forms.

1. Form Classification: Forms are categorized as Simple (static structure) or Dynamic (conditional fields that change based on user input).
2. State Graph Construction: The system employs a State Graph to model form interactions.
   • Nodes: Represent distinct form states.
   • Edges: Represent user interactions that transition the form from one state to another.
3. Recursive State Exploration (Algorithm 1):
   • The system iterates through all interactable elements in a form.
   • It captures the HTML state, simulates an interaction, and captures the new HTML.
   • If the HTML changes, a new state is saved, and the algorithm recursively explores further interactions within that new state.
   • Backtracking Mechanism: Since Selenium drivers do not natively support "undoing" actions, the system maintains a list of interactions to replay the sequence and return to the previous state after recursion.
4. Script Generation: Once all states are explored, the LLM generates Selenium scripts for each leaf node (final state) to test specific configurations.

3. Key Contributions

1. Novel Graph-LLM Integration for Navigation: A new approach combining screen transition graphs with LLMs to automate the generation of navigation test scenarios, ensuring comprehensive path coverage.
2. State Graph-Based Form Automation: A methodology using state graphs to model and explore complex conditional forms, enabling the automated generation of Selenium scripts for all possible form configurations.
3. Comprehensive Benchmark Dataset: The creation of a robust evaluation dataset comprising:
   • 2,000 Rule-based Synthetic Forms: 1,000 simple and 1,000 dynamic forms with varying complexities and "required" field logic.
   • 133 Real-world Forms: Collected from diverse domains (e-commerce, CMS, social networks) to test practical applicability.

4. Experimental Results

The system was evaluated using both closed-source (GPT-4o, GPT-4o-mini) and open-source models (DeepSeek Distill, Qwen, Llama).

• Site-Navigation Performance:
  • GPT-4o-mini achieved the highest overall performance, excelling in clarity, accuracy, and completeness.
  • Larger models generally outperformed smaller ones, though robustness (handling unexpected errors) remained a challenge across all models.
• Form-Filling Performance:
  • GPT-4o demonstrated superior accuracy across all settings (Simple, Dynamic, Real-world), particularly in handling structured and complex scenarios.
  • Dynamic Forms: Most models showed a significant drop in performance on dynamic forms compared to simple ones, highlighting the difficulty of handling complex conditional logic.
  • Field Type Analysis: The system performed well on common fields (text, checkbox, date) but struggled with hidden and file fields, as well as complex validation rules in real-world forms.
• Coverage: The system significantly improved test coverage by systematically exploring state spaces that manual testing often misses.

5. Significance

This work advances the state of web application testing by:

• Unifying Workflows: Bridging the gap between navigation modeling and form validation into a single, scalable framework.
• Handling Complexity: Providing a systematic solution for the "combinatorial explosion" of states in dynamic forms through recursive state exploration.
• Reducing Human Effort: Automating the creation of Selenium scripts, thereby reducing the time-consuming and error-prone nature of manual test case generation.
• Benchmarking: Establishing a new standard dataset for evaluating form-interaction testing, facilitating future research in AI-driven software testing.

The paper concludes that while challenges remain in handling edge cases and specialized field types, the integration of graph-based modeling with LLMs offers a powerful, adaptive solution for modern web testing.