Web Agents Should Use Typed Actions Instead of Click-Based Browsing
This position paper advocates for replacing brittle, low-level click-based web interactions with a semantic layer of typed "web verbs" to create more reliable, auditable, and reproducible agentic web systems.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
The Big Idea: Stop Clicking, Start Speaking "Verb"
Imagine you are trying to teach a robot how to order a pizza.
The Old Way (Click-Based Browsing):
Right now, most web agents (AI robots) try to learn how to order pizza by watching a human click buttons. The robot has to figure out: "Okay, I need to move my mouse 400 pixels to the right, click the red button, wait for the screen to load, scroll down three inches, and type 'pepperoni'."
This is like teaching someone to drive by giving them a list of instructions for every single muscle movement: "Turn your head left, move your foot 2 inches, press the pedal." It works, but it's incredibly fragile. If the pizza shop moves the "Order" button two inches to the left, or changes the color of the menu, the robot gets confused and crashes. It has to re-learn the whole dance every time.
The New Way (Typed Actions / Web Verbs):
The authors of this paper argue that we should stop teaching robots to click. Instead, we should give them a menu of "Web Verbs."
Think of a Web Verb like a magic button or a command card. Instead of telling the robot how to click, you tell it what to do using a clear, structured command.
- Old Command: "Move mouse to X, Y, click."
- New Verb:
search_hotel(destination="Anchorage", check_in="2026-06-01")
This command is like a pre-packaged meal. The robot doesn't need to know how to chop the onions or fry the steak (the complex clicking and scrolling); it just asks for the "Hotel Search" meal, and the system handles all the messy details behind the scenes.
Why Do We Need This?
The paper points out three main problems with the "clicking" approach and how "verbs" fix them:
1. Reliability (The "Fragile House" Problem)
- The Problem: When a robot clicks buttons, it's building a house of cards. If the website changes the layout slightly (like moving a button), the whole house collapses. The robot gets lost.
- The Verb Solution: A Verb is like a sturdy brick. Even if the website changes its paint job or rearranges the furniture, the "Search Hotel" brick still works because it's built to handle those changes internally. The robot just asks for the brick; it doesn't need to know how the brick was made.
2. Efficiency (The "Stop-Start" Problem)
- The Problem: To do a simple task like booking a trip, a clicking robot might have to take 50 tiny steps: click here, wait, scroll, type, click there. It has to "think" and "look" at the screen after every single step. This is slow and expensive.
- The Verb Solution: With verbs, the robot can say, "Find me a hotel," and "Find me a flight," and "Book them." It does the whole job in three big steps instead of 50 tiny ones. It's like ordering a full meal at a restaurant instead of cooking every ingredient from scratch.
3. Verifiability (The "Black Box" Problem)
- The Problem: If a clicking robot makes a mistake, it's hard to tell why. Did it click the wrong button? Did it misread the text? It's a messy trail of actions that is hard to check.
- The Verb Solution: Verbs are like a receipt. When the robot uses a verb, it gets a clear, structured answer (e.g., "Here is the list of hotels with prices"). We can easily check the input (what it asked for) and the output (what it got). If something goes wrong, we know exactly which "brick" failed, not which specific pixel the robot clicked.
How It Works in Real Life (The Examples)
The paper tests this with two examples:
Travel Planning: A user wants to find hotels near museums and rank them by distance.
- Clicking Robot: Gets confused. It tries to draw a single line connecting all the museums and hotels, which doesn't actually calculate the distance correctly. It loses the math in the middle of all the clicking.
- Verb Robot: Uses a
get_directionsverb. It loops through the list, asks the verb for the distance between each hotel and museum, adds them up, and sorts them. It follows the math perfectly because the "distance" is a clear number, not a visual guess.
Furniture Shopping: A user wants to buy a bed, desk, and lamp under a $1,000 budget, maximizing the rating.
- Clicking Robot: Picks items one by one greedily. It might pick a great bed, then a great desk, and suddenly realize it has no money left for a lamp. It fails the global goal.
- Verb Robot: Uses verbs to get a list of all items with their prices and ratings. It then runs a simple computer program to check every possible combination to find the best set that fits the budget. It solves the puzzle logically.
The Call to Action
The authors aren't saying AI models need to get "smarter." They are saying the interface needs to change.
They propose that websites should expose these "Verbs" to the world.
- For Developers: Instead of just building a website for humans, build a "Verb Layer" for robots. This could be a simple API (a direct line to the server) or a script that automates the browser clicks for you.
- For the Community: Just like we have standards for how websites look (HTML), we need standards for how robots talk to websites. We need a universal dictionary of "Web Verbs" so robots know exactly what commands are available.
Summary Analogy
- Current Web Agents: Are like a person trying to drive a car by manually turning the steering wheel, pressing the gas, and pumping the brakes with their feet, while looking at a map. If the road changes, they crash.
- Proposed Web Agents: Are like a passenger in a self-driving car. They just tell the car, "Take me to the airport." The car (the Verb) handles the steering, braking, and navigation. It's safer, faster, and you know exactly where you are going.
The paper argues that to make the "Agentic Web" (a web where robots do things for us) reliable, we must stop giving robots a steering wheel and start giving them a destination.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.