MyAG: A Graph-Based Framework for Designing and Analyzing Composable LLM Agent Systems

Imagine a world where computers don't just answer questions but actually do things. They can browse the web, book flights, or solve complex puzzles by acting like digital assistants. This is the realm of AI Agents. Think of them not as simple chatbots, but as little robots with a brain (a Large Language Model) and a set of hands (tools to interact with the world). But here's the tricky part: building these robots is messy. You have to decide what tools they have, how they think step-by-step, and what they do when they get stuck. It's like trying to build a car where the engine, the steering wheel, and the GPS are all glued together in a way that makes it impossible to swap them out. If you want to change how the car drives, you might have to rebuild the whole engine. Researchers are trying to figure out how to build these agents so they are flexible, efficient, and easy to fix, because right now, it's hard to tell if a robot is failing because it's "dumb" or because the instructions were just poorly written.

Enter MyAG, a new toolkit created by Zhisong Zhang from the City University of Hong Kong. The paper presents MyAG as a "graph-based framework," which is a fancy way of saying it organizes agent design into three separate, clear blueprints instead of one giant, tangled mess. The authors suggest that by separating the "who" (the components), the "how" (the workflow), and the "what if" (the search strategy), developers can mix and match parts like Lego bricks. They found that this separation makes it much easier to test different strategies without rewriting the whole system. In their experiments, they showed that this approach allows for a detailed look at the trade-offs between how well an agent does a task and how much time or money it costs to do it. They didn't claim to have solved all AI problems, but they did demonstrate that their method helps researchers see exactly where an agent is spending its time and where it might be wasting resources.

The Three Blueprints of a Digital Brain

To understand MyAG, imagine you are directing a movie. You have three distinct jobs, and MyAG insists you keep them separate so you can fix one without ruining the others.

1. The Component Graph: The Cast and Crew
This is your list of who is in the movie. In the AI world, these are the agents (the actors making decisions), the environments (the sets they act on, like a web browser), and the tools (the props they use). In MyAG, this is drawn as a map showing who can talk to whom. For example, an "Action Agent" might be connected to a "Browser Environment," meaning the agent can click buttons on a webpage. The cool thing here is that you can reuse the same actor for different scenes. You don't need to build a new actor just because the scene changes; you just tell the existing one what to do next.

2. The Workflow Graph: The Script
This is the script that tells the actors when to speak and what to do next. It's the flow of the story. Does the agent check the weather, then book a ticket? Or does it loop back and try again if it fails? MyAG lets you draw this script as a flowchart. You can have loops, branches (if this happens, go left; if that happens, go right), and stop conditions. The best part? You can keep the same cast (Component Graph) but swap the script (Workflow Graph) to see if a different story structure works better.

3. The Search Graph: The "What If" Rehearsals
This is where it gets really interesting. Sometimes, a robot needs to explore different paths before deciding on the best one. Maybe it tries a door, realizes it's locked, and then tries a window. The Search Graph records all these "what if" scenarios. It keeps track of every branch the agent takes, every dead end, and every successful path. This allows the system to use strategies like "Best-First Search" (always picking the most promising path) or "Rollback" (going back to a previous step if things go wrong). It's like having a director who can say, "Cut! Let's try that scene again, but this time, the character runs instead of walks," without having to rebuild the entire movie set.

Building Towers with Lego Blocks

One of the paper's big ideas is Hierarchical Composition. Imagine you are building a giant robot. Instead of trying to design every tiny screw and wire at once, you build a hand, then a leg, then a head, and finally put them together. MyAG lets you do this with AI agents. You can build a small "Web Browsing System" that is really good at finding papers on the internet. Then, you can wrap that whole system up and treat it like a single tool inside a bigger "Research Assistant" system.

The paper explains that these "sub-systems" can be stateless (resetting every time they are used, like a calculator) or stateful (remembering what happened before, like a human assistant who remembers your preferences). This modularity means you can build complex agents by snapping together smaller, tested pieces, rather than starting from scratch every time.

The Stopwatch and the Wallet: Measuring Efficiency

The authors also care about efficiency. They argue that just because an agent gets the right answer doesn't mean it's a good agent; it might have taken 100 tries and cost a fortune in computer time. MyAG includes a built-in stopwatch and wallet.

They measure two main things:

• LLM Cost ($C_L$): How much the "brain" costs. This is calculated based on how many words (tokens) the AI reads and writes. They use a formula where they weigh the input and output tokens, allowing users to set prices based on how expensive the AI model is.
• Environment Cost ($C_E$): How much the "hands" cost. This measures the time it takes to do things in the real world, like waiting for a webpage to load or scrolling down a page.

By tracking these costs, the researchers can see the performance-efficiency trade-off. For example, they found that a simple "Greedy" strategy (just doing the first thing that comes to mind) is the fastest and cheapest. However, a "Rollback" strategy (which lets the agent go back and try again) might be slightly more expensive but often gets a better result. The "Best-of-N" strategy (trying many paths at once and picking the best one) can be very accurate but is very slow and costly.

What the Experiments Showed

The team tested MyAG on two real-world tasks: answering complex questions (using the GAIA benchmark) and navigating websites (using Mind2Web-Live). They ran these tests with different strategies to see how they stacked up.

• Speed vs. Smarts: The Greedy strategy was the most efficient, taking the least time and using the fewest computer tokens because it didn't waste time checking alternatives.
• The Power of Second Chances: The Rollback strategy performed the best overall when given enough time. It allowed the agent to admit mistakes and try again, which led to higher accuracy, even though it cost a bit more in time and tokens.
• The Cost of Complexity: Strategies like Best-First and Best-of-N could improve performance, but they came with a heavy price tag. They required a lot more computer power and time. The paper notes that the success of these complex methods depends heavily on having a good "judge" to decide which path is best.

They also broke down exactly where the time was spent. For web navigation, they found that "scrolling" was surprisingly cheap and fast, while "waiting" for a page to load was the most expensive action, taking up a huge chunk of the time even though it happened less often. This kind of detailed analysis helps developers know exactly where to optimize their agents.

The Bottom Line

MyAG isn't a magic wand that solves all AI problems, and the authors are clear that it's designed as a research tool, not a finished product for building commercial apps with millions of users. It doesn't handle things like security or distributed networks yet. However, it successfully proves that separating the "who," the "how," and the "what if" makes it much easier to design, test, and understand AI agents. By giving researchers a clear way to visualize and measure their agents, MyAG helps them figure out exactly how to build smarter, faster, and more cost-effective digital assistants. The paper suggests that this approach is a solid step forward in making AI agent systems more transparent and manageable.