Agentization of Digital Assets for the Agentic Web: Concepts, Techniques, and Benchmark

1. Problem Statement

The Agentic Web envisions a decentralized ecosystem where autonomous agents collaborate to solve complex tasks. However, the scalability of this ecosystem is currently bottlenecked by the lack of a scalable method to continuously supply domain-specialized agents. Manual agent construction is costly, slow, and difficult to diversify.

While vast amounts of Digital Assets (e.g., code repositories, documents, services) exist as carriers of valuable knowledge and functionality, they remain static "data islands." The core challenge is Agentization: the automated process of transforming these static assets into interoperable, A2A (Agent-to-Agent) protocol-compliant agents.

The authors identify three primary technical hurdles in automating this process, particularly for code repositories:

1. Inconsistent Environments: Dependencies often conflict, making it difficult to reproduce execution environments.
2. Unstructured Skills: Valuable capabilities are hidden in undocumented code and must be extracted into atomic, reusable actions.
3. Semantic Gap: Even if code runs, it lacks clear, discoverable interfaces (e.g., Agent Cards) required for other agents to understand and invoke it.

2. Methodology

A. The Agentization Framework

The paper formalizes the A2A-Agentization process as a four-stage pipeline that transforms a raw repository ($D_{repo}$) into a compliant agent ($A$):

1. Environment Setup ($E$): Synthesizing a reproducible environment (e.g., container images, dependency resolution) to ensure the asset's logic runs deterministically.
   • $E = G_{env}(W_{dep}, W_{conf})$
2. Skill Extraction as Tools ($T$): Identifying, wrapping, and validating functional units from the codebase into executable tools.
   • $T = G_{tool}(D_{repo}, E)$
3. Inner Agent Instantiation ($A_{in}$): Constructing the agent's cognitive architecture (e.g., a ReAct loop) that integrates the extracted tools to plan and execute tasks.
   • $A_{in} = G_{inner}(D_{repo}, E, T)$
4. Final Agentization (Agent Card $C$): Generating a self-description registry (Agent Card) that details the agent's identity and available tools, enabling discovery by orchestrators.
   • $C = G_{card}(D_{repo}, T)$
   • Final Agent: $A = \{A_{in}, C\}$

B. The Agentization Agent

To execute this framework, the authors propose the A2A-Agentization Agent, an autonomous system designed to resolve the three technical hurdles. It performs end-to-end repository development, converting static code into an active, A2A-compliant agent.

C. The Benchmark: A2A-Agentization Bench

To rigorously evaluate this capability, the authors introduce A2A-Agentization Bench, the first benchmark explicitly designed for this domain.

• Dataset: Curated from 35 diverse GitHub repositories across 9 domains (e.g., Visual Processing, Document Automation, Chemistry, Finance), containing 522 evaluation instances.
• Task Types:
  • Single-Repo Tasks: Evaluate Capability Inheritance (fidelity). Can the agent correctly execute specific skills extracted from the repository?
  • Multi-Repo Tasks: Evaluate Collaborative Execution (interoperability). Can agents be orchestrated to solve workflows requiring multiple distinct agents?
• Evaluation Metrics:
  • Fidelity: Measured by Execution Success Rate (SR) on single-repo tasks.
  • Interoperability: Measured by Specification Quality (Skill F1-Score comparing generated Agent Cards to ground truth) and Orchestration Success Rate (ability of a coordinator to route tasks correctly).

3. Key Contributions

1. Conceptual Formalization: Defined "Agentization of Digital Assets" as a repository-level autonomous development task, formalizing the transformation from static data to A2A-compliant agents.
2. A2A-Agentization Agent: Proposed a fully automated framework that systematically handles environment setup, skill extraction, and semantic description generation.
3. A2A-Agentization Bench: Created a comprehensive benchmark with 35 repositories and 522 tasks, establishing the first standard for evaluating the fidelity and interoperability of automated agent generation.
4. Empirical Analysis: Provided a rigorous evaluation of state-of-the-art coding agents (Claude Code, Codex, OpenHands, EnvX) in this new context, identifying specific failure modes.

4. Experimental Results & Analysis

The authors evaluated four frameworks (Claude Code, Codex CLI, OpenHands, EnvX) using the benchmark.

• Stage 1 (Process Success): Claude Code and EnvX achieved a 100% Agentization Success Rate (Pass@1), successfully deploying agents with valid schemas. Other baselines struggled with environment configuration, requiring multiple attempts.
• Stage 2 (Capability Inheritance): Claude Code achieved the highest overall Execution Success Rate (36.9%), followed closely by EnvX (35.1%). This highlights that robust environment setup (EnvX's strength) is critical for execution fidelity.
• Stage 3 (Collaborative Execution):
  • Specification Quality: EnvX and Claude Code achieved the highest Skill F1-Scores (~66% and ~63%), indicating better self-description capabilities.
  • Orchestration: Higher specification quality correlated with better orchestration success. However, OpenHands achieved the highest Final Execution Success Rate (46.2%) in multi-agent scenarios despite lower orchestration scores, suggesting its execution engine is highly robust even when task routing is imperfect.

Critical Failure Modes Identified:

1. Environment Pre-configuration: Failures often stem from unresolved system dependencies, leading to "Startup Failures" or "Unusable Skills." This forces agents into costly runtime debugging loops (e.g., manually patching paths or reinstalling 150+ packages).
2. Skill Construction: Agents frequently hallucinate APIs or generate incorrect function signatures (e.g., type mismatches), causing valid semantic inputs to be rejected.
3. Capability Specification: Poor Agent Cards (vague descriptions or generic labels) prevent orchestrators from distinguishing specific agent roles, leading to dispatch failures in complex workflows.

5. Significance

This work bridges the gap between isolated software tools and a fully interconnected Agentic Web. By demonstrating that digital assets can be systematically transformed into interoperable agents, the paper provides a scalable supply-side solution for the Agentic Web.

The findings suggest that while automated agentization is feasible, future research must prioritize:

• Robust Environment Pre-configuration to eliminate runtime debugging overhead.
• Reliable Skill Construction to ensure extracted tools match actual repository implementations.
• Clear Self-Description (Agent Cards) to enable effective multi-agent collaboration and discovery.

The introduction of A2A-Agentization Bench sets a new standard for evaluating not just how well an agent uses tools, but how well it can be created from existing codebases to join the agentic ecosystem.