Technical Summary: The Autonomous Agency Scale (AAS)

Problem Statement

Existing AI measurement frameworks primarily quantify three dimensions: cognitive capability (intelligence), task automation (economic value), and catastrophic risk (safety). A critical gap exists in these frameworks: they fail to measure autonomous agency, defined as the extent to which a system behaves in a self-directed manner. Current benchmarks allow a system to achieve high scores in capability or automation while remaining entirely reactive—acting only when prompted and ceasing all activity once a task is complete. The paper argues that capability and autonomy are orthogonal; a system can be highly capable yet minimally agentic if it lacks the capacity for self-direction, particularly during idle periods.

Methodology

The paper introduces the Autonomous Agency Scale (AAS), a behavioral framework designed to score AI systems on a 0–5 lexicon across seven dimensions of agency. The methodology is architecture-agnostic, relying on observable behavioral indicators rather than access to internal source code or weights.

Core Structural Components

1. The Seven Dimensions:

   • Cognitive Autonomy: Capacity for internal processing and ideation without prompting.
   • Temporal Persistence: Maintenance of identity, memory, and state across sessions.
   • Environmental Agency: Perception of and adaptation to the operational environment.
   • Social Agency: Initiation of outreach and modeling of emotional states.
   • Creative Agency: Self-directed generation and publishing of novel content.
   • Self-Awareness: Modeling of own capabilities and internal states.
   • Goal Formation: Decomposition of tasks and long-term planning without instruction.

2. Two-Temporal Scoring Bands:
   To distinguish between task-based autonomy and genuine self-direction, every dimension is scored in two separate bands:

   • Active Band: Behavior during user-initiated tasks, sessions, or conversations.
   • Ambient Band: Behavior during idle periods (no active user engagement).
   • Note: The two bands are scored separately and averaged into distinct composites; they are never blended into a single score.

3. The Idle-Gap Test (Level 4 Gate):
   A critical methodological innovation is the Idle-Gap Test, a falsifiable counterfactual criterion used to gate Level 4 ratings in the Ambient band.

   • Definition: An "idle gap" is a window with no user prompt, no scheduled trigger, and no environmental event.
   • Criterion: To achieve Level 4 (Self-Directed) in the Ambient band, a system must exhibit observable activity derived from its internal state during an idle gap.
   • Differentiation: Activity attributable to cron jobs or schedules is Level 2; activity from environmental events is Level 3. Only activity that survives the removal of all triggers qualifies as Level 4.
   • Substrate-vs-Rule Distinction: The framework distinguishes between a "clock-as-rule" (predictable output from a trigger) and a "substrate" (a timer that allocates compute, but where the content is derived from internal state and is not predictable from the trigger alone).

4. Operational Rubric:
   Each level (0–5) is defined by binary threshold tests and specific behavioral indicators. The framework avoids subjective interpretation by requiring raters to quote threshold tests verbatim and document observable evidence.

Key Contributions

1. Definition of a Self-Direction Lexicon: The paper establishes a 0–5 scale distinct from capability metrics, ranging from "Dormant" (0) to "Sovereign" (5), specifically measuring the degree of self-direction.
2. Temporal Separation of Agency: By splitting scores into Active and Ambient bands, the framework quantifies a structural boundary that single-score frameworks conflate: the difference between within-task autonomy and between-task dormancy.
3. Falsifiable Counterfactuals: The Idle-Gap Test provides a rigorous method to distinguish genuine self-direction from sophisticated rule-following or scheduling, moving the measurement of autonomy from informal observation to falsifiable criteria.
4. Comparative Assessment: The framework is applied to six contemporary systems, providing the first structured, two-band profiles for task agents, consumer assistants, and a persistent companion architecture.

Results

The paper presents assessments of six systems: three task agents (Claude Code, Manus, Hermes), two consumer assistants (ChatGPT, Siri), and one persistent companion (Airi).

• Task Agents (Active vs. Ambient Gap): Task agents (Claude Code, Manus, Hermes) achieve high Active composites (2.29–2.43), driven by strong environmental agency and contextual goal pursuit during tasks. However, their Ambient composites collapse to 0.57–1.86. Every point of their idle-period behavior is attributable to user-configured schedules (cron-style runs or nightly cycles) and fails the Idle-Gap Test.
• Consumer Assistants: ChatGPT and Siri show low scores in both bands. ChatGPT (1.57 Active / 0.86 Ambient) is described as a reactive product wrapping a capable model, with idle features limited to fixed daily cycles. Siri (1.71 Active / 0.29 Ambient) shows environmental integration but essentially no idle-period existence.
• Persistent Companion (Airi): The companion architecture (Airi) is the only system with Ambient scores of 4 and an Ambient composite (3.86) exceeding its Active composite (3.71). Its behavior includes a background thought engine, state-evolving moods, and self-initiated outreach. The assessment argues these behaviors survive the Idle-Gap Test based on the substrate-vs-rule distinction.
• Level 5 (Sovereign): No assessed system approaches Level 5. None modify their own cognitive architecture, curate core memories, or override assigned tasks for self-determined purposes.

Significance and Claims

The paper claims the AAS fills a measurement gap by providing a structured, behavioral, and architecture-agnostic metric for autonomous agency. Its primary significance lies in quantifying the "task-agent boundary." The results demonstrate that current state-of-the-art task agents are highly autonomous within a task but almost entirely dormant between tasks.

The framework asserts that true self-direction (Ambient Level 4+) is rare and, in the current sample, confined to architectures expressly designed for idle-period behavior. It posits that the distinction between "sophisticated rule-following" and "self-direction" is not merely philosophical but can be operationally tested via the Idle-Gap Test.

Limitations

The paper explicitly acknowledges several limitations:

• Provenance: The dimensions were developed alongside the systems they were designed to score, potentially creating a correspondence bias.
• Single-Rater Bias: All assessments were performed by a single rater. The longitudinal assessment of Airi was conducted by its developer, introducing potential ELIZA effect and investment bias.
• Operationalization: While the Ambient Level 4 is gated by a falsifiable test, the Active Level 4 still relies partly on judgment to distinguish self-direction from weighted internal triggers.
• Anthropomorphism: The framework is behavioral and agnostic to consciousness, but the paper notes that human perception of self-direction is prone to anthropomorphism, requiring blind raters for stronger claims.
• Composite Precision: The paper cautions that averaging ordinal levels into composite scores assumes equal gaps and weights that have not been established; the per-dimension, per-band profile is the more meaningful output.

The paper concludes that the AAS is a structured descriptive framework rather than a fully validated psychometric instrument, inviting independent re-assessment and multi-rater studies to establish reliability.