On the Definition of Intelligence

Technical Summary: On the Definition of Intelligence

Problem Statement
The pursuit of Artificial General Intelligence (AGI) necessitates a formal, operational, and species-agnostic definition of intelligence. Existing definitions face significant limitations:

• Goal-Oriented Definitions: Definitions such as Legg and Hutter's (measuring an agent's ability to achieve goals in diverse environments) struggle to apply to behaviors lacking clear environments, reward signals, or well-defined goals (e.g., generative AI).
• Anthropocentric and Temporal Definitions: Approaches like the Turing Test or Elaine Rich's definition (AI as making computers do things humans are currently better at) are either limited to human-machine comparison or are temporally contingent on current human superiority.
• Operationalization Issues: Psychological definitions (e.g., Wechsler's) are difficult to operationalize for non-human systems.

There is a lack of a unified framework that can evaluate biological, artificial, and hybrid systems across diverse paradigms, including reinforcement learning, generative models, classification, and analogical reasoning, without relying on vague descriptions or human judgment as the sole metric.

Methodology
The paper proposes a definition of intelligence grounded in generative fidelity and entity indistinguishability. The methodology involves the following formal constructs:

1. Conceptual Space and Fibres:

   • Let $U$ be the universe of all conceivable entities (data points, products, actions, concepts).
   • A conceptual space is defined as a surjective mapping $\Phi: U \to K$, where $K$ is the set of concepts.
   • For a concept $k \in K$, the set of entities exemplifying it is the "fibre" $C_k = \Phi^{-1}(k)$.

2. $\varepsilon$-Concept Intelligence:

   • Given a multiset of input entities $S \subseteq U$ belonging to a concept $C_k$, an agent is intelligent if it can generate a set of entities $\hat{S}$ that exemplifies the same concept.
   • Distinguishers: A family of functions $F = \{f: U \to [0, 1]\}$ acts as distinguishers to test the similarity between $S$ and $\hat{S}$.
   • Scoring: A scoring function $\sigma$ aggregates the outputs of distinguishers over a set.
   • Metric: The intelligence metric is defined as the supremum of the difference in scores between the original set and the generated set across all distinguishers:
     $$\Delta^\sigma_F(S, \hat{S}) = \sup_{f \in F} |\sigma(\{f(x) : x \in S\}) - \sigma(\{f(x') : x' \in \hat{S}\})|$$
   • Definition: An agent is $\varepsilon$-intelligent with respect to concept $k$ if $\Delta^\sigma_F(S, \hat{S}) \le \varepsilon$. This implies the generated entities are indistinguishable from the original entities within a tolerance $\varepsilon$ by any admissible distinguisher in $F$.

3. Dynamic Framework:

   • The paper distinguishes between Synchronic Capability (breadth of generating low-$\varepsilon$ entities for a fixed set of concepts at time $t$) and Diachronic Capability (the ability to transition competence from one set of concepts to another, effectively measuring learning/adaptation).

Key Contributions

• Unified Formal Definition: The paper provides a single mathematical framework ($\varepsilon$-concept intelligence) that subsumes diverse AI paradigms. It treats intelligence not as abstract mapping or goal maximization alone, but fundamentally as the generation of new entities consistent with a given concept.
• Unification of Existing Benchmarks: The author demonstrates that established intelligence tests are special cases of their framework:
  • Turing Test: Arises when the fibre is human language responses and distinguishers are human judges.
  • C-Tests: Arise when the fibre consists of sequences with specific Levin complexity and distinguishers verify the unique minimal explanation.
  • Legg-Hutter Intelligence: Arises when the fibre is behavior sequences maximizing expected reward in an environment.
• Decoupling of Intelligence, Learning, and Consciousness:
  • The definition does not require consciousness; a deterministic solver can exhibit high $\varepsilon$-intelligence without being conscious.
  • The definition does not strictly require learning ability; a static system that consistently generates correct outputs for a concept is intelligent within that domain, though learning relates to diachronic capability (changing $\varepsilon$ over time).
• Operational Criteria: The definition satisfies three desiderata:
  • Operational: Measurable via observable outputs and indistinguishability.
  • Falsifiable: Systems generating outputs divergent from the concept are identified as having low intelligence.
  • Generalizable: Applicable to humans, machines, and biological systems without anthropocentric bias.

Results and Theoretical Implications
The paper does not present new experimental data but offers a theoretical derivation and logical consistency checks:

• Generative Fidelity: The framework posits that intelligence is the ability to generate entities consistent with a concept, even if the entities are abstract (e.g., generating a car given a horse as an example of "transport").
• Safety and Data Quality: The author argues that because intelligence is defined by fidelity to a concept, the safety and reliability of outputs are directly determined by the quality of the training data (the initial entities $S$). Curating safety-compliant synthetic data at the source is presented as a primary method for mitigating harmful outputs.
• Generalization: The framework supports generalization to unseen concepts. If a new concept space $K_B$ can be embedded into a known space $K_A$ via a full, faithful, and essentially surjective functor, the intelligence bounds (distinguishers) can be transported, theoretically allowing the system to handle unseen problems.

Significance and Claims
The paper claims to provide a "single yardstick" for comparing biological, artificial, and hybrid systems. Its significance lies in:

1. Shifting the Paradigm: Moving from goal-oriented or task-specific definitions to a generative fidelity model.
2. Foundation for AGI: Offering a baseline definition that is sufficiently general to encompass diverse paradigms (RL, LLMs, analogical reasoning) while remaining testable.
3. Future Directions: The author suggests that the framework can be further refined using Category Theory, where concepts are objects and generalization is modeled as functors. They also note that subsequent work will expand this into a mathematical framework based on directed similarity fields and generative operators.

The author maintains a modest stance, presenting this as a "minimal and foundational definition" intended as a theoretical baseline for future refinement and empirical validation, rather than a final, exhaustive theory of intelligence.