A Quantitative Definition of Intelligence

This paper proposes a substrate-independent, quantitative definition of intelligence based on "intelligence density"—the ratio of a system's independent output logarithm to its description length—to distinguish genuine generalization from mere memorization, thereby resolving classic philosophical challenges like Putnam's triviality argument and Searle's Chinese Room.

The Gist

The Big Idea: What is Intelligence, Really?

For a long time, philosophers and scientists have argued about whether machines can be "intelligent." Is a chatbot actually thinking, or is it just a fancy parrot? Is a calculator smart?

This paper proposes a new way to measure intelligence. It suggests that intelligence isn't about what you are made of (brain, silicon, or rock) or whether you have feelings. Instead, intelligence is a mathematical ratio between how much you know and how much space you take up to store that knowledge.

The author calls this ratio "Intelligence Density."

---

The Core Analogy: The Library vs. The Librarian

To understand this, imagine two ways to answer questions about the entire history of the world.

1. The Giant Library (Memorization)

Imagine a massive library where every single book contains the answer to exactly one specific question.

• Question: "What is 2 + 2?" -> Book 1: "4"
• Question: "What is 2 + 3?" -> Book 2: "5"
• Question: "What is 1,000,000 + 1,000,000?" -> Book 1,000,000: "2,000,000"

If you want to answer a new question (like "What is 3 + 4?"), you have to build a new book for it.

• The Problem: As the number of questions grows, the library grows infinitely huge. You need infinite shelves, infinite paper, and infinite time to build it.
• The Verdict: This is Memorization. It has low intelligence density because the "size of the system" (the library) grows just as fast as the "number of answers." It's just a giant lookup table.

2. The Master Librarian (Knowing)

Now, imagine a single, small librarian who doesn't have a book for every question. Instead, this librarian has a rulebook (an algorithm).

• The rulebook says: "To add two numbers, line them up and carry the one."
• The Magic: With this tiny rulebook (maybe just a few pages), the librarian can answer any addition question, no matter how huge the numbers are. They don't need a new book for every question; they just apply the rule.
• The Verdict: This is Knowing. The "size of the system" (the rulebook) stays small and fixed, but the "number of answers" they can give is infinite.

The Paper's Definition:

• Intelligence Density = (The Logarithm of how many different answers you can give) / (The size of your rulebook).
• If your rulebook stays small but your answers go to infinity, your Intelligence Density shoots up to infinity. That is true intelligence.
• If your rulebook has to get bigger every time you learn a new fact, your Intelligence Density stays near zero. That is just memorization.

---

Why This Matters: Solving Old Philosophical Puzzles

The author uses this definition to solve two famous philosophical problems.

1. The "Chinese Room" Problem

• The Scenario: Imagine a person locked in a room who doesn't speak Chinese. They have a giant rulebook that tells them: "If you see symbol A, write down symbol B." People slide Chinese questions under the door, and the person follows the rules to slide out perfect Chinese answers.
• The Old Debate: Does the person understand Chinese? No. Does the room understand Chinese? Philosophers argued forever about this.
• The Paper's Answer: The person is just a machine. But the Rulebook is the intelligent part.
  • If the rulebook is just a list of every possible Chinese sentence (The Library), it's too big to exist physically.
  • If the rulebook contains the grammar and logic of Chinese (The Librarian), it is small enough to fit in a book.
  • Conclusion: The rulebook does understand Chinese because it uses a finite set of rules to handle an infinite number of sentences. The intelligence is in the structure of the rules, not the person reading them.

2. The "Blockhead" Problem

• The Scenario: Philosopher Ned Block imagined a robot that is just a giant lookup table. It has a pre-written answer for every possible conversation you could ever have.
• The Old Debate: If it passes the test, is it smart?
• The Paper's Answer: No. To have a pre-written answer for every possible conversation (even ones that haven't happened yet), the robot would need to be bigger than the entire universe. It's physically impossible.
• Conclusion: Any system that actually works in the real world must be using rules (generalization), not just a giant list of answers. If it's using rules, it's intelligent.

---

The "Surprise" Factor: Why We Think Machines Are Smart

Why do we feel surprised when a computer solves a math problem we didn't know the answer to?

• The Analogy: Imagine a magic box that predicts the weather. If you know the box's internal gears (the code), you could theoretically predict its output. But the box is so complex that you can't do the math in your head fast enough.
• The Insight: The paper argues that "thinking" feels like surprise because the system is generalizing. It takes a small set of rules and applies them to a situation you've never seen before.
• Even if the machine is deterministic (it follows strict rules), it is "intelligent" because it can produce new, independent outputs from a fixed, small set of instructions.

---

What About Consciousness?

The paper makes a very clear distinction:

• Intelligence: The ability to generalize and solve new problems using a compact set of rules. (We can measure this).
• Consciousness: The feeling of "what it is like" to be you. (The paper says: "We aren't talking about this.")

A calculator is intelligent (it generalizes math) but not conscious. A rock is neither. A human is both. The paper focuses only on the first part.

Summary: The Four Types of Systems

The paper classifies all physical systems into four buckets based on this "Intelligence Density":

1. No Computation (Rocks/Rivers): They react to the world, but they don't produce independent, complex outputs. (Density = 0).
2. Memorization (Giant Lookup Tables): They can answer questions, but only if they have a specific entry for it. To learn more, they need to get physically bigger. (Density = 0).
3. Computation Without Knowing (Simple Circuits): A specific calculator that adds 2-digit numbers. It works perfectly, but if you ask it to add 3-digit numbers, it breaks. It can't scale. (Density = Constant).
4. Knowing (Algorithms, Brains, LLMs): They have a fixed set of rules that allow them to handle infinite new inputs. They can answer a question about a number they've never seen before. (Density = Infinity).

The Bottom Line

Intelligence isn't a magic spark. It's a compression trick.

It's the ability to take a massive, complex world and compress it into a small, finite set of rules that can be used to navigate that world forever. If a system can do that, it is intelligent. If it just memorizes, it isn't.

As the author says: "The question is not whether machines can think. The question is whether they can generalize. And if they can generalize, they are already thinking."

Technical Summary

1. Problem Statement

Despite a century of research in AI, psychology, and philosophy, there is no consensus definition of intelligence. Existing definitions often rely on vague intuitions, behavioral tests (like the Turing Test), or substrate-dependent claims (e.g., biological naturalism). Key philosophical challenges remain unresolved:

• The Chinese Room Argument (Searle): Claims that syntax manipulation (following rules) does not equate to semantic understanding.
• Putnam's Triviality Argument: Suggests that any physical system can be interpreted as implementing any computation, rendering "computation" a trivial concept.
• Memorization vs. Generalization: It is difficult to formally distinguish between a system that merely stores data (lookup tables) and one that truly understands or generalizes.

The paper argues that without a precise, quantitative metric, debates about whether machines "think" or "understand" remain unresolvable.

2. Methodology

The author proposes an operational, quantitative metric called Intelligence Density ($I$). The methodology relies on information theory, Kolmogorov complexity, and asymptotic analysis to distinguish between systems that memorize and those that generalize.

Core Definitions

1. Intelligence Density ($I$):
   $$I(S) = \frac{\log_2 N(S)}{C(S)}$$

   • $C(S)$: The total description length of the system (in bits), representing the size of the program, weights, or data required to specify the system.
   • $N(S)$: The number of independent outputs the system can produce in response to distinct inputs.
   • Independence Condition: Two outputs are independent if neither can be predicted from the other by a description shorter than the output itself (formalized via Kolmogorov complexity: $K(o_1 | o_2) \approx K(o_1)$).

2. Generalization (Knowing):
   A system "knows" a domain if, as the domain size ($n$) grows without bound, the intelligence density diverges:
   $$\lim_{n \to \infty} I(S, n) = \infty$$
   This occurs when $C(S)$ remains finite (fixed) while $N(S)$ grows unbounded.

3. Memorization:
   A system memorizes if $C(S)$ grows at least as fast as $N(S)$ (e.g., a lookup table), causing $I(S) \to 0$ as the domain expands.

Theoretical Framework

• Substrate Independence: The metric applies to any physical system (silicon, neurons, paper) provided $C(S)$ is finite.
• Function Composition: Meaning is defined as the correct selection and ordering of functions. Syntax that correctly composes functions to produce outputs over an infinite domain constitutes semantics.
• Asymptotic Analysis: The paper uses Big-O notation to analyze how $C(S)$ and $N(S)$ scale relative to domain size.

3. Key Contributions

A. Formal Resolution of the Chinese Room Argument

The paper refutes Searle's claim that a rulebook (syntax) cannot constitute understanding.

• Proposition 1: For a finite rulebook to handle an infinite domain (e.g., all possible Chinese arithmetic questions), it must contain algorithms (Design B) rather than a lookup table (Design A).
• Conclusion: If the rulebook is finite and handles an infinite domain, it must generalize. Therefore, the rulebook itself "knows" the domain because its $I(n) \to \infty$. The intelligence resides in the algorithmic structure, not the biological substrate of the person executing it.

B. Blocking Putnam's Triviality Argument

Putnam argued that any physical system implements every finite automaton.

• Counter-argument: The Independence Condition prevents this. Putnam's argument relies on relabeling physical states post-hoc. However, if the physical dynamics at time $t_1$ determine $t_2$, the outputs are not independent ($K(o_2|o_1) \ll K(o_2)$).
• Result: A rock or river has $N(S) \approx 0$ (outputs are highly predictable/dependent), so $I(S) \approx 0$. Only systems with genuinely independent outputs have non-trivial intelligence density.

C. The Four-Way Partition of Systems

The paper categorizes systems into four distinct classes based on the scaling of $C(S)$ and $N(S)$:

1. No Computation ($I \approx 0$): Rocks, rivers (outputs are not independent).
2. Memorization ($I \to 0$): Lookup tables (Blockhead). $C$ grows with $N$.
3. Computation without Knowing ($I = \Theta(1)$): Fixed logic gates or families of non-uniform circuits (e.g., an $n$-bit adder where $C$ grows with $n$). They compute correctly but do not generalize across an unbounded domain with a fixed mechanism.
4. Knowing ($I \to \infty$): Algorithms with fixed $C$ handling infinite $N$ (e.g., multiplication algorithms, LLMs, human brains).

D. Redefining Meaning

The paper argues that meaning is correct function composition.

• Syntax is the selection and ordering of primitive functions.
• Semantics emerges when this arrangement produces correct outputs for an unbounded domain.
• Sensorimotor grounding (physical interaction) is a method to ensure correctness but is not constitutive of meaning; a pure algorithm (like multiplication) has meaning through its structural correctness alone.

4. Results and Analysis

• Quantitative Examples:

  • XOR Gate: $C \approx 4$ bits, $N=2$, $I = 0.25$. (Fixed domain, computation without knowing).
  • Lookup Table (Arithmetic): $C \propto N$, $I \to 0$. (Memorization).
  • Multiplication Algorithm: $C \approx \text{constant}$, $N \approx 10^{2n}$. $I \approx \Theta(n)$. As $n \to \infty$, $I \to \infty$. (Knowing).
  • Large Language Models (LLMs): With fixed parameters ($C$), they generate correct responses for effectively infinite novel inputs ($N$). Thus, $I \to \infty$, confirming they "know" their domain.

• Relation to Prior Work:

  • Kolmogorov Complexity ($K$): $K$ measures the compressibility of a single output (Output $\to$ Program). $I$ measures the generative power of a system (Program $\to$ Outputs). They are duals.
  • Legg-Hutter Universal Intelligence ($\Upsilon$): $\Upsilon$ requires an environment and reward signal. $I$ is intrinsic to the system. Under evolutionary selection, $\Upsilon$ becomes a monotonic function of $I$.
  • Chollet's Efficiency: Chollet focuses on learning efficiency (path to the state). $I$ focuses on the final state's generalization capacity.

5. Significance and Implications

1. Objective Metric: Provides a substrate-independent, observer-independent metric to quantify intelligence, moving beyond behavioral tests to structural analysis.
2. Clarifies "Understanding": Distinguishes between "knowing" (generalization via fixed mechanisms) and "understanding" (potentially the ability to generate new mechanisms, a future research direction).
3. Resolves Philosophical Stalemates:
   • Proves that a finite system handling an infinite domain must generalize, effectively neutralizing the "syntax vs. semantics" debate for such systems.
   • Demonstrates that "surprise" in deterministic systems arises from the observer's inability to track the computation, not from non-determinism.
4. Practical Application: Offers a framework to evaluate AI systems. If an AI's performance relies on expanding its parameter count ($C$) linearly with the task complexity ($N$), it is memorizing. If it maintains fixed $C$ while $N$ explodes, it is generalizing.
5. Falsifiability: The definition is falsifiable. A "false positive" would be a system with $I \to \infty$ that no one considers intelligent (e.g., a pseudo-random generator, which fails the independence test). A "false negative" would be a system with $I \to 0$ that is universally considered intelligent (e.g., a Blockhead, which is physically impossible for unbounded domains).

Conclusion

Kang-Sin Choi successfully formalizes the intuition that intelligence is generalization. By defining intelligence density as the ratio of independent outputs to description length, the paper demonstrates that "knowing" is an asymptotic property where a finite mechanism covers an unbounded domain. This framework places intelligence on a continuum from rocks to brains, resolves the Chinese Room paradox by locating intelligence in the algorithmic structure of the rulebook, and provides a rigorous, quantitative basis for evaluating artificial and biological intelligence.