🤖 AI

Structural Ranking of the Cognitive Plausibility of Computational Models of Analogy and Metaphors with the Minimal Cognitive Grid

This paper applies the Minimal Cognitive Grid framework to systematically evaluate and rank the cognitive plausibility of leading computational models of analogy and metaphor, including SME, CogSketch, METCL, and LLMs, using a formalized quantitative approach based on functional/structural ratio, generality, and performance match.

Original authors: Alessio Donvito, Antonio Lieto

Published 2026-05-06

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Alessio Donvito, Antonio Lieto

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a judge at a talent show, but instead of singers or dancers, the contestants are computer programs trying to act like human thinkers. Specifically, they are trying to solve puzzles involving analogies (like "A is to B as C is to D") and metaphors (like "My job is a jail").

The authors of this paper, Alessio Donvito and Antonio Lieto, created a special scoring system called the Minimal Cognitive Grid (MCG). Think of this grid as a three-legged stool. For a computer to sit comfortably on the stool and be considered a "cognitively plausible" model (meaning it thinks like a human, not just looks like it), it needs to balance on all three legs.

Here is how the three legs work, using simple analogies:

Leg 1: The Blueprint Match (Functional/Structural Ratio)

The Question: Does the computer build its answer the same way a human brain does, or does it just get the right answer by magic?

The Analogy: Imagine two people trying to build a birdhouse.
- Person A (The Structural Model): Uses wood, nails, and a hammer, following the exact same steps a carpenter would. They might be slow, but their process is human-like.
- Person B (The Functional Model): Uses a 3D printer or a laser cutter. They get the exact same birdhouse, but the machine inside is totally different from a human's hands.
The Paper's Claim: The authors argue that for a computer to be a good model of how we think, it needs to use the "wood and nails" approach (structural).
- SME and CogSketch: These are like the carpenters. They use specific rules (like "one thing maps to one thing") that mimic how humans make analogies. They get high scores here.
- LLMs (like GPT-4): These are like the 3D printers. They are amazing at making the birdhouse (getting the right answer), but they do it by guessing patterns from a massive library of books, not by following human-like logical steps. They get very low scores here because their "internal machinery" is alien to human thought.
- METCL: This is a middle-ground carpenter. It uses a different set of blueprints (focusing on categorization rather than mapping), so it scores in the middle.

Leg 2: The Swiss Army Knife (Generality)

The Question: Can the computer do many different types of thinking, or is it a one-trick pony?

The Analogy:
- The Specialist: A master chess player who can beat a grandmaster but can't tie their own shoes or cook an egg.
- The Generalist: Someone who can play chess, cook, drive a car, and solve math problems.
The Paper's Claim: The authors look at whether these systems can handle math, visual puzzles, language, and even physical movement (sensory/motor skills).
- LLMs: These are the ultimate Generalists. They can read, write, do math, and look at pictures. They score highest here.
- SME and CogSketch: These are the Specialists. They are great at specific logic puzzles (like visual patterns) but can't really do math or understand complex sentences. They score low here.
- METCL: Also a specialist, focused mostly on language metaphors.

Leg 3: The Mirror Test (Performance Match)

The Question: When the computer gets it wrong, does it make the same kind of mistakes a human would?

The Analogy: Imagine taking a test.
- If you get a question wrong because you misunderstood the concept, that's a "human" mistake.
- If you get it wrong because the computer glitched or guessed randomly, that's a "machine" mistake.
- The goal is to see if the computer's "glitches" look like human confusion.
The Paper's Claim:
- SME and CogSketch: They are excellent at this. When they fail a visual puzzle (like the Raven's Progressive Matrices), they fail on the exact same hard puzzles that humans struggle with. They even take a similar amount of time to think. They score very high.
- LLMs: They often get the right answer, but when they fail, their mistakes look different from humans. They might get confused by a slight change in wording that wouldn't trip up a person. They score lower here.

The Final Scoreboard

The authors combine these three scores to see who wins the "Cognitive Plausibility" title. They decided that Leg 1 (The Blueprint) is the most important because the main goal is to understand how humans think, not just to get the right answer.

The Winner (Under strict rules): CogSketch and SME. Even though they are "one-trick ponies" (low generality), they think like humans. They are the best models for studying the human mind.
The Runner-Up: METCL. It sits in the middle, doing some human-like things but not as comprehensively as the top two.
The "Magic" Box: LLMs (GPT-4, etc.). They are incredibly smart and versatile (high generality), but they don't think like humans. If you want a tool to solve problems, they are great. But if you want a model to explain why humans think the way they do, the paper says they are not the best fit.

The Big Takeaway

The paper concludes that being "smart" (getting the right answer) is different from being "cognitively plausible" (thinking like a human).

LLMs are like a super-actor who can memorize a script perfectly and sound exactly like a human, but they don't actually feel the emotions.
SME/CogSketch are like a method actor who actually feels the emotions and thinks through the character's logic, even if they can't memorize as many lines.

The authors built this grid to help scientists decide which computer program is the best "method actor" for studying the human mind. They found that while the "super-actors" (LLMs) are impressive, the "method actors" (SME/CogSketch) are still the gold standard for understanding the mechanics of human thought.

Technical Summary: Structural Ranking of the Cognitive Plausibility of Computational Models of Analogy and Metaphors with the Minimal Cognitive Grid

Problem Statement
The creation and interpretation of analogies and metaphors are foundational to human semantic competence. While numerous computational models have been proposed to replicate these capabilities, there is a lack of systematic, quantitative frameworks to evaluate their cognitive plausibility—specifically, the extent to which their internal mechanisms align with human cognitive processes rather than merely producing functionally equivalent outputs. Current Artificial Intelligence systems, particularly Large Language Models (LLMs), often achieve high performance on analogy tasks but operate via mechanisms (e.g., statistical pattern recognition) that differ fundamentally from human cognition. The paper addresses the need for a standardized metric to distinguish between "functional" artifacts (which mimic output) and "structural" models (which mimic internal cognitive mechanisms), thereby assessing their explanatory power in cognitive science.

Methodology: The Minimal Cognitive Grid (MCG)
The authors employ and formally operationalize the Minimal Cognitive Grid (MCG), a framework proposed by Lieto (2021) to rank artificial systems based on their structural accuracy relative to biological or cognitive targets. The methodology involves a quantitative assessment across three distinct dimensions:

Functional/Structural Ratio (FSR): This dimension evaluates the balance between functional replication (output matching) and structural implementation (mechanism matching). The authors define a metric, $FSRM$, based on a set of structural constraints derived from two primary cognitive theories:
- Structure-Mapping Theory (SMT): Constraints include one-to-one mapping, parallel connectivity, systematicity (preference for interconnected relational structures), and inferential projection.
- Categorization Theory of Metaphor (CTM): Constraints include categorization (interpreting metaphors as class-inclusion statements) and property selection (transferring salient features).
  Models are scored based on whether they implement these mechanisms structurally ( $s_i=1$ ) or merely functionally ( $s_i=0$ ), weighted by the theoretical importance of each constraint.
Generality (GM): This dimension assesses the breadth of cognitive functions a model can simulate, grounded in the Cattell-Horn-Carroll (CHC) taxonomy of human cognitive abilities. The authors evaluate models across five domains: Quantitative Knowledge, Fluid Reasoning, Visual Processing, Language/Verbal Knowledge, and Sensory/Motor Abilities. Scores are assigned based on full coverage (1), partial coverage (0.5), or absence (0), with a specific weighting scheme that emphasizes the importance of Sensory/Motor abilities in embodied cognition.
Performance Match (PMM): This dimension quantifies the alignment between model behavior and human behavior, considering not just accuracy but also error patterns and response times. The score is a weighted sum of:
- Accuracy Delta: The deviation from human baseline performance.
- Error Patterns: Whether the model replicates the distribution of human errors (e.g., failing on the same difficult items).
- Response Times: The similarity in execution latency compared to human reaction times.

Key Contributions

Formal Operationalization: The paper provides a rigorous mathematical operationalization of the MCG, transforming qualitative theoretical distinctions into quantitative scores ($FSRM$, $GM$, $PMM$) that allow for direct comparison of disparate AI architectures.
Unified Cognitive Plausibility Metric (CPM): The authors introduce a composite score, $CPM = \lambda \cdot FSR'_M + \mu \cdot GM + \nu \cdot PMM$ , which integrates the three dimensions. They propose a weighting scheme ( $\lambda=0.5, \mu=0.25, \nu=0.25$ ) that prioritizes structural fidelity, arguing that explanatory power in cognitive science depends primarily on the alignment of internal mechanisms with human theory.
Systematic Comparative Analysis: The framework is applied to four distinct classes of systems:
- SME (Structure-Mapping Engine) & CogSketch: Symbolic systems based on SMT.
- METCL: A logic-based model for metaphor generation based on CTM.
- LLMs: Including GPT-3.5, GPT-4, and GPT-4o.

Results
The evaluation yields distinct profiles for each system:

SME and CogSketch: Achieve the highest Structural Scores ( $FSR'_M \approx 0.606$ ) due to their explicit implementation of SMT constraints (one-to-one mapping, systematicity). They also demonstrate the highest Performance Match ( $PMM \approx 0.923$ ) on Raven's Progressive Matrices, replicating human accuracy and error patterns. However, they score very low on Generality ( $GM \approx 0.063–0.125$ ) as they lack capabilities in quantitative reasoning, language processing, and sensory-motor tasks.
METCL: Occupies an intermediate position. It scores moderately on Structural Fidelity ( $FSR'_M \approx 0.407$ ) by implementing CTM constraints (categorization, property selection) but lacks SMT-based structural constraints. Its Generality is limited ( $GM \approx 0.125$ ), and Performance Match is moderate ( $PMM \approx 0.714$ ).
LLMs: Exhibit the highest Generality ( $GM \approx 0.500–0.800$ ), covering quantitative, fluid, visual, and linguistic domains. However, they show very low Structural Fidelity ( $FSR'_M \approx 0.109$ ), failing to implement SMT constraints like systematicity or one-to-one mapping in a mechanistic sense. Their Performance Match is mixed ( $PMM \approx 0.578$ ), often showing high accuracy but divergent error patterns (e.g., failing on distractor-rich tasks where humans succeed) and lacking temporal alignment data.

Under the proposed non-equal weighting scheme (prioritizing structural plausibility), CogSketch and SME rank highest in overall cognitive plausibility. Under an equal weighting scheme, LLMs surpass METCL due to their superior generality, a result the authors describe as "paradoxical" given LLMs' lack of cognitively inspired constraints.

Significance and Claims
The paper claims that the MCG offers a "blueprint" for a generalizable, reproducible comparison of AI systems based on their biological or cognitive plausibility, rather than just their functional output.

Explanatory Power: The authors argue that the Functional/Structural Ratio is the critical dimension for cognitive science because it assesses whether a model's success is grounded in mechanisms relevant to the phenomenon (e.g., relational mapping) rather than statistical generalization.
Distinction from Behaviorism: The framework aims to avoid "renewed behaviorism" by penalizing systems that achieve human-level performance through non-human-like internal processes.
Scalability: The methodology is presented as a scalable tool for assessing current and future AI generations, providing a common measurement language across symbolic, sub-symbolic, and hybrid architectures.
Limitations: The authors modestly acknowledge limitations, including the uneven availability of benchmark data (LLMs are tested on more datasets than SME/CogSketch), the lack of response time data for some systems, and the fact that constraint weights are currently based on theoretical relevance rather than data-driven derivation. They suggest future work should focus on expanding datasets and refining the weighting schemes to further extend the framework's generalizability.