💬 NLP

Parallelograms Strike Back: LLMs Generate Better Analogies than People

This study demonstrates that large language models generate higher-quality four-term word analogies than humans by more consistently adhering to the geometric "parallelogram" relational structure, suggesting that the model's perceived failure in human cognition stems from human inconsistency rather than the model's inadequacy.

Original authors: Qiawen Ella Liu, Raja Marjieh, Jian-Qiao Zhu, Adele E. Goldberg, Thomas L. Griffiths

Published 2026-03-20

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Qiawen Ella Liu, Raja Marjieh, Jian-Qiao Zhu, Adele E. Goldberg, Thomas L. Griffiths

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are playing a word game called "Complete the Pattern." The game looks like this:

King is to Queen as Man is to ___?

Most people would instantly say "Woman." This is a classic analogy. For decades, scientists thought our brains solved these puzzles like a geometry teacher: they imagined words as dots on a map, and the relationship between them as a straight line (a vector). If you draw a line from "King" to "Queen," you should be able to draw the exact same line starting from "Man" to land on "Woman." This is called the Parallelogram Model.

However, a few years ago, researchers found that humans are actually terrible at following these perfect geometric lines. When asked to come up with the answers, people often just grab the first word that sounds similar to the third word (e.g., "Man" is similar to "Boy," so maybe the answer is "Boy?"). It seems like our brains are lazy shortcuts artists, not geometry masters.

Enter the "Robot" (LLMs)

The authors of this paper asked a fascinating question: Is the geometry model actually the right way to solve these puzzles, or are we just bad at doing it?

To find out, they pitted Humans against Large Language Models (AI) (like the smart chatbots you might use today). They gave both groups thousands of these word puzzles and asked a third group of humans to judge: "Which answer makes more sense?"

Here is what they discovered, explained through a few simple metaphors:

1. The Robots are Better at the "Geometry"

When the AI generated answers, human judges rated them as better than the answers humans gave.

The Metaphor: Imagine a game of darts. Humans are throwing darts while looking at the board with one eye closed; they often hit the wall or the floor. The AI, however, is throwing darts with a laser sight. The AI's answers fit the "perfect geometric shape" (the parallelogram) much more often than human answers do.
The Twist: The AI didn't necessarily "think" in geometry. It just happened to produce answers that looked like perfect geometry to us.

2. The "Long Tail" of Bad Human Answers

Why did the AI win? Was it because the AI was a genius and humans were stupid? No.

The Metaphor: Think of a bakery. Humans are like a chaotic kitchen where 80% of the time, the baker makes a perfect loaf of bread. But 20% of the time, they burn the bread, drop it on the floor, or serve a rock.
The AI, on the other hand, is like a robot baker that always makes a decent loaf. It rarely makes a masterpiece, but it almost never makes a disaster.
The Result: When you average all the answers, the AI wins because humans have a "long tail" of terrible, random answers dragging their average down. If you only compare the best answer humans gave against the best answer the AI gave, they are actually tied. The AI's advantage comes from being consistent, not from being a genius.

3. The "Clever Word" Factor

Humans tend to pick words that are very common and easy to think of (like "Boy" or "Child"). The AI, however, often picked slightly more obscure or "fancier" words (like "Immaturity" instead of "Youth").

The Metaphor: If you ask a human to describe a "sad feeling," they might say "sad." If you ask an AI, it might say "melancholy."
Humans actually prefer the AI's "fancier" words. We think the AI's answers are smarter because they use less common words that still fit the pattern perfectly.

4. The Big Conclusion: We Know What a Good Analogy Looks Like

The most surprising finding is about the "Parallelogram Model" itself.

Old Idea: "The parallelogram model is wrong because humans don't use it."
New Idea: "The parallelogram model is right about what makes a good analogy, but humans are just bad at making them."

The Takeaway:
We humans are like drivers who know exactly what a "perfectly parked car" looks like, but we are terrible at actually parking the car ourselves. We get distracted, we rush, and we hit the curb. The AI, however, is a self-driving car that parks perfectly every single time.

The study proves that the "geometric" way of thinking about analogies isn't broken; it's just that our messy human brains struggle to follow the rules, while the AI follows them effortlessly. We value the perfect pattern, even if we can't always create it.

1. Problem Statement

The paper addresses a fundamental debate in cognitive science regarding the nature of word analogies (e.g., $A:B::C:D$ ).

The Parallelogram Model: A classic geometric theory (Rumelhart & Abrahamson, 1973) posits that analogies are solved by vector arithmetic in a semantic space ( $D \approx C + (B - A)$ ).
The Challenge: Recent work (Peterson et al., 2020) suggested this model fails to capture human analogy generation, arguing that humans rely more on local similarity heuristics (e.g., choosing a word similar to $C$ ) rather than relational constraints.
The Question: Does the parallelogram model fail because it is an incorrect model of analogy, or because humans are cognitively limited and fail to produce relation-preserving analogies consistently? The authors investigate this by comparing human performance against Large Language Models (LLMs), which may have different cognitive constraints.

2. Methodology

Data and Stimuli

Dataset: The study utilized 846 four-term analogy stems from Peterson et al. (2020), sourced from Green et al. (2010), Kmiecik & Morrison (2013), and SemEval-2012 Task 2.
Models: Six state-of-the-art LLMs were tested:
- Closed-source: GPT-5-mini, o4-mini.
- Open-source: DeepSeek-V3.1-671B, Qwen3-235B, Qwen3-32B, Gemma-3-27B.
Generation Protocol: Each analogy was prompted with four semantically equivalent phrasings, repeated 10 times per prompt (temperature = 1.0), yielding 40 responses per model per analogy. This generated ~203,040 LLM responses.
Human Baseline: Compared against 26,265 human responses from the original Peterson et al. dataset.

Evaluation Metrics

Human Rating: 390 participants rated the relational similarity of $C:D$ to $A:B$ on a 7-point scale.
Geometric Modeling (GloVe): Responses were mapped to 300-dimensional GloVe embeddings. Three rules were tested to predict completions:
- Parallelogram: Rank by cosine similarity to $v(B) - v(A) + v(C)$ .
- Local Similarity ( $C:D$ ): Rank by similarity to $C$ alone.
- Nearest-Neighbor (NN): Rank by similarity to the closer of $B$ or $C$ .
Predictive Analysis: The authors used Cumulative Proportion Retrieved (CPR) to measure how well these rules captured the distribution of responses.
Regression Analysis: Multiple regression models were built to predict the rating difference (LLM vs. Human) using four predictors:
- $\Delta$ Parallelogram alignment.
- $\Delta$ Local similarity ( $C:D$ or NN).
- $\Delta$ Log word frequency (lexical accessibility).
Internal Representation Check: The authors extracted residual stream embeddings from open-source models to see if internal LLM representations aligned with the parallelogram model better than GloVe did.

3. Key Results

A. LLMs Generate "Better" Analogies

Rating Advantage: Human raters consistently judged LLM-generated analogies as higher quality (more relationally similar) than human-generated ones.
- GPT-5-mini showed the largest advantage ( $b = 0.18, p < .001$ ).
- Qwen3-32B was the only model not significantly better than humans.
Distributional Differences: The LLM advantage is driven by the long tail of human responses. Humans produced a wide variety of low-frequency, weak completions. LLMs were more concentrated (85% of LLM responses were modal vs. 64% for humans).
- Crucial Finding: When restricting the comparison to only modal (most frequent) responses, the LLM advantage disappeared. Humans and LLMs performed equally well on their most common answers.

B. The Parallelogram Model "Strikes Back"

Prediction Accuracy: While local similarity heuristics ( $C:D$ ) were the best overall predictors for both humans and LLMs (replicating Peterson et al.), the Parallelogram model performed significantly better for LLMs than for humans.
Gap Reduction: The performance gap between the parallelogram model and local similarity heuristics was more than twice as large for humans as it was for LLMs. This suggests LLMs adhere more strictly to relational constraints than humans do.

C. Drivers of Quality

Regression analysis revealed that two factors predicted why LLM analogies were rated higher:

Parallelogram Alignment: Higher alignment with the vector difference $(B-A)$ in GloVe space strongly predicted higher ratings ( $\beta \approx 0.20, p < .001$ ).
Lexical Accessibility: LLMs used lower-frequency (less accessible) words than humans. Counter-intuitively, these less common words were rated as better analogies.
- Note: Local similarity ( $C:D$ ) was actually a negative predictor; being too similar to $C$ without satisfying the relation lowered ratings.

D. Internal vs. External Representations

GloVe vs. Internal: Parallelogram alignment in the external GloVe space predicted human ratings better than the LLMs' own internal residual stream embeddings.
Implication: LLMs do not necessarily "think" in geometric vectors internally to solve these tasks. Instead, they generate outputs that happen to satisfy the relational constraints that humans value, even if those constraints are not explicitly encoded in the model's internal geometry.

4. Key Contributions

Re-evaluation of the Parallelogram Model: The paper challenges the conclusion that the parallelogram model is a poor account of analogy. Instead, it suggests the model is a valid account of what constitutes a good analogy, but humans often fail to execute it due to cognitive constraints (accessibility, retrieval failures).
LLM vs. Human Cognitive Profiles: It demonstrates that LLMs are more consistent in producing relation-preserving analogies, while humans exhibit a "long tail" of weak, locally-similar responses.
Decoupling Generation from Evaluation: The study shows that while LLMs generate better analogies, they do so without necessarily relying on internal geometric representations that mirror the external GloVe space.
Modal Response Analysis: It introduces a critical nuance that the "LLM advantage" is largely an artifact of human inconsistency (long tail) rather than LLM superiority in generating the best possible answer.

5. Significance

Cognitive Science: The findings suggest that the "failure" of the parallelogram model in human studies may be a failure of human production rather than a failure of the model itself. Humans prefer geometrically consistent analogies but struggle to generate them under cognitive load.
AI Evaluation: It highlights that LLMs can simulate high-quality relational reasoning even if their internal mechanisms differ from the geometric models used to evaluate them.
Future Directions: The paper suggests that studying AI systems can provide new insights into human cognition, specifically by isolating the "ideal" relational performance from the noise of human cognitive limitations.

In summary, the paper argues that LLMs generate better analogies because they consistently satisfy relational constraints (parallelograms) and use less accessible vocabulary, whereas humans often default to easier, locally similar words that fail the relational test. The parallelogram model remains a robust predictor of human judgment of quality, even if it is a poor predictor of human generation.