💰 quantitative finance

Using Large Language Models for Idea Generation in Innovation

This study demonstrates that while AI-generated product ideas exhibit lower novelty and diversity compared to human-generated ones, they significantly outperform human ideas in average purchase intent and are seven times more likely to rank among the top 10% of high-quality concepts.

Original authors: Lennart Meincke, Karan Girotra, Gideon Nave, Christian Terwiesch, Karl T. Ulrich

Published 2026-07-31

📖 5 min read🧠 Deep dive

CC0 1.0

Original authors: Lennart Meincke, Karan Girotra, Gideon Nave, Christian Terwiesch, Karl T. Ulrich

Original paper dedicated to the public domain under CC0 1.0 (http://creativecommons.org/publicdomain/zero/1.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are standing in a vast, foggy field called "The Idea Landscape." Your goal is to find the single most valuable treasure hidden somewhere in this field. In the world of business and innovation, this treasure is a brilliant new product idea that people will love to buy. For a long time, the only way to find these treasures was to send out teams of human explorers. These humans would wander around, digging up ideas one by one. The rule of the game has always been that you don't need every single idea to be perfect; you just need to find a few absolute gems. If you dig up 100 ideas and 99 are rocks but one is a diamond, you've won. But what if you could hire a magical, super-fast robot that could dig up 1,000 ideas in the time it takes a human to dig up ten? That is the question scientists are asking about Artificial Intelligence (AI). Specifically, they are testing "Large Language Models" (LLMs)—super-smart computer programs trained on almost everything ever written on the internet. These programs are great at writing stories and answering questions, but can they be as creative as humans when it comes to inventing new things? This paper sets out to see if these digital dreamers can actually out-invent real people.

The researchers from top universities decided to put this to the test with a specific challenge: invent a new physical product for college students that costs $50 or less. They gathered three different groups of "idea generators." The first group was a team of university students who had taken a product design class before AI tools like ChatGPT were widely available. The second group was a computer program (OpenAI's GPT-4) asked to generate ideas with a simple instruction, like a blank canvas (this is called "zero-shot" prompting). The third group was the same computer program, but this time it was shown a few examples of great ideas first to help it get in the mood (this is called "few-shot" prompting).

To see who did the best, the researchers didn't just ask experts; they asked regular people. They showed hundreds of these product ideas to a large group of college-aged people and asked, "How likely are you to buy this?" They turned these answers into a "purchase score" to measure quality. They also used computer tools to check how similar the ideas were to each other and asked people to rate how "new" or "novel" the ideas felt.

Here is what they found, and it's a bit surprising. First, the AI was actually better at the main job: making ideas people wanted to buy. On average, the ideas generated by the computer were rated higher than the ideas made by the students. The computer that got a few examples to start with (few-shot) did the best of all. It's like the robot had a better "ear" for what customers wanted.

However, there was a catch. While the AI was better at making good ideas, it wasn't as good at making different ideas. The computer's ideas tended to look and sound very similar to each other. If you imagine the human students exploring the whole foggy field, the AI seemed to stay in one or two specific spots, digging up many variations of the same few treasures. The computer's ideas were also rated as slightly less "novel" or unique by the people reading them. It's as if the robot was very good at polishing existing concepts but struggled to think outside the box in the way humans sometimes do.

But here is the most exciting part. In innovation, the average idea doesn't matter as much as the best idea. The researchers looked at the top 10% of all the ideas they collected—the absolute best ones. They found that the AI was seven times more likely to produce an idea that landed in this top 10% compared to the humans. For every one great idea the students came up with, the computer came up with seven.

The authors suggest that this is actually a conservative estimate. They didn't even count how much faster the computer was. A human might take 15 minutes to come up with five ideas, while the computer could generate 200 ideas in that same time. So, the computer isn't just better at quality; it's a productivity monster.

The paper concludes that while AI might not be as wild or diverse as a human brainstorming session, it is a powerhouse for finding high-quality, marketable ideas. It suggests that in the future, companies might not need to spend so much time trying to come up with ideas from scratch. Instead, they could let the AI generate hundreds of high-quality options quickly, and then have humans focus on picking the best ones and making them even better. The robot isn't replacing the human dreamer, but it is giving them a super-powered shovel to dig up diamonds much faster.

Technical Summary: Using Large Language Models for Idea Generation in Innovation

Problem Statement
The paper addresses a fundamental question in innovation management: Can Large Language Models (LLMs), specifically OpenAI's GPT-4, effectively generate new product ideas that compete with or surpass human-generated ideas? While LLMs are known for producing coherent text, they are often criticized for "hallucinations" and inconsistencies. The authors propose that in the domain of creativity, where the value of a process is determined by the quality of the best idea rather than the average consistency of all outputs, the stochastic nature of LLMs might be a strength rather than a weakness. The study seeks to empirically test the hypothesis that LLMs can serve as superior ideators compared to humans, specifically within the context of generating physical product ideas for college students priced at $50 or less.

Methodology
The research employs a comparative experimental design involving three distinct pools of product ideas:

Human-Generated Pool: 200 ideas collected from university students in a product design course (conducted in 2021, prior to the widespread availability of LLMs).
Zero-Shot AI Pool: 100 ideas generated by GPT-4 (model gpt-4-0314) using a standard prompt without examples.
Few-Shot AI Pool: 100 ideas generated by GPT-4 using the same prompt but augmented with six examples of high-rated student ideas to facilitate in-context learning.

The experimental setup utilized a "temperature" parameter of 0.7 to maintain randomness. To overcome context window limitations, the authors implemented a summarization strategy to compress previously generated ideas, allowing the model to generate a continuous stream of 100 ideas without repeating itself due to memory constraints.

The evaluation of these ideas involved three distinct studies:

Study 1 (Quality Assessment): 400 ideas were evaluated by human respondents (college-aged individuals recruited via Prolific) using a standard five-box purchase intent survey. The responses were weighted to calculate a "purchase intent probability," serving as the primary metric for idea quality.
Study 2 (Diversity and Novelty):
- Flexibility/Overlap: Text mining was used to calculate the cosine similarity of text embeddings (using Google's Universal Sentence Encoder) between all pairs of ideas. Ideas with a similarity score above a threshold of $\theta = 0.8$ were classified as overlapping. A capture-recapture model was applied to estimate the total number of unique, discoverable ideas in the solution space for each pool.
- Originality: Human raters assessed the perceived novelty of the ideas on a 0–1 scale.
Study 3 (Top-Tier Performance): The authors analyzed the distribution of idea quality, specifically focusing on the top 10% (the 90th percentile) of ideas to determine which source produced the highest-quality outliers. Quantile regression was used to examine the relationship between idea source and quality across different percentiles.

Key Results
The study yields three primary findings:

Superior Average Quality: AI-generated ideas demonstrated significantly higher average purchase intent than human-generated ideas. The average purchase intent was 0.40 for humans, 0.46 for zero-shot AI, and 0.49 for few-shot AI. Statistical analysis confirmed that both AI conditions outperformed humans, with few-shot prompting yielding the highest average quality.
Reduced Novelty and Diversity: Despite higher average quality, AI-generated ideas were perceived as less novel (Human: 0.41; Zero-shot AI: 0.37; Few-shot AI: 0.36). Furthermore, text mining revealed that AI-generated ideas exhibited higher pairwise similarity. The fraction of overlapping ideas was 0% for humans, 5% for zero-shot AI, and 7% for few-shot AI. The capture-recapture model estimated that the human process could discover approximately 966 unique ideas (zero-shot) and 680 (few-shot), whereas the human pool contained only unique ideas within the sample, suggesting a broader solution landscape for human ideation.
Dominance in Top-Tier Ideas: The most significant finding relates to the quality of the best ideas. Among the top 10% of all ideas (the top 40 out of 400), 35 (87.5%) were generated by GPT-4. Specifically, AI-generated ideas were seven times more likely to appear in the top decile compared to human-generated ideas. Quantile regression confirmed that GPT-4's advantage is most pronounced in the upper tail of the quality distribution.

Key Contributions
The paper contributes to the innovation literature by:

Providing empirical evidence that LLMs can outperform humans in generating high-quality product ideas, challenging the notion that AI is merely a tool for consistency rather than creativity.
Demonstrating that while AI may reduce the diversity and novelty of the idea pool (leading to a less explored solution landscape), it significantly increases the probability of generating "exceptional" ideas.
Establishing a 7:1 advantage ratio for AI in generating top-tier ideas, suggesting that the value of AI in innovation lies in its ability to produce high-quality outliers rather than a high volume of average ideas.

Significance and Claims
The authors claim that their findings provide a "conservative estimate" of AI's potential, as the study did not account for the massive productivity advantage of LLMs (generating hundreds of ideas in minutes versus hours for humans). The paper argues that the traditional innovation model, which values the "best" idea over the "average" idea, aligns perfectly with the strengths of LLMs.

The significance of the work lies in its suggestion that the role of human ideators may shift from generation to evaluation and refinement. If LLMs can reliably produce a high volume of high-quality ideas at a fraction of the cost, organizations may need to reallocate resources toward sorting, ranking, and prioritizing these ideas rather than focusing solely on the initial ideation phase. The authors conclude that while human involvement remains crucial for ensuring novelty and driving the direction of ideation, AI represents a substantial and transformative benefit for new product development.

Technical Summary: Using Large Language Models for Idea Generation in Innovation

More like this