Anchorless Diversification for Parallel LLM Ideation

Technical Summary: Anchorless Diversification for Parallel LLM Ideation

Problem Statement
Large Language Models (LLMs) are increasingly deployed to generate pools of candidate ideas for creative tasks (e.g., story premises, product concepts, slogans). While parallel inference is attractive for scaling these pools without increasing model parameters, a critical failure mode exists: contemporary LLMs often concentrate probability mass around familiar patterns due to pretraining and preference optimization. This leads to "semantic collapse," where parallel generations cluster in the same semantic basins, creating an illusion of exploration while actually narrowing the available space for downstream human-AI co-creation.

The core challenge is to diversify these candidate pools effectively without sacrificing output quality or incurring prohibitive inference costs. Existing methods often rely on "anchored" strategies—using observed seed ideas, shared state, or iterative regeneration—which introduce hidden costs (seed generation, selection, feedback loops) that are frequently excluded from cost comparisons. This paper investigates whether "anchorless" methods—those that do not depend on observed seed ideas or sequential coordination—can rival anchored baselines in achieving diversity while maintaining the efficiency of parallel generation.

Methodology
The authors evaluate three creative task families: short stories (4 prompts), Alternative Uses Tasks (AUT, 5 objects), and slogans (3 prompts). They test three frontier models: GPT-5.4, Claude Sonnet 4.6, and Gemini 2.5 Pro.

The study compares two single-stage (anchorless) methods against four two-stage (anchored) methods under two instruction strategies:

1. Neutral: "Make the response novel and appropriate for the task."
2. Diverge: Adds a population-referential instruction: "Try to make it stand out from other responses that might be generated for this same task."

Generation Methods:

• Anchorless:
  • indep: Independent generation from the prompt alone.
  • strat (Semantic Direction Stratification): A single planning call asks the model to propose five broad semantic directions for the task. Subsequent generations are then distributed evenly across these directions without seeing seed examples.
• Anchored (Two-Stage):
  • repr: Generates a seed pool, selects three representative anchors (medoid + farthest-first), and regenerates candidates conditioned on avoiding these anchors.
  • self: Regenerates each candidate conditioned on avoiding its own seed output.
  • peer1/peer2: Regenerates candidates conditioned on avoiding their own seed output plus one or two peer outputs from the seed pool.

Evaluation Metrics:

• Diversity: Mean pairwise semantic distance ($D_{pair}$), nearest-neighbor distance, medoid distance, MST dispersion, and normalized semantic-region entropy ($D_{ent}$).
• Quality: Task-specific automatic proxies (MAoSS for stories, CLAUS for AUT, and a phrase-level non-template metric for slogans), standardized within task.
• Cost: Full-pipeline token accounting, including planning calls, seed generation, and regeneration steps.
• Analysis: Pool-size rarefaction (AUC and first-hit metrics) to measure how quickly diversity accumulates as a pool is sampled.

Key Results

1. Divergent Instructions as a Low-Cost Baseline: Applying the "diverge" instruction to independent generation (indep-diverge) significantly improves semantic diversity across all models and tasks while preserving or slightly improving quality proxies. It incurs minimal overhead ($R_{tok} \approx 1.1$), making it a strong, low-cost baseline.
2. Semantic Direction Stratification: The strat-diverge method (planning + divergent instruction) achieves the highest tier of diversity, often matching or exceeding the strongest anchored methods (like peer2-diverge). Crucially, it does so with a single planning call and parallel generation, avoiding the heavy token costs of seed-and-regenerate pipelines.
3. Token Efficiency: When accounting for full-pipeline costs, strat-diverge yields the best diversity–quality–compute frontier. For example, on GPT-5.4, strat-diverge achieves a diversity gain per 100k tokens (0.295) that significantly outperforms peer2-diverge (0.157), despite peer2 having a similar raw diversity gain.
4. Rarefaction Dynamics: While local anchor methods (self, peer) accumulate semantic-region entropy slightly faster than strat-diverge once the seed pool is generated, strat-diverge reaches the target diversity level with far fewer total tokens because it eliminates the seed-generation phase.
5. Robustness: Results hold across different embedding models, quality metrics (including LLM-as-a-judge for slogans), and task families.

Significance and Claims
The paper claims to establish practical, anchorless baselines for open-ended LLM ideation. It argues that:

• Direct divergent instructions are a powerful, low-cost intervention that leverages a model's implicit knowledge of likely responses to avoid redundancy without observing a seed pool.
• Semantic direction stratification allows models to externalize a "coarse map" of the response space, converting latent task variation into an explicit allocation structure. This enables broad parallel exploration without the overhead of stateful coordination or seed generation.
• Full-pipeline accounting is essential for evaluating diversification methods. While anchored regeneration remains powerful, its advantage shrinks when the costs of seeding and regeneration are included.
• These findings support an "autostratification" view of LLM ideation, where models can be steered toward broader candidate pools through inference-time controls that preserve the appeal of parallel generation, potentially mitigating AI-induced idea diversity collapse in downstream applications like scientific discovery or drug design.

The authors remain modest, noting limitations in relying on automatic quality proxies, the sensitivity of embedding-based metrics, and the need for future work to test these methods across longer-form outputs, stricter factual constraints, and multimodal tasks.