Leveraging Speech to Identify Signatures of Insight and Transfer in Problem Solving

Technical Summary: Leveraging Speech to Identify Signatures of Insight and Transfer in Problem Solving

Problem and Motivation
Insight-based problem solving is often characterized by sudden, non-obvious transitions in understanding, frequently accompanied by a subjective "Aha" experience. However, insight itself is not directly observable, and solving an insight problem does not guarantee a discrete moment of restructuring. While theoretical frameworks suggest that insight involves a shift from an initial problem representation to an alternative "problem space" (e.g., relaxing constraints or decomposing chunks), it remains unclear how these internal shifts manifest in observable behavior, particularly in how solvers approach similar problems in the future. This work investigates the distinguishing characteristics of speech produced during successful problem solving and how these verbal patterns evolve when a discovered solution structure can be transferred to subsequent, similar problems.

Methodology
The study employed a think-aloud protocol with 189 participants solving a sequence of five matchstick-arithmetic problems. In this paradigm, participants must transform an invalid Roman numeral equation into a valid one by moving a single matchstick.

• Experimental Design: Participants were randomly assigned to one of two between-subjects conditions:
  • Same Condition: All five problems shared the same underlying transformation type (e.g., changing a numeral or operator), allowing the discovered structure to be reused.
  • Different Condition: Each problem required a distinct transformation type, preventing the reuse of a specific structural insight.
• Procedure: Participants solved problems online while verbalizing their thoughts. Audio was recorded continuously. Trials were categorized into three phases relative to performance: pre-success (incorrect attempts), first-success (the first correct solution, serving as the operational marker for restructuring), and post-success (subsequent correct trials).
• Data Analysis Pipeline:
  • Performance: Accuracy and response times were analyzed using mixed-effects models.
  • Speech Output: Audio was segmented to calculate speech density (proportion of time speaking) and speech rate (words per second).
  • Semantic Content: Utterances were embedded using the text-embedding-3-large model. Logistic regression classifiers were trained to distinguish trials based on correctness, phase, and experimental condition.
  • Reasoning Moves: A large language model (GPT-5.1) annotated utterances into seven mutually exclusive categories: Proposal, Evaluation, Categorization, Restatement, Affect, Filler, and Meta & Other.

Key Results

1. Performance and Transfer:

   • Participants in the Same condition showed significantly greater improvement in both accuracy and speed across trials compared to the Different condition.
   • After the first success, participants in the Same condition solved subsequent problems with 83.7% accuracy and faster response times (mean 89.9s) compared to the Different condition (44.0% accuracy, mean 179.0s). This confirms that the discovered structure was successfully transferred only when problem types repeated.

2. Verbal Signatures of First Success:

   • Increased Speech: On the first-success trial, participants spoke more (higher speech density) and faster than on pre-success trials, regardless of the experimental condition.
   • Semantic Shift: Speech on correct trials was distinguishable from incorrect trials based on semantic content. Furthermore, speech on the first-success trial differed semantically from pre-success trials in both conditions.
   • Narrowed Search: At the moment of first success, participants considered fewer distinct candidate moves in both conditions, suggesting a narrowing of the search space.

3. Verbal Signatures of Transfer (Post-Success):

   • Condition Divergence: Crucially, changes in speech patterns after the first success were observed only in the Same condition.
   • Continued Speech Increase: In the Same condition, participants continued to speak more and faster on post-success trials compared to the first-success trial. In the Different condition, speech patterns stabilized after the first success.
   • Emergence of Categorization: Participants in the Same condition significantly increased the frequency of Categorization utterances (explicitly naming the problem type or noting similarity to prior problems) after the first success. This did not occur in the Different condition.
   • Refined Proposals: In the Same condition, participants further reduced the number of distinct candidate moves considered after the first success, whereas those in the Different condition did not.

Key Contributions

• Scalable Analysis of Insight: The study demonstrates the feasibility of using automated speech recognition and natural language processing to analyze think-aloud protocols at scale, moving beyond simple accuracy metrics to capture the dynamics of reasoning.
• Verbal Markers of Transfer: The paper identifies specific verbal signatures that distinguish mere performance improvement from genuine transfer. Specifically, the ability to spontaneously categorize a problem and the continued increase in speech density and rate are hallmarks of transferable insights.
• Accessibility of Insight: The findings suggest that while the precursors of insight may be difficult to articulate, the resulting structural understanding becomes increasingly accessible for verbal report as it is reused.

Significance and Claims
The authors claim that successful restructuring in problem solving does more than improve task performance; it fundamentally alters how people talk about a problem. The study posits that the hallmark of a transferable insight is its accessibility for verbal report. When a solver encounters a problem type that reuses a discovered structure, they not only solve it faster but also shift their reasoning strategy to include explicit categorization and a more focused search process.

The paper remains modest regarding its scope, acknowledging that matchstick arithmetic is a constrained paradigm and that the first success serves as a proxy for restructuring rather than a direct measure of subjective "Aha" moments. The authors suggest that future work could combine these speech analyses with subjective ratings of insight to further validate the link between verbal signatures and the internal experience of restructuring. Ultimately, the work argues that what people say while solving problems reveals patterns of success and transfer that performance data alone cannot capture.