Technical Summary: Semi-supervised Chain-of-Thought Learning (Semi-CoT)

1. Problem Definition

The paper addresses a gap in the current application of Chain-of-Thought (CoT) reasoning. While CoT has proven effective for eliciting reasoning capabilities in Large Language Models (LLMs) during inference, existing methods typically treat reasoning chains as transient prompts that are discarded after generating a final answer. Consequently, the vast amount of unlabeled data available in domains like mathematics and logic remains underutilized because high-quality CoT annotations are expensive to produce manually.

The authors define a new problem setting called Semi-supervised Chain-of-Thought Learning. The goal is to leverage a small set of labeled questions and a larger set of unlabeled questions to construct pseudo reasoning supervision. Unlike standard semi-supervised learning which generates pseudo-labels (final answers only), this setting requires generating a complete pseudo-supervision pair consisting of a pseudo reasoning chain ($\hat{r}_u$) and a pseudo answer ($\hat{y}_u$) for each unlabeled question ($x_u$):
$$x_u \rightarrow (\hat{r}_u, \hat{y}_u)$$

The core challenge is ensuring the reliability of these generated chains. A model may produce a correct answer via an invalid derivation, or an incorrect answer via a plausible-looking chain. Therefore, a mechanism is required to filter out unreliable pseudo-CoTs before they are used as supervision or demonstrations.

2. Methodology: Semi-CoT Framework

The authors propose Semi-CoT, a minimal framework that operates without fine-tuning model parameters. It functions as a prompt-level semi-supervised method designed to test whether unlabeled questions can provide reliable reasoning signals. The process involves three main stages:

A. Pseudo-CoT Generation

For each unlabeled question, the LLM samples $K$ distinct reasoning paths (CoT outputs) using a standard trigger (e.g., "Let's think step by step"). Each output yields a reasoning trace and a final answer.

B. Semantic Entropy Reliability Gate

To filter unreliable generations, the framework employs semantic entropy as a selection signal.

1. Normalization: Extracted answers are normalized (e.g., removing formatting symbols for numerical answers).
2. Clustering: Answers are grouped into semantic clusters.
3. Entropy Calculation: The normalized semantic entropy $H_{sem}(x_u)$ is computed based on the distribution of these clusters.
   • Low entropy indicates high agreement among the $K$ sampled paths.
   • High entropy indicates disagreement and uncertainty.
4. Selection: A pseudo-CoT is accepted only if $H_{sem}(x_u) \leq \delta$, where $\delta$ is a predefined threshold. The accepted pseudo-answer is the majority vote, and the pseudo-reasoning chain is selected from the paths leading to that majority answer.

C. Pseudo Reasoning Bank and Inference

Accepted pseudo-CoTs are stored in a pseudo reasoning bank $\mathcal{B}$, each tagged with a reliability weight ($w_u = 1 - H_{sem}(x_u)$). During inference on a test question:

• The system retrieves $M$ demonstrations from $\mathcal{B}$.
• Two retrieval strategies are tested: Random Selection and TF-IDF Retrieval (based on lexical similarity).
• The retrieved demonstrations are appended to the test question as a prompt, and the LLM generates the final reasoning and answer.

3. Key Contributions

The paper makes the following specific contributions:

• Problem Formulation: It defines Semi-supervised Chain-of-Thought Learning, shifting the view of CoT from a purely inference-time technique to a source of semi-supervised learning signals.
• Framework Proposal: It introduces Semi-CoT, a pipeline that generates multiple pseudo-CoTs, filters them using semantic entropy, and utilizes the high-confidence results as reasoning demonstrations.
• Empirical Evidence: It provides pilot evidence that entropy-based selection yields high-precision pseudo-CoTs across multiple arithmetic reasoning datasets.
• Limitation Identification: It identifies that pseudo-CoT reliability (ensured by entropy) and demonstration relevance (ensured by retrieval) are distinct challenges. High precision alone does not guarantee performance gains if the demonstrations are irrelevant to the test question.

4. Experimental Results

The authors conducted pilot experiments on four arithmetic reasoning benchmarks: AQuA, SVAMP, GSM8K, and MultiArith.

Pseudo-CoT Reliability

The semantic entropy gate successfully selected high-quality pseudo-CoTs. The pseudo-answer precision (accuracy of the selected pseudo-answers against hidden gold labels) was:

• MultiArith: 100%
• AQuA: 97.14%
• SVAMP: 96.30%
• GSM8K: 91.36%

Task Performance

The impact on final task accuracy was modest and varied by dataset:

• SVAMP: +2% improvement over Zero-shot CoT.
• GSM8K: +1% improvement over Zero-shot CoT.
• AQuA: Showed mild negative transfer (performance decreased compared to Zero-shot CoT), despite high pseudo-answer precision.
• MultiArith: All methods reached 100% accuracy, indicating a ceiling effect in the pilot setting.

Retrieval Strategy

Comparing Random Selection vs. TF-IDF Retrieval for demonstration selection showed no significant difference in the pilot subsets. This suggests that simple lexical similarity is insufficient to ensure the relevance of pseudo-CoTs to the test question.

5. Significance and Claims

The paper positions itself as a technical report establishing a new problem setting rather than a definitive solution. Its significance lies in:

1. Conceptual Shift: It reframes CoT as a form of self-training where model-generated reasoning can serve as pseudo-supervision, bridging the gap between inference-time prompting and semi-supervised learning.
2. Validation of Signals: It demonstrates that semantic entropy is a viable, model-agnostic signal for filtering reliable reasoning chains from unlabeled data, achieving high precision without human annotation.
3. Conservative Conclusion: The authors modestly conclude that while unlabeled questions can provide reliable pseudo reasoning signals, reliable selection is not sufficient for consistent performance improvement. The current implementation highlights that demonstration relevance is a critical, unresolved factor. Future work must address stronger retrieval mechanisms, reasoning-pattern matching, or parametric student training to fully realize the potential of Semi-CoT.

The paper explicitly avoids claiming that the current prompt-level implementation is a state-of-the-art solution, noting that the observed gains are small and that negative transfer remains a risk if relevance is not managed.