💬 NLP

COFT: Counterfactual-Conformal Decoding for Fair Chain-of-Thought Reasoning in Large Language Models

COFT is a training-free, decoding-time method that reduces societal biases in large language model chain-of-thought reasoning by combining counterfactual logit fusion with conformal calibration, achieving significant bias reduction while preserving task utility and requiring no model retraining.

Original authors: Arya Fayyazi, Mehdi Kamal, Massoud Pedram

Published 2026-06-01

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Arya Fayyazi, Mehdi Kamal, Massoud Pedram

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart, well-read librarian (the Large Language Model) who helps you write stories, answer questions, or solve problems. This librarian has read millions of books from the internet. While they are incredibly knowledgeable, they have also absorbed some of the messy, unfair stereotypes found in those books (like thinking certain jobs are only for men, or making assumptions about people based on their names).

When you ask the librarian to "think out loud" (Chain-of-Thought reasoning) before giving an answer, they might accidentally whisper these unfair stereotypes into their thinking process, even if the final answer looks okay.

COFT (Chain of Fair Thought) is a new "traffic cop" system that stands next to the librarian while they are thinking. It doesn't retrain the librarian or change their brain; it just watches what they are about to say and gently steers them toward fairer choices in real-time.

Here is how COFT works, broken down into three simple steps using a creative analogy:

The Analogy: The "Double-Check" Kitchen

Imagine the librarian is a chef preparing a complex dish (the answer). Sometimes, the recipe calls for an ingredient that represents a sensitive topic (like a specific nationality or gender). If the chef uses that ingredient, the dish might taste biased.

COFT acts like a twin chef working in a parallel kitchen.

Step 1: The "Blindfold" Test (Counterfactual Masking)

Before the main chef writes down the next word of the recipe, COFT creates a "masked" version of the prompt.

Real World: The prompt says, "The nurse (who is a woman) finished her rounds..."
COFT's Masked Version: It replaces the sensitive word "woman" with a neutral placeholder like [MASK]. So it becomes, "The nurse (who is [MASK]) finished her rounds..."

The system now runs the librarian's brain twice: once with the real word, and once with the masked word. This is like asking the chef, "If I didn't know the gender, would you still choose this next ingredient?"

Step 2: The "Blender" (Logit Fusion)

The system takes the two lists of potential next words (one from the real prompt, one from the masked prompt) and blends them together.

If the real prompt strongly suggests a biased word (e.g., "nurse" + "she"), but the masked prompt suggests a neutral word, the "blender" mixes them.
This creates a compromise list where the unfair, biased options are dialed down, and the neutral options are boosted. It's like mixing a strong, spicy sauce with a mild one to get a flavor that isn't too extreme.

Step 3: The "Safety Gate" (Conformal Filtering)

This is the most unique part. COFT doesn't just guess; it uses a mathematical "safety gate" called Conformal Prediction.

Imagine a security guard at the door of the kitchen. This guard has a pre-calculated rule: "Only let ingredients through if both the real chef and the masked chef agree they are safe."
If a word is only popular in the biased version but unpopular in the neutral version, the guard blocks it.
If a word is popular in both, it gets the green light.

This gives a statistical guarantee: COFT can promise, "We are 95% sure that the word we just let through is fair, regardless of the sensitive details in the prompt."

Why is this a big deal?

No Brain Surgery: Most ways to fix bias require "retraining" the model, which is like sending the librarian back to school for years to unlearn bad habits. COFT is training-free. It works on the model exactly as it is, right now.
No Extra Brains: Some methods require hiring a second AI (a classifier) to check for bias. COFT doesn't need a second AI; it just uses the librarian's own "twin" (the masked version) to do the checking.
It Keeps the Good Stuff: The paper shows that while COFT cuts out the bias (reducing it by about 30–55%), it doesn't ruin the quality of the answers. The librarian still solves math problems and writes good stories just as well as before.
It's Auditable: Because COFT makes a clear, step-by-step decision on every single word, you can look at the logs and see exactly why a word was chosen or rejected. It's not a "black box."

The Cost

The only downside is speed. Because COFT has to run the model twice (once for the real prompt, once for the masked prompt) and then blend the results, it takes a little more time—roughly equivalent to adding one extra step to the cooking process (about 10% slower). However, the paper argues this small cost is worth it for the safety and fairness it provides.

In Summary

COFT is a real-time fairness filter that sits between you and the AI. It asks the AI to imagine a world where sensitive details (like race or gender) are hidden, compares that imagination to reality, and only allows the AI to speak words that make sense in both worlds. It ensures the AI's "thinking process" stays fair without needing to retrain the AI or hire extra helpers.

Technical Summary: COFT (Counterfactual-Conformal Decoding for Fair Chain-of-Thought Reasoning)

1. Problem Statement

Large Language Models (LLMs) trained on web corpora internalize and often amplify societal biases, particularly during Chain-of-Thought (CoT) generation. While the final output of a model may appear neutral, the intermediate reasoning steps can reveal harmful associations token-by-token. Existing mitigation strategies face significant limitations:

Training-time methods (e.g., fine-tuning, RLHF, data curation) are computationally expensive, require access to model weights, and risk degrading general-domain performance or encoding annotator biases.
Inference-time steering (e.g., using auxiliary classifiers or expert ensembles) introduces latency, relies on the blind spots of external classifiers, and lacks rigorous statistical guarantees.
Representation-space debiasing (e.g., nullspace projection) often applies global, prompt-agnostic corrections that may suppress legitimate context-dependent semantics and fail to capture non-linear bias manifestations.

Crucially, current approaches lack per-step statistical guarantees at decode time. They do not specify which tokens can be safely emitted while keeping the risk of bias under control, nor do they enforce local counterfactual parity (stability of output under hypothetical changes to sensitive spans in the prompt) at every decoding step.

2. Methodology: COFT

COFT (Chain of Fair Thought) is a training-free, inference-only decoding framework designed to enforce counterfactual fairness for frozen causal language models. It operates in three distinct stages:

Stage I: Counterfactual Masking

COFT constructs a "masked" version of the input prompt, $\hat{p}$ , by replacing sensitive spans (e.g., gender, race, religious identifiers) with neutral sentinel tokens (e.g., [MASK]).

Implementation: The masking operator is deterministic and length-preserving. If a sensitive span tokenizes into $k$ tokens, it is replaced by $k$ sentinel tokens. This ensures the factual and masked branches remain perfectly aligned in absolute position, allowing for valid token-level comparisons of logits at identical autoregressive prefixes.
Dual-Branch Execution: The model $f_\theta$ is run twice in parallel: once on the factual prompt $p$ and once on the masked prompt $\hat{p}$ , conditioned on the same generated prefix $w_{<t}$ .

Stage II: Counterfactual Logit Fusion

To attenuate attribute-driven biases, COFT performs a convex interpolation of the logit distributions from the factual ( $z^F_t$ ) and masked ( $z^{CF}_t$ ) branches:
$\tilde{z}_t = (1 - \lambda) z^F_t + \lambda z^{CF}_t$
where $\lambda \in [0, 1]$ is a tunable fusion parameter.

Effect: This operation creates a geometric blend of the two probability distributions. Theoretically, it linearly interpolates the log-odds between tokens, effectively "blinding" the model to sensitive attributes while preserving the high-probability regions of the distribution. This fusion occurs before certification to ensure the subsequent filtering step operates on distributions that already agree on their high-mass regions.

Stage III: Dual-Branch Split-Conformal Filtering

COFT applies a split-conformal prediction (CP) procedure to certify a set of candidate tokens at each step $t$ .

Nonconformity Score: A score is defined for each token $v$ based on the minimum probability it holds in both the fused factual distribution ( $\tilde{\pi}_t$ ) and the masked distribution ( $\pi^{CF}_t$ ):
$s_t(v) = 1 - \min\{ \tilde{\pi}_t(v), \pi^{CF}_t(v) \}$
Calibration: Offline, on a disjoint calibration set, the $(1-\alpha)$ -quantile $q_t$ of these scores is computed.
Certification: At test time, a token is admitted to the candidate set $C_t$ only if its score is below the threshold ( $s_t(v) \le q_t$ ), which is equivalent to requiring the token to have sufficient probability in both the factual and masked views simultaneously.
Sampling: The final token is sampled from the fused distribution $\tilde{\pi}_t$ restricted to the certified set $C_t$ . If $C_t$ is empty (a rare event), a deterministic fallback (e.g., argmax) is used.

3. Key Contributions

Training-Free Fair Decoding: COFT provides a mechanism to enforce fairness on frozen LMs without retraining, gradient access, or auxiliary classifiers.
Per-Step Marginal Guarantees: Unlike previous methods, COFT offers distribution-free, marginal validity guarantees (under the assumption of exchangeability) for the stability of emitted tokens. It certifies that the true next token is included in the candidate set with probability at least $1-\alpha$ .
Local Counterfactual Parity: The method operationalizes counterfactual fairness at the token level, ensuring that the decoding process remains stable when sensitive attributes in the prompt are hypothetically altered.
Dual-Branch Architecture: By fusing logits and requiring joint support from both factual and masked views, COFT effectively attenuates bias while preserving task utility.

4. Experimental Results

The authors evaluated COFT across six models (including LLaMA-2, Mistral, Mixtral, and Qwen2) and multiple bias benchmarks (StereoSet, CrowS-Pairs, BBQ, BOLD, Utrecht, COMPAS) alongside standard utility tasks (GSM8K, StrategyQA, ARC-easy, PIQA).

Bias Mitigation: COFT reduced standard bias metrics by 30–55% (median 38%) compared to vanilla decoding and outperformed strong inference-time baselines (e.g., Self-Debiased Decoding, DExperts, Dual-Threshold Conformal Decoding). For instance, on the BBQ dataset, bias reduction reached 34–41%.
Utility Preservation: Task accuracy on reasoning and QA benchmarks remained unchanged within run-to-run noise margins (differences $\le 0.2$ points). Language quality metrics (Perplexity, MAUVE) were indistinguishable from vanilla decoding.
Efficiency: The computational overhead is modest, equivalent to one additional cached forward pass per step. Throughput is reduced by approximately 10–25%, and peak memory usage increases negligibly ( $\le 0.8$ GB).
Ablations: Removing logit fusion resulted in higher residual bias, confirming that fusion is the primary mechanism for attenuating attribute-driven log-odds. Removing the dual-branch conformal filter resulted in a loss of counterfactual stability guarantees.

5. Significance and Claims

The paper positions COFT as a clear, auditable path to safer CoT generation. Its primary significance lies in providing verifiable safety bounds for frozen models without the prohibitive compute costs of retraining.

Auditable Safety: By providing per-step marginal guarantees, COFT allows practitioners to deploy models in sensitive domains (e.g., healthcare, legal, hiring) with a quantified measure of risk.
Robustness: The method is robust to distribution shifts (with coverage degrading gracefully based on density ratios) and maintains performance across diverse model architectures.
Limitations: The authors modestly note that algorithmic interventions are not a panacea. The guarantees rely on the specific definition of sensitive attributes and the assumption of exchangeability between calibration and test data. Furthermore, the dual-branch decoding introduces inference latency, which must be weighed against the benefits of bias reduction. The method is most appropriate for scenarios where the reasoning trace itself must be auditable.

In summary, COFT demonstrates that it is possible to significantly reduce societal biases in LLM reasoning traces while preserving utility, using a lightweight, training-free procedure grounded in conformal prediction theory.