Position: Logical Soundness is not a Reliable Criterion for Neurosymbolic Fact-Checking with LLMs

1. Problem Statement

The paper addresses a critical limitation in current neurosymbolic fact-checking systems that integrate Large Language Models (LLMs) with formal logic.

• Current Paradigm: Existing approaches often use LLMs to translate natural language claims into formal logical formulae (e.g., First-Order Logic). These formulae are then verified against a trusted knowledge base using formal inference rules. The underlying assumption is that if a claim is logically sound (validly derived from true premises), it is factually supported and acceptable.
• The Gap: The authors argue that this assumption is flawed because logical soundness does not equate to factual reliability in the context of human communication.
• Core Issue: There is a systematic divergence between formal logical operations and human pragmatic inference. A conclusion can be logically valid based on strict rules but still be misleading because it triggers human inferences (implicatures) that are unsupported by the original premises. Current logic-based systems fail to detect these "technically true but misleading" statements.

2. Methodology and Theoretical Framework

The paper does not propose a new experimental dataset or a specific algorithmic implementation. Instead, it employs a theoretical and analytical approach grounded in cognitive science and linguistics to deconstruct the limitations of logic-based verification.

• Cognitive Science Basis: The authors draw on the Theory of Mental Models (Johnson-Laird) and Gricean Pragmatics (Grice, Levinson). They posit that humans do not process language as strict logical operators; instead, they infer meaning based on context, conversational maxims (e.g., Quantity, Quality), and default assumptions.
• Typology of Divergence: The authors construct a typology of cases where formal logic yields valid conclusions that induce ungrounded human inferences. They analyze specific logical operations and contrast the formal output with the "human inference" (the implied meaning).
• Counter-Argument Analysis: The paper systematically evaluates and refutes alternative views, such as:
  • Adding an auxiliary criterion to logical soundness.
  • Ignoring compound statements (assuming malicious actors only use simple assertions).
  • Relying on human users to inspect logic traces.

3. Key Contributions

A. Theoretical Refutation of Logical Soundness as a Sole Criterion

The primary contribution is the argument that logical soundness is an insufficient metric for fact-checking. The authors demonstrate that a claim can be:

1. Factually True: Derived from true premises via valid rules.
2. Logically Sound: Valid within the formal system.
3. Misleading: Because it implies a falsehood to a human reader due to pragmatic interpretation.

B. Typology of Misleading Logical Derivations

The paper provides a concrete table (Table 1) illustrating how specific logical rules generate misleading human inferences:

| Logical Operation | Formal Derivation | Example Claim | Misleading Human Inference (Implicature) | Cognitive/Pragmatic Basis |
| :--- | :--- | :--- | :--- :--- |
| Disjunction Introduction ($A \vdash A \lor B$) | "Tariffs will go up 10% or 100%." | Implies: "It is possible tariffs will go up 100%." | Humans interpret "A or B" as a conjunction of default possibilities, inferring $B$ is a real possibility even if only $A$ is true. | Mental Models Theory |
| Conjunction Introduction ($A \vdash A \land A$) | "Tariffs will go up 10%, and 10%." | Implies: "Tariffs will go up 10% twice." | Humans interpret repetition as distinct events (Gricean Maxim of Quantity) and temporal sequence (Conjunction Buttressing). | Gricean Maxims / Levinson |
| Material Implication (Positive Paradox) | "If rates don't drop, tariffs rise." | Implies: "If rates do drop, tariffs won't rise." | Humans interpret conditionals as biconditionals ("If and only if"), a phenomenon known as Conditional Perfection. | Pragmatic Strengthening |
| Vacuous Truth | "If tariffs don't rise, market floods." | Implies: "If tariffs do rise, market won't flood." | Same as above; the negation of the antecedent is incorrectly inferred to negate the consequent. | Conditional Perfection |
| Modal Axioms | "It is possible X." | Implies: "It is possible NOT X." | Humans infer uncertainty or alternative possibilities from statements of possibility, violating strict modal logic. | Gricean Maxim of Quality |

C. Proposed Paradigm Shift: LLMs as Features, Not Bugs

The authors advocate for a fundamental shift in how LLMs are utilized in neurosymbolic systems:

• Current View: LLMs' tendency to make human-like logical errors is a "bug" to be fixed by forcing alignment with formal logic.
• Proposed View: These tendencies are a feature. LLMs should be leveraged to model human pragmatic inference.
• New Architecture: Instead of using LLMs solely to translate text to logic, LLMs should be used to validate the outputs of formal components. The LLM acts as a "pragmatic filter" to detect if a logically sound conclusion would mislead a human reader.

4. Results and Findings

• Systematic Failure of Logic-Only Systems: The analysis confirms that logic-based fact-checkers will systematically pass misleading claims (like the "10% or 100%" tariff example) because the claims are formally valid.
• Human Inference Divergence: The paper establishes that human reasoning relies heavily on implicature (what is suggested) rather than just entailment (what is strictly proven).
• Training Implications: Optimizing LLMs to strictly conform to formal logic (e.g., via logic-conforming training data or reward models) may actually degrade their ability to detect misinformation, as it suppresses the very pragmatic sensitivities needed to spot misleading implications.

5. Significance and Implications

• Redefining Fact-Checking: The paper challenges the field to move beyond "truth" (factual accuracy) to "reliability" (resistance to misleading interpretation). A fact-checking system must detect half-truths and implicatures, not just falsehoods.
• Neurosymbolic Design: It suggests that future neurosymbolic systems should be hybrid: using formal logic for rigorous verification of atomic facts, but relying on LLMs' "human-like" reasoning capabilities to audit the communicative intent and pragmatic impact of those facts.
• LLM Evaluation: It calls for a re-evaluation of how LLM reasoning is benchmarked. Errors that look like logical failures (e.g., inferring a biconditional from a conditional) should be studied as predictive features of human cognition rather than just performance deficits.

Conclusion

The paper concludes that logical soundness is a necessary but insufficient condition for fact-checking. To effectively combat misinformation, systems must account for the gap between formal logic and human pragmatics. The authors propose that LLMs should not be forced to abandon their human-like reasoning patterns but should instead be harnessed to bridge this gap, acting as a complementary layer to formal logic to detect misleading conclusions that strict logic misses.