MAT-3: A Bounded Averaged-Projection Operator for Pre-Inference Cognitive-Affective Guidance

Technical Summary: MAT-3 – A Bounded Averaged-Projection Operator for Pre-Inference Cognitive-Affective Guidance

Problem Statement
Current methods for influencing the affective behavior of Large Language Models (LLMs) typically rely on weight adaptation (fine-tuning), prompting, or direct manipulation of internal activations. These approaches often lack explicit bounds, are difficult to audit before inference, or alter the model's internal state in ways that are not easily reversible or verifiable. The paper addresses a specific, narrower question: Can a local geometric correction be defined outside a target model's weights, bounded explicitly, audited prior to inference, and selectively withheld when its geometric basis is unreliable?

Methodology: The MAT-3 Operator
The proposed solution, MAT-3, is an external, pre-inference operator that constructs a "complementary vector" ($z_{aux}$) to accompany an original input ($x$) and an auxiliary embedding ($z$). It does not modify the target model's weights or directly edit internal activations. Instead, it operates on the representation geometry of the embedding space.

1. Local Geometry Estimation:

   • Given an embedding $z$, the method identifies $k$ nearest reference vectors to form a local neighborhood.
   • It computes a local mean ($\mu_z$) and an unbiased sample covariance matrix ($C_z$), regularized by a parameter $\lambda_{cov}$.
   • Using the top $r$ eigenvectors of the regularized covariance, it defines a local affine set $\mathcal{A}_z$ and an orthogonal projector $P_z$.

2. Averaged Projection:

   • The operator calculates the transverse component of the input relative to the local affine set: $a_{\perp} = (I - P_z)(z - \mu_z)$.
   • It applies an averaged projection to reduce this transverse component. The update rule is:
     $$z_{aux} = z - \eta a_{\perp}$$
   • The effective weight $\eta$ is determined by a nominal control weight $\eta_0$ (derived from $\lambda_{ctrl}$) and a hard intervention radius $\epsilon$. Specifically, $\eta = \min(\eta_0, \epsilon / \|a_{\perp}\|)$. This ensures the correction $\|z_{aux} - z\|$ never exceeds $\epsilon$.

3. MDEI State Integration:

   • The system utilizes the MDEI (Internal State Dynamics Model), which maps embeddings to a 3D state space ($c, \iota, \tau$) representing cognitive-valence, intensity, and cognitive-tension.
   • This state $u$ dynamically calibrates the intervention radius $\epsilon$ and control weight $\lambda_{ctrl}$, linking the geometric intervention to a dynamical systems framework.
   • Stability is analyzed using coercive Lyapunov functions, ensuring that under specific conditions, the state dynamics remain bounded and converge to absorbing sets.

4. Safe Abstention Mechanism:

   • MAT-3 includes a rigorous set of validity checks (Table 1). If any condition fails—such as insufficient neighborhood size, poor matrix conditioning, insufficient eigengap, out-of-distribution (OOD) distance, or violation of angular/semantic bounds—the operator abstains.
   • In cases of abstention, $z_{aux}$ is set to the original $z$, ensuring no unverified correction is applied.

Key Theoretical Contributions

• Boundedness: The operator guarantees that the magnitude of the intervention is strictly bounded by $\epsilon$ (Theorem 1).
• Transverse Contraction: Under frozen geometry, MAT-3 is non-expansive in the full space but strictly contractive in transverse directions. The distance to the affine set decreases exactly at a rate of $(1-\eta)^n$ (Theorem 2).
• Adaptive Stability: A sufficient condition ($\epsilon_{adapt} < \eta$) is provided to preserve transverse contraction even when the local geometry adapts to the input, though this relies on controlling the variation of the projector.
• Semantic Guarantees: A local semantic bound is established, ensuring that if the semantic map is Lipschitz continuous, the semantic distance change is bounded by the intervention radius.

Results and Validation
The paper explicitly states that it does not report downstream LLM accuracy, emotional competence, safety improvements, or reasoning benefits. The validation is strictly algebraic and synthetic:

• Synthetic Setup: A 2D spiral manifold embedded in $\mathbb{R}^{10}$ with Gaussian noise was used.
• Performance: Out of 2,000 queries, 1,967 passed all validity checks. The intervention radius $\epsilon$ was respected with floating-point precision.
• Convergence: The observed transverse distance reduction matched the theoretical $(1-\eta)^n$ rate with a maximum error of $3.89 \times 10^{-16}$.
• Unit Tests: The implementation passed tests for projector symmetry, idempotency, orthogonal equivariance, and the specific abstention logic for eigengap and OOD failures.

Significance and Claims
The paper positions MAT-3 as a mathematically explicit, auditable mechanism for pre-inference guidance. Its significance lies in:

1. Separation of Concerns: It separates covariance regularization from intervention cost and operates externally to the model weights.
2. Safety via Abstention: It introduces "safe abstention" as a core feature, preventing corrections when the local geometric basis is unreliable (e.g., OOD inputs or ill-conditioned neighborhoods).
3. Theoretical Rigor: It provides exact convergence rates and stability conditions under frozen geometry, distinguishing between what is proven (boundedness, transverse contraction) and what remains empirical (downstream utility).

The author concludes that while the operator's algebraic properties and implementation are validated, the translation of these properties into improved language-model behavior remains an open empirical question. The next necessary step is a downstream evaluation on frozen LLMs (e.g., Llama, Mistral) comparing MAT-3 against baselines, fine-tuning, and other intervention methods, with a focus on metrics like Macro-F1, calibration, and abstention rates.