ATLAS: Constitution-Conditioned Latent Geometry and Redistribution Across Language Models and Neural Perturbation Data

1. Problem Statement

The paper addresses a fundamental gap in understanding post-training (specifically constitution-conditioned training) in Large Language Models (LLMs). While standard evaluation focuses on output shifts (e.g., refusal rates, benchmark scores), it remains unclear whether these training interventions leave behind a recoverable, reusable internal organization (latent geometry).

The core research question is: Does constitution-conditioned post-training realize a specific, local representational geometry in a source model that can be re-identified in a different, unadapted target model and corroborated in a biological substrate, even if the exact location (occupancy) and behavioral expression shift?

2. Methodology: The ATLAS Framework

The authors introduce ATLAS, a "geometry-first" program designed to trace constitution-induced hidden-state structures across models and substrates. The methodology relies on a fixed experimental arc involving three stages:

• Source (Gemma 3 1B IT): A model trained with constitution-conditioned post-training. The authors identify a "source-local chart" (a local representational organization) and define a broader "source-defined family" of latent structures.
• Target (Phi-4 Mini Instruct): An unadapted model (no adapter transfer). The frozen source-defined family is searched for within Phi's hidden states to see if a "target-local realisation" exists.
• Substrate (ALM8): Held-out mouse frontal-cortex perturbation data. The same frozen family is tested to see if it acts as a "held-out bridge realisation" in biological neural data.

Key Technical Concepts:

• Local Chart: A local representational organization defined by its tangent structure, occupancy distribution, and behavioral coupling.
• Source-Defined Family: The broader frozen export unit (a cluster of latent subspaces) anchored by the source chart, used for downstream testing.
• Redistribution: The core finding that the family recurs across systems, but its local coordinates, occupancy, and behavioral expression shift. It is a recurrence without exact identity.
• Geometry Contract: A strict set of metrics to validate findings:
  • Tangent/Chart Structure: Measured via basis angles and empirical tangent angles (e.g., basis_angle_mean_deg).
  • Occupancy: Measured via displacement norms (e.g., occ_w2_sq_norm).
  • Behavioral Coupling: Measured via AUC and mean score gaps on adjudicated datasets.

3. Key Contributions

1. Source-Local Chart Realization: Demonstrated that constitution-conditioned post-training on Gemma creates a recoverable, behavior-linked source-local chart.
2. Family-Level Export Unit: Established that the exportable unit is not a single "compact patch" but a broader source-defined family. While a single patch suffices for some behaviors, the family is required for robust downstream transfer.
3. Cross-Model Re-identification: Successfully re-identified the frozen Gemma-defined family in an unadapted Phi-4 model using a fully adjudicated confirmatory protocol.
4. Cross-Substrate Corroboration: Validated the same family in held-out ALM8 mouse frontal-cortex data, proving the geometry survives transfer to biological substrates.
5. Defining the Boundary (Redistribution): Proved that while the geometry recurs, it does so via redistribution (structural recurrence with shifted occupancy) rather than exact identity. The paper explicitly rules out "exact cross-system identity" and "target-side mediation theorems."

4. Key Results

A. Source-Side (Gemma)

• Chart Visibility: The anchored chart captured 310/320 reviewed source rows and 84/84 score-flip rows.
• Family vs. Patch: While the chart is visible, "compact exact-patch sufficiency" failed. The behavior is carried by a broader family (constitutions-v2-high_effective_mi), not a single isolated patch.
• Metric: Flip-minus-no-flip Grassmann chordal separation was +0.0385, indicating distinct movement for harmful vs. safe routes.

B. Target-Side (Phi-4 Mini)

• Re-identification: The frozen family was found in Phi's Layer 24 (reasoning lane).
• Performance: On a fully adjudicated confirmatory set (32 informative-constitution rows vs. 32 null-random-control rows):
  • AUC: 0.984 (95% CI [0.952, 1.000]).
  • Mean Gap: 5.50.
• Interpretation: The result shows strong structural correspondence but blocked occupancy support (the exact slot identity did not match). This confirms "family-level correspondence with redistribution."

C. Biological Corroboration (ALM8)

• Held-Out Validation: Tested across 5 held-out mouse folds.
• Performance:
  • Mean AUC: 0.72.
  • Mean Fold Gap: 4.50.
  • Success Rate: 5/5 folds were positive.
• Interpretation: The family recurs in biological data, but local expression shifts outward (e.g., from "late-reasoning" to "answer-heavy" spans), reinforcing the redistribution hypothesis.

D. Limiting Boundary (MCQ)

• In a Multiple-Choice Question (MCQ) task, a nearby target-local signal was found, but it failed the source-faithful closure test.
• Result: AUCs were negative (0.43/0.45), and the failure mode was displacement (centroid shift) rather than simple rotation. This proves that local localization alone is insufficient for the paper's main claim; structural closure is required.

5. Significance and Implications

• Beyond Output Metrics: The paper shifts the focus from "does the model behave differently?" to "does the model have a different internal geometry?" It proves that post-training leaves a mesoscopic family of local subspaces that are recoverable.
• Redistribution over Identity: The findings challenge the idea of exact "neural reuse" across models. Instead, constitutions induce a recoverable latent geometry that maintains its organizational logic (tangent structure) while its physical location (occupancy) and behavioral expression shift.
• Auditability: The framework suggests that frozen latent targets can be audited and perturbed across models. If a specific family is identified in a source model, it can be located in a target model to test for alignment or safety, even without fine-tuning.
• Biological Validity: The successful transfer to mouse neural data (ALM8) suggests that these "constitutions" (abstract rules) induce latent structures that are not just artifacts of transformer architecture but may reflect more general principles of representational organization found in biological systems.

Conclusion:
ATLAS demonstrates that constitution-conditioned post-training realizes a recoverable, behavior-linked latent geometry. However, this geometry does not transfer as an exact copy (identity) but as a redistributed family that preserves its structural core while adapting its local occupancy and expression to the new model or substrate. This provides a new, geometry-first lens for understanding model alignment, safety, and cross-system transfer.