Semantic Structure of Feature Space in Large Language Models

1. Problem Statement

While recent research has established that specific semantic features (e.g., deception, refusal, sycophancy) in Large Language Models (LLMs) are encoded as linear directions in hidden states, the prevailing approach treats these features in isolation. This perspective assumes features are orthogonal (independent). However, psychological research (specifically the "semantic differential" tradition) suggests that human semantic associations are highly intercorrelated and can be reduced to a low-dimensional subspace (e.g., Evaluation, Potency, Activity).

The paper addresses a critical gap: Do the geometric relations between semantic features in LLMs mirror human psychological associations? Specifically, it investigates whether LLM feature spaces are composed of independent axes or if they exhibit meaningful non-orthogonality (entanglement) that reflects human cultural and cognitive structures.

2. Methodology

The study employs a multi-step methodology combining survey data, activation analysis, and intervention experiments across Llama 3 models (3B and 70B) and Qwen3 models.

A. Data Construction

• Human Baseline: A survey of 1,750 respondents rated 360 words on 32 semantic scales (e.g., beautiful-ugly, soft-hard). This dataset serves as the ground truth for human semantic associations.
• LLM Feature Vectors (Words): For each of the 360 words, feature vectors were extracted from the LLM's hidden states.
  • Protocol: Four prompts were used per word (e.g., "Tell me about {word}", "Define {word}").
  • Extraction: Activations were collected from specific layers (Layer 8 for 3B, Layer 40 for 70B) and averaged across tokens and prompts to create a robust single vector per word.
• LLM Semantic Axes: 32 semantic axes were constructed using a contrastive pair design.
  • For each scale (e.g., good-bad), 10 antonym pairs were selected.
  • Feature vectors for each antonym were generated, and the axis vector was calculated as the mean difference between positive and negative antonym vectors:
    $$\vec{f}_{axis} = \frac{1}{10} \sum_{j=1}^{10} (\vec{v}_{pos_j} - \vec{v}_{neg_j})$$

B. Analytical Techniques

1. Projection Correlation: Projected word feature vectors onto the semantic axes and correlated these projections with human survey ratings.
2. Geometric Alignment: Calculated Cosine Similarities between the semantic axes themselves to test if the axes are non-orthogonal. These geometric similarities were then correlated with the pairwise correlations found in the human survey data.
3. Dimensionality Reduction: Applied Principal Component Analysis (PCA) to both the survey data and the LLM semantic axes to determine if the variance in both spaces is captured by a low-dimensional subspace (specifically the first 3 components). Canonical Correlation Analysis (CCA) was used to align the subspaces of the LLM and human data.
4. Steering Interventions: To test practical implications, the authors performed "steering" interventions.
   • Mechanism: Added a scaled semantic axis vector ($\alpha \hat{f}$) to the residual stream during the forward pass of a prompt asking the model to rate a word.
   • Measurement: Measured the change in the model's probability of selecting a specific antonym (spillover effect) on off-target scales.
   • Hypothesis: If axes are geometrically aligned, steering along one axis should cause predictable spillover effects on other axes proportional to their cosine similarity.

3. Key Contributions

1. Validation of Human-Like Geometry: Demonstrated that the geometric structure of LLM feature spaces is not random or purely orthogonal but closely mirrors human psychological semantic associations.
2. Non-Orthogonality of Features: Proved that semantic axes in LLMs are meaningfully non-orthogonal. The cosine similarity between two axes in the model predicts the correlation between those scales in human surveys.
3. Low-Dimensional Subspace: Showed that a substantial portion of the variance in LLM semantic features (like in human data) lies within a low-dimensional subspace (approx. 3 dimensions), reproducing the "Evaluation-Potency-Activity" structure found in classic psychology.
4. Predictive Steering Spillover: Established a causal link between geometric alignment and model behavior. Steering a model along one semantic axis induces predictable "spillover" effects on other axes, the magnitude of which is determined by the cosine similarity between the axes.

4. Key Results

• Projection Correlation: There is a strong positive correlation ($r > 0.3$ to $r > 0.8$) between the projection of word vectors on LLM semantic axes and human ratings of those words. This holds for both small (3B) and large (70B) models.
• Axis Correlation: The cosine similarities between LLM semantic axes are highly predictive of the correlations between those scales in the human survey ($r \approx 0.8$). This confirms that the model's internal geometry encodes human cultural associations.
• PCA Alignment:
  • Human survey data reduced effectively to 3 dimensions.
  • LLM semantic axes also showed that the first three principal components explained a majority of the variance (>45% for 3B, >33% for 70B).
  • CCA revealed high correlations between the 3D subspaces of the LLM and the human survey, indicating the models "think" about these concepts in a structurally similar way to humans.
• Steering Spillover:
  • Steering a word toward a specific attribute (e.g., beautiful) caused the model to rate the word higher on geometrically similar attributes (e.g., soft).
  • There is a strong linear relationship ($r \approx 0.65 - 0.68$) between the cosine similarity of two axes and the magnitude of the spillover effect.
  • Note: The 70B model showed smaller spillover magnitudes than the 3B model, suggesting larger models are more resilient to steering, but the structural relationship (geometry predicting spillover) remains consistent.

5. Significance and Implications

• Beyond Isolation: The findings challenge the view of LLM features as a catalog of independent "knobs." Instead, features must be understood through their geometric relations and the subspaces they form.
• AI Safety and Control: If safety-critical features (e.g., true-false, safe-dangerous) lie on a shared low-dimensional subspace, controlling one may inadvertently affect others due to geometric entanglement. This complicates "feature steering" but also offers a way to audit models by understanding these relational structures.
• Representation Engineering: The study validates that techniques used on static word embeddings (like analyzing angles and cultural dimensions) are applicable and fruitful for deep LLM representations.
• Cognitive Alignment: The results suggest that despite being trained on raw text, LLMs develop internal representations that structurally align with human cognitive models of the world, reinforcing the idea that LLMs capture human semantic structures.

In conclusion, the paper argues that the geometry of the feature space is humanly meaningful. Understanding the angles between features and the subspaces they define is essential for interpreting, auditing, and controlling Large Language Models.