A Closer Look into LLMs for Table Understanding

1. Problem Statement

While Large Language Models (LLMs) have achieved significant success in table understanding tasks (e.g., Table Question Answering, Fact Verification), their internal mechanisms remain a "black box." Existing research predominantly focuses on performance optimization through prompt engineering, supervised fine-tuning (SFT), or reinforcement learning (RL). There is a lack of empirical understanding regarding:

• How LLMs allocate attention across different parts of a table and the input prompt.
• The depth of layers required to reach stable predictions for tabular data compared to other reasoning tasks.
• Whether Mixture-of-Experts (MoE) models develop specialized "table experts."
• How input formats (Markdown vs. HTML) and reasoning strategies (Direct vs. Chain-of-Thought) influence internal representations.

2. Methodology

The authors conducted a comprehensive empirical study involving 16 LLMs across three categories:

1. General-purpose Instruct LLMs: (e.g., Llama-3.1-8B, Qwen2.5-7B).
2. Specialist Tabular LLMs: Fine-tuned via SFT or RL (e.g., TAMA, Table-R1).
3. Mixture-of-Experts (MoE) Models: (e.g., DeepSeek-V2-Lite, Qwen3-30B-A3B).

Experimental Setup:

• Data: 2,000 samples from four benchmarks: WTQ, HiTab, AITQA (Table QA), and TabFact (Fact Verification).
• Input Segmentation: Inputs were divided into three segments: System Prompt, Table Content, and User Question.
• Analysis Dimensions:
  • Attention Dynamics: Measured segment attention ratios, table attention entropy (focus concentration), and attention contribution (L2 norm of value-weighted representations).
  • Effective Depth: Used LogitLens to decode hidden states at each layer, measuring KL divergence and top-5 token overlap to determine when predictions stabilize.
  • MoE Analysis: Tracked expert activation frequencies and overlap between table and math tasks.
  • Intervention Studies: Masked attention weights in specific layer ranges to test causal impact; compared Markdown vs. HTML formats; compared Direct Answering vs. Chain-of-Thought (CoT).

3. Key Contributions & Findings

A. Three-Phase Attention Pattern

The study reveals a consistent three-phase workflow across different model architectures:

1. Early Layers (Broad Scanning): The model broadly scans the entire table content and system prompt. Attention entropy is high (dispersed).
2. Middle Layers (Localization): The model concentrates attention on query-relevant cells. Attention entropy drops significantly, and the model focuses on specific cells (e.g., the "Gold" column and "China" row).
3. Late Layers (Amplification): The model amplifies the contributions of the localized content to generate the final answer.

• Insight: Accurate localization in middle layers matters more than the total volume of attention.

B. Effective Depth and Task Specificity

• Tabular vs. Math: Table tasks require more layers for output distribution refinement (measured by KL divergence) than math reasoning tasks.
• Crystallization Point: While the distribution refinement takes longer for tables, the depth at which the final answer content "crystallizes" (Top-k overlap) is comparable to math tasks.
• Implication: Early-exit strategies (common in math reasoning) may be less effective for table tasks, as deep processing is required for fine-grained distribution adjustments.

C. MoE Expert Specialization

• Middle-Layer Specialization: MoE models activate a distinct set of table-specific experts primarily in the middle layers.
• Separation of Duties: These table experts show minimal overlap with math-reasoning experts. Early and late layers tend to share general-purpose experts across different domains.
• Pattern: This mirrors the attention entropy findings, suggesting a modular division of labor where middle layers handle domain-specific reasoning.

D. Impact of Input and Reasoning Strategies

• Input Format (Markdown vs. HTML): HTML tables induce higher attention entropy in early layers due to verbose syntax (tags like <tr>, <td>), dispersing attention. However, internal representations converge in deeper layers, explaining why performance gaps between formats are minimal.
• Reasoning Strategy (CoT): Chain-of-Thought prompting increases attention allocation to table content, particularly in middle and late layers. Table-specific fine-tuning further amplifies this effect, leading to significant performance gains (e.g., Table-R1-7B improved from ~70% to ~82% with CoT).

E. Causal Validation

• Masking Experiments: Masking attention in early-to-middle layers causes severe performance degradation (44–65% accuracy drop), confirming that the "broad scanning" and "localization" phases are functionally essential.
• Attention-Answer Correspondence: When middle layers (low entropy) successfully attend to the correct answer cell, prediction accuracy is significantly higher. Conversely, misfocused attention in these layers is a strong predictor of errors.

4. Significance and Future Directions

• Interpretability: This work provides the first systematic view of how LLMs process structured tabular data, moving beyond unstructured text analysis.
• Optimization Guidelines:
  • Inference: Targeted interventions in later layers could improve table understanding.
  • Architecture: MoE models could be optimized by ensuring robust routing to specialized experts in middle layers.
  • Efficiency: Low-attention table regions identified in early layers could potentially be pruned to create efficient tabular LLMs.
• Future Work: The authors suggest extending this analysis to larger models (>32B), Multimodal LLMs (image-based tables), and more complex tasks like data analysis and summarization.

Conclusion

The paper establishes that LLMs do not treat tables as simple text sequences but employ a structured, multi-stage processing pipeline. The transition from broad scanning to precise localization and finally to amplification is a fundamental mechanism for table understanding, which is further enhanced by specialized expert routing in MoE models and reasoning strategies like CoT. These insights offer a roadmap for improving the interpretability and efficiency of future table-aware AI systems.