🤖 AI

EyeLayer: Integrating Human Attention Patterns into LLM-Based Code Summarization

The paper proposes EyeLayer, a lightweight module that integrates human eye-gaze patterns as attention priors into LLM-based code summarization, significantly improving performance across diverse model families by enhancing semantic focus without disrupting existing representations.

Original authors: Jiahao Zhang, Yifan Zhang, Kevin Leach, Yu Huang

Published 2026-02-27

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jiahao Zhang, Yifan Zhang, Kevin Leach, Yu Huang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a brilliant but slightly naive robot how to write a summary of a complex piece of software code. You give the robot millions of examples of code and their summaries, and it learns the patterns. It gets pretty good, but sometimes it misses the point. It might summarize a function by listing every single variable name, missing the actual purpose of the code, because it doesn't know what a human programmer would consider important.

This is the problem the paper EyeLayer tries to solve.

Here is the simple breakdown of what they did, using some everyday analogies.

1. The Problem: The Robot vs. The Human Eye

When a human developer reads code to understand it, they don't stare at every single letter equally. Their eyes dart around. They might pause for a long time on a specific function name (because it's the core logic), glance quickly at a variable name, and skip over some comments.

The Robot's approach: "I will read every word with equal intensity and try to guess the meaning."
The Human's approach: "I will zoom in on the important parts and skim the rest."

The researchers asked: Can we teach the robot to "look" at the code the way a human does?

2. The Solution: EyeLayer (The "Glasses" for the AI)

They built a small, lightweight add-on module called EyeLayer. Think of this module as a pair of smart glasses that the AI puts on before it reads the code.

These glasses don't just show the code; they overlay a "heat map" of where a human would likely look.

If a human would stare intently at a specific line of code, the glasses tell the AI: "Hey, pay extra attention here! This is the most important part."
If a human would glance over a section, the glasses tell the AI: "You can skim this part."

3. How It Works: The "Spotlight" Analogy

The paper uses a fancy mathematical concept called a "Multimodal Gaussian Mixture," but let's translate that into a Spotlight analogy.

Imagine the code is a dark stage.

Standard AI: Uses a giant, flat floodlight that illuminates the whole stage equally. It sees everything but highlights nothing specific.
EyeLayer: Turns on three movable spotlights.
- One spotlight might focus tightly on the main function name (the "center" of attention).
- Another might cast a wider glow over the variables used in that function.
- A third might highlight a specific loop or condition.

The AI learns to move these spotlights based on what a human eye-tracking study (where they watched 27 real developers read code) told them. It learns that humans usually focus on the "center" of the logic and the "edges" of the context, but ignore the noise in between.

4. The Training: Learning from a Small Group

You might think you need millions of humans staring at code to teach the AI. Surprisingly, the researchers only used data from 27 professional developers.

Think of it like teaching a chef. You don't need a million people to taste the soup to know if it's good. You just need a few expert chefs to tell you, "Add more salt here, and less pepper there."

The AI learned the pattern of how experts focus, not the specific words they said.
Once it learned this "focus pattern," it could apply it to any code, even code it had never seen before.

5. The Results: A Smarter Summary

When they tested this "smart glasses" system on different types of AI models (from small ones to massive ones), the results were great:

Better Summaries: The summaries were more accurate and captured the real meaning of the code, not just a list of words.
Universal Help: It worked on different types of AI architectures, proving that "human focus" is a universal clue for understanding code.
Efficiency: It didn't require rebuilding the whole AI; it was just a small add-on layer (like a plugin) that made the existing AI much smarter.

The Big Takeaway

The paper proves that human intuition is a superpower for AI.

By simply teaching the AI where to look based on how humans actually read code, the AI becomes a much better writer of summaries. It's like giving the AI a mentor who whispers, "Look here, this is the important part," and suddenly, the AI understands the code much better.

In short: EyeLayer is a tool that teaches AI to read code with "human eyes," resulting in summaries that humans actually find useful.

1. Problem Statement

Code summarization—the automatic generation of natural language descriptions for source code—is critical for software maintenance and comprehension. While Large Language Models (LLMs) have achieved significant success in this domain through supervised fine-tuning (SFT) on code-summary pairs, they often struggle to align with human cognitive processes. Specifically, LLMs may fail to prioritize the specific code regions that human developers focus on during comprehension, leading to summaries that are lexically fluent but semantically incomplete or misaligned with human intent.

The core research question is: Can human expertise, specifically attention patterns derived from eye-tracking data, be integrated into LLMs to enhance code summarization performance? Previous attempts (e.g., EyeTrans) were limited to smaller models or specific architectures, leaving a gap in understanding how human attention signals transfer to modern, large-scale LLMs.

2. Methodology: EyeLayer

The authors propose EyeLayer, a lightweight, plug-and-play architectural module that integrates human eye-gaze data into LLMs without altering the base model's architecture. The approach relies on the insight that human attention during code reading is uneven, concentrating intensely on semantically critical regions while peripherally attending to context.

Core Components:

Multimodal Gaussian Mixture: EyeLayer models human attention as a mixture of Gaussian distributions. Instead of a single attention focus, it predicts multiple "modes" (peaks of attention), each defined by a center ( $\mu$ ) and spread ( $\sigma$ ). This captures the complex, multi-focal nature of human reading.
Sparse Gating Network: A lightweight network determines how many Gaussian modes (1–3) are active for a given code snippet based on its complexity. Simple code may activate a single mode, while complex code activates multiple.
Causal-Aware Attention Redistribution:
- For Decoder-Only Models (e.g., LLaMA, Qwen): EyeLayer is injected via hooks at an intermediate layer. It intercepts hidden states, predicts a Gaussian attention distribution, and applies a residual perturbation to the hidden states. This "softly" redirects the model's focus toward human-attended regions while strictly preserving the causal (left-to-right) autoregressive constraint required for generation.
- For Encoder-Only Models (e.g., CodeBERT): The module operates after the encoder, modulating the contextualized hidden states before they are passed to the decoder.
Dual-Dataset Training Strategy:
- Primary Task: Supervised Fine-Tuning on a large CodeXGLUE dataset (Java code to docstring summaries).
- Auxiliary Task: Alignment on a sparse EyeTrans dataset (27 developers, 625 samples) containing eye-tracking fixations mapped to Abstract Syntax Tree (AST) nodes.
- Alignment: A three-stage pipeline maps AST nodes (from eye-tracking data) to subtoken indices (model input) to create a supervision signal for the Gaussian parameters.
Optimization: The system uses Projecting Conflicting Gradients (PCGrad) to resolve conflicts between the main generation loss and the auxiliary alignment loss, ensuring human signals act as a regularizer rather than disrupting the primary learning objective.

3. Key Contributions

Framework for Cognitive Priors: Proposes the first framework to integrate human cognitive priors (eye-tracking) into large-scale LLMs for code summarization, bridging the gap between human attention behavior and LLM attention mechanisms.
Multimodal Gaussian EyeLayer: Designs a novel, lightweight module that uses learnable Gaussian mixtures to redistribute code embeddings. This decouples gaze-informed redistribution from intrinsic attention weights, allowing scalable integration into billion-parameter models.
Systematic Evaluation: Conducts a comprehensive evaluation across five models (CodeBERT, LLaMA-3.2-1B/3B, Qwen3-1.7B/4B) covering both encoder-only and decoder-only architectures.
Open Source: Releases implementation scripts and datasets to facilitate reproducibility.

4. Experimental Results

The authors evaluated EyeLayer against strong SFT baselines using standard metrics (BLEU-4, ROUGE-L, METEOR, BERTScore).

Performance Gains: EyeLayer consistently outperformed baselines across all models.
- BLEU-4 improvements ranged from +1.04% to +1.98% across models.
- METEOR improvements were particularly notable, reaching +5.43% on Qwen3-1.7B.
- The largest relative gains were observed in smaller models (e.g., LLaMA-1B, Qwen-1.7B), suggesting that human attention signals provide a valuable inductive bias when model capacity is limited.
Ablation Studies (RQ4): The multimodal design (multiple Gaussian modes) significantly outperformed a single-mode variant. This confirms that human attention during code comprehension is not unimodal; it requires modeling multiple distinct focus areas (e.g., method signatures, variable declarations, control flow) simultaneously.
Integration Depth (RQ2): Performance peaked when EyeLayer was inserted into later-middle layers (e.g., layer 13 of a 16-layer model). This suggests that cognitive priors are most effective when semantic representations are mature but before the final generation stage.
Generalization (RQ3): The method successfully transferred to encoder-only architectures (CodeBERT), demonstrating that human attention patterns encode architecture-agnostic cues of semantic importance.

5. Significance and Implications

Human-in-the-Loop AI: EyeLayer demonstrates that even sparse, cognitively grounded data (from just 27 developers) can measurably improve the performance of massive LLMs. This validates the hypothesis that human expertise contains complementary signals that pure data-driven training misses.
Interpretability and Trust: By aligning model attention with human gaze patterns, the resulting summaries are more likely to reflect the regions developers actually care about. This could lead to more interpretable AI tools that build greater trust with software engineers.
Efficiency: The module is lightweight and does not require retraining the entire LLM from scratch, making it a practical solution for enhancing existing models.
Future Directions: The paper suggests that scaling eye-tracking data and incorporating dynamic signals (like saccades and attention switches) could further refine these models, potentially extending to other software engineering tasks like bug localization and code review.

In conclusion, EyeLayer establishes a new paradigm for "Human-Centered AI" in software engineering, proving that integrating human cognitive patterns into LLMs yields tangible improvements in code understanding and summarization.