DeAR: Fine-Grained VLM Adaptation by Decomposing Attention Head Roles
The paper proposes DeAR, a fine-grained adaptation framework for Vision-Language Models that decomposes attention heads into functional roles (Attribute, Generalization, and Mixed) using a Concept Entropy metric to selectively isolate task-specific learning from generalization capabilities, thereby achieving superior performance across diverse tasks while preserving zero-shot robustness.