When Does Context Help Machine Vision? Decomposing the Risk and Reliability of Contextual Conditioning in Vision-Language Models
This paper systematically analyzes contextual conditioning in vision-language models across diverse architectures and datasets, revealing that while domain-level prompting is consistently beneficial, full per-image contextual conditioning carries high variance and overfitting risks, with its utility primarily driven by model scale and baseline accuracy.