Hospitality-VQA: Decision-Oriented Informativeness Evaluation for Vision-Language Models

This paper introduces a formal framework for "informativeness" and a corresponding hospitality-specific VQA dataset to evaluate Vision-Language Models, revealing that while current models struggle with decision-oriented reasoning, their performance significantly improves with modest domain-specific finetuning.

The Gist

Imagine you are trying to buy a house or book a vacation rental. You don't just read the text description; you stare at the photos. You ask yourself: "Is the room big enough for my bed?" "Can I actually see the ocean from the window?" "Is there a place to put my luggage?"

You aren't looking for a robot to tell you, "This is a chair." You are looking for a robot that understands how useful that chair is for your vacation.

This paper, "Hospitality-VQA," is about teaching AI to stop being a simple photo describer and start being a helpful travel advisor. Here is the breakdown in plain English:

1. The Problem: The "Tourist" vs. The "Traveler"

Current AI models (Vision-Language Models) are like tourists who just take snapshots. If you show them a hotel room, they say, "I see a bed, a lamp, and a window." That's factually correct, but it doesn't help you decide if you should book the room.

You, the traveler, need to know:

• Is the room layout clear enough to move around? (Spatial Legibility)
• Does the room actually support activities like working or sleeping? (Activity Affordance)
• Is the view blocked by a wall, or is it open and airy? (Contextual Openness)
• Is the building shown in a way that makes sense, or is it a weird close-up? (Geometric Completeness)

The authors realized that existing AI benchmarks only test if the AI knows what objects are in a picture, not if the picture is good enough to make a decision.

2. The Solution: A New "Ruler" for Photos

To fix this, the team created a new framework called Hospitality Informativeness. Think of this as a four-point ruler used to grade hotel photos based on how helpful they are for a guest.

They built a massive new dataset called Hospitality-VQA containing 5,000 real hotel photos. Instead of asking the AI, "What color is the sofa?" they asked decision-based questions like:

• "Can you see the whole room, or is it cut off?"
• "Is there a desk where I could work?"
• "How much of the outside view is visible?"

3. The Experiment: Testing the AI

The researchers took 8 of the smartest AI models available (like GPT-4o, Gemini, and others) and put them through a test using this new dataset.

The Results were surprising:

• The "General Knowledge" Gap: The AI models were great at identifying the room type (e.g., "This is a bedroom"). But when asked about the usefulness of the room (e.g., "Is the layout clear enough to navigate?"), they struggled badly. It's like a student who can memorize the periodic table but can't do basic chemistry.
• The "Fine-Tuning" Fix: The AI wasn't "stupid"; it just hadn't been trained for this specific job. When the researchers gave the AI a "crash course" (a small amount of extra training) specifically on these hotel questions, its performance skyrocketed. It went from guessing to being a reliable travel assistant.

4. Why This Matters

This paper is a wake-up call for the AI industry. Just because an AI can describe a picture doesn't mean it understands why humans care about that picture.

• For Hotels: They can use this to automatically pick the best photos to show customers, ensuring the images actually help people feel confident about booking.
• For Travelers: It paves the way for AI assistants that can say, "Don't book this room; the photo shows the bed is too close to the bathroom door, and the view is blocked," rather than just saying, "Here is a photo of a bed."

The Bottom Line

The authors built a new test to see if AI can judge the quality of information in a photo, not just the quantity of objects. They found that current AI needs a little bit of specialized training to stop being a "tourist" and start being a "travel agent."

Technical Summary

1. Problem Statement

While Vision-Language Models (VLMs) have achieved impressive results in general multimodal understanding, their applicability to decision-oriented domains like hospitality remains underexplored.

• The Gap: Existing VQA benchmarks focus on factual correctness (e.g., "What color is the slide?") rather than decision utility. In hospitality, users rely on images to evaluate spatial layout, functional affordances (e.g., sleeping areas), and context (e.g., views) to make booking decisions.
• The Limitation: Current VLMs often generate vague descriptions or fail to infer higher-level spatial organization and functional semantics. They lack the specific reasoning required to determine if an image provides sufficient evidence for a consumer to make an informed accommodation choice.
• The Challenge: How to formalize and quantify the specific visual evidence that supports user decisions in hospitality settings.

2. Methodology

A. The "Hospitality Informativeness" Framework

The authors propose a formal framework to quantify how much decision-relevant information an image-question pair provides. This is decomposed into four fundamental visual axes:

1. Spatial Legibility (SL): The count of distinct planar surfaces (floor, walls, ceiling) to measure spatial comprehension and volume.
2. Activity Affordance (AA): The presence of functional objects that support guest activities (e.g., desks, beds, seating) versus purely decorative elements.
3. Contextual Openness (CO): The ratio of non-facility elements (sky, nature, background) to assess environmental context and balance.
4. Geometric Completeness (GC): The visibility of a building's canonical faces (front, side, roof) to evaluate 3D form integrity.

Note: For Room Interior images, two additional semantic attributes are defined: View Type and Room Type.

B. The Hospitality-VQA Dataset

To operationalize this framework, the authors constructed a new dataset:

• Data Collection: 5,000 images randomly sampled from nol.yanolja.com (a major Korean travel platform).
• Annotation Schema:
  • Hierarchical Labels: Main facility type (e.g., Room Interior, Outdoor Facility) and sub-facility labels.
  • Axis Annotation: Five expert annotators labeled images based on the four axes (and semantic attributes where applicable).
  • Consensus Protocol: High-agreement labels (4/5 annotators) were accepted as ground truth; disagreements were resolved via discussion.
• Task Formulation: The dataset generates 19,729 QA pairs by mapping the annotations into fixed instruction-answer templates. Unlike open-ended generation, tasks are formulated as classification problems to align with discrete decision-making attributes.
• Distribution: The dataset covers diverse facility types (Room Interior, Indoor/Outdoor Facilities, Exteriors) and exhibits skewed distributions toward high visual informativeness, reflecting professional listing photography.

C. Experimental Setup

• Models Evaluated: Eight state-of-the-art VLMs, including proprietary models (GPT-5, GPT-4o-mini, Gemini 2.5 Pro, GLM-4.1V) and open-weight models (Qwen2.5-VL, LLaVA-NeXT, Gemma-3).
• Evaluation Strategy:
  • Zero-shot: Testing general capabilities.
  • Domain Adaptation: Fine-tuning Qwen2.5-VL models (3B and 7B) using LoRA on the Hospitality-VQA dataset to align outputs with the specific informativeness axes.
• Metrics: Exact-match accuracy on categorical labels (e.g., counting planes, identifying facility types, discretized scenery proportions).

3. Key Contributions

1. Formalization of Informativeness: Defined a domain-grounded framework with four interpretable axes (SL, AA, CO, GC) that quantify decision-relevant visual cues in hospitality imagery.
2. Hospitality-VQA Benchmark: Created the first VQA dataset specifically designed for decision-oriented evaluation, covering diverse facility types and aligned with the informativeness axes.
3. Empirical Analysis: Provided a comprehensive benchmark of eight VLMs, revealing that while general facility recognition is strong, fine-grained decision-relevant reasoning is weak without domain adaptation.

4. Results

A. Zero-Shot Performance

• Facility Recognition: Most models achieved high accuracy (>90%) in identifying the main facility type (e.g., distinguishing a room from an exterior).
• Informativeness Axes: Performance dropped significantly for the specific axes:
  • Spatial Legibility (SL): Relatively stable but varied (e.g., GPT-4o-mini: 97.12%, Qwen2.5-VL-7B: 43.88%).
  • Activity Affordance (AA) & Geometric Completeness (GC): These were the weakest areas for most models (e.g., GPT-4o-mini scored only 38.21% on AA and 8.82% on GC).
  • Conclusion: General VLMs are not intrinsically "decision-aware"; they struggle to extract the specific structural and functional cues needed for booking decisions.

B. Impact of Domain Adaptation (Fine-tuning)

Fine-tuning the Qwen2.5-VL models using LoRA yielded substantial improvements:

• Overall Gains: Significant accuracy increases across all tasks.
  • Main & Sub Facility: +36.79% (3B) and +21.22% (7B).
  • Spatial Legibility: +26.61% (3B) and +53.24% (7B).
  • Geometric Completeness: +25.00% (3B) and +26.47% (7B).
• Implication: The dataset provides a strong supervisory signal that allows models to learn the specific visual patterns associated with hospitality decision-making.

5. Significance and Future Directions

• Bridging the Gap: This work shifts the focus from "what is in the image" to "how useful is the image for a decision," addressing a critical gap in hospitality AI.
• Practical Application: The framework and dataset enable B2C applications (displaying more appealing images to improve booking rates) and B2B applications (curating and ranking property images based on user appeal).
• Limitations: The study focuses on static images from a specific platform and does not yet incorporate text reviews, pricing, or video. It also abstracts aesthetic qualities (ambiance, style) which are subjective but influential.
• Future Work: The authors suggest extending this to domain-aware representation learning, prompt optimization, and modeling human-preferred attractiveness.

In summary, Hospitality-VQA demonstrates that while general VLMs are powerful, they require domain-specific grounding to perform the fine-grained, decision-oriented reasoning necessary for the hospitality industry. The proposed benchmark and framework serve as a foundational tool for developing next-generation multimodal intelligence in this sector.