💻 computer science

A Multi-Attribute Latent Space for Visual Analysis of Watches

This paper presents an interactive visual-analysis system that enables open-ended exploration of large wristwatch collections by integrating heterogeneous visual and semantic attributes into a unified multi-attribute latent space, thereby overcoming the limitations of traditional metadata-based filtering in e-commerce interfaces.

Original authors: Kai Lawonn, Tobias Günther, Monique Meuschke

Published 2026-06-29

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Kai Lawonn, Tobias Günther, Monique Meuschke

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are walking into a massive, high-end watch store. The shelves are packed with tens of thousands of timepieces. If you ask the clerk, "Show me all the blue watches under $500," they can do that easily because they have a computer database. But what if you say, "I want something that feels like this specific vintage diver watch I saw in a magazine, but maybe from a different brand, and it needs to have a clean, minimalist face"?

That is the problem this paper solves. Current online stores are great at checking boxes (metadata), but they are terrible at helping you explore based on vibes (visual similarity).

Here is how the authors built a system to fix that, explained simply:

1. The Problem: Mixing Apples, Oranges, and Watches

If you try to teach a computer to find similar watches by just smashing all the data together (brand, price, color, shape, and type) into one big list, the computer gets confused. It's like trying to measure the distance between two cities by adding up their population, their temperature, and their street names. One factor might accidentally dominate the math, making two watches look similar just because they are both expensive, even if they look nothing alike.

The authors realized you need to treat different "flavors" of similarity separately:

The "Type" (The Job): Is it a diver? A dress watch? A chronograph? This is the big category.
The "Color" (The Vibe): Is the face blue, black, or silver?
The "Design" (The Details): How are the hands shaped? Are there little sub-dials? What does the texture look like?

2. The Solution: A "Smart Map" of Watches

Instead of a list or a grid, the team built an interactive 2D map (a latent space). Think of this map as a magical floor plan of the watch store.

The Global Layout (The Neighborhoods): They forced the map to have seven distinct "neighborhoods" arranged in a circle (like a heptagon). Each corner represents a major watch type (e.g., the "Diver District," the "Dress Watch Downtown"). This ensures that if you are looking for a diver, you know exactly which part of the map to go to.
The Local Neighborhoods (The Street Level): Inside the "Diver District," the map doesn't just group them randomly. It arranges them based on color and design. If you walk through the Diver District, you will find all the blue-dial divers clustered together, and right next to them, the ones with the specific "clean face" design you like.

3. How the Computer "Sees" the Watches

To build this map, the computer had to learn how to look at a picture of a watch and understand it, just like a human expert would. They used three special tools:

The "Face Cutter" (U-Net): First, the computer uses a neural network to cut out just the watch face (the dial) from the picture, ignoring the strap or the background.
The "Type Spotter" (Vision Transformer): It looks at the whole watch and guesses the type (e.g., "That's a GMT watch").
The "Color & Pattern Scanner":
- It breaks the watch face down into a palette of 128 standard colors to see how much blue, silver, or black is there.
- It uses a technique called HOG (Histogram of Oriented Gradients) to map the "shape" of the design—where the lines, hands, and numbers are located.

4. The Magic Math: The "Multi-Attribute" UMAP

The core of their invention is a new way of using a math tool called UMAP (which is usually used to flatten 3D data into 2D).

Usually, UMAP tries to flatten data based on one single definition of "similarity." The authors changed the rules. They told the math: "Don't just look at one thing. Look at the color graph AND the design graph separately, then combine them with a knob you can turn."

The Knob: If you want to find watches that look similar in color, you turn the knob to favor color. If you want to find watches with similar designs, you turn it to favor design. If you want a mix, you set it in the middle.
The Result: The map rearranges itself instantly. You can see a cluster of watches that are all blue, or a cluster of watches that all have the same "tactical" look, even if they are different brands.

5. How You Use It (The Interactive System)

The paper describes a system where you can:

Zoom and Pan: Fly around the map to see the big picture or zoom in to see tiny details.
Hover for Details: Hover your mouse over a dot to see the watch's brand, price, and size.
Filter: Use standard filters (like "Price < $2000") to shrink the map down to only what you can afford.
Search by Example: This is the coolest part. You can upload a photo of a watch you like (maybe one you took with your phone). The system cuts out the face, analyzes it, and drops a pin on the map right next to the most similar watches in the database. It's like saying, "Put this watch on the map, and show me its neighbors."

6. Did It Work?

The authors tested this with two groups: Watch Experts (collectors who know everything) and Novices (people who just like cool watches).

Experts loved it because it helped them find subtle design similarities across different brands that they usually miss.
Novices found it intuitive. They could start with a vague idea ("I want a blue diver") and the map helped them discover specific models they didn't know existed.

The Bottom Line

This paper presents a system that turns a boring database of watch photos into a visual playground. It separates the "big categories" (like watch types) from the "small details" (like color and design) so you can explore a massive collection not just by checking boxes, but by following your visual curiosity. It proves that when you have a lot of different kinds of data, you need a special kind of map that respects those differences, rather than smushing them all together.

Technical Summary: A Multi-Attribute Latent Space for Visual Analysis of Watches

Problem Statement
The paper addresses a limitation in current catalog and e-commerce interfaces for complex product domains like wristwatches. While users can filter by metadata (e.g., brand, price), these systems lack support for open-ended exploration of visual similarity, stylistic alternatives, and mixed aesthetic-functional criteria. A standard concatenated feature vector for dimensionality reduction is problematic because watch type, dial color, and dial structure possess different semantics, dimensionalities, and statistical distributions. Simply concatenating these features often allows one attribute to dominate the distance metric, or a single learned embedding to obscure which attribute drives a specific neighborhood. The authors propose a system that allows users to explore large watch collections by balancing global semantic structure (watch type) with local visual similarity (color and design).

Methodology
The system constructs a multi-attribute latent space using a modified UMAP (Uniform Manifold Approximation and Projection) framework, supported by a pipeline of deep learning models for attribute extraction.

Data and Attribute Extraction:
- Dataset: Approximately 33,000 watch images from Chrono24 were processed.
- Dial Segmentation: A U-Net model segments the watch dial from the case and strap, isolating the face for analysis.
- Watch Type Classification: A Vision Transformer (ViT) predicts the watch type (e.g., Chronograph, Diver, Dress) from the full image.
- Color Representation: A two-stage approach extracts dominant colors. First, $k$ -means clustering identifies image-specific dominant colors in CIELAB space. Second, a global reference palette of 128 colors is derived from the entire dataset. Each watch is represented by a 128-dimensional vector encoding the proportion of pixels matching these reference colors.
- Design Representation: Histograms of Oriented Gradients (HOG) are computed on the segmented dial (grayscale) to capture structural cues like subdials, hands, and indices, resulting in a 324-dimensional descriptor.
Multi-Attribute UMAP Embedding:
- Instead of concatenating descriptors, the authors construct separate fuzzy neighborhood graphs for each attribute (color and design).
- Objective Function: The standard UMAP cross-entropy objective is extended to combine these attribute-specific graphs. A weighted sum of membership strengths ( $\gamma_f p^f_{ij}$ ) creates a unified high-dimensional graph.
- Class-Aware Constraints: To preserve global semantic structure (Requirement W1), the authors introduce a "decaying class-aware layout term." This term attracts points toward vertices of a regular heptagon representing the seven watch types. This term is strong initially to establish global structure but decays during optimization to allow local visual neighborhoods (color/design) to form without being overridden by the type constraint.
- Membership-Constrained Neighborhoods: The graph construction includes a parameter to control the influence of same-type neighbors versus general nearest neighbors, allowing the system to balance type-awareness with visual similarity.
Interactive Visualization System:
- The system provides a 2D spatial map where users can pan, zoom, and hover to inspect details (brand, price, dimensions).
- Search-by-Example: Users can upload a reference image. The system segments the dial, extracts attributes, and inserts the new point into the fixed latent space by finding its nearest neighbors in the high-dimensional attribute space and positioning it accordingly, preserving the existing structure.
- Hybrid Filtering: Metadata filters (price, case size) are coordinated with the visual map, allowing users to narrow visual discoveries by practical constraints.

Key Contributions

Multi-Attribute UMAP Variant: A formulation that combines attribute-specific fuzzy neighborhood graphs with user-controllable weights, augmented by membership-constrained neighborhoods and a decaying class-aware layout term.
Design Rationale: A review-driven approach connecting domain requirements (global type overview, color-based similarity, design-based similarity) to specific latent-space mapping and interaction techniques.
Implementation: A complete system pipeline processing segmented dial images, watch-type predictions, and color/design descriptors into an interactive interface.
Evaluation: A qualitative pilot study with experts and novices, alongside parameter analysis and runtime measurements.

Results

Parameter Analysis: Quantitative evaluation using a Jaccard index and reconstruction error metric shows that the proposed graph-weighted embedding performs better than standard UMAP or concatenated approaches in preserving the intended balance of attributes. The method successfully allows users to shift the embedding focus between color and design.
Search-by-Example: Controlled retrieval tests with perturbed query images (rotation, scaling, brightness changes) indicate that the design-only (HOG) setting yields the highest Top-1 accuracy (88.2%), though the mixed setting remains robust.
User Study: Ten participants (4 experts, 6 novices) completed exploration tasks.
- Experts achieved 94% task accuracy, praising the spatial grouping of types and ease of comparison.
- Novices achieved 84% accuracy, finding the system effective for discovering aesthetically related designs, though some initially struggled with interpreting spatial proximity versus functional similarity.
- Both groups rated the system highly for usability (SUS scores >73) and user experience (UEQ-S), noting that the system supported a progressive workflow from global orientation to local inspection.

Significance and Claims
The paper claims that the system supports the discovery and comparison of watches by revealing stylistic relationships across brands and within types. The authors explicitly state that the results are qualitative evidence of usage patterns and design implications rather than statistical proof of effectiveness, given the small sample size.

The primary significance lies in the design implications for visualizing multi-attribute latent spaces in heterogeneous collections:

Balancing Structure and Similarity: Distinguishing between attributes that define global structure (type) and those defining local similarity (visual features) prevents misleading embeddings.
Progressive Exploration: Supporting multi-scale navigation (overview to detail) is crucial for large collections.
Hybrid Filtering: Integrating non-visual metadata filters with spatial embeddings supports both exploratory discovery and targeted search.
Search-by-Example: Providing an intuitive entry point for similarity search reduces cognitive load for non-experts.

The authors conclude that while demonstrated on horology, the approach is applicable to other domains with heterogeneous attributes (e.g., fashion, industrial design). They acknowledge limitations in scalability assessment and the need for broader domain studies, and they note that the current evidence does not claim the system replaces expert judgment or precise filtering but rather complements them.