Technical Summary: Virtual Ring Try-On

Problem Statement

The paper addresses the challenge of providing a realistic, accessible, and efficient virtual try-on experience for jewelry, specifically rings, within the context of online shopping. Existing solutions suffer from significant limitations:

• Overlay-based approaches are computationally lightweight but lack realism because they fail to adapt to finger orientation, depth, or lighting, resulting in static, unnatural appearances.
• Augmented Reality (AR) 3D solutions offer immersion but are hardware-dependent, often requiring high-end devices. They frequently suffer from latency, jitter, and misalignment, causing rings to appear "floating" rather than attached to the finger.
• Generative and Diffusion models (e.g., SDEdit, SmartBrush) can produce high-quality images but require massive GPU resources, extensive training data, and precise prompt engineering. Crucially, these stochastic methods often alter unmasked regions (such as the user's hand or background) or fail to reproduce the exact reference ring design, which is unacceptable for e-commerce fidelity.

The core problem is the lack of a system that balances visual realism (correct orientation, scaling, and attachment) with computational efficiency (running on mid-range consumer smartphones without GPU dependency).

Methodology

The proposed system utilizes a hybrid pipeline combining computer vision for detection and lightweight geometric transformations for rendering. It avoids deep generative algorithms in favor of deterministic geometric computation. The workflow consists of four primary stages:

1. Hand Landmark Detection:

   • The user captures a hand image via a React Native interface.
   • MediaPipe is employed to detect 21 hand landmark points.
   • Based on the target finger, a Region of Interest (RoI) is defined using specific landmarks (base and tip of the finger) to establish the finger's reference axis.

2. Ring Object Localization:

   • A custom YOLOv8 object detection model, trained on a dataset of 600–700 ring images with varying orientations and styles, is used to detect the ring.
   • The model generates a bounding box around the ring, which is then cropped.
   • Background removal is applied to isolate the ring pixels.

3. Geometric Alignment and Scaling:

   • Orientation: Vector algebra is used to compute the angular discrepancy ($\theta$) between the finger's reference vector ($\vec{f}$) and the ring's principal axis ($\vec{r}$). The ring is rotated by $\theta$ to align with the finger's pose.
   • Scaling: The ring is resized based on the ratio of the finger width ($w_f$) to the ring width ($w_r$). This scaling factor ($s$) preserves the ring's aspect ratio to ensure perceptual realism.

4. Image Synthesis and Rendering:

   • The aligned and scaled ring image is superimposed onto the hand's RoI using alpha blending.
   • A Gaussian blur is applied to the composite to simulate lighting and shadows, enhancing the realism of the attachment.
   • The final output is displayed on the user's screen.

Key Contributions

• Lightweight Architecture: The system operates without the need for high-end GPUs, specialized AR hardware, or large-scale generative models, making it deployable on standard mid-range consumer smartphones.
• Geometric Fidelity: Unlike static overlays, the system dynamically adjusts the ring's rotation and scale based on the specific geometry of the user's finger, ensuring the ring appears "fixed" rather than floating.
• Input Preservation: The approach guarantees that the original hand image and the specific reference ring image remain unchanged, avoiding the stochastic alterations common in generative AI methods.
• Hybrid Detection Pipeline: The integration of MediaPipe for hand landmarks and a custom YOLOv8 model for ring isolation provides a robust foundation for precise placement.

Experimental Results

The authors conducted a comparative analysis against industry platforms (Candere by Kalyan) and generative models (Google AI Studio/Banana Pro, OpenAI ChatGPT).

• Visual Quality:
  • Generative Models: While Google AI Studio maintained the hand background, it failed to reproduce the exact reference ring, creating a different-looking design. ChatGPT regenerated the entire image, altering the background and hand consistency.
  • AR Solutions (Candere): The output appeared unrealistic due to size inaccuracies and a "floating" effect caused by a lack of proper rotation and occlusion handling.
  • Proposed System: Successfully preserved the exact input ring and hand images. The ring appeared correctly attached, with appropriate rotation, scaling, and lighting effects (via Gaussian blur).
• Performance Metrics:
  • Processing Time: The proposed approach took approximately 34.42 seconds per image. This was slower than simple overlay methods (3.32 sec) but significantly faster than generative models like ChatGPT (2 min 23 sec) and competitive with other complex pipelines (Candere: 40.02 sec; Google AI Studio: 38.64 sec).
  • Fidelity: The system achieved high fidelity to the input ring design and hand geometry, a metric where generative models scored "Low" due to unwanted alterations.

Significance and Claims

The paper claims that this work represents a new direction in virtual jewelry visualization by demonstrating that geometry-based solutions can achieve a balance between realism and efficiency that current market solutions cannot.

The authors assert that their system redefines the virtual try-on experience for the digital marketplace by:

1. Eliminating the dependency on expensive hardware and GPU-intensive generative pipelines.
2. Ensuring that the user's hand and the specific product image remain authentic and unaltered.
3. Providing a computationally efficient, accessible solution that maintains high visual quality through adaptive geometric alignment rather than heavy image regeneration.

The system is presented as a practical, deployable alternative for e-commerce platforms seeking to increase purchase certainty without the barriers of high-end hardware or complex AI infrastructure.