⚡ electrical engineering

From Slices to Structures: Unsupervised 3D Reconstruction of Female Pelvic Anatomy from Freehand Transvaginal Ultrasound

This paper introduces TVGS, an unsupervised framework that leverages adapted Gaussian Splatting and joint pose optimization to reconstruct high-fidelity 3D female pelvic anatomy from freehand 2D transvaginal ultrasound sweeps without requiring external tracking hardware.

Original authors: Max Krähenmann, Sergio Tascon-Morales, Fabian Laumer, Julia E. Vogt, Ece Ozkan

Published 2026-02-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Max Krähenmann, Sergio Tascon-Morales, Fabian Laumer, Julia E. Vogt, Ece Ozkan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: Building a 3D Puzzle Without the Box Picture

Imagine you have a 3D object, like a statue, but you can only see it through a very narrow, flat window. Every time you move the window slightly, you get a new, thin "slice" of the statue. If you take 100 of these slices, you have a lot of information, but they are just flat pictures. Usually, to turn these flat pictures into a 3D model, you need a robot arm to tell you exactly where the window was for every single picture, or a special camera that tracks your hand movements.

This paper introduces a new way to build that 3D model without any robots or trackers.

The authors created a system called TVGS (Transvaginal Gaussian Splatting). It takes a series of freehand ultrasound images (taken by a doctor moving a probe by hand) and figures out how to stack them into a 3D volume using only math and computer power.

How It Works: The "Cloud of Dots" Analogy

Instead of building the 3D model out of tiny cubes (like LEGO bricks), which can look blocky and jagged, this method builds the anatomy out of fuzzy, glowing clouds.

The Fuzzy Clouds (Gaussians): Think of the 3D shape of the uterus as being made of thousands of invisible, soft, 3D fog clouds. Each cloud has a center, a shape (is it a sphere or a flat pancake?), a brightness, and a level of "see-through-ness."
The Magic Camera (The Rasterizer): The computer has a special tool that knows how to look at these clouds from the angle of the ultrasound probe. It asks: "If I look at these clouds from this specific angle, do they look like the ultrasound image I just took?"
The Guess-and-Check Game:
- The computer starts by guessing where all the clouds are.
- It renders a fake image based on those guesses.
- It compares the fake image to the real ultrasound slice.
- If the fake image doesn't match, the computer nudges the clouds (moving them, changing their shape, or dimming them) to make the match better.
- It does this thousands of times until the clouds arrange themselves perfectly to recreate the anatomy.

The Special Challenges & Solutions

The paper highlights a few specific problems with ultrasound that make this hard, and how they solved them:

Problem: The "No-Overlap" Issue.
- Analogy: Imagine taking photos of a cake by slicing it. If you slice it, take a photo, slice again, and take another photo, the two photos don't overlap much. In normal photography, you have many angles that overlap to help you figure out depth. Ultrasound slices are very thin and often don't overlap.
- Solution: The authors created a custom "rasterizer" (a rendering engine) specifically designed for these thin slices. It understands that the data is sparse and fills in the gaps using the fuzzy clouds, which can stretch out to cover the empty space between slices.
Problem: The Shaky Hand.
- Analogy: When a doctor moves the probe by hand, their hand might tremble or move unevenly. If the computer assumes the hand moved in a perfect straight line, the 3D model will look warped.
- Solution: The system doesn't just build the clouds; it also learns the exact path the hand took. It treats the position of every slice as a "variable" that can be tweaked. If the 3D model looks blurry because a slice is slightly out of place, the computer shifts that slice's position to make the picture sharp. It essentially "self-corrects" the doctor's hand movements.
Problem: Dead Clouds.
- Analogy: Sometimes, the computer places a "cloud" in empty space where there is no anatomy. That cloud does nothing and wastes energy.
- Solution: The system has a "gardener" that periodically checks the clouds. If a cloud isn't contributing to the image (it's too dark or too far away), the system deletes it and plants a new one in a more useful spot.

The Results: Fast and Accurate

The paper tested this on both computer-generated models and real patient data.

Speed: They compared their method to other advanced 3D reconstruction techniques (like Neural Radiance Fields). Those other methods are like slow, heavy construction cranes, taking over an hour to build a model. The new method is like a high-speed 3D printer, finishing the job in 8 to 16 minutes.
Quality: The resulting 3D models clearly show important structures like the uterus and the endometrium (the lining of the uterus).
No Extra Hardware: The biggest win is that this works with standard 2D ultrasound machines. You don't need expensive 3D probes or magnetic trackers attached to the patient.

Summary

In short, this paper shows that you can take a shaky, freehand video of 2D ultrasound slices and turn it into a high-quality 3D model using a "cloud of fuzzy dots" approach. The computer figures out both the shape of the anatomy and the path the doctor took, all without needing any special tracking equipment. It's a faster, cheaper, and more flexible way to see inside the body in 3D.

Technical Summary: Unsupervised 3D Reconstruction of Female Pelvic Anatomy from Freehand Transvaginal Ultrasound

Problem Statement
Volumetric ultrasound (US) offers significant potential for improving diagnostic accuracy in obstetrics and gynecology, particularly for pelvic anatomy. However, widespread clinical adoption is hindered by the reliance on specialized hardware (mechanically swept probes or external tracking systems) and restrictive acquisition protocols. Existing 3D systems are often cumbersome, expensive, and suffer from lower resolution compared to standard 2D imaging. Furthermore, recent machine learning approaches for reconstructing 3D volumes from freehand 2D sweeps typically depend on supervised training with ground-truth pose data or external tracking, which is often unavailable in clinical settings. There is a need for a method that can perform accurate 3D reconstruction from standard, freehand 2D transvaginal ultrasound (TVS) sweeps without external sensors, tracking hardware, or pre-trained pose estimators.

Methodology: TVGS (Transvaginal Gaussian Splatting)
The authors propose TVGS, an unsupervised framework that adapts the principles of Gaussian Splatting to the unique physics and geometry of ultrasound imaging. The core methodology involves the following components:

Representation: Anatomy is modeled as a collection of anisotropic 3D Gaussians. Each Gaussian is defined by a mean position ( $\mu$ ), a covariance matrix ( $\Sigma$ ) describing shape and orientation, an opacity ( $o$ ), and a scalar intensity ( $I$ ).
Slice-Aware Differentiable Rasterizer: Unlike optical Gaussian Splatting which relies on perspective projection, TVGS utilizes a custom CUDA-optimized differentiable rasterizer tailored for US. This rasterizer treats each ultrasound slice as a planar cross-section. It computes pixel intensity by aggregating contributions from 3D Gaussians based on their spatial proximity to the slice plane, using Mahalanobis distance to account for anisotropic shapes. This approach handles the sparse, non-overlapping nature of TVS sweeps.
Joint Optimization: The framework performs end-to-end optimization of both the anatomical structure (Gaussian parameters) and the slice poses. Slice poses are initialized based on a smooth angular sweep assumption but are treated as learnable parameters. During backpropagation, gradients flow through the rasterizer to refine both the Gaussian parameters and the rigid body transformations of the slices, allowing the system to self-calibrate against irregular probe motion (jitter).
Initialization and Density Control: To ensure stable convergence, the method employs "on-slice" initialization, distributing Gaussian centers directly on the image planes rather than randomly in 3D space. Additionally, a density control mechanism prunes "inactive" Gaussians (those with low intensity-opacity products) and re-seeds them in active regions to maintain computational efficiency and volumetric capacity.
Training Objective: The system is optimized using a hybrid loss function combining L1 and SSIM losses, driven purely by image-level supervision from the input 2D slices.

Key Contributions

Unsupervised Sensorless Reconstruction: The first application of Gaussian Splatting for 3D US that requires no external tracking, ground-truth trajectories, or learned pose estimators.
US-Adapted Gaussian Splatting: A novel volumetric framework that adapts Gaussian Splatting to US physics, specifically addressing the lack of view redundancy and the planar nature of slices.
Custom Differentiable Rasterizer: A high-performance, CUDA-optimized rasterizer designed for slice-based rendering under highly anisotropic sampling conditions (dense in-plane, sparse through-plane).
Joint Pose and Scene Optimization: A strategy that refines slice poses alongside scene parameters, enhancing robustness to the irregular probe motion typical of freehand acquisition.

Experimental Results
The method was validated on both synthetic data (simulated uterus models with ground-truth poses) and real-world clinical TVS data from 10 patients.

Performance Metrics: On synthetic data, the method achieved high structural similarity (SSIM ~~0.97–0.98) and PSNR (~~28–30 dB). On real clinical data, it outperformed a state-of-the-art implicit neural representation baseline (ImplicitVol) across all metrics (SSIM, PSNR, LPIPS).
Efficiency: A significant advantage of TVGS is computational speed. While the ImplicitVol baseline required approximately 3 hours to converge on real data, TVGS achieved superior results in 11–16 minutes.
Robustness: The method demonstrated robustness to moderate motion jitter (up to 2.5% deviation) through joint pose optimization. However, performance degraded at extreme jitter levels (5%), where the geometric prior of the fixed-pose baseline proved safer.
Generalization: The model showed strong cross-view generalization, capable of reconstructing anatomical structures (e.g., uterus, endometrium) from a single view (sagittal or transversal) and rendering them in the orthogonal view.
Rendering Speed: The framework supports real-time rendering, achieving frame rates of ~55 FPS for scenes with 50,000 Gaussians and ~229 FPS for smaller scenes.

Significance and Claims
The paper claims that TVGS demonstrates that accurate 3D reconstruction from 2D ultrasound images can be achieved through purely computational means, eliminating the need for specialized hardware. By leveraging geometric assumptions and optimizing anisotropic Gaussian primitives directly from 2D slices, the method offers a scalable, memory-efficient, and hardware-free alternative to conventional 3D systems.

The authors position this work as a step toward enabling new opportunities for AI-assisted analysis and diagnosis in gynecology. They note that while challenges remain—specifically regarding the alignment of multiple orthogonal views and achieving true real-time performance during the scanning process—the method provides a foundation for data-driven, volumetric reconstruction that lowers the barrier to clinical deployment for low-cost ultrasound imaging. The work does not claim to replace existing 3D systems immediately but offers a viable path for generating high-fidelity 3D volumes from standard 2D equipment without operator-dependent tracking.