Panoramic Voltage-Sensitive Optical Mapping of Contracting Hearts using Cooperative Multi-View Motion Tracking with 12 to 24 Cameras

This paper presents a high-resolution, panoramic optical mapping system utilizing 12 to 24 cameras and cooperative multi-view motion tracking to reconstruct the 3D deforming surface of beating hearts, enabling simultaneous high-fidelity imaging of electrical action potentials and mechanical contractions across the entire ventricular surface.

The Gist

Imagine trying to take a high-definition video of a tiny, frantic dancer spinning and jumping inside a glass ball. Now, imagine that dancer is a rabbit's heart, beating so hard it changes shape with every jump, and you need to see not just the dance moves, but the invisible electrical sparks that tell the muscles when to jump.

For decades, scientists could only film this dance if they gave the heart a "muscle relaxant" drug to stop it from moving. They could see the electricity, but they couldn't see how the heart actually felt or moved in response. It was like watching a movie of a dancer frozen in mid-air.

This paper introduces a revolutionary new way to watch the heart dance while it's actually dancing. Here is how they did it, explained simply:

1. The "Soccer Ball" Studio

The researchers built a custom camera studio shaped like a soccer ball.

• The Setup: Instead of a flat room, they built a spherical chamber with 24 windows.
• The Cameras: They placed 12 high-speed cameras (like GoPros, but much faster) around the ball, looking in from every angle.
• The Lights: They added 48 tiny LED lights (like a disco ball) to shine on the heart from every direction, ensuring no shadows were cast.
• The Heart: A living rabbit heart sits right in the center, beating away in a warm, oxygenated bath.

2. The "Magic Paint" and the "Ghost" Problem

To see the electricity, they painted the heart with a special fluorescent dye. When the heart's electrical signal fires, the paint glows brighter or dimmer.

• The Problem: Because the heart is squeezing and twisting so fast, the paint moves around. If you just film it, the movement creates "ghosting" or blurry streaks that look like electrical signals but aren't. It's like trying to read a street sign while driving past it at 100 mph; the letters blur together.
• The Old Way: Scientists used to stop the heart (freeze the dancer) to read the sign.
• The New Way: They used a clever trick called Ratiometric Imaging. They flashed two different colored lights (blue and green) very quickly. The blue light acts as a "control" to measure how much the heart moved, while the green light measures the electricity. By comparing the two, they can mathematically subtract the movement and see only the true electrical signal.

3. The "Digital Puppeteer" (3D Tracking)

This is the most impressive part. The heart isn't just moving; it's twisting, stretching, and changing shape in 3D space.

• The Solution: The computer acts like a digital puppeteer. It takes the video from all 12 cameras and builds a 3D "wireframe" model of the heart (like a digital skeleton made of thousands of tiny triangles).
• The Magic: As the real heart moves, the computer tracks every single triangle of the digital skeleton in real-time. It essentially says, "Okay, that triangle moved 2 millimeters to the left and twisted 5 degrees."
• The Result: The computer then "warps" the video to follow the heart. It's like having a camera that sticks to the heart's surface, moving and twisting with it, so the image stays perfectly steady even though the heart is thrashing around.

4. What They Discovered

With this setup, they could finally see the full story of the heartbeat:

• The Spark and the Squeeze: They watched the electrical wave (the spark) race across the heart, and exactly 30 milliseconds later, they saw the muscle squeeze (the dance move). They could measure exactly how fast the squeeze happened.
• The Vortex: When they induced a dangerous irregular rhythm (fibrillation), they saw "electrical tornadoes" (vortices) spinning on the heart's surface. Amazingly, they saw that the heart muscle itself twisted in a matching "mechanical tornado" right underneath the electrical one.
• The Drug Test: They tested a drug that blocks potassium channels. They saw the electrical signal get "stuck" longer (prolonged), and the heart muscle reacted by squeezing harder and relaxing faster.

Why This Matters

Think of the heart as a complex machine where the wiring (electricity) and the engine (muscle) are deeply connected.

• Before: We could only look at the wiring while the engine was turned off.
• Now: We can watch the wiring and the engine working together in real-time, in 3D, at a resolution so high we can see tiny blood vessels on the surface.

This technology is like upgrading from a blurry, black-and-white photo of a frozen dancer to a 4K, 3D, slow-motion movie of a dancer in a full costume, captured from every angle simultaneously. It opens the door to understanding heart diseases, testing new drugs, and perhaps one day, designing better treatments for heart failure and arrhythmias.

Technical Summary

Here is a detailed technical summary of the paper "Panoramic Voltage-Sensitive Optical Mapping of Contracting Hearts using Cooperative Multi-View Motion Tracking with 12 to 24 Cameras."

1. Problem Statement

Voltage-sensitive fluorescence imaging is the gold standard for mapping cardiac action potentials (AP) at high spatial and temporal resolution. However, a critical limitation has persisted for decades: motion artifacts.

• The Challenge: In a beating heart, mechanical contraction causes rapid tissue deformation and displacement. This motion creates artifacts in optical signals that are indistinguishable from the physiological voltage changes, rendering data unusable without correction.
• Current Limitations: To avoid this, researchers typically use pharmacological agents (e.g., Blebbistatin) to paralyze the heart, preventing contraction. This decouples electrophysiology from mechanics, making it impossible to study the critical electromechanical coupling (how electrical waves trigger mechanical strain and vice versa) in a physiological state.
• Existing Solutions: Previous attempts to map contracting hearts using multi-camera systems were limited by:
  • Low spatial resolution (often requiring fiducial markers glued to the tissue).
  • Inability to reconstruct the 3D surface dynamically in real-time (often requiring a post-experiment static reconstruction of a paralyzed heart).
  • Limited field of view (usually capturing only a portion of the ventricle).

2. Methodology

The authors developed a novel Panoramic Multi-View Optical Mapping System capable of imaging the entire 3D surface of a strongly contracting heart.

A. Hardware Setup

• Imaging Chamber: A custom 3D-printed, spherical "soccer-ball" shaped chamber (truncated icosahedron) with 24 windows. This geometry allows for even illumination and unobstructed imaging from all angles.
• Cameras: The system utilizes 12 high-speed CMOS cameras (Basler acA720-520um) arranged around the chamber. While the chamber supports up to 24 cameras, 12 were used to achieve dense coverage (at least 3 cameras viewing any single point simultaneously).
• Illumination: Up to 48 high-power RGBW LEDs provide illumination.
  • Ratiometric Mode: Alternating green and blue excitation pulses (odd/even frames) allow for the separation of voltage signals from motion-induced illumination changes.
  • Continuous Mode: Continuous green excitation for high-speed wavefront tracking.
• Perfusion: Isolated rabbit hearts are perfused via a Langendorff system within the chamber, maintaining physiological conditions (37°C, pH 7.4) without pharmacological uncoupling agents.

B. Computational Pipeline

The core innovation lies in the Cooperative Multi-View Motion Tracking algorithm:

1. Static 3D Reconstruction: Using COLMAP, raw 3D meshes are generated for each frame from the multi-view images. These are uncorrelated and vary in topology.
2. Template Mesh Tracking: A fixed-topology template mesh (approx. 2,000–10,000 triangular faces) is created from an initial frame.
3. Cooperative Vertex Tracking: A patch-based algorithm (adapted from Klaudiny et al.) tracks the vertices of the template mesh across time. It minimizes an error function combining:
   • Image Component: Normalized cross-correlation of pixel neighborhoods around vertices across different camera views.
   • Mesh Component: Distance constraints to the raw static mesh to prevent drift.
   • Cooperative Optimization: Vertices are optimized simultaneously, using neighbor displacements to refine the solution.
4. Ratiometric Correction: In ratiometric mode, motion is tracked using the blue channel (which is voltage-insensitive) to avoid tracking artifacts caused by fluorescence intensity changes during the AP. This motion data is then applied to the green channel (voltage-sensitive) to correct the signal.
5. 2D Refinement: A secondary 2D optical flow step (Farnebäck algorithm) is applied to the warped images to correct residual motion between mesh vertices.

3. Key Contributions

• First Fully Panoramic 3D Mapping: The system captures the entire 360° ventricular surface of a beating heart, resolving approximately 0.5 to 1.0 million pixels (spatial resolution ~120 µm/pixel).
• Marker-Free Dynamic Reconstruction: Unlike previous methods, this system requires no fiducial markers and no post-experiment static reconstruction (no need to paralyze the heart to get a 3D shape). It generates dynamic 3D reconstructions automatically during the experiment.
• Simultaneous Electromechanical Measurement: It successfully measures electrical activation, action potential duration (APD), and mechanical strain (tissue deformation) simultaneously at high resolution.
• Cost-Effectiveness: By utilizing industrial-grade low-cost cameras (~$400 each) rather than expensive scientific cameras ($10k–$50k), the system makes high-resolution 3D optical mapping accessible to more laboratories.

4. Results

The system was validated across three physiological states in isolated rabbit hearts:

A. Sinus Rhythm

• Electromechanical Coupling: The system visualized the propagation of AP waves followed by mechanical contraction.
• Strain Mapping: Measured epicardial strain, showing local area decreases of 10–20% (contraction) and increases of 5–15% (tension).
• Timing: Observed a clear temporal separation between the electrical depolarization and the onset of mechanical contraction (~30 ms delay).
• Torsion: Detected rotational and torsional motion of the ventricles around the apex during contraction.

B. Paced Rhythms

• Focal Waves: Successfully tracked focal action potential waves originating from a stimulation electrode.
• Electromechanical Waves: Identified a distinct mechanical wavefront that follows the electrical wavefront with a delay of ~5.0 ± 2.4 ms.
• Wave Speed: Measured propagation speeds of electrical waves (0.55 m/s) and mechanical waves (0.49 m/s), demonstrating they are comparable but distinct phenomena.

C. Ventricular Fibrillation (VF)

• Motion Artifact Suppression: Demonstrated that even with minimal motion (VF1), motion artifacts can obscure data; the tracking algorithm successfully recovered clean signals.
• Vortex Dynamics: Visualized electromechanical vortices. Phase maps revealed phase singularities (rotors) in the electrical activity that co-localized with vortex-like mechanical deformation patterns (curls) in the tissue.
• Frequency Dependence: Showed that slower VF frequencies (10 Hz) resulted in stronger mechanical contractions and larger deformations compared to fast VF (20 Hz).

5. Significance and Impact

• Unprecedented Resolution: Provides the highest resolution (spatial and temporal) 3D electromechanical data of a beating heart to date.
• Physiological Insight: Enables the study of mechano-electric feedback (how stretch affects electrical properties) in a native, contracting environment, which was previously impossible.
• Disease Modeling: Offers a powerful tool for investigating arrhythmia mechanisms (e.g., re-entry, rotors) and the effects of pharmacological interventions (e.g., potassium channel blockers) on both electrical and mechanical function.
• Model Calibration: The high-fidelity data can be used to calibrate and validate complex computational models of cardiac electromechanics.
• Accessibility: The low-cost design democratizes advanced optical mapping, potentially accelerating research into cardiac physiology and disease mechanisms.

In conclusion, this work represents a paradigm shift in cardiac imaging, moving from static, paralyzed heart studies to dynamic, holistic 3D mapping of the beating heart, bridging the gap between electrophysiology and biomechanics.