Percept-Aware Surgical Planning for Visual Cortical Prostheses with Vascular Avoidance

This paper presents a percept-aware surgical planning framework that optimizes electrode placement for cortical visual prostheses by formulating it as a differentiable constrained optimization problem, which simultaneously maximizes perceptual reconstruction fidelity and adheres to critical vascular safety and anatomical feasibility constraints.

The Gist

Imagine you are trying to restore sight to someone who has been blind for a long time. Scientists have developed a "brain camera" called a cortical visual prosthesis. Instead of using eyes, this device sends tiny electrical signals directly to the visual part of the brain (V1). When these signals hit the brain, the person "sees" glowing dots of light called phosphenes. By arranging these dots correctly, the brain can piece them together to form shapes, letters, or faces.

However, there is a huge problem: Where exactly should the doctors put the electrodes?

The brain is a wrinkly, 3D maze (like a giant, folded walnut), and it is crisscrossed by delicate blood vessels. If a surgeon puts an electrode in the wrong spot, the person might see a blurry mess, or worse, the surgery could damage a blood vessel and cause a stroke.

This paper introduces a new, smart way to plan these surgeries using a video game-like simulation. Here is how it works, broken down into simple concepts:

1. The Old Way: "Covering the Map"

Previously, surgeons tried to place electrodes like they were laying down tiles on a floor. They would try to cover as much of the "visual map" as possible, avoiding big blood vessels just by looking at a map.

• The Flaw: Just because you cover the whole map doesn't mean the picture looks good. It's like trying to read a book where the letters are scattered randomly across the page. You have all the letters, but they don't form words.

2. The New Way: "The Percept-Aware Optimizer"

The authors created a computer program that acts like a super-smart architect. Instead of just covering the map, it asks: "If I put an electrode here, what will the patient actually see? Will they be able to read a '7' or recognize a cat?"

• The "Differentiable" Magic: The computer uses a special kind of math that allows it to "learn" from its mistakes. It simulates the surgery, sees the blurry result, and then says, "Okay, if I move this electrode just a tiny bit to the left, the '7' looks sharper." It does this millions of times in seconds until it finds the perfect arrangement.
• The Goal: It doesn't just want to cover the area; it wants to minimize the error in the final image, specifically for tasks like reading or recognizing objects.

3. The Safety Guardrails: "The Blood Vessel Avoidance"

The brain is full of tiny, fragile rivers (blood vessels). If an electrode touches one, it's a disaster.

• The Analogy: Imagine you are placing pins on a balloon, but you must stay at least 1 inch away from any red lines drawn on the balloon.
• The Solution: The computer has a "safety zone" rule. It knows exactly where the blood vessels are. If the "perfect" spot for a clear image is too close to a vessel, the computer automatically shifts the electrode to a safe spot, finding a balance between safety and clarity.

4. The "Thread" Trick: Doing More with Less

Surgeons can only poke a certain number of holes into the brain (insertions) without causing damage.

• The Innovation: The paper also tested using "threads" (like a fishing line with multiple hooks on it) instead of single pins.
• The Result: Even with the same number of holes in the brain, using these multi-electrode threads allowed them to place more "pixels" of light, creating a much clearer picture without needing more surgery.

The Bottom Line

The researchers tested this on a digital brain model using pictures of handwritten numbers (like reading) and natural scenes (like recognizing a dog).

• Without this new planning: The images were blurry and hard to understand.
• With this new planning: The images became much sharper, and the "virtual vision" was much better at helping a computer (or a human brain) recognize what it was seeing.

In short: This paper gives surgeons a GPS and a simulator for brain surgery. It moves beyond just "hitting the target area" to "hitting the target area in a way that actually makes a clear picture," all while strictly obeying the rule: "Do not touch the blood vessels." This paves the way for future devices that could give blind people much clearer, more useful sight.

Technical Summary

1. Problem Statement

Cortical visual prostheses aim to restore sight by electrically stimulating neurons in the primary visual cortex (V1). While high-density, flexible neural interfaces are emerging, a critical challenge remains: determining the optimal 3D placement of electrodes within the folded cortex.

• Current Limitations: Existing surgical planning strategies rely on anatomical landmarks and retinotopic maps to maximize visual field coverage. However, these approaches are geometric and coverage-driven rather than optimized for perceptual fidelity (how well the user actually "sees" the image).
• Safety Constraints: Surgical planning must also adhere to strict safety constraints, particularly avoiding blood vessels (vasculature) to prevent damage, and ensuring electrodes remain within the gray matter.
• The Gap: Previous work has either optimized stimulation parameters for fixed implants or optimized placement based on coverage. No unified framework exists that jointly optimizes electrode placement for task-relevant perceptual outcomes while simultaneously satisfying anatomical and vascular safety constraints.

2. Methodology

The authors propose a percept-aware optimization framework that treats electrode coordinates as learnable parameters within a differentiable forward model.

A. Formulation

The problem is formulated as a constrained multi-objective optimization in 3D brain space ($\Omega$):
$$E^* = \arg \min_{E \subset \Omega} \mathbb{E}_{T \sim D} [L_{percept}(P(E), T)] + \lambda_{vasc}L_{vasc}(E) + \lambda_{cortex}L_{cortex}(E)$$
Where:

• $E$: Set of electrode coordinates.
• $L_{percept}$: Task-level perceptual error (minimizing the difference between the target image and the simulated phosphene).
• $L_{vasc}$: Vascular safety penalty.
• $L_{cortex}$: Anatomical feasibility penalty (gray matter confinement).

B. Key Components

1. Anatomical Representation:

   • Uses the FreeSurfer fsaverage cortical template to model realistic folded gray matter geometry.
   • Integrates a cerebrovascular atlas to define prohibited regions.
   • Uses Bayesian retinotopic priors to map cortical coordinates to visual field locations.

2. Differentiable Forward Model:

   • Retinotopic Mapping: Estimates the visual field coordinate of an electrode via distance-weighted interpolation of nearest retinotopic sites.
   • Phosphene Generation: Models the resulting visual percept as a sum of isotropic Gaussians. The amplitude is sampled from the target image at the mapped location.
   • Differentiability: The entire pipeline (from electrode coordinates to the final image) is differentiable, allowing the use of gradient-based optimization (Adam optimizer) to update electrode positions directly.

3. Safety Constraints (Differentiable Penalties):

   • Vascular Avoidance: A hinge-style penalty is applied if an electrode is within a 300 µm safety margin of a blood vessel. This creates strong gradients near vessels to push electrodes away.
   • Gray Matter Confinement: A penalty is applied for coordinates outside the gray matter mask, ensuring electrodes target the appropriate cortical layers (proxy for Layer 4).

4. Experimental Setup:

   • Datasets: MNIST (handwritten digits for reading tasks) and CIFAR-10 (natural images).
   • Baselines: Compared against "Visual Field Tiling" (uniform grid) and "Visual Field Coverage" (maximizing area coverage).
   • Metrics: Structural Similarity Index (SSIM), Mean Squared Error (MSE), and downstream classification accuracy (using ResNet-50).

3. Key Contributions

1. Percept-Aware Framework: A novel end-to-end optimization system that directly minimizes predicted task-relevant perceptual error rather than just geometric coverage.
2. Integrated Safety Constraints: Explicit integration of vascular avoidance and anatomical feasibility into the optimization loop, ensuring clinically viable solutions.
3. Device Co-Optimization: Demonstration of the framework's ability to co-optimize multi-electrode thread configurations (where multiple electrodes share a single insertion point) under fixed surgical budgets.

4. Results

The framework was evaluated on simulated reading and natural image tasks using realistic cortical geometry.

• Improved Fidelity: Percept-aware optimization consistently outperformed baseline strategies.
  • MNIST: Reduced Median MSE by 67.7% compared to tiling and 33.4% compared to coverage-based methods. Classification accuracy improved by 62.6% over tiling.
  • CIFAR-10: Substantially outperformed tiling and matched or exceeded coverage-based optimization.
• Safety vs. Performance Trade-off:
  • Without constraints, many electrodes violated the 300 µm vascular margin.
  • With vascular constraints, all margin violations were eliminated.
  • Crucially, this safety enforcement resulted in negligible performance loss (SSIM decreased by only 1.7% on MNIST and 4.4% on CIFAR-10).
• Threaded Configurations: Under a fixed insertion budget (128 insertions), optimizing for multi-electrode threads improved perceptual quality compared to single-electrode placements, proving the method can guide device architecture design.

5. Significance and Impact

• Paradigm Shift: Moves surgical planning from a "coverage-centric" approach to a "percept-centric" approach, directly linking surgical decisions to functional visual outcomes.
• Safety-First Optimization: Demonstrates that high-fidelity vision restoration is achievable without compromising patient safety, solving the trade-off between performance and vascular risk.
• Computer-Assisted Planning: Provides a foundation for next-generation computer-assisted surgical planning tools that can utilize patient-specific anatomy (fMRI, angiography) to generate personalized, optimized electrode maps.
• Scalability: The framework is computationally efficient (optimization takes <10 minutes on a single GPU) and generalizable to different percept models and device architectures.

Limitations & Future Work:
The study utilized a standard template (fsaverage) rather than subject-specific anatomy, though the framework is designed to support it. The percept model uses simplified Gaussian phosphenes; future work will incorporate more complex biophysical models.