Closing the loop on-sky with a vector-Zernike wavefront sensor using a convolutional neural network as phase reconstructor

Technical Summary: Closing the Loop on-Sky with a Vector-Zernike Wavefront Sensor Using a CNN

Problem Statement
The next generation of giant segmented mirror telescopes requires adaptive optics (AO) systems capable of reaching fundamental resolution limits. While Fourier Filtering Wavefront Sensors (FFWFSs), such as the Zernike Wavefront Sensor (ZWFS), offer superior sensitivity for detecting faint guide stars, they suffer from inherent non-linearities. Specifically, the ZWFS signal exhibits a periodic response ($I(\phi) \sim \sin(\phi)$ and $\cos(\phi)$), causing phase wrapping for large aberrations. This wrapping introduces ambiguities where the estimated phase may have the wrong sign, potentially driving the control loop unstable. Although the Vector-ZWFS (v-ZWFS) was developed to extend the dynamic range by combining two masks with different phase shifts, it still fails when aberrations exceed its extended linear range. Traditional linear reconstruction techniques are limited to small residual phases and cannot resolve these non-linear ambiguities effectively.

Methodology
The authors propose using a Convolutional Neural Network (CNN) as a non-linear phase reconstructor for the v-ZWFS. Unlike linear methods that rely on local information, the CNN leverages the full spatial context of the detector image to reconstruct the phase and resolve wrapping ambiguities in a single step.

• System Configuration: The study utilized OZIRIIS, the second-stage AO system of the PAPYRUS instrument at the Observatoire de Haute-Provence (OHP). The system features a v-ZWFS operating in the H-band (1.5–1.7 µm) with two masks (phase shifts $\delta_1 = 0.3\pi$ and $\delta_2 = -0.8\pi$) and a 97-actuator deformable mirror (DM).
• Simulation and Training: The CNN was trained entirely in simulation. The forward model accurately simulated the v-ZWFS signal propagation, including DM misregistration (translational offsets, rotations, magnification) and atmospheric turbulence modeled via Kolmogorov/Von-Karman power spectral densities.
• Dataset Generation: The training dataset included wavefronts with varying $r_0$ (8–20 cm), wind speeds, and noise levels (photon and detector noise). To ensure robustness, the training employed "curriculum learning," shifting from $f^{-11/3}$ to $f^{-11/6}$ power laws to balance energy distribution across modes, and included random pupil wandering (up to 2 pixels) to prevent overfitting to specific pupil positions.
• Network Architecture: The CNN accepted two channels (one per pupil) and consisted of six convolutional layers followed by adaptive average pooling and a fully connected layer, outputting the first 50 Karhunen-Loève modal coefficients. Inference time was optimized to 260 µs on an NVIDIA A40 GPU.
• Loss Function and Strategy: A critical contribution was the development of a two-step training strategy (open-loop followed by closed-loop fine-tuning) and a specific loss function: the mean of the logarithm of the residual phase variance. This loss function compresses high-error regimes and expands low-error regimes, prioritizing the stability required for the final stages of loop convergence, which standard Mean Squared Error (MSE) or Mean Absolute Error (MAE) losses failed to achieve in closed-loop simulations.

Key Results

• Simulation Performance: In simulated closed-loop tests, the CNN maintained residual phase variance below 10% of the input for variances up to 80 rad², whereas the linear reconstructor diverged for inputs above 5 rad². The CNN successfully corrected phases where the linear method failed to converge.
• On-Sky Demonstration: The system was successfully closed on-sky under observing conditions where the linear reconstructor could not achieve a stable flat wavefront.
  • In favorable seeing conditions, the CNN achieved a Strehl Ratio (SR) of 32.7% in a two-stage loop (first stage + v-ZWFS/CNN), compared to 22.0% for the linear reconstructor.
  • Most notably, the authors demonstrated closing the loop using only the v-ZWFS/CNN as the first stage under full atmospheric turbulence (seeing 2.2 arcseconds). This configuration achieved an SR of 46.2%, comparable to a well-optimized first stage, a feat the linear method could not accomplish due to the large input phase variance (33 rad²).
• Robustness: The CNN operated successfully across a broader range of seeing conditions than the linear method, including runs where the linear approach failed to converge entirely.

Significance and Claims
The paper claims this work represents the first on-sky closed-loop demonstration of a v-ZWFS using a CNN as a phase reconstructor. The authors conclude that non-linear reconstructors can significantly extend the use cases of highly sensitive but non-linear wavefront sensors like the ZWFS. By removing the constraint of small residual phases, the v-ZWFS becomes viable for a wider range of observing conditions and instrument configurations, including operation as a primary sensor under full atmospheric turbulence.

The authors emphasize that the success of this deployment relied less on the specific CNN architecture and more on three critical factors: an accurate instrument model (including DM misregistration), a physically informed loss function (log-variance) that encodes the importance of small residuals, and a two-step training strategy ensuring closed-loop stability. They assert that this approach demonstrates that simulation-trained non-linear reconstructors can be deployed directly on-sky without retraining on real data, a crucial capability for future extremely large telescopes.