Beyond a symmetry-restricted VEV ansatz: transverse stability of an $SU(5)$ special-subgroup vacuum

Technical Summary: Beyond a Symmetry-Restricted VEV Ansatz

Problem Statement
In Grand Unified Theories (GUTs), extended Higgs sectors are often employed to realize multistage symmetry breaking chains, which can lead to intermediate gauge phases relevant for cosmological defects (e.g., monopole erasure via the Langacker–Pi mechanism). A standard method for identifying candidate vacuum configurations is to restrict Vacuum Expectation Values (VEVs) to ansätze that preserve a chosen subgroup (symmetry-restricted VEV ansatz). While efficient for locating stationary points, this approach leaves two critical questions unresolved:

1. Does the restricted ansatz span the entire field subspace preserving the proposed subgroup?
2. Is a stationary configuration found within this subspace stable against fluctuations outside that subspace (transverse stability)?

The principle of symmetric criticality ensures that restricted configurations are stationary points of the full potential if they are stationary within the restricted subspace, but it does not guarantee that the Hessian is positive definite in directions normal to the fixed subspace. This paper addresses these questions in a concrete renormalizable $SU(5)$ GUT model containing an adjoint scalar $\Phi \in \mathbf{24}_H$ and a complex symmetric tensor $S \in \mathbf{15}_H$.

Methodology
The authors analyze the scalar potential $V(\Phi, S)$, which includes three independent mixed quartic portal interactions ($\alpha, \beta, \gamma$) coupling the adjoint and symmetric sectors.

• Orbit Variable Formalism: The potential is decomposed using radial variables and dimensionless orbit variables ($k, \ell, u, v, p, q$) that characterize the gauge-invariant structure of the field configurations. This allows the potential to be organized by homogeneous degree ($V_2, V_3, V_4$).
• Symmetry-Restricted Analysis: The authors first analyze the "two-block" family of VEVs for $S$ that preserves the intermediate gauge group $H_{sp} = S(O(3)_C \times O(2)_L)$ on a standard $D_{32}$ adjoint background. In this subspace, the portal couplings $\beta$ and $\gamma$ appear only as a sum $\sigma = \beta + \gamma$.
• Transverse Stability Analysis: The paper derives the complete spectrum of fluctuations in the directions orthogonal to the symmetry-restricted subspace. Specifically, it calculates the mass matrix for the "color–weak" cross-block fluctuations, which decompose into two sixfold physical families.
• Full-Field Validation: To verify analytic results, the authors perform direct numerical Hessian calculations in the full 54-dimensional real scalar field space. They compare restricted candidate branches (including isotropic, block-diagonal, and off-diagonal configurations) and test local stability without restricting the field variations.

Key Contributions and Results

1. Decoupling of Reduced Potential and Transverse Curvature:
   The study demonstrates that on the symmetry-restricted two-block field space, two distinct mixed quartic invariants (associated with $\beta$ and $\gamma$) become identical. Consequently, the reduced potential and the Hessian restricted to the subspace depend only on the combination $\sigma = \beta + \gamma$. However, the full Hessian in the transverse directions retains separate dependence on $\gamma$. This creates a scenario where two different sets of couplings can induce the exact same potential energy and stationary point within the restricted subspace but yield different curvatures in the transverse directions.

2. Minimum-to-Saddle Transition:
   The authors identify two bounded-from-below parameter choices that share the same reduced potential and stationary configuration. In one case, the configuration is a local minimum. In the other, where $\gamma$ crosses a critical value while $\sigma$ is held fixed, the configuration becomes a saddle point with six negative physical modes (tachyonic color–weak fluctuations). This establishes a precise local-stability boundary in parameter space that is invisible to restricted ansatz analyses.

3. Analytic Spectrum of Cross-Block Modes:
   The paper derives the complete analytic spectrum for the color–weak cross-block fluctuations. The masses for the two sixfold families ($\eta = \pm 1$) are shown to depend on the block amplitudes ($s_C, s_L$), the adjoint VEV ($\phi$), and the couplings $\kappa_2$ and $\gamma$. The condition for stability is found to be sensitive to $\gamma$, even when the diagonal ansatz suggests stability.

4. Benchmark Validation:
   Using specific benchmark points, the authors verify that the analytic spectrum derived from the reduced variables matches the results of direct Hessian calculations in the full 54-dimensional space. They confirm that the "minimum" and "saddle" distinctions hold in the unrestricted theory.

5. Thermal Analysis Scope:
   The paper includes a leading-order thermal mass analysis to determine the ordering of stationary branches at the onset of condensation. However, the authors explicitly limit the scope of this analysis, noting it does not address later phase transitions, bubble nucleation, defect network evolution, or gravitational wave production.

Significance
The paper's primary significance lies in demonstrating the limitations of symmetry-restricted VEV ansätze for determining vacuum stability in GUTs. It provides a concrete mechanism where a configuration that appears stable (and is a local minimum) within a symmetry-preserving subspace is actually a saddle point in the full theory due to transverse instabilities driven by portal interactions.

The authors emphasize that their results concern local vacuum geometry rather than a certified global phase diagram. The work serves as a rigorous check on the stability of intermediate gauge phases proposed for cosmological scenarios (such as monopole erasure), highlighting that the "reduced potential" often lacks the coupling information necessary to determine stability against fluctuations that break the specific symmetry of the ansatz. The findings suggest that for models relying on specific intermediate subgroups, a full Hessian analysis or a careful examination of transverse modes is essential to avoid misidentifying saddle points as stable vacua.