Anti-topological crystal and non-Abelian liquid in twisted semiconductor bilayers

Technical Summary: Anti-topological Crystal and Non-Abelian Liquid in Twisted Semiconductor Bilayers

Problem Statement
Twisted bilayer MoTe$_2$ ($t$MoTe$_2$) hosts topological minibands where electron-electron interactions drive a rich phase diagram. Recent theoretical proposals suggest that at half-filling of the second moiré band ($n=3/2$ or $5/2$ depending on spin polarization), the system may realize a non-Abelian fractional Chern insulator (FCI). This state is motivated by the resemblance of the second miniband's wavefunctions to the first-excited Landau level (1LL), which supports non-Abelian states at half-filling. However, this resemblance is sensitive to microscopic details, such as the moiré potential landscape at small twist angles. A critical open question is identifying the energetic competitors to this proposed non-Abelian liquid, particularly electron crystals, which are known to compete with topological liquids in other regimes (e.g., the lowest Landau level or the lowest moiré band at $n=1/3$).

Methodology
The authors investigate the phase diagram of $t$MoTe$_2$ at half-filling of the second miniband using two distinct theoretical frameworks:

1. The Adiabatic Model: An approximation where the one-body Hamiltonian is treated as a particle in a periodic magnetic field and scalar potential, effectively mimicking Landau levels modulated by the moiré pattern.
2. The Lowest-Harmonic Continuum Model: A more microscopic model from which the adiabatic model is derived, retaining the full band structure details.

The study employs two primary numerical techniques:

• Band-projected Exact Diagonalization (ED): Calculations are performed on finite clusters (e.g., 24–28 particles) with full spin/valley polarization. The authors analyze many-body energy spectra, ground state degeneracies, and overlaps with model wavefunctions (Pfaffian and anti-Pfaffian). To detect crystallization in the presence of 1LL-like wavefunction suppression of density fluctuations, they introduce "modified" correlation functions (structure factor and pair correlation) that replace the projected density operator of the second band with an operator projected onto the 1LL basis.
• All-band Hartree-Fock (HF) Calculations: Self-consistent unrestricted HF calculations are performed in a plane-wave basis without projecting to specific sub-bands. This allows for the study of spontaneous symmetry breaking and band mixing effects, including the evolution of Chern numbers across band inversions.

Key Contributions and Results

1. Discovery of the Anti-topological Crystal
The central finding is the identification of a novel electron crystal phase, termed the "anti-topological crystal," which competes closely with the non-Abelian FCI.

• Structure: In both the adiabatic and lowest-harmonic models, this crystal phase exhibits a $2 \times 2$ enlargement of the moiré unit cell (quadrupling), evidenced by four-fold ground state quasidegeneracy with crystal momenta at the $m$-points of the moiré Brillouin zone.
• Topological Character: The defining feature is its total many-body Chern number ($C_{tot}$).
  • In the lowest-harmonic model, the system forms a crystal with $C_{tot} = 0$. This occurs because the first non-interacting band is fully occupied (contributing $C_1 = +1$), while the half-filled second band contributes $C_2 = -1$. The cancellation results in a topologically trivial insulator despite the underlying bands having non-zero Chern numbers.
  • In the adiabatic model, the authors find a richer phase diagram near the non-Abelian window. While a $C_{tot} = 3/2$ non-Abelian FCI appears at $\theta \approx 2.6^\circ$, the system transitions to crystal phases at slightly larger angles ($\theta \gtrsim 2.7^\circ$) and smaller angles ($\theta \lesssim 2.4^\circ$). These crystal phases possess $C_{tot} = 0$, $C_{tot} = 1$, and $C_{tot} = 2$, depending on the specific twist angle and model parameters.
• Distinction from Previous Work: Unlike the Quantum Anomalous Hall (QAH) crystal proposed in Ref. [32] (which occurs at half-filling of a single $C=+1$ band and has $C_{tot}=+1$), the anti-topological crystal occurs at $n=3/2$ filling of two $C=+1$ bands and has a vanishing total Chern number due to the cancellation of contributions.

2. Phase Diagram and Competing States

• Adiabatic Model: The phase diagram as a function of twist angle $\theta$ shows a window of non-Abelian FCI behavior (characterized by six-fold ground state degeneracy and high overlap with Pfaffian/anti-Pfaffian states) centered around $\theta \approx 2.6^\circ$. Flanking this window are crystal phases. The Pfaffian overlap is consistently higher than the anti-Pfaffian, suggesting the Pfaffian is the dominant non-Abelian candidate in the thermodynamic limit.
• Lowest-Harmonic Model: In this more microscopic model, the non-Abelian FCI phase is not observed. Instead, the system exhibits a robust crystal phase with $C_{tot}=0$ over a wide range of angles ($2.0^\circ \leq \theta \leq 2.65^\circ$). At larger angles ($\theta > 2.65^\circ$), the system transitions to a metallic Landau Fermi liquid.
• Robustness: The Hartree-Fock study confirms that the anti-topological crystal ($C_{tot}=0$) is robust against band mixing and persists across a quadratic band inversion where the non-interacting Chern number of the second band changes from $-1$ to $+1$.

3. Mechanism of Crystallization
The authors utilize a toy model (half-filled 1LL in a weak periodic potential) to understand the origin of these phases. They find that a weak periodic potential can induce a transition from a non-Abelian state to a crystal. However, the toy model reproduces the $C_{tot}=1$ and $C_{tot}=2$ crystals found in the adiabatic model but fails to reproduce the $C_{tot}=0$ anti-topological crystal. This suggests that the anti-topological crystal relies on stronger deviations from the conventional Landau level limit and specific features of the $t$MoTe$_2$ band structure not captured by the simple toy model.

Significance and Claims
The paper claims to establish the anti-topological crystal as a distinct and robust competitor to the non-Abelian FCI in twisted semiconductor bilayers.

• Novelty: It identifies a new type of electron crystal that exists in systems with multiple Chern bands at filling factors $n > 1$. Its topological triviality ($C_{tot}=0$) arises not from trivial bands, but from the precise cancellation of topological contributions from a full band and a half-filled band.
• Experimental Relevance: The authors suggest their theory offers a possible explanation for recent experimental observations of an insulating state near $n=3/2$ in finite magnetic fields, which was previously unexplained. They note that while a $C=2$ Chern insulator is observed at $n=2$, the $n=3/2$ state could be this anti-topological crystal.
• Theoretical Impact: The work highlights that the competition between topology and crystallization is more complex in multi-band systems than in single-band Landau level physics, where the "anti-topological" cancellation mechanism is unique to the specific filling and band structure of $t$MoTe$_2$.

The authors remain modest regarding the non-Abelian nature of the FCI in the lowest-harmonic model, noting that their calculations do not find evidence for it in that specific model, whereas the adiabatic model supports it. The primary contribution is the rigorous characterization of the competing crystal phases and the definition of the "anti-topological" class.