Reassessment of ammonia self- and air-broadened half-widths in the HITRAN database

Technical Summary: Reassessment of Ammonia Self- and Air-Broadened Half-Widths in the HITRAN Database

Problem Statement
Accurate pressure-broadening parameters for ammonia (NH$_3$) are critical for reliable line-by-line simulations in atmospheric sensing, combustion diagnostics, and planetary spectroscopy. The current treatment of NH$_3$ self- ($\gamma_{self}$) and air-broadened ($\gamma_{air}$) Lorentz half-widths in the HITRAN database relies heavily on the Nemtchinov et al. [31] empirical correlation, originally developed for the 10-$\mu$m region. While this correlation captures rotational trends within its measured range, it exhibits unphysical behavior when extrapolated to higher rotational quantum numbers ($J''$ and $K''$), often yielding negative or unrealistically small half-widths. To prevent these nonphysical values, HITRAN implementations (from HITRAN2012 through 2024) employ a hybrid approach: applying the Nemtchinov polynomial for low-to-moderate $J''$ transitions and "clamping" or assigning fixed default values for high-$J''$ transitions.

This hybrid treatment introduces artificial discontinuities and suppresses the measured rotational dependence of the linewidths, particularly for high-$J''$ transitions which become increasingly significant in high-temperature applications such as combustion and exoplanet atmospheres. Furthermore, the existing dataset used for the original correlation is now over two decades old, while numerous recent Fourier-transform infrared (FTIR) and laser-based studies have provided broader constraints across multiple vibrational bands and branches.

Methodology
To address these limitations, the authors compiled a comprehensive dataset of published laboratory measurements, comprising 1,317 self-broadened and 1,231 air-broadened half-widths at or reduced to 296 K. This dataset spans multiple vibrational bands ($\nu_1, \nu_2, \nu_3, \nu_4$, and various combinations/overtone bands) and covers rotational quantum numbers up to $J'' = 16$.

The study proceeded through the following steps:

1. Data Compilation and Verification: Measurements were gathered from various sources, including corrections for known errors in previous literature (e.g., pressure-unit handling errors in Fabian et al. [47]). Air-broadening coefficients not directly measured were inferred using the dry-air approximation ($\gamma_{air} = 0.79\gamma_{N_2} + 0.21\gamma_{O_2}$).
2. Analysis of Rotational Dependence: The analysis confirmed that the dominant dependence of the linewidths is rotational (functions of the branch-dependent index $m$ and lower-state projection quantum number $K''$), rather than specific to vibrational bands, branches, or inversion states.
3. Development of New Correlations: Constrained third-degree polynomial correlations were fitted for $\gamma_{self}$ and $\gamma_{air}$ as functions of $m$ and $K''$. The regression utilized relative-uncertainty weighting to prioritize high-precision measurements.
4. Physical Constraints: The fitting process imposed strict physical constraints over the intended HITRAN domain ($0 \le K'' \le J'' \le 22$):
   • Predicted coefficients must remain strictly positive.
   • Coefficients must decrease with increasing $J''$ at fixed $K''$.
   • Coefficients must increase with increasing $K''$ at fixed $J''$.
     These constraints were designed to prevent the negative extrapolations seen in the original Nemtchinov polynomial and the artificial plateaus introduced by HITRAN's clamping rules.

Key Results
The new constrained correlations demonstrate significant improvements over both the current HITRAN2024 values and the original extrapolated Nemtchinov correlation:

• Statistical Improvement:

  • Air-broadening: The Mean Absolute Percentage Error (MAPE) decreased from 10.95% (HITRAN2024) and 9.28% (Nemtchinov) to 6.80% for the new fit. The uncertainty-weighted MAPE (wMAPE) improved from 8.64% to 3.63%.
  • Self-broadening: The MAPE decreased from 23.61% (HITRAN2024) and 12.13% (Nemtchinov) to 10.89%. The wMAPE improved from 5.87% to 3.78%.
  • High-$J$ Transitions: The improvement is most pronounced for high-$J''$ transitions ($J'' > 8$), which are critical for high-temperature applications. For self-broadening in this regime, the MAPE dropped from 55.30% (HITRAN) to 10.62% (new fit).

• Spectral Validation:

  • Foreign-Gas Broadening: Comparisons with Pacific Northwest National Laboratory (PNNL) NH$_3$/N$_2$ spectra showed that the new $\gamma_{air}$ coefficients produce simulated absorption profiles that better match measured peak heights and shapes, particularly in the $\nu_2$ Q-branch and high-$J$ R-branch regions where HITRAN previously overestimated broadening (leading to underestimated peak heights).
  • Self-Broadening: Comparisons with pure-NH$_3$ spectra from Alturaifi and Petersen [45] revealed that the new $\gamma_{self}$ coefficients correct characteristic "gull-wing" residuals in the line shapes, indicating a more accurate reproduction of pressure-broadened profiles, especially for high-$J$ Q-branch transitions.

Significance and Claims
The paper claims to provide a "physically constrained, database-ready replacement" for the current HITRAN NH$_3$ broadening treatment. The primary significance lies in resolving the conflict between the need to avoid nonphysical extrapolations and the need to preserve measured rotational trends. By replacing HITRAN's hybrid clamping/default rules with a single, continuous, and physically bounded empirical correlation, the authors argue that line-by-line simulations will achieve higher fidelity, particularly for:

1. High-Temperature Applications: Combustion diagnostics and exoplanet atmospheres (e.g., WASP-43b) where high-$J$ transitions dominate.
2. Atmospheric Retrievals: Improved accuracy in satellite and ground-based NH$_3$ abundance retrievals.

The authors modestly note that while the new correlations improve the representation of rotational dependence, they do not claim to resolve all discrepancies, as some residual differences in spectral simulations persist due to other factors such as line-strength uncertainties and the complexity of overlapping transitions in congested spectral windows. The work is presented as a necessary update to the database to reflect the expanded experimental literature accumulated since the original Nemtchinov correlation was established.