Noisy Nonreciprocal Pairwise Comparisons: Scale Variation, Noise Calibration, and Admissible Ranking Regions

1. Problem Statement

Pairwise comparisons are a cornerstone of decision analysis, preference modeling, and multicriteria evaluation. In an ideal setting, comparison matrices are reciprocal ($a_{ji} = -a_{ij}$ in additive form) and additively consistent ($a_{ik} = a_{ij} + a_{jk}$), allowing for a direct derivation of a global ranking vector.

However, in practice, observed matrices are often nonreciprocal and inconsistent. The standard approach treats nonreciprocity as a defect, immediately "brutally" projecting the matrix onto the space of reciprocal matrices (discarding the symmetric part) to extract a ranking.

The Core Issue: This paper argues that nonreciprocity is not always pure noise. It may arise from two distinct sources:

1. Random Perturbation: Hesitation, fatigue, or judgment errors.
2. Structured Scale Variation: A systematic shift in the evaluation scale (e.g., a decision-maker evaluating items differently depending on context or order).

Blindly projecting onto reciprocal matrices discards the structured scale variation, potentially treating meaningful information as noise. This leads to an overestimation of uncertainty and a distorted view of how strongly the data supports a specific ranking.

2. Methodology

The author proposes a structured additive model that decomposes the observed comparison matrix $X = (x_{ij})$ into three distinct components:

$$x_{ij} = \underbrace{(u_i - u_j)}_{\text{Latent Ranking}} + \underbrace{(s_i + s_j)}_{\text{Scale Deformation}} + \underbrace{\varepsilon_{ij}}_{\text{Residual Noise}}$$

Where:

• $u = (u_1, \dots, u_n)$ is the latent ranking vector (with $\sum u_i = 0$).
• $s = (s_1, \dots, s_n)$ is the scale-deformation vector (with $\sum s_i = 0$).
• $\varepsilon_{ij}$ represents random perturbations, assumed to follow a Gaussian distribution (justified by the Central Limit Theorem as an accumulation of small effects).

A. Estimation Strategy

The method utilizes the decomposition of the observed matrix $X$ into its antisymmetric part ($K$) and symmetric part ($H$):

• $K_{ij} = \frac{x_{ij} - x_{ji}}{2} \approx u_i - u_j + \text{noise}$
• $H_{ij} = \frac{x_{ij} + x_{ji}}{2} \approx s_i + s_j + \text{noise}$

Step 1: Separate Estimation
Instead of projecting $X$ directly, the model estimates $u$ and $s$ separately using Least Squares (LS) on the complete comparison graph:

• Latent Ranking ($\hat{u}$): Derived from the antisymmetric part $K$. The estimator is the row average of $K$: $\hat{u}_i = \frac{1}{n}\sum_j K_{ij}$.
• Scale Deformation ($\hat{s}$): Derived from the symmetric part $H$. The LS estimator is given by $\hat{s}_i = \frac{r_i - \bar{r}}{n-2}$, where $r_i$ is the row sum of $H$.

Step 2: Residual Calibration
Once the structured components ($\hat{u}$ and $\hat{s}$) are fitted, a residual matrix $\hat{E}$ is computed:
$$\hat{E}_{ij} = x_{ij} - (\hat{u}_i - \hat{u}_j + \hat{s}_i + \hat{s}_j)$$
This residual matrix contains only the unexplained random noise. Two indicators are calculated on $\hat{E}$ to calibrate the noise level:

1. Residual Reciprocity Indicator ($IR^2$): Measures the remaining asymmetry in the residuals.
2. Residual Triangular Indicator ($IC^2$): Measures the remaining violation of transitivity.

Under a Gaussian working model, these indicators provide explicit estimators for:

• $\hat{\sigma}$: The effective standard deviation of the residual noise.
• $\hat{\rho}$: The correlation between opposite residual judgments ($\varepsilon_{ij}$ and $\varepsilon_{ji}$).

B. Decision Diagnostics

The paper introduces two indices to determine if the scale deformation is "moderate" (i.e., does not alter the latent ranking):

1. Global Ratio ($\Lambda$): The ratio of the Frobenius norm of the scale deformation to the latent ranking component ($\|\hat{S}\|_F / \|\hat{U}\|_F$).
2. Ranking Impact Index ($\Gamma$): Compares the maximum scale deformation to the smallest gap between latent scores ($\Gamma = 2\|\hat{s}\|_\infty / \text{gap}(\hat{u})$). If $\Gamma \ll 1$, the ranking is stable; if $\Gamma \approx 1$ or $>1$, the scale deformation threatens the ranking order.

C. Probabilistic Ranking Regions

Rather than outputting a single ranking, the method assigns probabilities to ranking regions (sets of score vectors inducing a specific strict ordering).

• A Gaussian distribution is placed on the latent vector $U^*$ centered at $\hat{u}$ with covariance derived from $\hat{\sigma}$ and $\hat{\rho}$.
• Monte Carlo simulation is used to estimate the probability that the true ranking falls into specific regions (e.g., "Alternative A is ranked 1st").

3. Key Contributions

1. Novel Decomposition Model: Introduces an additive model that explicitly separates latent ranking, structured scale deformation, and random noise, challenging the view that nonreciprocity is solely an error.
2. Explicit Estimators: Provides closed-form Least Squares estimators for both the ranking vector and the scale-deformation vector.
3. Noise Calibration Framework: Develops a method to estimate residual noise levels ($\hat{\sigma}$) and dependence ($\hat{\rho}$) after removing structured effects, leading to more accurate uncertainty quantification.
4. Diagnostic Indicators: Defines $\Lambda$ and $\Gamma$ to quantify whether nonreciprocity is negligible, moderate, or influential, providing a decision-support tool to judge the validity of the ranking.
5. Probabilistic Output: Shifts from a point estimate to a probability distribution over ranking regions, offering a nuanced view of decision confidence.
6. Comparison with "Brutal" Projection: Demonstrates that while both the proposed method and brutal reciprocal projection yield the same central ranking estimate (in the least-squares setting), they differ fundamentally in uncertainty calibration.

4. Results and Findings

• Central Ranking Consistency: In the least-squares setting, the estimated latent ranking vector $\hat{u}$ is identical whether one uses the structured approach or the brutal reciprocal projection (which discards the symmetric part).
• Uncertainty Distortion: The "brutal" projection treats structured scale deformation as random noise. Consequently, it inflates the estimated noise level ($\hat{\sigma}$) and produces a more diffuse probability distribution over ranking regions.
• Admissibility Failure: Numerical illustrations show that under high noise, brutal reciprocal projection can produce a reciprocal matrix with a sign pattern that is inadmissible for any strict ranking (creating cycles), whereas the structured approach preserves the underlying ranking logic.
• Scale Deformation Impact: When scale deformation is moderate but non-negligible, the structured method correctly identifies it as a systematic effect, maintaining a tighter, more accurate uncertainty bound. The brutal method misinterprets this structure as high noise, leading to overly conservative (diffuse) decision support.

5. Significance

This paper offers a paradigm shift in handling nonreciprocal pairwise comparisons:

• From Correction to Interpretation: Instead of blindly "fixing" nonreciprocity, the method interprets it. It distinguishes between "noise" (random error) and "signal" (systematic scale variation).
• Improved Decision Support: By providing calibrated probabilities for ranking regions, decision-makers receive not just an ordering, but a measure of how strongly the data supports that ordering.
• Robustness: The approach prevents the loss of meaningful information (the symmetric part) and avoids the artificial inflation of uncertainty that occurs when structured effects are mistaken for random noise.
• Practical Applicability: The use of explicit estimators and Monte Carlo simulation makes the framework computationally feasible for real-world decision problems involving multiple criteria and alternatives.

In summary, Magnot's work argues that nonreciprocity is a feature, not just a bug, and that a structured decomposition of comparison data leads to more reliable and interpretable decision outcomes than traditional projection methods.