Evidence governance and boundary diagnosis in domain-specific university education quality evaluation: a robust fuzzy decision-support study

Technical Summary: Evidence Governance and Boundary Diagnosis in Domain-Specific University Education Quality Evaluation

Problem Statement
University quality evaluation increasingly relies on heterogeneous evidence sources, including institutional documents, expert judgments, student experiences, digital traces, and performance indicators. A central problem identified in the literature is that this evidence is rarely complete or unambiguous, yet evaluation systems frequently transform it into single scores or rigid categories. This reductionist approach obscures uncertainty and treats evaluation as a purely measurement exercise rather than a governance process. Consequently, institutions may face "performative compliance" where they manage indicators rather than improve practice, and boundary cases (institutions near category thresholds) are often misinterpreted as sharp quality distinctions when they are actually fragile. The paper argues that quality evaluation should be reframed as evidence governance: the institutional process of selecting, translating, weighting, and interpreting heterogeneous evidence to justify action, rather than merely generating a ranking.

Methodology
The study employs a robust fuzzy decision-support framework applied to an anonymised dataset of 29 university cases from a domain-specific education quality evaluation. The evaluation covers four dimensions: organisational leadership, resource support, staff development, and educational environment. The methodology consists of three linked analytical elements:

1. Uncertainty-Preserving Coding: Instead of assigning precise scores, evidence is represented as fuzzy-number intuitionistic fuzzy values. Each dimension is coded as a triplet for membership (positive evidence) and a triplet for non-membership (negative evidence), preserving the interval and uncertainty of expert judgments.
2. Order-Weight Robustness Checks: The study utilizes a Fuzzy-Number Intuitionistic Fuzzy Weighted Maclaurin Symmetric Mean (FNIFWMSM) aggregation model. Unlike simple additive models, this allows for interaction among dimensions (e.g., a weakness in one area cannot always be fully compensated by a strength in another). The model tests robustness by varying:
   • Interaction Order ($k$): Comparing second, third, and fourth-order specifications to determine how dimension interactions affect the final score.
   • Weight Fusion ($\beta$): Combining subjective expert weights and data-driven CRITIC weights using a compromise coefficient ($\beta$) ranging from 0 to 1 to test if results depend on specific weighting preferences.
   • Perturbation: Applying Gaussian noise and case deletion (rank reversal tests) to assess stability.
3. Boundary Diagnosis: After classification, the framework calculates the boundary distance ($D_i$) for each case relative to category thresholds. It identifies cases near thresholds and diagnoses the specific dimension responsible for their fragile position, transforming a ranking exercise into a diagnostic tool for institutional improvement.

Key Results

• Stability of Structure: Under the main specification (third-order interaction, $\beta=0.50$), the three-category structure (A+, A, A-) is stable. Robustness checks confirm that the level structure is preserved across different interaction orders and weight fusion scenarios (Kendall correlations ranging from 0.897 to 1.000).
• Fragility of Boundary Cases: The analysis reveals that aggregate scores can be misleading. For instance, Case A5 (Score: 0.8586, Category: A+) and Case A9 (Score: 0.8569, Category: A) have nearly identical scores but fall on opposite sides of the A+/A threshold.
  • A5 is constrained primarily by resource support.
  • A9 is constrained primarily by staff development.
• Diagnostic Value: The framework demonstrates that a simple category label is insufficient. Boundary diagnosis shows that A5 requires consolidation in resource support to secure its A+ status, while A9 requires staff development improvements to move upward. Similarly, Case A13 (Category A) and Case A28 (Category A-) are shown to be constrained by staff development, preventing them from reaching higher categories despite otherwise strong performance.
• Noise Sensitivity: Under stronger noise perturbation, level consistency decreases slightly (to ~88.67%), indicating that near-boundary cases remain sensitive to data disturbance. The authors interpret this not as a flaw, but as a necessary disclosure of uncertainty that should trigger secondary expert intervention rather than automated closure.

Key Contributions
The paper claims three primary contributions:

1. Conceptual: It advances the shift from "quality scoring" to evidence governance. It posits that evaluation is not a neutral measurement but a process of producing categories through coding, weighting, and thresholding. Making these steps visible constitutes a form of governance that enhances accountability.
2. Analytical: It revalues boundary cases. Rather than treating them as inconvenient outliers, the study argues they are analytically valuable because they reveal where categories are most fragile and where governance attention is most needed. The framework turns these cases into sites of organisational learning.
3. Methodological: It demonstrates how computational tools (fuzzy logic, symmetric aggregation) can support social science research without replacing human judgment. The model functions as a transparent evidence pathway and accountability mechanism, supporting interpretation rather than acting as an automated decision-maker.

Significance and Claims
The paper positions its work within the broader context of "responsible quantification" and AI-era evidence use. It claims that as universities increasingly rely on data and computational tools, evaluation systems must be designed to support institutional learning, accountability, and fair governance rather than just classification.

The authors assert that their framework does not reject measurement but rejects the assumption that measurement can stand outside interpretation. By exposing uncertainty and linking category labels to specific improvement priorities (e.g., distinguishing between resource constraints and staff development issues), the study aims to move evaluation away from "reward-punishment" logic toward a "governance conversation." The significance lies in providing a method to make uncertain evidence auditable and to ensure that technical tools support, rather than obscure, institutional judgment and transparency.