💬 NLP

A categorical error sensitivity index (ISEC): A preventive ordinal decision-support measure for irrecoverable errors in manual data entry systems

This paper introduces the Categorical Error Sensitivity Index (ISEC), a scalable, vector-based ordinal metric that integrates semantic, morphological, and empirical data to proactively identify and rank structural risks of irrecoverable categorical errors in manual data entry systems, thereby enhancing decision-making for small and medium-sized enterprises.

Original authors: Ricardo Raúl Palma, Mauro Anibal Benetti, Fabricio Orlando Sanchez Varretti

Published 2026-05-13

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Ricardo Raúl Palma, Mauro Anibal Benetti, Fabricio Orlando Sanchez Varretti

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are running a small shop. You have a notebook where you write down every item you sell. To keep things simple, you use short codes like "RedShirt," "BlueShirt," or "RedHat."

Now, imagine a customer asks for a "RedShirt," but you accidentally write "RedShit." Or, you have two items: "Caba" (a city) and "Cba" (another city). If you type them wrong, the computer might mix them up.

In a big company with fancy robots, a computer would catch these mistakes. But in small businesses, humans type everything. Sometimes, the mistake looks so much like the right answer that the computer can't tell the difference. Once that mistake is saved, it's gone forever. It ruins your sales reports, and you might think you sold 100 "RedShits" instead of "RedShirts."

This paper introduces a tool called ISEC (Categorical Error Sensitivity Index). Think of ISEC as a "Fragility Detector" for your list of codes.

Here is how it works, using simple analogies:

1. The Problem: The "Look-Alike" Trap

The authors say that some categories are naturally "fragile."

The Analogy: Imagine two houses on a street. If they are far apart, it's easy to tell them apart. But if they are right next to each other, and you take one small step (a typo), you might accidentally knock on the wrong door.
The Paper's Claim: When two categories are very similar in how they sound (semantics) or how they are spelled (morphology), a tiny human error can make them indistinguishable. The paper calls this "Distance Compression." It's like squeezing two magnets together until they snap into the wrong spot.

2. The Solution: The "Fragility Score"

ISEC gives every pair of categories a score.

High Score: These two categories are dangerous. They look too much alike, or they are often used in ways that make them easy to confuse. If you use them, you are at high risk of a permanent mistake.
Low Score: These categories are safe. They are distinct enough that a typo won't cause confusion.

3. How It Calculates the Score (The Recipe)

The paper explains that ISEC doesn't just look at one thing; it mixes three ingredients:

Meaning (The "What"): Does "Apple" mean the same thing as "Aple"? (Semantic distance).
Spelling (The "How"): How many keystrokes does it take to turn "Apple" into "Aple"? (Morphological distance).
Popularity (The "How Often"): If you sell 1,000 "Apples" and only 1 "Aple," a mistake is much more damaging because it affects more data.

The Magic Ingredient: The paper adds a special "penalty" for certain types of typos. For example, on a keyboard, the letter 'G' is right next to 'T'. If you type 'G' instead of 'T', it's a very easy mistake. ISEC knows this and says, "Hey, these two are extra dangerous because they are neighbors on the keyboard."

4. Why It's Fast (The "Smart Search")

Checking every single pair of items in a huge list is like trying to find a specific needle in a haystack by looking at every single piece of hay one by one. It takes forever.

The Paper's Trick: ISEC uses a "smart search" (Vector Database). Instead of checking everything, it quickly finds the "neighbors" (the items that look most similar) and only checks those.
The Result: The paper claims this makes the process 195 times faster. It turns a task that would take months into one that takes minutes.

5. Real-World Tests

The authors tested this tool on three very different things:

Government Court Records: They found that short abbreviations for cities (like "CBA" vs. "CABA") were extremely risky. A single missing letter could mix up legal records.
Supermarket Inventory: They found that similar-sounding food items (like "Chicharrón" in different sections) were often confused, even if they were in different categories.
Metal Parts (ISO Codes): They tested a list of metal tool codes. They found that tiny typos could turn one valid metal part code into a completely different, but still valid, metal part code.

The Big Takeaway

The paper argues that Data Quality isn't just about fixing mistakes after they happen. It's about designing your lists so that mistakes can't happen in the first place.

ISEC is a preventive tool. It tells a business owner: "Don't use these two codes together. They are too close. Change one of them before you start typing, or your future reports will be wrong."

It shifts the focus from "Cleaning up the mess" to "Building a stronger foundation."

Technical Summary: A Categorical Error Sensitivity Index (ISEC)

1. Problem Statement

Small and Medium-sized Enterprises (SMEs) often rely on manual data entry for categorical information (e.g., SKUs, supplier names, incident types) due to limited resources for automated validation. This reliance creates a structural vulnerability known as categorical fragility. When nominal categories exhibit semantic or morphological proximity, manual entry errors can result in irrecoverable distortions.

Unlike missing data or format violations, these errors are "irrecoverable" because:

They produce syntactically valid categories.
They do not violate database constraints.
They remain undetected during aggregation.
The original intended label cannot be reconstructed ex post.

When an erroneous entry falls equidistant between two valid categories (or closer to an incorrect one) due to a small stochastic perturbation (typo), the classification decision becomes probabilistically indeterminate. This leads to structured misinformation in Decision Support Systems (DSS), where Key Performance Indicators (KPIs) become internally consistent but semantically distorted. Current solutions fail because they either evaluate morphological and semantic dimensions in isolation or rely on heavy computational resources and standard dictionaries unsuited for SME-specific jargon and abbreviations.

2. Methodology: The ISEC Framework

The paper introduces the Categorical Error Sensitivity Index (ISEC), an ordinal composite score designed to rank pairs of categories based on their structural susceptibility to confusion before errors occur. ISEC shifts the paradigm from reactive data cleaning to preventive risk modeling.

2.1 Mathematical Formulation

For a pair of distinct categories $(c_i, c_j)$ , the index is defined as:

$ISEC(i,j) = \frac{1 + FMN(i,j)}{DSN(i,j)^\alpha \times CMP(i,j)^{1-\alpha}}$

Where:

$FMN(i,j)$ (Normalized Mean Frequency): The base-10 logarithm of the average empirical frequency of the two categories. This weights the index by operational exposure; high-frequency pairs pose greater risk if confused.
$DSN(i,j)$ (Normalized Semantic Distance): Derived from the cosine similarity of word embeddings ( $1 - \text{CosineSimilarity}$ ), normalized to the range $[0, 1]$ . It measures semantic dissimilarity.
$CMP(i,j)$ (Penalized Mean Cost): A morphological cost metric based on a weighted Damerau–Levenshtein algorithm.
- It includes a Mean Transformation Cost ($CM$): The average weighted cost of edit operations (insertion, deletion, substitution, transposition).
- It includes a Penalty Cost ($CP$): A linear accumulation of insertions, deletions, and substitutions (excluding transpositions to preserve linear growth with string length).
- $k$ Parameter: A tunable factor ( $k \ge 0$ ) that amplifies the penalty for cumulative structural divergence.
$\alpha$ Parameter: A tuning parameter ( $0 \le \alpha \le 1$ ) balancing semantic vs. morphological weight. $\alpha \to 1$ prioritizes meaning (long descriptions), while $\alpha \to 0$ prioritizes structure (short codes/SKUs).

2.2 Customization and Governance

ISEC utilizes character-level transformation cost matrices rather than unitary costs. This allows organizations to encode specific error tendencies:

Keyboard Proximity: Substitutions between adjacent keys (e.g., 'G' and 'T' on QWERTY) can be assigned lower costs.
Visual Confusion: Confusing characters (e.g., '0' and 'O') can be weighted differently.
Governance Surface: By adjusting the cost matrices and the parameter $k$ , organizations explicitly define their risk tolerance and assumptions about human-machine interaction.

2.3 Computational Optimization

To address the $O(N^2)$ complexity of brute-force distance calculation for large catalogs, ISEC employs a Hybrid Search Strategy using Vector Databases (e.g., ChromaDB with HNSW algorithms):

Semantic Filtering (Breadth-First): Retrieves the Top- $K$ semantic neighbors for a category in $O(\log N)$ time.
Morphological Calculation (Greedy): Computes the full $CMP$ only against these $K$ candidates in $O(K)$ time.
This reduces overall complexity to approximately $O(\log N + K)$ , enabling scalable processing.

3. Empirical Validation

The authors validated ISEC across three heterogeneous datasets:

Judicial Administrative Records (Argentina):
- Data: 1.06 million records involving provincial abbreviations (e.g., "caba" vs. "cba").
- Result: Identified high-risk pairs where single-character differences in high-frequency abbreviations created irrecoverable confusion. Processing time was 34 seconds.
Retail Supermarket Inventory:
- Data: 25,638 product descriptions across 14 ontological groups.
- Result: Detected semantic collisions across different groups (e.g., "Bizcocho achiras" in Bakery vs. Sweets) that traditional semantic models missed due to structural token overlap. Processing time dropped from an estimated 187 days (brute-force) to 12 minutes.
Synthetic ISO-Based Metalworking Catalog:
- Data: 1,000 valid ISO codes (ISO 1832) with no historical frequency (FMN set to 1).
- Result: Confirmed "silent" vulnerabilities where minor typographical noise generated valid but incorrect codes (e.g., transpositions in codes like 'AAGX' vs 'AGAX').

4. Key Contributions

Ex-Ante Structural Risk Modeling: ISEC treats categorical fragility as a measurable geometric property of the taxonomy itself, independent of whether errors have already occurred.
Ordinal Sensitivity Ranking: Rather than predicting absolute error probabilities (which requires unrealistic behavioral assumptions), ISEC provides a robust ordinal ranking of vulnerability for prioritization.
Unified Dimensional Integration: It bridges the gap between morphological distance (typographical effort) and semantic distance (meaning), which are often treated separately in literature.
Configurable Governance: The framework allows organizations to encode empirical error tendencies and risk preferences directly into the metric via cost matrices and the $k$ parameter.
Technological Sovereignty: The solution is built on open-source stacks, making it accessible to SMEs without proprietary software or massive training corpora.

5. Significance and Claims

The paper argues that information robustness (the ability of a categorical system to preserve semantic separability under bounded stochastic perturbation) is a distinct dimension of data quality that precedes record-level accuracy.

Preventive vs. Reactive: ISEC enables the detection of latent structural risks before they propagate into managerial metrics, preventing "structured misinformation" where dashboards appear coherent but are semantically distorted.
Decision-Theoretic Design: By identifying fragile taxonomic configurations, ISEC supports the redesign of User Interfaces (UI), the consolidation of categories, and the implementation of targeted validation rules.
Scalability: The hybrid search architecture makes the analysis of large, heterogeneous datasets feasible for organizations with limited computational resources.

The authors conclude that while ISEC currently relies on manually defined cost matrices, it provides a scalable, proactive instrument for data governance that transforms categorical fragility from an operational nuisance into a manageable structural risk. Future work suggested includes automating cost matrix learning via Hidden Markov Models and integrating ISEC into open-source data governance platforms.