💻 computer science

A Hybrid Semantic–Lexical Framework for Intelligent Data Quality Enhancement Using Ontology Reasoning and NLP-Based Validation

This paper proposes a hybrid semantic–lexical framework that integrates ontology-based reasoning with NLP-driven lexical validation to significantly enhance contextual anomaly detection and intelligent data correction in heterogeneous textual datasets, as demonstrated by superior performance on polluted healthcare data compared to traditional lexical-only approaches.

Original authors: Ismail El Gayar, Hesham Hassan, Lamia Abo Zaid

Published 2026-09-23

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Ismail El Gayar, Hesham Hassan, Lamia Abo Zaid

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the modern world, data is the fuel that powers everything from hospital records to financial markets. Yet, this fuel is often dirty. Just as a car engine sputters on contaminated gasoline, intelligent computer systems fail when fed inaccurate, incomplete, or confusing information. This problem is especially acute with text, where a simple typo or a misplaced word can change the entire meaning of a record. Traditional methods for cleaning this data have relied on checking for obvious errors, like missing numbers or words that don't fit a strict list. However, these methods often miss deeper problems where the words are spelled correctly but make no sense in their specific context. To solve this, researchers are turning to a combination of two powerful tools: a structured map of how the world works, known as an ontology, and the ability of computers to understand human language, known as natural language processing.

A team of researchers has developed a new system that merges these two approaches to create a smarter way to clean data. Their work, published in a recent study, introduces a hybrid framework designed to detect and fix errors in messy text datasets, particularly those found in healthcare. The system does not just look for typos; it understands the meaning behind the words. By combining a formal knowledge base that defines how medical concepts relate to one another with advanced language analysis, the framework can spot inconsistencies that older methods would overlook. For instance, it can identify that a patient's symptom description is syntactically correct but medically impossible given their other recorded conditions.

The researchers tested their system using a dataset derived from COVID-19 health records, a field where data accuracy is critical for tracking diseases and making life-saving decisions. They deliberately introduced various types of errors into the data, including missing values, misspelled medical terms, and semantically confusing entries, to simulate the kind of mistakes that happen in real-world record-keeping. The system was then tasked with finding these errors and suggesting corrections. Unlike previous methods that might only check if a word is spelled correctly or if a number is within a certain range, this new framework checks if the information fits the broader story of the patient's record. It uses a structured map of medical knowledge to understand that certain symptoms belong to specific diseases, and it uses language analysis to catch subtle spelling variations that a human might miss.

The results of the experiment were clear. The hybrid system significantly outperformed approaches that relied on language analysis alone. When the researchers tested the system without the structured knowledge map, it struggled to distinguish between words that were merely unusual and words that were actually wrong in a medical context. However, once the system was equipped with the ontology, its ability to detect true errors improved dramatically. In one specific test case, the system's accuracy in identifying correct data points rose from 60 percent to over 96 percent after the knowledge map was fully integrated. This suggests that the combination of understanding the rules of a domain and analyzing the text itself is far more effective than using either method in isolation.

The study also measured how stable the system was when run multiple times. The results showed very little variation in performance, indicating that the framework is reliable and consistent. It managed to find errors and suggest fixes with high precision, meaning that when it flagged a record as problematic, it was almost certainly right. While the system did not catch every single error, the ones it did catch were highly trustworthy, reducing the risk of false alarms that could waste human time. The researchers noted that the system works best when the underlying knowledge map is complete and accurate, highlighting that the quality of the data depends heavily on the quality of the knowledge used to clean it.

This work represents a significant step forward in how we handle the vast amounts of text data generated every day. By teaching computers to understand not just the words but the relationships between them, the researchers have created a tool that can clean data with a level of intelligence previously reserved for human experts. The framework proved particularly effective in the complex and high-stakes environment of healthcare, where a single error can have serious consequences. The study concludes that while the system is not perfect and relies on the quality of its knowledge base, it offers a scalable and robust solution for improving data quality in heterogeneous environments. As data continues to grow in volume and complexity, such intelligent systems will become essential for ensuring that the information driving our decisions is as clean and reliable as possible.

Technical Summary: A Hybrid Semantic–Lexical Framework for Intelligent Data Quality Enhancement

Problem Statement

The increasing reliance on data-driven systems and intelligent analytical platforms has highlighted a critical vulnerability: the degradation of data quality within heterogeneous digital environments. Traditional data quality approaches predominantly rely on statistical validation, rule-based cleansing, and syntactic correction. While effective for structured numerical data, these methods often fail to capture semantic inconsistencies and contextual anomalies within textual datasets.

In heterogeneous textual environments, data quality issues arise from human entry errors, inconsistent formatting, migration failures, and semantic ambiguities. Existing frameworks frequently address isolated dimensions of quality (e.g., completeness or accuracy) or treat semantic validation and lexical correction as separate tasks. Consequently, semantically invalid but syntactically correct records often remain undetected, leading to unreliable analytics and the "Garbage In, Garbage Out" (GIGO) phenomenon in machine learning applications. There is a distinct lack of unified frameworks that integrate deep semantic understanding with intelligent textual correction.

Methodology

The paper proposes an Intelligent Dynamic Framework that unifies ontology-based semantic reasoning with Natural Language Processing (NLP)-based lexical validation within a context-aware architecture. The framework operates through a six-layer pipeline:

Input Layer: Ingests heterogeneous textual datasets and a semantic knowledge base (ontologies and knowledge graphs).
Semantic Knowledge Layer: Performs ontology modelling (using OWL), defines semantic constraints, and constructs knowledge graphs to represent domain entities, relationships, and contextual rules.
NLP Processing Layer: Applies text preprocessing (tokenization, normalization) and utilizes the Damerau–Levenshtein distance algorithm to compute lexical similarity. This identifies typographical errors, insertions, deletions, substitutions, and transpositions.
Hybrid Validation Layer: The core innovation where semantic and lexical validation occur simultaneously.
- Semantic Validation: Verifies contextual consistency against ontology constraints and knowledge graph dependencies.
- Lexical Validation: Evaluates textual similarity and detects syntactic anomalies.
- Integration: A weighted scoring mechanism ( $V(d_i) = \alpha V_s(d_i) + (1-\alpha) V_l(d_i)$ ) combines semantic compatibility and lexical distance to classify records as valid or anomalous.
Correction Layer: Performs semantic-aware correction. Instead of relying solely on predefined rules, it generates candidate corrections ranked by both lexical similarity and semantic compatibility with the ontology.
Output Layer: Produces the corrected dataset, detailed anomaly reports, and a list of undefined semantic entities requiring human review.

The framework is formally defined as $F = \langle D, K, M, V, C, O \rangle$ , where $D$ is the dataset, $K$ is the knowledge base, $M$ is mapping, $V$ is validation, $C$ is correction, and $O$ is the output.

Key Contributions

The paper outlines four primary contributions:

Hybrid Framework: A unified architecture combining ontology-based semantic reasoning with NLP-based lexical analysis for intelligent contextual data correction.
Automated Semantic Validation: A mechanism capable of identifying semantic inconsistencies and contextual anomalies in heterogeneous textual datasets, moving beyond surface-level syntactic checks.
Intelligent Correction Process: A correction engine that integrates lexical similarity analysis with semantic knowledge constraints to rank and select the most contextually appropriate corrections.
Comprehensive Evaluation: An experimental study demonstrating the framework's effectiveness using controlled, polluted healthcare datasets.

Experimental Results

The framework was evaluated using polluted healthcare datasets derived from COVID-19 data sources (Kaggle), containing approximately 100 records and 1,500 textual values. Controlled anomalies (missing values, typos, semantic inconsistencies) were injected at a rate of ~40%.

Performance Metrics: The study utilized Precision, Recall, and F1-score.
Ablation Study: The research compared three levels:
- Level 1 (Lexical Only): Precision of 0.60.
- Level 2 (Hybrid with Ontology): Precision increased to 0.731.
- Level 3 (Enhanced Semantic Knowledge): Precision reached 0.9615, with Recall at 0.625 and F1-score at 0.7575.
Stability: Statistical stability analysis across repeated executions showed low standard deviations (Precision $\pm$ 0.01, Recall $\pm$ 0.02, F1-score $\pm$ 0.01), indicating robust and consistent performance. The mean values obtained during these repeated executions were Precision 0.96, Recall 0.94, and F1-score 0.95.
Efficiency: While the worst-case computational complexity approaches $O(n^2)$ , practical execution time was maintained at acceptable levels for medium-scale datasets due to semantic filtering and ontology-based candidate restriction.

Significance and Claims

The authors claim that integrating ontology-based semantic reasoning with NLP-based lexical analysis significantly improves contextual anomaly detection and semantic-aware correction performance compared to lexical-only or rule-based approaches.

The paper asserts that the framework provides a scalable and context-aware solution for intelligent data quality enhancement. Its significance lies in its ability to detect "semantically inconsistent yet syntactically valid" records—a common failure point for traditional methods. By leveraging domain-specific ontologies (specifically the CODO ontology for COVID-19 data), the framework reduces false positives and ensures that corrections are not just textually similar but semantically compatible with the domain context.

The authors maintain a modest tone regarding limitations, acknowledging that the framework's effectiveness is dependent on the completeness and quality of the underlying ontology. They note that the current evaluation was limited to medium-scale datasets and that future work is required to address large-scale enterprise environments and dynamic data streams. The paper concludes that while the approach is promising, it requires continuous ontology updates to reflect evolving domain semantics.