Technical Summary: Research on Readability Rating Methods for English Texts Based on Artificial Intelligence

1. Problem Statement

The paper addresses the limitations of current automated readability assessment systems, which struggle to balance semantic richness, interpretability, and computational efficiency. Traditional formula-driven methods (e.g., Flesch-Kincaid, Gunning Fog) rely on shallow linguistic features like sentence length and syllable counts, failing to capture semantic meaning, discourse continuity, and complex sentence structures. Conversely, while deep learning and transformer-based models (e.g., BERT, RoBERTa) excel at contextual understanding, they often require significant computational resources, lack interpretability, and may ignore explicit linguistic cues crucial for readability estimation. Existing hybrid approaches often fail to effectively integrate deep semantic embeddings with interpretable, handcrafted linguistic features within a unified regression framework.

2. Methodology

The study proposes a Hybrid RoBERTa–XGBoost Framework designed to predict continuous readability scores for English texts. The methodology follows a structured pipeline:

• Dataset: The research utilizes the CLEAR Corpus (CommonLit Ease of Readability), a benchmark dataset containing 4,724 annotated English text samples ranging from Grade 3 to Grade 12 levels. The data was split into an 80/20 training (3,779 samples) and testing (945 samples) set.
• Preprocessing: Raw text underwent normalization (lowercasing), removal of non-linguistic special characters, sentence segmentation, and tokenization using the RoBERTa tokenizer.
• Feature Extraction (Dual-Stream):
  1. Semantic Features: A RoBERTa-Base model was utilized to generate 768-dimensional semantic embeddings. The model was configured with a learning rate of 1.00E-05, a maximum sequence length of 512, a batch size of 16, and trained for 10 epochs; however, the paper explicitly states the model was employed in a frozen state without additional fine-tuning to extract contextual hidden representations. The [CLS] token representation was extracted to capture the global contextual meaning of each passage.
  2. Linguistic Features: 39 handcrafted features were engineered to capture structural properties, including average word length, sentence length, lexical density, punctuation ratios, conjunction ratios, and syntactic complexity.
• Feature Fusion: Both feature sets were normalized using Standard Scaling to ensure uniform distribution. The 768-dimensional semantic vectors and 39-dimensional linguistic vectors were concatenated to form a unified 807-dimensional feature vector.
• Model Training: The fused features were fed into an XGBoost Regressor. The model was optimized using Grid Search Cross-Validation (5-fold) to tune hyperparameters such as learning rate, tree depth, and the number of estimators. The objective function minimized squared error while applying regularization (L1 and L2) to prevent overfitting.

3. Key Contributions

The paper outlines four primary contributions:

1. Framework Development: The creation of an intelligent, hybrid AI-based regression framework for predicting continuous readability scores on English texts using the CLEAR Corpus.
2. Workflow Articulation: A comprehensive documentation of the end-to-end workflow, including dataset collection, preprocessing (handling missing values, text cleaning, tokenization), and the specific extraction of both RoBERTa semantic embeddings and engineered linguistic features.
3. Feature Fusion Strategy: The implementation of a specific fusion technique that combines RoBERTa-based semantic representations with linguistically designed features via scaling and concatenation, followed by XGBoost regression with Grid Search Cross-Validation for hyperparameter tuning.
4. Performance Evaluation: A rigorous evaluation of the proposed solution against baseline models (Linear Regression, Support Vector Regression, Random Forest) and ablation studies using standard regression metrics (RMSE, MAE, R²).

4. Experimental Results

The proposed model demonstrated superior performance compared to all baseline and ablation variants:

• Primary Metrics: The full hybrid model achieved an RMSE of 0.0980, an MAE of 0.0794, and an R² score of 0.9908.
• Comparative Performance:
  • vs. Baselines: The proposed model outperformed Linear Regression (R²: 0.944), Support Vector Regression (R²: 0.762), and Random Forest (R²: 0.965).
  • vs. Ablation Studies: The full model significantly outperformed models using only semantic features (RoBERTa + Precomputed: R² 0.882), only linguistic features (R² 0.685), or Only Fine-tuned RoBERTa (R² 0.722) as listed in the ablation study (Table 3).
• Error Analysis: Residual analysis indicated that errors were normally distributed around zero with minimal bias, confirming the model's reliability.
• Feature Importance: Analysis revealed that syntactic features, specifically Conjunction Ratio (0.1802) and Preposition Ratio (0.0946), were among the most influential predictors, validating the importance of integrating structural cues with semantic embeddings.

5. Significance and Claims

The paper claims that the integration of deep semantic embeddings with interpretable linguistic features significantly enhances prediction accuracy and generalization capability. The study asserts that this hybrid approach successfully bridges the gap between the contextual understanding of transformer models and the structural interpretability of traditional linguistic analysis.

The author positions the framework as a scalable solution for:

• Educational Content Adaptation: Tailoring learning materials to specific student levels.
• Manuscript Evaluation: Assisting in the assessment of text complexity for publishing.
• Natural Language Processing Applications: Providing a robust method for automated text leveling and content recommendation.

The paper concludes that the proposed method offers a scientifically founded basis for automated linguistic analysis, achieving high accuracy (capturing >99% of variance) while maintaining computational reliability through the use of optimized ensemble learning. Future work is suggested to explore cross-linguistic robustness, integration of Explainable AI (XAI) modules like SHAP, and the use of lightweight models (e.g., DistilRoBERTa) for pipeline efficiency.