Linking Hadith Narrator Identities Across Heterogeneous Arabic Biographical Databases: A Multi-Signal Entity Resolution Pipeline

Technical Summary: Linking Hadith Narrator Identities Across Heterogeneous Arabic Biographical Databases

Problem Statement
The computational analysis of Islamic Hadith literature is currently hindered by the fragmentation of narrator data. While large-scale digital corpora exist—such as the Sanadset 650K (containing 650,986 Hadith records with narrator chains) and biographical databases like Hadithtransmitters (hawramani) and Muslimscholars.info—these resources operate in isolation. They lack common identifiers and utilize divergent name representations (e.g., varying nasab chains, kunyah vs. ism forms, and orthographic variations). Existing entity resolution approaches for Arabic are largely designed for modern, Wikipedia-aligned knowledge bases and fail to address the specific challenges of historical Islamic names, such as variable-length nasab chains and morphological case variations. Consequently, social network analyses of transmission chains (sanad) are constrained to single collections without access to cross-source biographical metadata (e.g., death years, reliability grades), and disambiguation systems operate within closed-world assumptions that do not facilitate linking across independent databases.

Methodology
The paper proposes a two-phase entity resolution pipeline designed to link narrator names from Sanadset to hawramani, and subsequently link hawramani to muslimscholars. The pipeline relies on domain-specific Arabic normalization and a multi-signal scoring framework.

1. Arabic Name Normalization: Before any comparison, a shared normalization function norm(t) is applied to all three databases. This process:

   • Strips diacritics (tashkeel) using explicit, disjoint Unicode ranges to avoid accidentally removing consonants.
   • Unifies five Alef variants into a single bare Alef.
   • Normalizes Ta marbuta to ha and Alef maqsura to ya.
   • Removes Tatweel and collapses whitespace.

2. Candidate Generation: To avoid computationally prohibitive $O(n \times m)$ comparisons, the system employs a bigram prefix index. Records are indexed by pairs of consecutive normalized tokens. Candidates are retrieved as the union of records sharing at least one bigram with the query name, reducing the comparison burden by orders of magnitude.

3. Phase 1: Sanadset $\rightarrow$ hawramani (Name-Only):

   • Constraint: Sanadset entries lack biographical metadata (no death years or grades).
   • Method: Matching relies solely on name similarity using token-sorted fuzzy string matching (TSR). Tokens are sorted alphabetically to mitigate variations in the order of nasab chains.
   • Scoring: A similarity score $s_{name} \in [0, 1]$ is computed. Links are established if $s_{name} \ge 0.80$.
   • Confidence: HIGH ($\ge 0.90$) and MEDIUM ($\ge 0.80$).

4. Phase 2: hawramani $\leftrightarrow$ muslimscholars (Multi-Signal):

   • Method: A weighted multi-signal function combines name similarity, death-year proximity, and reliability grade polarity.
   • Signals:
     • Name Similarity: Weighted at 0.70 (or 0.50 if death years are available on both sides).
     • Death-Year Proximity: Calculated as $s_{death}(\Delta y)$ based on the absolute difference in Hijri years. Scores range from 1.0 (exact match) to 0.0 (>10 years difference). If death years are missing, a neutral score of 0.50 is applied.
     • Grade Polarity: Reliability grades are collapsed into positive, negative, or unknown classes (scores 1.0, 0.0, 0.5).
     • Source Bonus: A small bonus ($\delta_{src} = 0.05$) is added if the hawramani record has five or more source citations.
   • Dynamic Weighting: Because death years are sparse in hawramani (available for <0.1% of matched pairs in the initial run, though the paper notes improved extraction covers 27.2% in the results), the system dynamically adjusts weights. For the vast majority of pairs, the effective weight distribution is ~70% name, 10% death (neutral), and 10% grade.
   • Confidence: HIGH ($\ge 0.85$), MEDIUM ($\ge 0.65$), and LOW ($\ge 0.45$).

5. Graph Construction: The linked data is used to construct a directed transmission graph where edges represent student-to-teacher relationships. Node metadata is enriched with cross-source biographical details. Chain confidence for transitive links (Sanadset $\rightarrow$ hawramani $\rightarrow$ muslimscholars) is determined by the minimum confidence tier of the constituent links.

Key Results

• Phase 1 (Sanadset $\rightarrow$ hawramani): Out of 185,216 unique narrator name variants in Sanadset, 94,628 (51.1%) were successfully linked.
  • HIGH confidence: 39,938 (21.6%).
  • MEDIUM confidence: 54,690 (29.5%).
  • 47.7% of narrators had no bigram candidates, typically due to highly abbreviated forms or absence from the target database.
• Phase 2 (hawramani $\leftrightarrow$ muslimscholars): 95,573 links were established, covering 94.7% of the 100,915 hawramani entries.
  • HIGH confidence: 18,245 (18.1%).
  • MEDIUM confidence: 71,546 (70.9%).
  • LOW confidence: 5,782 (5.7%).
  • The dominance of MEDIUM confidence is attributed to data sparsity (lack of death years on both sides), forcing reliance on name similarity.
• Transmission Graph: The pipeline produced a directed graph with 185,216 nodes and 814,093 edges. This is approximately 90 times larger than the next largest published Hadith narrator graph. The graph exhibits scale-free properties with power-law degree distributions.
• Enrichment: 93,588 narrators achieved three-source alignment (Sanadset $\rightarrow$ hawramani $\rightarrow$ muslimscholars), enabling the enrichment of the graph with cross-source metadata.

Significance and Contributions
The paper claims three primary contributions that address a gap identified in prior surveys regarding the lack of cross-source narrator identity resolution at scale:

1. A Two-Phase Pipeline: A novel pipeline combining a name-only phase (necessary for metadata-poor sources) and a multi-signal phase (leveraging death years and grades where available), utilizing domain-specific Arabic normalization.
2. Annotated Link Corpora: The release of two stratified link corpora (sanad_links and narrator_links) categorized by confidence tiers, which serve as the first large-scale resources for this specific cross-source linking task.
3. Enriched Transmission Graph: The construction of the largest published Hadith transmission graph (185k nodes), enriched with biographical metadata from multiple sources, allowing for analyses previously impossible with isolated datasets.

The authors emphasize that the pipeline operates entirely on structured name strings and categorical metadata, making it applicable to any pair of Arabic biographical databases. They note that while confidence thresholds were set by inspection rather than formal ground-truth calibration (due to the lack of a manually verified evaluation set), the resulting resources provide a foundational starting point for downstream research in Hadith authentication and digital Islamic humanities. The work explicitly positions itself as solving an "open-world" linking problem, distinct from the "closed-world" disambiguation tasks addressed by previous models like AR-Sanad v2.