CVPD at QIAS 2026: RAG-Guided LLM Reasoning for Al-Mawarith Share Computation and Heir Allocation

1. Problem Statement

The paper addresses Islamic Inheritance Law (QIlm al-Maw¯ar¯ıth), a high-precision, multi-stage legal reasoning task. The system must:

1. Identify eligible heirs and those blocked (hajb).
2. Assign fixed and residual shares.
3. Handle global adjustments ('Awl for deficit or Radd for surplus).
4. Output a consistent post-taş¯ıl (distribution) calculation.

Key Challenges:

• Complexity: The task requires strict adherence to legal logic and precise arithmetic.
• Variability: Rules differ across Sunni madh¯ahib (schools of jurisprudence) and national civil-law codifications.
• Data Imbalance: Training data is dominated by simple cases, while evaluation sets contain a high proportion of complex scenarios (e.g., Radd), leading to a train-test mismatch.
• Output Constraints: Models must produce structured, machine-readable JSON outputs with specific schema compliance, not just natural language answers. Standard LLMs often fail at maintaining stage-wise consistency and numerical accuracy.

2. Methodology

The authors propose a Retrieval-Augmented Generation (RAG) pipeline designed for structured prediction, consisting of four main components:

A. Knowledge Base Construction (Symbolic Data Synthesis)

To overcome data scarcity and imbalance, the team created a rule-grounded synthetic corpus of ~100,000 unique cases.

• Symbolic Calculator: A deterministic engine solves inheritance problems step-by-step (eligibility, blocking, shares, adjustments) before generating text. This ensures 100% jurisprudential and arithmetic validity.
• Multi-view Rendering: Each case is rendered into four views: Arabic problem text, Q&A, explicit reasoning traces, and structured JSON. The Q&A view serves as the primary document for indexing.

B. Hybrid Retrieval & Reranking

The system retrieves relevant precedents to guide the LLM.

• Hybrid Search: Combines Dense Retrieval (semantic embeddings) and Sparse Retrieval (BM25 lexical search).
• Fusion: Uses Reciprocal Rank Fusion (RRF) with weights $\alpha=0.7$ (dense) and $\beta=0.3$ (sparse) to merge candidate sets.
• Reranking: A Cross-Encoder reranks the top candidates to select the most relevant $K$ documents (context) for the generation prompt.
• Source Ablation: Experiments showed that curated PDF jurisprudential texts significantly outperform web-scraped data in semantic quality and retrieval success rates.

C. Prompted Structured Generation

• Model: Uses Qwen3.5-9B, selected as the strongest open-source base for legal reasoning.
• Prompting Strategy: The prompt injects the query and top-$K$ retrieved precedents, instructing the model to:
  1. Perform explicit reasoning in a <thinking> block.
  2. Output the final answer in a strict <answer> JSON block.
• Decomposition: The generation is modeled as $P(y_{answer} | q, D_K, y_{think})$, forcing the model to justify its logic before committing to the structured output.

D. Output Parsing & Validation

A deterministic post-processing layer ensures schema compliance:

• Validation Constraints: Checks for critical keys ($C_{keys}$), field types ($C_{types}$), legal label values ($C_{labels}$), and mass coherence ($C_{mass}$).
• Mass Coherence: Ensures the sum of heir shares equals 100% (within a tolerance $\epsilon=5$).
• Fallbacks: If non-critical fields are missing, rule-based defaults are applied to ensure submission completeness.

3. Key Contributions

1. Synthetic Data Strategy: Introduced a symbolic-first approach to generate 100k high-fidelity, mathematically consistent training examples, addressing the train/dev distribution mismatch.
2. Hybrid RAG Pipeline: Demonstrated that combining dense/sparse retrieval with cross-encoder reranking and curated PDF sources is superior to web-based retrieval for high-stakes legal reasoning.
3. Deterministic Validation: Showed that enforcing schema constraints and mass coherence after generation is critical for achieving high scores in structured prediction tasks, compensating for LLM hallucinations.
4. State-of-the-Art Performance: Achieved the top rank on the QIAS 2026 leaderboard.

4. Results

• Leaderboard Performance: The system achieved a MIR-E (Mawarith Inheritance Reasoning Evaluation) score of 0.935, ranking 1st on the official blind-test leaderboard (beating the 2nd place by 0.004 points).
• Baseline Comparison: Off-the-shelf models (e.g., Qwen 3.5 9B without RAG) scored only 0.347, highlighting the necessity of the retrieval and validation pipeline.
• Ablation Insights:
  • Source Quality: PDF-only retrieval yielded a combined score of 0.718, significantly outperforming Web-only (0.463).
  • Error Analysis: Remaining errors were primarily in complex blocking logic (42%) and fraction arithmetic (31%), rather than language understanding.

5. Significance and Conclusion

The paper establishes that for high-precision Arabic legal reasoning:

• Model Choice is Secondary to Engineering: While strong base LLMs are necessary, the reliability of the system depends more on retrieval grounding (curated legal texts) and deterministic post-generation validation than on model size alone.
• Structured Prediction is Key: Success in legal domains requires explicit control over output schemas and intermediate reasoning steps, not just generative fluency.
• Future Directions: The authors suggest that future improvements should focus on neuro-symbolic constraints (e.g., explicit blocking graphs) to handle complex multi-generation interactions and rare arithmetic edge cases, rather than simply scaling up language models.

The full pipeline and code are open-sourced, providing a robust framework for legal AI applications in Arabic.