A Multi-Layer Testing Framework for Automated Data Quality Assurance in Cloud-Native ELT Pipelines
This paper presents a unified, multi-layer testing framework for cloud-native ELT pipelines that integrates orchestration-level validation, declarative dbt tests, and LLM-generated semantic tests, demonstrating through controlled experiments that this approach achieves a 128.57% improvement in anomaly detection over manual baselines while maintaining operational practicality.