Sandesh Gawande

Developing reliable AI models requires high-quality data. Due to the shortcomings of traditional methods, there is a need to adopt data reliability engineering. This approach involves applying principles from manufacturing and considering systems as data factories. It entails significant adjustments to people, processes, and tools. - Data Testing: Validates data through migration testing, pipeline certification, and big data reconciliation. - Data Monitoring: Tracks data pipeline health using business rules, and handles exceptions in real time. - Data Observability: Uses AI/ML for anomaly detection, compliance measurement, and defect rate prediction. The talk will highlight these pillars—Data Testing, Data Monitoring, and Data Observability—showcasing their importance in preventing errors, ensuring compliance, and optimizing AI performance. Case studies and industry practices will demonstrate how to engineer quality data effectively.