Day 2 | HALL 2 - Practical Insights and Best Practices

Discover how redBus built a smart data platform component that transforms raw user interactions into queryable Parquet files on S3. Learn how its auto-schema detection and evolution capabilities enable efficient debugging, analytics, and model training at scale.

In the era of Responsible AI, the ability to build scalable and efficient data engineering pipelines is critical—especially in data-sensitive domains like Insurtech. This session will explore how to architect robust pipelines that handle personalized data at scale while ensuring compliance, transparency, and fairness. We’ll delve into practical approaches for integrating Responsible AI principles into data workflows, tackling challenges like data privacy, bias mitigation, and auditability. Attendees will gain insights into real-world use cases and design patterns that enable secure, scalable, and ethical AI applications in the insurance space.

In this session, Prakash Patidar will explore the architecture of next-gen data platforms that are unified, reliable, scalable, and AI-driven. He will discuss how real-time data processing and AI technologies are transforming data systems, enabling smarter decision-making at scale. With a focus on automation, data quality, and real-time insights, this talk will showcase how businesses can leverage AI to unlock the true potential of their data, especially in the fast-evolving world of crypto and DeF.

This session is to outline strategic data engineering practices that drive innovation and business growth. Emphasizing scalability, data quality, and operational efficiency. The session highlights how modern data platforms, automation, and governance empower data-driven decision-making across the organization.

In today’s data-driven financial ecosystem, speed, scale, and accuracy are non-negotiable. This session explores the key principles and real-world practices behind building scalable data pipelines that power meaningful financial insights. From ingesting high-volume transactional data to ensuring real-time processing and analytics, we’ll dive into architectural patterns, technology choices, and performance considerations that enable organizations to unlock value from their data. Whether you’re modernizing legacy systems or designing from the ground up, this talk will equip you with practical takeaways to build resilient, future-ready data pipelines tailored for the dynamic demands of the financial domain.

In today’s data-driven world, ensuring resilience in data architectures is crucial for maintaining seamless operations, minimizing downtime, and enabling real-time decision-making. This session explores the key principles of designing fault-tolerant, scalable, and adaptive data ecosystems that can withstand failures and disruptions.

Key focus areas include strategies for high availability, redundancy, disaster recovery planning, and leveraging distributed computing and cloud-native technologies to enhance resilience. Real-world insights will showcase how resilient data architectures can optimize performance, improve reliability, and support business continuity in an evolving technological landscape

Real-time data pipelines enable organizations to ingest, process, and act on streaming data with minimal latency, ensuring they can make informed decisions instantly.
This session will explore the core components of real-time data pipelines, including event-driven architectures, stream processing frameworks, and scalable data ingestion technologies. We’ll discuss real-world applications such as fraud detection, predictive analytics, IoT monitoring, and personalized customer experiences.
By the end of the session, attendees will gain a deeper understanding of how real-time data pipelines drive agility, enhance operational efficiency, and unlock new business opportunities. Whether you’re a data engineer, architect, or business leader, this talk will equip you with the knowledge to harness the power of real-time data for smarter decision-making.

As organizations increasingly rely on complex data ecosystems, maintaining data quality and trust has become paramount to business success. This discussion explores the critical components and best practices for implementing robust data quality frameworks within modern data architectures. We examine how organizations can establish automated testing, monitoring, and governance processes to ensure data reliability across the entire pipeline – from ingestion to consumption. Special attention is given to emerging tools and methodologies that enable continuous data quality assessment while scaling with growing data volumes and complexity.