Day 1 | HALL 2 - Practical Insights and Best Practices

As data-driven operations scale, data modeling has become increasingly complex and fast-paced, often clashing with the demands of agile delivery. This session explores how Generative AI is tackling long-standing challenges such as source data understanding and gaps in data catalogs. It highlights how GenAI is automating tasks like data profiling, metadata generation, schema design, and mapping, unlocking new levels of speed and accuracy. Real-world examples from Tiger Analytics illustrate the impact, along with a perspective on the evolving role of GenAI as a co-pilot in building scalable, business-aligned data architectures.

Data Engineering and Technical Product Capabilities at AMEX empowering critical business and risk decisioning at AMEX.

In his session, Koushik will reveal how organizations can transform from passive data collectors to strategic data orchestrators, unlocking unprecedented business growth and competitive advantage.
Drawing from his learnings of proven frameworks being adopted by industry-leading financial institutions and companies, he will outline the foundational pillars necessary to build a high-impact data strategy—starting with organizational alignment, where data teams are integrated into business units and held accountable for P&L outcomes.

Koushik will also highlight the role of experimentation, scalable infrastructure, and predictive analytics in optimizing every stage of the customer journey and the common pitfalls like misaligned KPIs, siloed teams, and resistance to change—offering practical solutions to overcome them.

The session will deliver an overall theme that winning with data is not about tools alone; it’s about aligning people, processes, and purpose.

In a world overflowing with data, the true challenge lies not in access—but in articulation. This session explores the craft of transforming raw, noisy data into compelling, credible, and impactful narratives. From choosing the right metrics to visual storytelling and context-driven publishing, we’ll unpack techniques that help data professionals elevate insight over information. Whether you’re building dashboards, whitepapers, or public datasets, discover how to publish data that informs, engages, and drives action.

Data engineers play a pivotal role in driving contextualized intelligence. They must ensure that data is enriched with the proper context to deliver meaningful insights.

This tech talk will explore the practical steps and tools data engineers can use to build a contextual layer, ensuring AI models can deliver strategic insights based on relevant business contexts.

AI is reshaping the landscape of data engineering in transformative ways. One key trend is the rise of AI-powered data pipelines, where machine learning automates tasks like data cleaning, transformation, and feature engineering, significantly accelerating development cycles. Alongside this, AI-driven data observability tools are improving data reliability by detecting anomalies and quality issues in real time. Intelligent data catalogs are also evolving, using AI to automatically generate metadata, track lineage, and simplify data discovery.
Natural language interfaces, such as Text-to-SQL systems powered by large language models, are making it easier for non-technical users to query databases using plain English. ETL and ELT processes are becoming smarter too, with AI automating schema mapping and transformation logic, reducing the need for manual coding. In parallel, synthetic data generation is gaining traction, enabling the creation of realistic datasets for testing, training, and privacy-sensitive use cases.
Real-time data processing is becoming more intelligent as AI models integrate directly into streaming frameworks like Kafka and Spark, supporting use cases like fraud detection and personalization. Governance is also being augmented by AI, with tools now capable of automatically detecting sensitive data like PII and enforcing compliance with regulations.

In this talk, I will dive into a real-time data pipeline solution I worked on for the Travel & Hospitality domain, powered by Apache Flink Stateful Functions. This solution efficiently handled over 800,000 streaming events per day, addressing critical use cases. I will share insights into the challenges faced, the strategies used to overcome them, and how we successfully scaled the pipeline to meet demanding business requirements.

For the second part of the session, I will explore the hot topic: Will Low-Code ETL Replace Data Engineers? I firmly believe that Low-Code ETL is not here to replace data engineers but to accelerate the development process. However, for this to succeed, ETL platforms must empower data engineers with the freedom and control to customize and adapt the platform to their specific needs. I’ll discuss examples like Prophecy, a Low-Code ETL platform that strikes a balance by acting as an IDE, generating open-source code, and keeping implementation transparent. This approach ensures that while development is expedited, the power remains firmly in the hands of data engineers.

The Unified Data platform is designed to address key limitations in data management for machine learning (ML) model development, training, and testing. Its primary focus is to improve turnaround time (TAT), enhance experimentation velocity, and provide a centralized solution for data needs.

Discover the engineering innovations behind KnowBe4’s agentic platform, which utilizes Data and Generative AI to empower cybersecurity defense. Learn how KnowBe4 developed AIDA (Artificial Intelligence Defense Agents) to significantly reduce administrative overhead for customers by automating the configuration of KnowBe4 products. AIDA intelligently understands specific customer requirements based on user behavior, leading to automated risk reduction. This session will delve into the cutting-edge technologies that power this feature, including the latest advancements in Artificial Intelligence, Datamesh architecture, and big data engineering tools. We will explore how these elements are integrated to deliver current GenAI capabilities and lay the foundation for future intelligent features aimed at proactively mitigating human-related cyber risks.

Building a future-forward health insurance data platform focuses on leveraging emerging technologies like AI, machine learning, and blockchain to revolutionize data management in the insurance sector. By integrating diverse data sources such as medical records, wearable devices, and claims data, the platform enables personalized policies, faster claims processing, and enhanced risk management. This approach not only increases operational efficiency but also improves customer satisfaction by providing tailored coverage and real-time insights