Day 2 | HALL 2 - Practical Insights and Best Practices

Real-time data platforms promise instant insights, but at scale, they often fail in unexpected and expensive ways. In large IoT ecosystems processing hundreds of GBs of data and ingesting over 100 million events per day across multiple real-time pipelines, even well-designed systems begin to show cracks. This session explores real-world production challenges such as query latency on billion-row datasets, ineffective partitioning and indexing strategies, data skew, hot partitions, and pipeline backpressure in high-throughput environments. It highlights why indexed queries can still take seconds, how traditional design patterns break under extreme scale, and how scaling decisions directly impact the balance between cost and performance. The talk also covers practical solutions, architectural trade-offs for real-time systems, techniques to improve pipeline reliability and observability, and approaches to optimize performance under continuous load. Drawing from hands-on experience operating large-scale IoT data platforms, this session offers battle-tested insights and actionable guidance for building systems that not only scale efficiently but also remain resilient under real-world pressure.

This session covers how organizations can use Generative AI in CI/CD pipelines while maintaining the right balance between fast delivery, strong security, and cost efficiency. It will highlight simple ways to scale GenAI in production without compromising performance or control.

Every enterprise is building something expensive. Lakehouses, real-time pipelines, RAG systems, agentic workflows — the data stack has never been more sophisticated, or more costly or more easier to build. Yet ask most data engineering teams what their platform costs per insight, per model inference, or per AI decision and you will struggle to find the right answer.
With AI advancing almost quarter by quarter, the next competitive advantage in data engineering is not a better pipeline framework or a smarter vector store — it is engineering economics and AI economics. The ability to measure, attribute, and optimise the cost of intelligence at scale. Whether you are a data engineer designing pipelines, a platform lead managing infrastructure, or head of AI/ Data / Analytics justifying AI investment to the board — this session will give you a concrete language for the conversation that every data team is avoiding: what does our intelligence layer actually cost, and is it worth it?

Target audience: Data Engineers, Platform Leads, CDOs, Head of Data
Category tags: FinOps, Cost Engineering, AI Infrastructure, Platform Economics
Key takeaway: A reusable framework for unit economics on data + AI platforms

AI Agents for Data Platforms: Discover, Model, Analyze
Building AI-ready data platforms involves transforming traditional data systems into intelligent, self-service ecosystems powered by automation and AI. The foundation lies in creating an automated data catalog that continuously ingests, organizes, and enriches metadata, enabling better data discovery, governance, and contextual understanding through platforms

Complementing this, AI-driven data modeling agents automate the identification of fact and dimension tables, generate optimized analytical schemas, and improve performance through intelligent recommendations. Together, these components create a unified, feedback-driven architecture that accelerates analytics, reduces manual effort, and enables scalable, AI-powered decision-making across the organization.

On top of this, self-service analytics using Text-to-SQL empowers business users to interact with data using natural language, leveraging advanced LLMs such as Qwen, Gemini, and OpenAI GPT models to generate accurate, context-aware queries.

A story of taming fragmented, unstructured healthcare data to unlock scalable AI and measurable patient impact.

Modern data stacks often promise a simple idea: a single query layer that can unify access across databases, data lakes, and SaaS sources. Engines like Trino make this vision technically feasible—allowing teams to query MySQL, MongoDB, S3-based lakehouses, and even spreadsheets through one interface.

This talk shares a practitioner’s perspective on what it actually takes to make that promise work inside a regulated fintech environment, where data residency, auditability, and access control are not optional. Drawing from real-world experience building a federated query layer over heterogeneous systems, it explores the gap between architectural elegance and operational reality.

The session will highlight challenges that emerge at scale: inconsistent semantics across sources, unpredictable query performance, governance complexities with role-based access, and the risk of turning federation into “distributed chaos.” It will also cover the practical guardrails required to make such a system usable—data contracts, curated layers, controlled abstractions, and user-focused enhancements like custom functions.

Rather than presenting federation as a silver bullet, this talk reframes it as a powerful but disciplined capability. Attendees will leave with a clearer understanding of when a unified query layer accelerates data access—and when it quietly amplifies complexity.

In a world of always-on connectivity, telecom systems generate massive streams of real-time data every second. This session explores how modern data platforms process, analyze, and act on this data instantly, powering everything from network optimization to personalized customer experiences at scale

DataOps gave us insights. DecisionOps makes them execute. The goal has changed from data to decisions & AI-native platforms are accelerating decision-making across the value chain. Let us hear about how is that advancing clinical research, transporting life saving medicines on time & saving lives!

Most startups bolt on privacy later. Designing systems where PII is isolated, tokenised, or abstracted early. This can include schema design, access control layers, and service boundaries.