In a remarkably short period, Generative AI has emerged as a transformative technology. But like any other Analytics capability, Gen AI will need strong integration with and rely on Data Platforms for high-quality data, processing at scale, and a secure way of serving. In this session, we will share thoughts on strategy for evolving existing & traditional Data Foundation Architectures to enable Generative AI capabilities within an enterprise.
The data engineering lifecycle has 3 profound stages – Data Input, Data Processing and Data Output. There are various problems that need to be tackled in various stages of Data Lifecycle. To name a few challenges – Data input mismatch, Data Relationship Management, Data Coding, Data Catalogue enrichment etc. These challenges can be bucketized into 3 profound stages – Input, Process and Output. At Genpact, have tried to create GenAI based solutions to tackle such challenges and showcased to Clients. While lot more solutions and use cases are being thought through, wanted to share a perspective of a few important solutions and their potential impact in Data Engineering space.
The world of Generative AI is evolving rapidly, yet its implementation into real-world applications lags behind. What could be the reasons for this, and could this trend reverse soon? You could be a Business Leader, Data Engineering Practitioner or Data Engineer/ML Engineer/Data Scientist; the developments in GenAI are poised to impact your field significantly.
In my keynote, The All-In GenAI Gamble: Who Should Bet Big and Who Should Check, I will provide insights from a practitioner’s perspective on GenAI. This session will help you understand how to stay relevant and adapt to this rapidly evolving world of GenAI.
This session covers how Data Engineering teams across industries can process from multiple sources and types of data, simplify complex data pipelines, code in language of choice with unlimited processing power for meaningful insights, build products and solutions on data to monetize, incorporating and supercharging AI-ML story in the entire spectrum.
Emerging trends and industry wide Innovation
Developer productivity from enterprise POV
Productivity and experience: Two sides of the same coin
Enterprise and developers considerations on Data accuracy, privacy and security
In this talk, Sai will explore how Intuit improved the efficiency of its Datalake in response to the challenges caused by the lack of governance over table creation. With a massive 300K hive tables and several thousand new tables added weekly, the Datalake had become inefficient, costly, and difficult to govern. He will delve into the challenges faced with data discovery, lack of ownership, and unused data, including the difficulty in finding necessary data due to repetitive table names and schemas. Intuit tackled these challenges by introducing Datalake Observability, a tool designed to provide usage states, access patterns, and recommendation algorithms for each table in the Datalake. This led to the removal of unused tables, streamlined discovery, and the establishment of ownership for many tables. Join him to explore how Intuit’s Datalake Observability improved the efficiency of the Datalake while boosting governance.
This topic explores the synergy between ClickHouse, an open-source column-oriented database management system and Apache Kafka, a distributed event streaming platform. Together, they offer a robust solution for real-time data ingestion, processing, and analytics. Kafka efficiently handles high-throughput data streams, while ClickHouse excels in storing and analyzing vast amounts of data with lightning-fast query performance. This duo is a formidable combination for businesses aiming for rapid data processing and actionable insights.
© AIM MEDIA HOUSE LLC 2022-2024