Ashwin Swarup Adurthi

This session will explore how data engineering practices come into play when operationalizing foundation models. We'll move beyond the model itself to understand how data engineers prepare and manage the data these models rely on, ensuring they function effectively in real-world applications. We will cover: Foundation models as data representations: We'll discuss how foundation models can be seen as powerful representations of underlying data, and how data engineering is crucial for leveraging this potential. Data pipelines for foundation models: The focus will shift to the role of data engineers in building and maintaining data pipelines that continuously feed the model with high-quality data. Fine-tuning foundation models: We'll explore how data engineers can collaborate with data scientists to prepare data for fine-tuning foundation models for specific tasks. Data management for foundation models: This section will delve into the data management challenges associated with foundation models, such as data versioning and monitoring.