Day 2 | Hall 3 - Closed Door Workshops

You shipped your LLM app. Congratulations. Now the hard part starts. Because production AI isn’t a launch — it’s a living system that drifts, degrades, and surprises you in ways traditional monitoring never prepared you for.

This workshop is about what happens after deployment. You’ll learn to instrument the full pipeline — prompt telemetry, retrieval accuracy, embedding quality, and model output evaluation — using tools like LangSmith and Weights & Biases. We’ll dig into the problems nobody warns you about: dataset drift, prompt regressions, and retrieval pipelines that silently rot.

Most importantly, you’ll build automated feedback loops that don’t just detect issues but make your system smarter over time. Bring experience with AI pipelines and monitoring tools. Leave with the operational playbook that keeps your LLM reliable long after the demo hype fades.

Everyone’s talking about prompts and models. Nobody’s talking about the pipes feeding them. Here’s the truth — your LLM is only as good as the data infrastructure behind it.

This workshop puts data engineers where they belong: at the center of the AI stack. You’ll build the pipelines that power retrieval-augmented generation — from ingestion and transformation to embedding generation and vector store delivery, using tools you already know like Airflow and dbt.

We’ll tackle the unsexy but essential stuff: dataset versioning, embedding lifecycle management, and keeping knowledge bases fresh so your LLM doesn’t hallucinate last quarter’s answers. Bring your Python or SQL skills and a basic understanding of embeddings. Leave knowing how to build the data backbone that makes AI actually work.