Software engineers today face a rapidly evolving AI-assisted landscape, where integrating AI capabilities into products is no longer a nice-to-have. In this full-day, hands-on workshop, participants will learn how to leverage pre-trained small and large foundation models, develop robust evaluation and retrieval pipelines, and architect AI agents.

The focus of the workshop will be on real-world application of these technologies rather than low-level model training. However, I'm keen to add an informal layer of data science on top of the content to help explain why AI does what it does, not just how, to give you a better understanding of the uncertainty involved in AI-powered applications.

Throughout the day, attendees will:

• Discover how to harness off-the-shelf LLM and multimodal APIs to automate code generation, document analysis, and data retrieval, without needing to train large models from scratch.
• Master evaluation techniques and metrics to benchmark AI components, ensure accuracy, and reduce unpredictable outputs through self-verification and validation.
• Build a minimal RAG pipeline and a simple AI agent, learning how to chunk documents, implement embedding-based search, and orchestrate tool calls for dynamic, context-aware behavior.
• Explore deployment best practices, from prompt security and versioning to batching, quantization, and monitoring, so that AI features can scale reliably in production environments while maintaining cost efficiency and compliance. Also a review of the state of the art in inference deployment and AI-assisted tooling.

Whether you're already experimenting with AI APIs or planning your first AI-enhanced feature, this workshop equips you with a pragmatic toolkit: you'll learn not only what these AI building blocks are, but how to integrate, evaluate, and deploy them within your existing engineering workflows.

Schedule

Please note that this schedule is subject to change. AI is moving fast!

1. Session 1 (Foundations & Application of Pre‐Trained Models)

Focus on how to leverage existing foundation models in software workflows. After a 15-minute introduction, the session covers a concise history of foundation models (why they matter), a high-level look at tokenization (to build intuition for prompt length and API costs), and then jumps into hands-on demos showing:

• Calling a hosted LLM API to generate code snippets or complete functions,
• Using a multimodal endpoint (e.g., classifying or captioning an image via an API),
• Iterating on prompts in real time to see how the model's outputs change,
• A quick introduction to RAG (setting the stage for Session 3).

2. Session 2 (Evaluating & Benchmarking AI Systems)

Engineers need to know when a model's output is "good enough." This session teaches:

• Basic sampling controls (temperature, top-k, top-p) so participants learn to dial in creativity vs. determinism,
• Hallucination mitigation: self-verification prompts and simple JSON validators to ensure structured outputs are well‐formed,
• How to interpret language modeling metrics at a practical level---e.g., when perplexity moves from 20 → 10, why does that matter?
• Working examples of small benchmarks: e.g., running a mini‐GLUE style evaluation on a handful of inputs to compare two public LLM APIs,
• Embeddings & similarity: loading a small embedding index of code comments to demonstrate semantic search in action,
• Sketching out a basic evaluation pipeline: "Take N sample prompts, collect model outputs, automatically score with a simple rubric, send borderline cases for human review."

3. Session 3 (Retrieval‐Augmented Generation & AI Agents)

Provides a deeper dive into why RAG is critical for production AI apps (scalability, cost, accuracy) and how to prototype a simple system:

• Covering inverted index (BM25) at a conceptual level so engineers understand why an index speeds up retrieval for large corpora;
• Showing embedding creation and vector similarity search with an open‐source library (e.g., FAISS),
• Building a minimal RAG pipeline: chunk a small document set (e.g., a GitHub repo's README, docs), retrieve top-k chunks, then call an LLM to answer a question;
• Transition to AI agents: how to think about an agent's "planner" and "executor," how to register "tools" (e.g., code interpreter, SQL executor) with the agent, and a short demo that chains retrieval + execution + response. This bridges to the final session on deployment.

4. Session 4 (Deployment, Optimization & Best Practices)

Engineers often struggle when moving from prototype to production. This session covers:

• Advanced Prompt Engineering & Security: designing prompts that minimize injection risks, version control for prompt templates, managing context windows to stay under token limits---complete with a live coding exercise transforming a naive prompt into a secure template;
• Inference Optimization: overview of strategies (batching, caching, prompt prefix caching) that reduce latency & cost, plus a brief explanation of quantized/mixed-precision inference and when to consider self-hosting vs. API;
• Monitoring & Observability: how to log request/response pairs, basic metrics to track (latency percentiles, token cost per request), and pointer to lightweight dashboards;
• Use Cases, Safety & Next Steps: lightning tour of popular AI-driven developer tools (e.g., GitHub Copilot), chatbot integrations, and low-code/ no-code platforms. Finally, a concise overview of AI safety (common attack vectors like prompt injection, data leakage) and references for deeper study.

Timing Summary

• 09:00--10:30 Session 1 (Foundations & Applied Prompts)

• 10:30--10:45 Morning Break

• 10:45--12:15 Session 2 (Evaluation & Benchmarking)

• 12:15--13:00 Lunch

• 13:00--14:30 Session 3 (RAG & Agents)

• 14:30--14:45 Afternoon Break

• 14:45--16:00 Session 4 (Deployment & Best Practices)