🔭Gen AI
LangSmith
Tracing, evaluation and monitoring for LLMs
Master LangSmith for debugging, testing, and monitoring your LLM applications. Covers tracing, dataset creation, prompt evaluation, and setting up observability pipelines.
25 lessons
12 hrs
Advanced
What you will learn
Build real-world projects from scratch
Write clean, production-ready code
Understand core concepts deeply with diagrams
Follow industry best practices
Get hands-on with code in every lesson
Access lifetime updates as the tech evolves
Curriculum
Module 1Tracing, From Zero to Reading a Real Trace Tree2 lessons
Module 2Building Real Evaluation Datasets2 lessons
A Dataset, Defined — Curated Examples vs Real Production Traces12 min
Turning a Real Production Failure Into a Dataset Example13 min
Module 3Evaluators and Experiments2 lessons
Three Kinds of Evaluators, Built and Tested Directly14 min
Running Experiments and Measuring Real Regressions13 min
Module 4Online Evaluation and Production Monitoring2 lessons
Online Evaluation — Scoring Real, Live Production Traces13 min
Monitoring Dashboards and Real, Actionable Alert Thresholds13 min
Module 5Prompt Versioning and A/B Comparison2 lessons
Real Prompt Versioning With push_prompt and pull_prompt13 min
A/B Comparing Two Prompt Versions Against the Same Dataset13 min
Module 6Closing the Production Loop2 lessons
The Complete Loop — Failing Trace to Fix to Redeploy13 min
Closing Synthesis — Instrumenting a Real Graph End to End14 min
12 total lessons across 6 modulesPreview Module 1 free