LLM Evaluation for Production Systems by Ammar Mohanna (.ePUB)
File Size: 55 MB
Practical LLM Evaluation for Production Systems: Measure, monitor, and improve reliable LLM systems across training and inference by Ammar Mohanna, Indrajit Kar, Feli Ralte
Requirements: .ePUB reader, 55 mb
Overview: Move beyond benchmarks and learn how to evaluate whether LLM-powered systems actually work in production. This book gives you practical frameworks, metrics, and operational strategies to measure reliability, safety, quality, latency, and cost across modern AI systems. Guided by experienced AI leaders and researchers, you’ll build evaluation pipelines that support real business decisions instead of isolated leaderboard scores.
The book takes a product-first approach to evaluation, treating it as a continuous operational capability rather than a one-time testing exercise. You’ll explore how evaluation changes across training, inference, and end-to-end system operation while learning how to connect metrics directly to deployment gates, rollback criteria, monitoring systems, and production reliability goals.
Using practical examples and real-world workflows, the book covers evaluation strategies for text LLMs, vision-language models, multimodal conversational systems, Mixture-of-Experts architectures, agentic systems, reasoning models, Text2SQL and Text2Cypher systems, retrieval pipelines, embedding models, OCR workflows, and guardrail SLMs.
By the end of this book, you’ll be able to design and operate reliable, safe, and cost-effective LLM-powered applications with confidence.
Genre: Non-Fiction > Tech & Devices
Free Download links: