Data Engineering with Medallion Architecture by Miki Eto (.ePUB)
File Size: 17.4 MB
Data Engineering with Medallion Architecture: Building scalable multi-cloud pipelines with auditable governance and automated DevOps by Miki Eto
Requirements: .ePUB reader, 17.4 MB | True EPUB
Overview: Data engineering fuels the AI revolution by transforming raw information into high-quality insights. This guide navigates the evolution from traditional warehousing to modern lakehouse systems, teaching you to build and safely operate the medallion architecture (bronze, silver, and gold layers) in production. This book explores the evolution from data warehousing to the rise of data mesh and lakehouse patterns. You will master medallion architecture and data vault for auditable and ROI-driven integration with AWS Step Functions and multi-cloud design across AWS, Azure, and GCP using Kafka, dbt, and Terraform, while implementing the Four-Gate Governance Model for secure operations. You will also implement critical MLOps workflows using AWS SageMaker and DevOps practices with GitHub Actions. The book concludes with expert migration protocols, Z-ordering optimization, and observability techniques to ensure your data platform remains high-performing and cost-effective. By the end of the book, you will confidently design and operate medallion architecture across cloud environments and implement governance frameworks that satisfy auditors. You will be a competent AI collaboration architect ready to orchestrate complex data lifecycles in the BFSI, healthcare, or retail sectors. You will possess the practical skills to deploy serverless streaming pipelines and maintain rigorous compliance. This book represents a fundamental change in how we think about data platform governance. The old model asked Who wrote this code? and trusted senior engineers while scrutinizing junior ones. The new model asks, Is this change safe? regardless of the author. Gates do not care whether a change came from a senior architect with twenty years of experience or an AI assistant that generated code in seconds. They evaluate the same criteria: Does it respect boundaries? Can we roll back? Is there evidence? Every change earns its way to production through the same objective evaluation. Welcome to the future of data engineering. The book is designed for data engineers, architects, and AI specialists. This book requires proficiency in SQL, Python, and cloud platforms like AWS. It targets professionals experienced in building systems who seek advanced mastery in production-grade medallion architectures and resilient, automated data pipelines.
Genre: Non-Fiction > Tech & Devices
Free Download links: