Data Engineering with Medallion Architecture: Building scalable multi-cloud pipelines with auditable governance and automated DevOps (English Edition)
暫譯: 使用獎牌架構進行數據工程：構建可擴展的多雲管道，實現可審計的治理和自動化的 DevOps (英文版)
Eto, Miki
- 出版商: BPB Publications
- 出版日期: 2026-03-11
- 售價: $1,630
- 貴賓價: 9.5 折 $1,548
- 語言: 英文
- 頁數: 252
- 裝訂: Quality Paper - also called trade paper
- ISBN: 9365894239
- ISBN-13: 9789365894233
-
相關分類:
Data-visualization、Power BI
海外代購書籍(需單獨結帳)
商品描述
Data engineering fuels the AI revolution by transforming raw information into high-quality insights. This guide navigates the evolution from traditional warehousing to modern lakehouse systems, teaching you to build and safely operate the medallion architecture (bronze, silver, and gold layers) in production.
This book explores the evolution from data warehousing to the rise of data mesh and lakehouse patterns. You will master medallion architecture and data vault for auditable and ROI-driven integration with AWS Step Functions and multi-cloud design across AWS, Azure, and GCP using Kafka, dbt, and Terraform, while implementing the Four-Gate Governance Model for secure operations. You will also implement critical MLOps workflows using AWS SageMaker and DevOps practices with GitHub Actions. The book concludes with expert migration protocols, Z-ordering optimization, and observability techniques to ensure your data platform remains high-performing and cost-effective.
By the end of the book, you will confidently design and operate medallion architecture across cloud environments and implement governance frameworks that satisfy auditors. You will be a competent AI collaboration architect ready to orchestrate complex data lifecycles in the BFSI, healthcare, or retail sectors. You will possess the practical skills to deploy serverless streaming pipelines and maintain rigorous compliance.
What you will learn
● Design medallion architecture with bronze, silver, and gold layers.
● Create audit trails that answer auditors in one click.
● Build scalable pipelines with Kafka, dbt, and Terraform.
● Deploy AI/ML models through the same governance gates.
● Migrate to the cloud without disrupting live operations.
● Implement data mesh and lakehouse patterns at scale.
● Reduce firefighting and increase deployment confidence.
Who this book is for
The book is designed for data engineers, architects, and AI specialists. This book requires proficiency in SQL, Python, and cloud platforms like AWS. It targets professionals experienced in building systems who seek advanced mastery in production-grade medallion architectures and resilient, automated data pipelines.
Table of Contents
Reading Guide
1. Evolution of Data Architecture
2. Understanding Data Mesh, Lakehouse, and Medallion
3. Data Integration Strategy, Business Impact, and ROI
4. Medallion Architecture in Multi-cloud
5. Building Scalable Data Pipelines
6. Data Governance and Compliance
7. MLOps for AI Model Deployment and Monitoring
8. DevOps and CI/CD for Data Engineering
9. Cloud Migration and Coexistence Strategies
10. Scaling Data Platforms with Optimization
商品描述(中文翻譯)
資料工程透過將原始資訊轉換為高品質的洞察，推動了人工智慧革命。本指南引導您從傳統資料倉儲演變到現代湖屋系統，教您如何在生產環境中構建並安全運行獎牌架構（銅層、銀層和金層）。
本書探討了從資料倉儲到資料網格和湖屋模式的演變。您將掌握獎牌架構和資料保險庫，以便與 AWS Step Functions 和跨 AWS、Azure 和 GCP 的多雲設計進行可審計和以 ROI 為驅動的整合，使用 Kafka、dbt 和 Terraform，同時實施四閘治理模型以確保安全運行。您還將使用 AWS SageMaker 和 GitHub Actions 的 DevOps 實踐來實施關鍵的 MLOps 工作流程。本書最後將介紹專家遷移協議、Z-ordering 優化和可觀察性技術，以確保您的資料平台保持高效能和具成本效益。
在本書結束時，您將能夠自信地設計和運行跨雲環境的獎牌架構，並實施滿足審計員要求的治理框架。您將成為一名合格的 AI 協作架構師，準備在 BFSI、醫療保健或零售行業中協調複雜的資料生命週期。您將具備部署無伺服器串流管道和維持嚴格合規的實用技能。
您將學到的內容：
● 設計具有銅層、銀層和金層的獎牌架構。
● 創建能夠一鍵回答審計員的審計追蹤。
● 使用 Kafka、dbt 和 Terraform 構建可擴展的管道。
● 通過相同的治理閘部署 AI/ML 模型。
● 在不干擾實時操作的情況下遷移到雲端。
● 大規模實施資料網格和湖屋模式。
● 減少緊急處理並提高部署信心。
本書的對象：
本書專為資料工程師、架構師和 AI 專家設計。此書要求具備 SQL、Python 和 AWS 等雲平台的熟練程度。它針對有系統建設經驗的專業人士，尋求在生產級獎牌架構和韌性、自動化資料管道方面的高級掌握。
目錄：
閱讀指南
1. 資料架構的演變
2. 理解資料網格、湖屋和獎牌
3. 資料整合策略、商業影響和 ROI
4. 多雲中的獎牌架構
5. 構建可擴展的資料管道
6. 資料治理和合規性
7. AI 模型部署和監控的 MLOps
8. 資料工程的 DevOps 和 CI/CD
9. 雲遷移和共存策略
10. 通過優化擴展資料平台
