Practical MLOps: Operationalizing Machine Learning Models

450 Pages·2021·75.15 MB·english

by  Noah Gift

Practical
MLOps
Operationalizing Machine Learning Models
Noah Gift & Alfredo Deza

Practical MLOps
Operationalizing Machine Learning Models
Noah Gift and Alfredo Deza
BBeeiijjiinngg BBoossttoonn FFaarrnnhhaamm SSeebbaassttooppooll TTookkyyoo
Practical MLOps
by Noah Gift and Alfredo Deza
Copyright © 2021 Noah Gift and Alfredo Deza. All rights reserved.
Printed in the United States of America.
Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.
O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are
also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional
sales department: 800-998-9938 or [email protected].
Acquisitions Editor: Rebecca Novack Indexer: WordCo Indexing Services, Inc.
Development Editor: Melissa Potter Interior Designer: David Futato
Production Editor: Daniel Elfanbaum Cover Designer: Karen Montgomery
Copyeditor: Kim Cofer Illustrator: Kate Dullea
Proofreader: Piper Editorial Consulting, LLC
September 2021: First Edition
Revision History for the First Edition
2021-09-14: First Release
See http://oreilly.com/catalog/errata.csp?isbn=9781098103019 for release details.
The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Practical MLOps, the cover image, and
related trade dress are trademarks of O’Reilly Media, Inc.
The views expressed in this work are those of the authors, and do not represent the publisher’s views.
While the publisher and the authors have used good faith efforts to ensure that the information and
instructions contained in this work are accurate, the publisher and the authors disclaim all responsibility
for errors or omissions, including without limitation responsibility for damages resulting from the use of
or reliance on this work. Use of the information and instructions contained in this work is at your own
risk. If any code samples or other technology this work contains or describes is subject to open source
licenses or the intellectual property rights of others, it is your responsibility to ensure that your use
thereof complies with such licenses and/or rights.
978-1-098-10301-9
[LSI]
Table of Contents
Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
1. Introduction to MLOps. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Rise of the Machine Learning Engineer and MLOps 2
What Is MLOps? 4
DevOps and MLOps 5
An MLOps Hierarchy of Needs 7
Implementing DevOps 8
Configuring Continuous Integration with GitHub Actions 13
DataOps and Data Engineering 15
Platform Automation 16
MLOps 17
Conclusion 20
Exercises 21
Critical Thinking Discussion Questions 22
2. MLOps Foundations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Bash and the Linux Command Line 23
Cloud Shell Development Environments 24
Bash Shell and Commands 26
List Files 26
Run Commands 26
Files and Navigation 27
Input/Output 27
Configuration 27
Writing a Script 28
Cloud Computing Foundations and Building Blocks 29
Getting Started with Cloud Computing 31
iii
Python Crash Course 33
Minimalistic Python Tutorial 36
Math for Programmers Crash Course 37
Descriptive Statistics and Normal Distributions 37
Optimization 41
Machine Learning Key Concepts 50
Doing Data Science 54
Build an MLOps Pipeline from Zero 56
Conclusion 63
Exercises 64
Critical Thinking Discussion Questions 64
3. MLOps for Containers and Edge Devices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
Containers 68
Container Runtime 69
Creating a Container 69
Running a Container 72
Best Practices 74
Serving a Trained Model Over HTTP 76
Edge Devices 80
Coral 81
Azure Percept 84
TFHub 85
Porting Over Non-TPU Models 86
Containers for Managed ML Systems 89
Containers in Monetizing MLOps 90
Build Once, Run Many MLOps Workflow 91
Conclusion 91
Exercises 92
Critical Thinking Discussion Questions 92
4. Continuous Delivery for Machine Learning Models. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
Packaging for ML Models 95
Infrastructure as Code for Continuous Delivery of ML Models 99
Using Cloud Pipelines 107
Controlled Rollout of Models 110
Testing Techniques for Model Deployment 112
Conclusion 115
Exercises 116
Critical Thinking Discussion Questions 116
iv | Table of Contents
5. AutoML and KaizenML. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
AutoML 118
MLOps Industrial Revolution 123
Kaizen Versus KaizenML 125
Feature Stores 127
Apple’s Ecosystem 131
Apple’s AutoML: Create ML 132
Apple’s Core ML Tools 136
Google’s AutoML and Edge Computer Vision 139
Azure’s AutoML 144
AWS AutoML 146
Open Source AutoML Solutions 151
Ludwig 151
FLAML 153
Model Explainability 154
Conclusion 158
Exercises 159
Critical Thinking Discussion Questions 159
6. Monitoring and Logging. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
Observability for Cloud MLOps 163
Introduction to Logging 164
Logging in Python 165
Modifying Log Levels 169
Logging Different Applications 170
Monitoring and Observability 172
Basics of Model Monitoring 174
Monitoring Drift with AWS SageMaker 175
Monitoring Drift with Azure ML 182
Conclusion 184
Exercises 185
Critical Thinking Discussion Questions 185
7. MLOps for AWS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
Introduction to AWS 188
Getting Started with AWS Services 189
MLOps on AWS 206
MLOps Cookbook on AWS 209
CLI Tools 211
Flask Microservice 218
AWS Lambda Recipes 223
AWS Lambda-SAM Local 223
Table of Contents | v
AWS Lambda-SAM Containerized Deploy 224
Applying AWS Machine Learning to the Real World 229
Conclusion 233
Exercises 234
Critical Thinking Discussion Questions 234
8. MLOps for Azure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
Azure CLI and Python SDK 236
Authentication 238
Service Principal 238
Authenticating API Services 240
Compute Instances 240
Deploying 242
Registering Models 243
Versioning Datasets 245
Deploying Models to a Compute Cluster 246
Configuring a Cluster 246
Deploying a Model 248
Troubleshooting Deployment Issues 251
Retrieving Logs 252
Application Insights 253
Debugging Locally 254
Azure ML Pipelines 257
Publishing Pipelines 259
Azure Machine Learning Designer 260
ML Lifecycle 262
Conclusion 263
Exercises 263
Critical Thinking Discussion Questions 264
9. MLOps for GCP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265
Google Cloud Platform Overview 265
Continuous Integration and Continuous Delivery 270
Kubernetes Hello World 272
Cloud Native Database Choice and Design 280
DataOps on GCP: Applied Data Engineering 282
Operationalizing ML Models 287
Conclusion 289
Exercises 291
Critical Thinking Discussion Questions 291
vi | Table of Contents
10. Machine Learning Interoperability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293
Why Interoperability Is Critical 294
ONNX: Open Neural Network Exchange 296
ONNX Model Zoo 297
Convert PyTorch into ONNX 299
Create a Generic ONNX Checker 301
Convert TensorFlow into ONNX 303
Deploy ONNX to Azure 307
Apple Core ML 310
Edge Integration 314
Conclusion 315
Exercises 316
Critical Thinking Discussion Questions 316
11. Building MLOps Command Line Tools and Microservices. . . . . . . . . . . . . . . . . . . . . . . . . 317
Python Packaging 319
The Requirements File 320
Command Line Tools 321
Creating a Dataset Linter 321
Modularizing a Command Line Tool 328
Microservices 331
Creating a Serverless Function 333
Authenticating to Cloud Functions 338
Building a Cloud-Based CLI 341
Machine Learning CLI Workflows 342
Conclusion 344
Exercises 345
Critical Thinking Discussion Questions 345
12. Machine Learning Engineering and MLOps Case Studies. . . . . . . . . . . . . . . . . . . . . . . . . 347
Unlikely Benefits of Ignorance in Building Machine Learning Models 348
MLOps Projects at Sqor Sports Social Network 350
Mechanical Turk Data Labeling 351
Influencer Rank 352
Athlete Intelligence (AI Product) 353
The Perfect Technique Versus the Real World 355
Critical Challenges in MLOps 357
Ethical and Unintended Consequences 358
Lack of Operational Excellence 358
Focus on Prediction Accuracy Versus the Big Picture 359
Final Recommendations to Implement MLOps 364
Data Governance and Cybersecurity 365
Table of Contents | vii
MLOps Design Patterns 366
Conclusion 367
Exercises 367
Critical Thinking Discussion Questions 368
A. Key Terms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369
B. Technology Certifications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375
C. Remote Work. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393
D. Think Like a VC for Your Career. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399
E. Building a Technical Portfolio for MLOps. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403
F. Data Science Case Study: Intermittent Fasting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409
G. Additional Educational Resources. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415
H. Technical Project Management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427
Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 431
viii | Table of Contents