Designing for scalability with Erlang/OTP

482 Pages·2016·10.868 MB·English

by  Cesarini, Francesco;Vinoski, Steve

Designing for
S calability with
Erlang/OTP
IMPLEMENTING ROBUST,
FAULT-TOLERANT SYSTEMS
Francesco Cesarini & Steve Vinoski

Designing for Scalability
with Erlang/OTP
Francesco Cesarini and Steve Vinoski
BBeeiijjiinngg BBoossttoonn FFaarrnnhhaamm SSeebbaassttooppooll TTookkyyoo
Designing for Scalability with Erlang/OTP
by Francesco Cesarini and Steve Vinoski
Copyright © 2016 Francesco Cesarini and Stephen Vinoski. All rights reserved.
Printed in the United States of America.
Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.
O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are
also available for most titles (http://safaribooksonline.com). For more information, contact our corporate/
institutional sales department: 800-998-9938 or [email protected].
Editor: Andy Oram Indexer: Lucie Haskins
Production Editor: Nicholas Adams Interior Designer: David Futato
Copyeditor: Christina Edwards Cover Designer: Karen Montgomery
Proofreader: Rachel Head Illustrator: Rebecca Demarest
May 2016: First Edition
Revision History for the First Edition
2016-05-11: First Release
See http://oreilly.com/catalog/errata.csp?isbn=9781449320737 for release details.
The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Designing for Scalability with Erlang/
OTP, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc.
While the publisher and the authors have used good faith efforts to ensure that the information and
instructions contained in this work are accurate, the publisher and the authors disclaim all responsibility
for errors or omissions, including without limitation responsibility for damages resulting from the use of
or reliance on this work. Use of the information and instructions contained in this work is at your own
risk. If any code samples or other technology this work contains or describes is subject to open source
licenses or the intellectual property rights of others, it is your responsibility to ensure that your use
thereof complies with such licenses and/or rights.
978-1-449-32073-7
[LSI]
To Alison, Peter and baby Bump for being patient and supportive.
—Francesco

To Dooley and Ed, for teaching me how, and to Cindy, Ryan, Erin, Andrew, and Jake,
for being why.
—Steve

To Joe, Mike, Robert, for making that phone call.
—Francesco & Steve

Table of Contents
Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Defining the Problem 2
OTP 4
Erlang 6
Tools and Libraries 7
System Design Principles 10
Erlang Nodes 11
Distribution, Infrastructure, and Multicore 12
Summing Up 13
What You’ll Learn in This Book 14
2. Introducing Erlang. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Recursion and Pattern Matching 21
Functional Influence 25
Fun with Anonymous Functions 25
List Comprehensions: Generate and Test 27
Processes and Message Passing 29
Fail Safe! 33
Links and Monitors for Supervision 34
Links 35
Monitors 37
Records 38
Maps 41
Macros 42
Upgrading Modules 43
ETS: Erlang Term Storage 45
v
Distributed Erlang 48
Naming and Communication 48
Node Connections and Visibility 49
Summing Up 51
What’s Next? 51
3. Behaviors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
Process Skeletons 53
Design Patterns 56
Callback Modules 57
Extracting Generic Behaviors 60
Starting the Server 62
The Client Functions 64
The Server Loop 66
Functions Internal to the Server 68
The Generic Server 69
Message Passing: Under the Hood 72
Summing Up 76
What’s Next? 76
4. Generic Servers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
Generic Servers 77
Behavior Directives 78
Starting a Server 80
Message Passing 82
Synchronous Message Passing 82
Asynchronous Message Passing 84
Other Messages 85
Unhandled Messages 86
Synchronizing Clients 88
Termination 89
Call Timeouts 91
Deadlocks 94
Generic Server Timeouts 95
Hibernating Behaviors 97
Going Global 97
Linking Behaviors 99
Summing Up 99
What’s Next? 100
5. Controlling OTP Behaviors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
The sys Module 101
vi | Table of Contents
Tracing and Logging 101
System Messages 103
Your Own Trace Functions 103
Statistics, Status, and State 105
The sys Module Recap 107
Spawn Options 108
Memory Management and Garbage Collection 109
Spawn Options to Avoid 113
Timeouts 114
Summing Up 114
What’s Next? 114
6. Finite State Machines. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
Finite State Machines the Erlang Way 118
Coffee FSM 119
The Hardware Stub 121
The Erlang Coffee Machine 122
Generic FSMs 125
A Behavior Example 127
Starting the FSM 127
Sending Events 131
Termination 139
Summing Up 141
Get Your Hands Dirty 141
The Phone Controllers 141
Let’s Test It 143
What’s Next? 145
7. Event Handlers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
Events 147
Generic Event Managers and Handlers 149
Starting and Stopping Event Managers 149
Adding Event Handlers 150
Deleting an Event Handler 152
Sending Synchronous and Asynchronous Events 153
Retrieving Data 156
Handling Errors and Invalid Return Values 158
Swapping Event Handlers 160
Wrapping It All Up 162
The SASL Alarm Handler 165
Summing Up 166
What’s Next? 167
Table of Contents | vii
8. Supervisors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
Supervision Trees 170
OTP Supervisors 174
The Supervisor Behavior 175
Starting the Supervisor 176
The Supervisor Specification 179
Dynamic Children 186
Non-OTP-Compliant Processes 194
Scalability and Short-Lived Processes 195
Synchronous Starts for Determinism 197
Testing Your Supervision Strategy 199
How Does This Compare? 200
Summing Up 201
What’s Next? 201
9. Applications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
How Applications Run 204
The Application Structure 206
The Callback Module 209
Starting and Stopping Applications 210
Application Resource Files 213
The Base Station Controller Application File 215
Starting an Application 216
Environment Variables 219
Application Types and Termination Strategies 221
Distributed Applications 222
Start Phases 226
Included Applications 228
Start Phases in Included Applications 228
Combining Supervisors and Applications 230
The SASL Application 231
Progress Reports 236
Error Reports 236
Crash Reports 237
Supervisor Reports 238
Summing Up 239
What’s Next? 240
10. Special Processes and Your Own Behaviors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
Special Processes 242
The Mutex 242
Starting Special Processes 244
viii | Table of Contents