Table Of ContentAI Crash Course
A fun and hands-on introduction to reinforcement
learning, deep learning, and artificial intelligence
with Python
Hadelin de Ponteves
BIRMINGHAM - MUMBAI
AI Crash Course
Copyright © 2019 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval
system, or transmitted in any form or by any means, without the prior written
permission of the publisher, except in the case of brief quotations embedded in
critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy
of the information presented. However, the information contained in this book
is sold without warranty, either express or implied. Neither the author, nor Packt
Publishing or its dealers and distributors, will be held liable for any damages caused
or alleged to have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the
companies and products mentioned in this book by the appropriate use of capitals.
However, Packt Publishing cannot guarantee the accuracy of this information.
Acquisition Editor: Jonathan Malysiak
Acquisition Editor – Peer Reviews: Suresh Jain
Content Development Editor: Alex Patterson
Project Editor: Kishor Rit
Technical Editor: Aniket Shetty
Proofreader: Safis Editing
Indexer: Rekha Nair
Presentation Designer: Sandip Tadge
First published: November 2019
Production reference: 1281119
Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham B3 2PB, UK.
ISBN 978-1-83864-535-9
www.packt.com
packt.com
Subscribe to our online digital library for full access to over 7,000 books and videos,
as well as industry leading tools to help you plan your personal development and
advance your career. For more information, please visit our website.
Why subscribe?
• Spend less time learning and more time coding with practical eBooks
and Videos from over 4,000 industry professionals
• Learn better with Skill Plans built especially for you
• Get a free eBook or video every month
• Fully searchable for easy access to vital information
• Copy and paste, print, and bookmark content
Did you know that Packt offers eBook versions of every book published, with PDF
and ePub files available? You can upgrade to the eBook version at www.Packt.com
and as a print book customer, you are entitled to a discount on the eBook copy. Get
in touch with us at [email protected] for more details.
At www.Packt.com, you can also read a collection of free technical articles, sign up
for a range of free newsletters, and receive exclusive discounts and offers on Packt
books and eBooks.
Contributors
About the author
Hadelin de Ponteves is the co-founder and director of technology at BlueLife
AI, which leverages the power of cutting-edge Artificial Intelligence to empower
businesses to make massive profits by optimizing processes, maximizing efficiency,
and increasing profitability. Hadelin is also an online entrepreneur who has created
50+ top-rated educational e-courses on topics such as machine learning, deep
learning, artificial intelligence, and blockchain, which have reached over 700,000
subscribers in 204 countries.
First and foremost, I would like to thank Jan Warchocki for his
amazing contribution to the book. Jan is one of my top talented
students on Udemy, and I had the pleasure to work with him on
the creation of this book. He provided very smart feedback on
every chapter, added many brilliant examples to illustrate the theory
and practice for each of the AI models, and even brought his own
practical application in the last chapter, covering the most advanced
AI model of this book. Thanks to Jan, the book is richer in relevant
AI applications, and more complete in the theory. I give him my
highest thanks, and I have no doubt he will become a very talented
AI scientist with whom I will have the pleasure to collaborate again
on other projects.
Then I would like to thank my business partner and dear friend
Kirill Eremenko, who encouraged me to write this book. First, Kirill
gave me the inspiration to write a complete book on AI. But most
importantly, this book would never have been written without him.
Indeed, I was able to create this book thanks to the many years of
collaboration I had with Kirill producing our online courses.
This book includes the top explanations and practical activities
of our courses, while seriously taking into account all of the most
recurrent feedback we collected from our students over the years.
I would also like to thank Alex Patterson, Jonathan Malysiak, and
David Barnes from the team at Packt. Alex did a tremendous job
improving the book, by reviewing and providing feedback on every
single paragraph, making sure everything is top quality from start
to finish. Jonathan was like a coach to me, helping me all the way
to make this book better and better. Sometimes some big challenges
had to be taken on during the creation of this book, and Jon was here
to encourage me every time to succeed in them. I am very grateful
to him for pushing me to write this book and achieve my highest
potential. David was the one to offer me this opportunity in the first
place. He came to me with a great idea, convincing me that it was
possible to convey the energy I have in my online courses into a
book. I accepted the challenge, and today I can say that he was right.
Thanks, David, for this brilliant idea.
And finally, I would like to thank the technical reviewers of this
book, who not only provided feedback on every chapter of the book,
but also made sure that all the code ran correctly. Thanks to them the
reader is better guided in the theory of the AI models, and also on
how to run all the different code in this book.
About the reviewers
Valeriy Babushkin is the director of Data Science at X5 Retail Group, where
he leads a team of 80+ people in the area of natural language processing, machine
learning, computer vision, data analysis, and A/B testing. Valeriy is a Kaggle
competition grand master, ranking globally in the top 30, and also leads a data
science team at Yandex. He studied cybernetics in Moscow Polytechnical University
and mechatronics at Karlsruhe University of Applied Sciences and worked with
Packt as an author of the Python Machine Learning Tips, Tricks, and Techniques course.
Shrinivas Shetty is a budding data science and machine learning enthusiast. He
is a Master of Science graduate from Stevens Institute of Technology with a focus
in data engineering and business intelligence and analytics. He has been working
in the field of AI for a year, and he helps businesses solve problems and improve
efficiency using AI and machine learning techniques. He provides software services
using programming tools like Python and Scala to develop data science projects and
deploy them using data engineering principles. Apart from doing computer science
and using AI and machine learning to solve business use cases, he indulges his
long-time passion for e-sports (DOTA 2 and CS: GO anyone?).
I would like to thank my family (mom, dad, and younger brother)
for being a constant source of support and encouragement. I would
also like to thank my dear friends for inspiring me to push through
my limitations. It is a privilege to be in your company and a fortune
to have your support.
Table of Contents
Preface vii
Chapter 1: Welcome to the Robot World 1
Beginning the AI journey 1
Four different AI models 2
The models in practice 3
Fundamentals 3
Thompson Sampling 3
Q-learning 3
Deep Q-learning 3
Deep convolutional Q-learning 4
Where can learning AI take you? 4
Energy 4
Healthcare 4
Transport and logistics 5
Education 5
Security 5
Employment 5
Smart homes and robots 5
Entertainment and happiness 6
Environment 6
Economy, business, and finance 6
Summary 7
Chapter 2: Discover Your AI Toolkit 9
The GitHub page 9
Colaboratory 11
Summary 15
[ i ]
Table of Contents
Chapter 3: Python Fundamentals –
Learn How to Code in Python 17
Displaying text 18
Exercise 18
Variables and operations 19
Exercise 20
Lists and arrays 20
Exercise 22
if statements and conditions 22
Exercise 23
for and while loops 24
Exercise 27
Functions 27
Exercise 28
Classes and objects 29
Exercise 31
Summary 31
Chapter 4: AI Foundation Techniques 33
What is Reinforcement Learning? 33
The five principles of Reinforcement Learning 34
Principle #1 – The input and output system 34
Principle #2 – The reward 35
Principle #3 – The AI environment 37
Principle #4 – The Markov decision process 37
Principle #5 – Training and inference 38
Training mode 38
Inference mode 39
Summary 40
Chapter 5: Your First AI Model – Beware the Bandits! 41
The multi-armed bandit problem 41
The Thompson Sampling model 42
Coding the model 43
Understanding the model 47
What is a distribution? 48
Tackling the MABP 52
The Thompson Sampling strategy in three steps 55
The final touch of shaping your Thompson Sampling intuition 56
Thompson Sampling against the standard model 57
Summary 58
[ ii ]
Table of Contents
Chapter 6: AI for Sales and Advertising –
Sell like the Wolf of AI Street 59
Problem to solve 59
Building the environment inside a simulation 61
Running the simulation 64
Recap 66
AI solution and intuition refresher 66
AI solution 67
Intuition 68
Implementation 68
Thompson Sampling vs. Random Selection 69
Performance measure 69
Let's start coding 69
The final result 74
Summary 76
Chapter 7: Welcome to Q-Learning 77
The Maze 78
Beginnings 78
Building the environment 79
The states 79
The actions 80
The rewards 81
Building the AI 85
The Q-value 85
The temporal difference 86
The Bellman equation 87
Reinforcement intuition 88
The whole Q-learning process 88
Training mode 89
Inference mode 89
Summary 90
Chapter 8: AI for Logistics – Robots in a Warehouse 91
Building the environment 94
The states 94
The actions 95
The rewards 95
AI solution refresher 96
Initialization (first iteration) 96
Next iterations 96
Implementation 97
Part 1 – Building the environment 98
Part 2 – Building the AI Solution with Q-learning 101
[ iii ]
