Table Of ContentLIFE OF AI
ARTIFICIAL INTELLIGENCE
BY
WILLIAM KRYSTAL
Copyright c 2020 by William Krystal, Inc. All rights reserved Published by William Krystal
Inc., Hoboken, New Jersey
Published simultaneously in Canada
No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means,
electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the
1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through
payment of the appropriate per‐copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923,
(978) 750‐8400, fax (978) 750‐4470, or on the web at www.copyright.com. Requests to the Publisher for permission should be
addressed to the Permissions Department, William Krystal, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748‐6011, fax
(201) 748‐6008, or online at http://www.williamkrystalepublisher.com/go/permissions.
Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book,
they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and
specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created
or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable
for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for
any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other
damages.
For general information on our other products and services or for technical support, please contact our Customer Care
Department within the United States at (800) 762‐2974, outside the United States at (317) 572‐3993 or fax (317) 572‐4002.
OVERVIEW OF ARTIFICIAL INTELLIGENCE
WHAT IS AI ?
IMPORTANCE OF AI
SPEECH RECOGNITION
UNDERSTANDING NATURAL LANGUAGE
COMPUTER VISION
EXPERT SYSTEMS
HEURISTIC CLASSIFICATION
EARLY WORK IN AI
LOGIC/MATHEMATICS
COMPUTATION
PSYCHOLOGY / COGNITIVE SCIENCE
BIOLOGY / NEUROSCIENCE
EVOLUTION
NATURAL LANGUAGE PROCESSING
COMMON TECHNIQUES
AI AND RELATED FIELDS
SEARCHING AND-OR GRAPHS
CONSTRAINT SATISFACTION SEARCH
FORWARD CHECKING
MOST-CONSTRAINED VARIABLES
HEURISTIC REPAIR
LOCAL SEARCH AND METAHEURISTICS
EXCHANGING HEURISTICS
ITERATED LOCAL SEARCH
TABU SEARCH
TRANSLATING BETWEEN ENGLISH AND LOGIC NOTATION
TRUTH TABLES
COMPLEX TRUTH TABLES
TAUTOLOGY
EQUIVALENCE
PROPOSITIONAL LOGIC
PREDICATE CALCULUS
FIRST-ORDER PREDICATE LOGIC
SOUNDNESS
COMPLETENESS
REASONING IN MODAL LOGIC
POSSIBLE WORLD REPRESENTATIONS
DEMPSTER- SHAFER THEORY
FUZZY SET THEORY
FUZZY
SET
FUZZY MEMBERSHIP
FUZZY OPERATIONS
FUZZY PROPERTIES
FUZZY RELATIONS
3.1 DEFINITION OF FUZZY RELATION
FORMING FUZZY RELATIONS
MAX-MIN AND MIN-MAX COMPOSITION
FUZZY SYSTEMS
FUZZY LOGIC
CLASSICAL LOGIC
FUZZIFICATION
FUZZY INFERENCE
FUZZY RULE BASED SYSTEM
DEFUZZIFICATION
CENTROID METHOD
PROBABILISTIC REASONING
BAYESIAN PROBABILISTIC INFERENCE
DEFINITION AND IMPORTANCE OF KNOWLEDGE
KNOWLEDGE BASED SYSTEMS
REPRESENTATION OF KNOWLEDGE
KNOWLEDGE ORGANIZATION
KNOWLEDGE MANIPULATION
MATCHING TECHNIQUES:
MEASURE FOR MATCHING
DISTANCE METRICS
MATCHING LIKE PATTERNS
THE RETE MATCHING ALGORITHM
NATURAL LANGUAGE PROCESSING :
OVERVIEW OF LINGUISTICS
MORPHOLOGICAL ANALYSIS
BNF
BASIC PARSING TECHNIQUES
AUGMENTED TRANSITION NETWORKS
CHART PARSING
SEMANTIC ANALYSIS
RULES FOR KNOWLEDGE REPRESENTATION
CONFLICT RESOLUTION
RULE-BASED EXPERT SYSTEMS
ARCHITECTURE OF AN EXPERT SYSTEM
THE EXPERT SYSTEM SHELL
KNOWLEDGE ENGINEERING
CLIPS (C LANGUAGE INTEGRATED PRODUCTION SYSTEM)
BACKWARD CHAINING IN RULE-BASED EXPERT SYSTEMS
CYC
AI DEEP LEARNING FRAMEWORKS FOR DS
HOW AI DL SYSTEMS WORK
AI MAIN DEEP LEARNING FRAMEWORKS
AI MAIN DEEP LEARNING PROGRAMMING LANGUAGES
HOW TO LEVERAGE DL FRAMEWORKS
ETL PROCESSES FOR DL
AI DEPLOYING DATA MODELS
AI ASSESSING A DEEP LEARNING FRAMEWORK
INTERPRETABILITY
MODEL MAINTENANCE
AI BUILDING A DL NETWORK USING MXNET
CORE COMPONENTS
DATASETS DESCRIPTION
CLASSIFICATION FOR MXNET
CREATING CHECKPOINTS FOR MODELS DEVELOPED IN MXNET
ARTIFICIAL INTELLIGENCE BUILDING AN OPTIMIZER BASED ON THE PARTICLE SWARM
OPTIMIZATION ALGORITHM
PSO ALGORITHM FOR AI
FIREFLY OPTIMIZER PSO
PSO VERSUS OTHER OPTIMIZATION METHODS
AI MAXIMIZING AN EXPONENTIAL EXPRESSION
AI BUILDING AN ADVANCED DEEP LEARNING SYSTEM
STANDARD GENETIC ALGORITHM
GAS IN ACTION FOR AI
AI ADVANCED BUILDING DEEP LEARNING SYSTEM
AI CNN COMPONENTS
DATA FLOW AND FUNCTIONALITY
CNN TRAINING PROCESS
AI VISUALIZATION OF A CNN MODEL
RECURRENT NEURAL NETWORKS
AI ALTERNATIVE FRAMEWORKS IN DS
AI EXTREME LEARNING MACHINES (ELMS)
AI MOTIVATION BEHIND ELMS
AI ARCHITECTURES OF ELMS
AI CAPSULE NETWORKS
AI MOTIVATIONS BEHIND CAPSNETS
AI FULLY CONNECTED LAYER
AI DYNAMIC ROUTING BETWEEN CAPSULES
FUZZY SETS
PYTHON
BIG DATA
HADOOP
APACHE SPARK
MACHINE LEARNING FOR AI
AI PERCEPTRON & NEURAL NETWORKS
ARTIFICIAL INTELLIGENCE DECISION TREES
AI SUPPORT VECTOR MACHINES
AI PROBABILISTIC MODELS
AI DYNAMIC PROGRAMMING AND REINFORCEMENT LEARNING
AI EVOLUTIONARY ALGORITHMS
AI TIME SERIES MODELS
ARTIFICIAL INTELLIGENCE THE NATURE OF LANGUAGE
MATHEMATICS FOR AI
ALGEBRAIC STRUCTURES FOR AI
LINEAR ALGEBRA FOR AI
INTERNET OF THINGS (IOT)
WHAT IS THE IOT?
IOT PROGRAMMING CONNECTED DEVICES
IOT DIGITAL SWITCHES
IOT USER DEFINED FUNCTIONS
ARDOS IOT
IOT PROGRAMMING RASPBERRY PI WITH C AND PYTHON
IOT PYTHON HELLOTHERE.PY
IOT PYTHON FUNCTIONS
IOT INSTALLATION OF VIM
IOT PROGRAMMING IN C
IOT INSTALLING WIRING PI
IOT RASPBERRY PI WITH RASPBIAN OPERATING SYSTEM
HOW TO SETUP THE RASPBIAN OPERATING SYSTEM
FILE SYSTEM LAYOUT IOT
IOT PROGRAMMING IN RASPBERRY
GALILEO, WINDOWS, AND THE IOT
IOT CREATING THE SERVER APPLICATIONS
IOT TEMPERATURE CONTROLLER
IOT CREATION OF TABLES AND CONTROLLERS
IOT SEEDING THE DATABASE
IOT CUSTOM APIS
CONCLUSION
PYTHON FOR ARTIFICIAL INTELLIGENCE
AGENTS AND CONTROL FOR AI
AI REPRESENTING SEARCH PROBLEMS
AI REASONING WITH CONSTRAINTS
DEEP LEARNING FOR AI
INTRODUCTION
1.2 HISTORICAL TRENDS IN DEEP LEARNING
1.2.1 THE MANY NAMES AND CHANGING FORTUNES OF NEURAL NETWORKS
1.2.2 INCREASING DATASET SIZES
1.2.3 INCREASING MODEL SIZES
1.2.4 INCREASING ACCURACY, COMPLEXITY AND REAL-WORLD IMPACT
APPLIED MATH AND MACHINE
LEARNING BASICS
LINEAR ALGEBRA
2.1 SCALARS, VECTORS, MATRICES AND TENSORS
2.3 IDENTITY AND INVERSE MATRICES
2.4 LINEAR DEPENDENCE AND SPAN
2.5 NORMS
2.6 SPECIAL KINDS OF MATRICES AND VECTORS
2.7 EIGENDECOMPOSITION
2.8 SINGULAR VALUE DECOMPOSITION
2.9 THE MOORE-PENROSE PSEUDOINVERSE
2.10 THE TRACE OPERATOR
2.11 THE DETERMINANT
2.12 EXAMPLE: PRINCIPAL COMPONENTS ANALYSIS
PROBABILITY AND INFORMATION THEORY
3.1 WHY PROBABILITY?
3.2 RANDOM VARIABLES
3.3 PROBABILITY DISTRIBUTIONS
3.3.1 DISCRETE VARIABLES AND PROBABILITY MASS FUNCTIONS
3.4 MARGINAL PROBABILITY
3.5 CONDITIONAL PROBABILITY
3.6 THE CHAIN RULE OF CONDITIONAL PROBABILITIES
3.7 INDEPENDENCE AND CONDITIONAL INDEPENDENCE
3.8 EXPECTATION, VARIANCE AND COVARIANCE
3.9 COMMON PROBABILITY DISTRIBUTIONS
3.9.1 BERNOULLI DISTRIBUTION
3.9.2 MULTINOULLI DISTRIBUTION
3.9.3 GAUSSIAN DISTRIBUTION
3.9.4 EXPONENTIAL AND LAPLACE DISTRIBUTIONS
3.9.5 THE DIRAC DISTRIBUTION AND EMPIRICAL DISTRIBUTION
3.9.6 MIXTURES OF DISTRIBUTIONS
3.10 USEFUL PROPERTIES OF COMMON FUNCTIONS
3.11 BAYES’ RULE
3.12 TECHNICAL DETAILS OF CONTINUOUS VARIABLES
3.13 INFORMATION THEORY
REINFORCEMENT LEARNING
1. INTRODUCTION
1.1 REINFORCEMENT LEARNING
1.2 EXAMPLES
1.3 ELEMENTS OF REINFORCEMENT LEARNING
1.4 AN EXTENDED EXAMPLE: TIC-TAC-TOE
1.5 SUMMARY
1.6 HISTORY OF REINFORCEMENT LEARNING
2. EVALUATIVE FEEDBACK
2.1 AN -ARMED BANDIT PROBLEM
2.2 ACTION-VALUE METHODS
2.3 SOFTMAX ACTION SELECTION
2.4 EVALUATION VERSUS INSTRUCTION
2.5 INCREMENTAL IMPLEMENTATION
2.6 TRACKING A NONSTATIONARY PROBLEM
2.7 OPTIMISTIC INITIAL VALUES
2.8 REINFORCEMENT COMPARISON
2.9 PURSUIT METHODS
2.10 ASSOCIATIVE SEARCH
2.11 CONCLUSIONS
3. THE REINFORCEMENT LEARNING PROBLEM
3.1 THE AGENT-ENVIRONMENT INTERFACE
3.2 GOALS AND REWARDS
3.3 RETURNS
3.4 UNIFIED NOTATION FOR EPISODIC AND CONTINUING TASKS
3.5 THE MARKOV PROPERTY
3.6 MARKOV DECISION PROCESSES
3.7 VALUE FUNCTIONS
3.8 OPTIMAL VALUE FUNCTIONS
3.10 SUMMARY
REINFORCEMENT LEARNING ALGORITHMS — AN INTUITIVE OVERVIEW
TERMINOLOGIES
MODEL-FREE VS MODEL-BASED REINFORCEMENT LEARNING
I. MODEL-FREE RL
I.1. POLICY OPTIMIZATION OR POLICY-ITERATION METHODS
I.1.1. POLICY GRADIENT (PG)
I.1.2. ASYNCHRONOUS ADVANTAGE ACTOR-CRITIC (A3C)
I.1.3. TRUST REGION POLICY OPTIMIZATION (TRPO)
I.1.4. PROXIMAL POLICY OPTIMIZATION (PPO)
I.2. Q-LEARNING OR VALUE-ITERATION METHODS
I.2.1 DEEP Q NEURAL NETWORK (DQN)
I.2.2 C51
I.2.3 DISTRIBUTIONAL REINFORCEMENT LEARNING WITH QUANTILE REGRESSION (QR-DQN)
I.2.4 HINDSIGHT EXPERIENCE REPLAY (HER)
I.3 HYBRID
II.1. LEARN THE MODEL
ASYNCHRONOUS ADVANTAGE ACTOR CRITIC (A3C) ALGORITHM
ADVANTAGES:
ROLE OF AI IN AUTONOMOUS DRIVING
VIRTUAL ASSISTANTS IN DESKTOP ENVIRONMENTS
VIRTUAL ASSISTANTS IN MOBILE CONTEXTS
VIRTUAL ASSISTANTS AND THE INTERNET OF THINGS
VIRTUAL ASSISTANTS AS A TYPE OF (DISEMBODIED) ROBOT
VIRTUAL ASSISTANTS AS SOCIAL ROBOTS
THE PLACE FOR AI IN AUTONOMOUS DRIVING:
AI IN “SAFETY-RELATED” AUTONOMOUS DRIVING
FUNCTIONALITIES IS BASED ON STANDARDS
VARIOUS APPROACHES AND PRODUCTS
COMMUNICATION DISPLAYS
IN-CAR VIRTUAL ASSISTANTS
GATEWAY TO IOT (E.G. HOME CONTROL)
CUSTOMISED INFOTAINMENT
PERSONAL HEALTH AND WELL-BEING
RECENT STRATEGIC DEVELOPMENTS
HIGHLIGHTS ON AI AND SELF-DRIVING CARS
HIGHLIGHTS ON AI AND SELF-DRIVING CARS
HIGHLIGHTS ON AI AND SELF-DRIVING CARS
HIGHLIGHTS ON AI AND SELF-DRIVING CARS
HIGHLIGHTS ON AI AND SELF-DRIVING CARS
SOCIAL ROBOTS, AI AND SELF-DRIVING CARS
SOCIAL ROBOTS, AI AND SELF-DRIVING CARS
AI & INDUSTRY 4.0
SOME RECENT DEVELOPMENTS
ARTIFICIAL INTELLIGENCE AND ROBOTICS
