MATLAB Automated Driving Toolbox User s Guide

1382 Pages·2021·40.893 MB·English

by  coll

Automated Driving Toolbox™
User’s Guide
R2021a
How to Contact MathWorks
Latest news: www.mathworks.com
Sales and services: www.mathworks.com/sales_and_services
User community: www.mathworks.com/matlabcentral
Technical support: www.mathworks.com/support/contact_us
Phone: [phone]
The MathWorks, Inc.
1 Apple Hill Drive
Natick, MA 01760-2098
Automated Driving Toolbox™ User’s Guide
© COPYRIGHT 2017–2021 by The MathWorks, Inc.
The software described in this document is furnished under a license agreement. The software may be used or copied
only under the terms of the license agreement. No part of this manual may be photocopied or reproduced in any form
without prior written consent from The MathWorks, Inc.
FEDERAL ACQUISITION: This provision applies to all acquisitions of the Program and Documentation by, for, or through
the federal government of the United States. By accepting delivery of the Program or Documentation, the government
hereby agrees that this software or documentation qualifies as commercial computer software or commercial computer
software documentation as such terms are used or defined in FAR 12.212, DFARS Part 227.72, and DFARS 252.227-7014.
Accordingly, the terms and conditions of this Agreement and only those rights specified in this Agreement, shall pertain
to and govern the use, modification, reproduction, release, performance, display, and disclosure of the Program and
Documentation by the federal government (or other entity acquiring for or through the federal government) and shall
supersede any conflicting contractual terms or conditions. If this License fails to meet the government's needs or is
inconsistent in any respect with federal procurement law, the government agrees to return the Program and
Documentation, unused, to The MathWorks, Inc.
Trademarks
MATLAB and Simulink are registered trademarks of The MathWorks, Inc. See
www.mathworks.com/trademarks for a list of additional trademarks. Other product or brand names may be
trademarks or registered trademarks of their respective holders.
Patents
MathWorks products are protected by one or more U.S. patents. Please see www.mathworks.com/patents for
more information.
Revision History
March 2017 Online only New for Version 1.0 (Release 2017a)
September 2017 Online only Revised for Version 1.1 (Release 2017b)
March 2018 Online only Revised for Version 1.2 (Release 2018a)
September 2018 Online only Revised for Version 1.3 (Release 2018b)
March 2019 Online only Revised for Version 2.0 (Release 2019a)
September 2019 Online only Revised for Version 3.0 (Release 2019b)
March 2020 Online only Revised for Version 3.1 (Release 2020a)
September 2020 Online only Revised for Version 3.2 (Release 2020b)
March 2021 Online only Revised for Version 3.3 (Release 2021a)
Contents
Sensor Configuration and Coordinate System Transformations
1
Coordinate Systems in Automated Driving Toolbox . . . . . . . . . . . . . . . . . . 1-2
World Coordinate System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-2
Vehicle Coordinate System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-2
Sensor Coordinate System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-4
Spatial Coordinate System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-7
Pattern Coordinate System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-7
Calibrate a Monocular Camera . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-9
Estimate Intrinsic Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-9
Place Checkerboard for Extrinsic Parameter Estimation . . . . . . . . . . . . . . 1-9
Estimate Extrinsic Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-12
Configure Camera Using Intrinsic and Extrinsic Parameters . . . . . . . . . . 1-12
Ground Truth Labeling and Verification
2
Get Started with the Ground Truth Labeler . . . . . . . . . . . . . . . . . . . . . . . . 2-2
Load Ground Truth Signals to Label . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-4
Load Timestamps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-4
Open Ground Truth Labeler App . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-4
Load Signals from Data Sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-4
Configure Signal Display . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-7
Label Ground Truth for Multiple Signals . . . . . . . . . . . . . . . . . . . . . . . . . . 2-9
Create Label Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-9
Label Video Using Automation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-13
Label Point Cloud Sequence Using Automation . . . . . . . . . . . . . . . . . . . . 2-15
Label with Sublabels and Attributes Manually . . . . . . . . . . . . . . . . . . . . . 2-18
Label Scene Manually . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-20
View Label Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-20
Save App Session . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-20
Export and Explore Ground Truth Labels for Multiple Signals . . . . . . . . 2-21
Sources vs. Signals in Ground Truth Labeling . . . . . . . . . . . . . . . . . . . . . 2-28
Keyboard Shortcuts and Mouse Actions for Ground Truth Labeler . . . . 2-30
Label Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-30
Frame Navigation and Time Interval Settings . . . . . . . . . . . . . . . . . . . . . 2-30
Labeling Window . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-30
iii
Cuboid Resizing and Moving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-31
Polyline Drawing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-32
Polygon Drawing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-32
Zooming, Panning, and Rotating . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-33
App Sessions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-34
Control Playback of Signal Frames for Labeling . . . . . . . . . . . . . . . . . . . 2-35
Signal Frames . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-35
Master Signal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-35
Change Master Signal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-36
Display All Timestamps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-37
Specify Timestamps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-38
Frame Display and Automation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-38
Label Lidar Point Clouds for Object Detection . . . . . . . . . . . . . . . . . . . . . 2-39
Set Up Lidar Point Cloud Labeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-39
Zoom, Pan, and Rotate Frame . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-40
Hide Ground . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-40
Label Cuboid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-41
Modify Cuboid Label . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-43
Apply Cuboids to Multiple Frames . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-44
Configure Display . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-44
Create Class for Loading Custom Ground Truth Data Sources . . . . . . . . 2-46
Custom Class Folder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-46
Class Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-46
Class Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-46
Method to Customize Load Panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-47
Methods to Get Load Panel Data and Load Data Source . . . . . . . . . . . . . 2-48
Method to Read Frames . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-50
Use Predefined Data Source Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-51
Tracking and Sensor Fusion
3
Visualize Sensor Data and Tracks in Bird's-Eye Scope . . . . . . . . . . . . . . . . 3-2
Open Model and Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-2
Find Signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-2
Run Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-6
Organize Signal Groups (Optional) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-8
Update Model and Rerun Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-8
Save and Close Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-8
Linear Kalman Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-11
State Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-11
Measurement Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-12
Linear Kalman Filter Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-13
Filter Loop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-13
Constant Velocity Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-14
Constant Acceleration Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-15
iv Contents
Extended Kalman Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-16
State Update Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-16
Measurement Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-16
Extended Kalman Filter Loop . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-17
Predefined Extended Kalman Filter Functions . . . . . . . . . . . . . . . . . . . . 3-18
Planning, Mapping, and Control
4
Display Data on OpenStreetMap Basemap . . . . . . . . . . . . . . . . . . . . . . . . . 4-2
Read and Visualize HERE HD Live Map Data . . . . . . . . . . . . . . . . . . . . . . . 4-7
Enter Credentials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-7
Configure Reader to Search Specific Catalog . . . . . . . . . . . . . . . . . . . . . . 4-8
Create Reader for Specific Map Tiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-8
Read Map Layer Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-11
Visualize Map Layer Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-12
HERE HD Live Map Layers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-15
Road Centerline Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-16
HD Lane Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-17
HD Localization Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-18
Rotations, Orientations, and Quaternions for Automated Driving . . . . . 4-19
Quaternion Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-19
Quaternion Creation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-19
Quaternion Math . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-21
Extract Quaternions from Transformation Matrix . . . . . . . . . . . . . . . . . . 4-23
Control Vehicle Velocity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-26
Velocity Profile of Straight Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-28
Velocity Profile of Path with Curve and Direction Change . . . . . . . . . . . . 4-32
Cuboid Driving Scenario Simulation
5
Create Driving Scenario Interactively and Generate Synthetic Sensor Data
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-2
Create Driving Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-2
Add a Road . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-2
Add Lanes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-3
Add Barriers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-4
Add Vehicles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-5
Add a Pedestrian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-7
Add Sensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-8
Generate Synthetic Sensor Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-13
Save Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-14
v
Keyboard Shortcuts and Mouse Actions for Driving Scenario Designer
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-16
Canvas Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-16
Road Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-16
Actor Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-16
Preview Actor Times of Arrival . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-19
Barrier Placement Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-20
Sensor Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-20
File Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-21
Prebuilt Driving Scenarios in Driving Scenario Designer . . . . . . . . . . . . 5-22
Choose a Prebuilt Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-22
Modify Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-41
Generate Synthetic Sensor Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-41
Save Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-42
Euro NCAP Driving Scenarios in Driving Scenario Designer . . . . . . . . . . 5-44
Choose a Euro NCAP Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-44
Modify Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-62
Generate Synthetic Detections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-62
Save Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-63
Cuboid Versions of 3D Simulation Scenes in Driving Scenario Designer
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-65
Choose 3D Simulation Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-65
Modify Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-69
Save Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-70
Recreate Scenario in Simulink for 3D Environment . . . . . . . . . . . . . . . . . 5-70
Create Reverse Motion Driving Scenarios Interactively . . . . . . . . . . . . . 5-72
Three-Point Turn Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-72
Add Road . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-72
Add Vehicle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-74
Add Trajectory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-75
Run Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-78
Adjust Trajectory Using Specified Yaw Values . . . . . . . . . . . . . . . . . . . . . 5-79
Generate INS Sensor Measurements from Interactive Driving Scenario
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-82
Import Road Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-82
Add Actor and Trajectory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-84
Smooth the Trajectory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-86
Add INS Sensor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-87
Simulate Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-88
Export and Explore Sensor Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-89
Import OpenDRIVE Roads into Driving Scenario . . . . . . . . . . . . . . . . . . . 5-91
Import OpenDRIVE File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-91
Inspect Roads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-92
Add Actors and Sensors to Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-96
Generate Synthetic Detections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-98
Save Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-99
Export Driving Scenario to OpenDRIVE File . . . . . . . . . . . . . . . . . . . . . . 5-101
Load Scenario File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-101
vi Contents
Export to OpenDRIVE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-102
Inspect Exported Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-103
Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-104
Import HERE HD Live Map Roads into Driving Scenario . . . . . . . . . . . 5-108
Set Up HERE HDLM Credentials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-108
Specify Geographic Coordinates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-108
Select Region Containing Roads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-110
Select Roads to Import . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-111
Import Roads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-113
Compare Imported Roads Against Map Data . . . . . . . . . . . . . . . . . . . . . 5-114
Save Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-114
Import OpenStreetMap Data into Driving Scenario . . . . . . . . . . . . . . . . 5-115
Select OpenStreetMap File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-115
Select Roads to Import . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-116
Import Roads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-117
Compare Imported Roads Against Map Data . . . . . . . . . . . . . . . . . . . . . 5-118
Save Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-119
Import Zenrin Japan Map API 3.0 (Itsumo NAVI API 3.0) into Driving
Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-121
Set Up Zenrin Japan Map API 3.0 (Itsumo NAVI API 3.0) Credentials . . 5-121
Specify Geographic Coordinates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-121
Select Region Containing Roads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-123
Select Roads to Import . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-124
Import Roads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-126
Compare Imported Roads Against Map Data . . . . . . . . . . . . . . . . . . . . . 5-127
Save Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-127
Create Driving Scenario Variations Programmatically . . . . . . . . . . . . . . 5-129
Generate Sensor Blocks Using Driving Scenario Designer . . . . . . . . . . 5-135
Test Open-Loop ADAS Algorithm Using Driving Scenario . . . . . . . . . . . 5-144
Test Closed-Loop ADAS Algorithm Using Driving Scenario . . . . . . . . . . 5-151
Automate Control of Intelligent Vehicles by Using Stateflow Charts . . 5-156
3D Simulation – User's Guide
6
Unreal Engine Simulation for Automated Driving . . . . . . . . . . . . . . . . . . . 6-2
Unreal Engine Simulation Blocks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-2
Algorithm Testing and Visualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-5
Unreal Engine Simulation Environment Requirements and Limitations
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-7
Software Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-7
Minimum Hardware Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-7
Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-7
vii
How Unreal Engine Simulation for Automated Driving Works . . . . . . . . . 6-9
Communication with 3D Simulation Environment . . . . . . . . . . . . . . . . . . . 6-9
Block Execution Order . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-9
Coordinate Systems for Unreal Engine Simulation in Automated Driving
Toolbox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-11
World Coordinate System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-11
Vehicle Coordinate System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-13
Choose a Sensor for Unreal Engine Simulation . . . . . . . . . . . . . . . . . . . . 6-17
Simulate Simple Driving Scenario and Sensor in Unreal Engine
Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-23
Depth and Semantic Segmentation Visualization Using Unreal Engine
Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-32
Visualize Sensor Data from Unreal Engine Simulation Environment . . . 6-37
Customize Unreal Engine Scenes for Automated Driving . . . . . . . . . . . . 6-45
Install Support Package for Customizing Scenes . . . . . . . . . . . . . . . . . . . 6-46
Verify Software and Hardware Requirements . . . . . . . . . . . . . . . . . . . . . 6-46
Install Support Package . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-46
Set Up Scene Customization Using Support Package . . . . . . . . . . . . . . . 6-46
Customize Scenes Using Simulink and Unreal Editor . . . . . . . . . . . . . . . 6-50
Open Unreal Editor from Simulink . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-50
Reparent Actor Blueprint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-51
Create or Modify Scenes in Unreal Editor . . . . . . . . . . . . . . . . . . . . . . . . 6-52
Run Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-54
Package Custom Scenes into Executable . . . . . . . . . . . . . . . . . . . . . . . . . 6-56
Package Scene into Executable Using Unreal Editor . . . . . . . . . . . . . . . . 6-56
Simulate Scene from Executable in Simulink . . . . . . . . . . . . . . . . . . . . . 6-57
Apply Semantic Segmentation Labels to Custom Scenes . . . . . . . . . . . . . 6-59
Create Top-Down Map of Unreal Engine Scene . . . . . . . . . . . . . . . . . . . . . 6-65
Capture Screenshot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-65
Convert Screenshot to Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-66
Place Cameras on Actors in the Unreal Editor . . . . . . . . . . . . . . . . . . . . . 6-68
Place Camera on Static Actor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-68
Place Camera on Vehicle in Custom Project . . . . . . . . . . . . . . . . . . . . . . 6-71
Prepare Custom Vehicle Mesh for the Unreal Editor . . . . . . . . . . . . . . . . 6-79
Step 1: Setup Bone Hierarchy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-79
Step 2: Assign Materials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-80
Step 3: Export Mesh and Armature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-82
Step 4: Import Mesh to Unreal Editor . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-83
Step 5: Set Block Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-84
viii Contents
Featured Examples
7
Configure Monocular Fisheye Camera . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-4
Annotate Video Using Detections in Vehicle Coordinates . . . . . . . . . . . . 7-10
Automate Ground Truth Labeling Across Multiple Signals . . . . . . . . . . . 7-18
Automate Ground Truth Labeling of Lane Boundaries . . . . . . . . . . . . . . 7-38
Automate Ground Truth Labeling for Semantic Segmentation . . . . . . . . 7-50
Automate Attributes of Labeled Objects . . . . . . . . . . . . . . . . . . . . . . . . . . 7-60
Evaluate Lane Boundary Detections Against Ground Truth Data . . . . . . 7-74
Evaluate and Visualize Lane Boundary Detections Against Ground Truth
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-86
Visual Perception Using Monocular Camera . . . . . . . . . . . . . . . . . . . . . . . 7-99
Create 360° Bird's-Eye-View Image Around a Vehicle . . . . . . . . . . . . . . 7-119
Train a Deep Learning Vehicle Detector . . . . . . . . . . . . . . . . . . . . . . . . . 7-139
Ground Plane and Obstacle Detection Using Lidar . . . . . . . . . . . . . . . . 7-149
Code Generation for Tracking and Sensor Fusion . . . . . . . . . . . . . . . . . 7-158
Forward Collision Warning Using Sensor Fusion . . . . . . . . . . . . . . . . . . 7-166
Adaptive Cruise Control with Sensor Fusion . . . . . . . . . . . . . . . . . . . . . . 7-179
Forward Collision Warning Application with CAN FD and TCP/IP . . . . 7-197
Multiple Object Tracking Tutorial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-203
Track Multiple Vehicles Using a Camera . . . . . . . . . . . . . . . . . . . . . . . . . 7-211
Track Vehicles Using Lidar: From Point Cloud to Track List . . . . . . . . . 7-218
Sensor Fusion Using Synthetic Radar and Vision Data . . . . . . . . . . . . . 7-237
Sensor Fusion Using Synthetic Radar and Vision Data in Simulink . . . 7-246
Autonomous Emergency Braking with Sensor Fusion . . . . . . . . . . . . . . 7-255
Visualize Sensor Coverage, Detections, and Tracks . . . . . . . . . . . . . . . . 7-267
Extended Object Tracking of Highway Vehicles with Radar and Camera
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-275
ix
Track-to-Track Fusion for Automotive Safety Applications . . . . . . . . . . 7-295
Track-to-Track Fusion for Automotive Safety Applications in Simulink 7-308
Visual-Inertial Odometry Using Synthetic Data . . . . . . . . . . . . . . . . . . . 7-312
Lane Following Control with Sensor Fusion and Lane Detection . . . . . 7-321
Track-Level Fusion of Radar and Lidar Data . . . . . . . . . . . . . . . . . . . . . . 7-331
Track-Level Fusion of Radar and Lidar Data in Simulink . . . . . . . . . . . 7-351
Track Vehicles Using Lidar Data in Simulink . . . . . . . . . . . . . . . . . . . . . 7-361
Grid-based Tracking in Urban Environments Using Multiple Lidars . . 7-370
Track Multiple Lane Boundaries with a Global Nearest Neighbor Tracker
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-384
Generate Code for a Track Fuser with Heterogeneous Source Tracks . 7-393
Highway Vehicle Tracking with Multipath Radar Reflections . . . . . . . . 7-403
Scenario Generation from Recorded Vehicle Data . . . . . . . . . . . . . . . . . 7-414
Lane Keeping Assist with Lane Detection . . . . . . . . . . . . . . . . . . . . . . . . 7-427
Model Radar Sensor Detections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-441
Model Vision Sensor Detections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-457
Radar Signal Simulation and Processing for Automated Driving . . . . . 7-475
Simulate Radar Ghosts due to Multipath Return . . . . . . . . . . . . . . . . . . 7-487
Create Driving Scenario Programmatically . . . . . . . . . . . . . . . . . . . . . . . 7-499
Create Actor and Vehicle Trajectories Programmatically . . . . . . . . . . . 7-518
Define Road Layouts Programmatically . . . . . . . . . . . . . . . . . . . . . . . . . . 7-529
Automated Parking Valet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-541
Automated Parking Valet in Simulink . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-570
Highway Trajectory Planning Using Frenet Reference Path . . . . . . . . . 7-577
Motion Planning in Urban Environments Using Dynamic Occupancy Grid
Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-592
Code Generation for Path Planning and Vehicle Control . . . . . . . . . . . . 7-606
Use HERE HD Live Map Data to Verify Lane Configurations . . . . . . . . . 7-615
x Contents