Stereo Camera Calibration Accuracy in Real-time Car Angles Estimation for Vision Driver ...

85 Pages·2017·11.79 MB·English

Stereo Camera Calibration Accuracy in Real-time Car
Angles Estimation for Vision Driver Assistance and
Autonomous Driving
Björn Algers
June 20, 2018
Master of science Thesis, Engineering physics
Stereo Camera Calibration Accuracy in Real-time
Car Angles Estimation for Vision Driver Assistance
and Autonomous Driving
Björn Algers
Supervisors: Ove Andersson
Dep. of Physics, Umeå Universitet
Simon Blomstrand
Veoneer AB
Examiner: Alexandr Talyzin
Dep. of Physics, Umeå Universitet
Department of Physics
Umeå University
SE-90187 Umeå, Sweden
Copyright (cid:13)c 2018 Björn Algers
i
Abstract
TheautomotivesafetycompanyVeoneerareproducersofhighenddrivervisualassistance
systems, but the knowledge about the absolute accuracy of their dynamic calibration
algorithms that estimate the vehicle’s orientation is limited.
In this thesis, a novel measurement system is proposed to be used in gathering reference
data of a vehicle’s orientation as it is in motion, more specifically the pitch and roll angle
ofthevehicle. Focushasbeentoestimatehowtheuncertaintyofthemeasurementsystem
is affected by errors introduced during its construction, and to evaluate its potential in
being a viable tool in gathering reference data for algorithm performance evaluation.
The system consisted of three laser distance sensors mounted on the body of the vehicle,
and a range of data acquisition sequences with different perturbations were performed
by driving along a stretch of road in Linköping with weights loaded in the vehicle. The
referencedatawerecomparedtocamerasystemdatawherethebiasofthecalculatedan-
gles were estimated, along with the dynamic behaviour of the camera system algorithms.
The experimental results showed that the accuracy of the system exceeded 0.1◦ for both
pitch and roll, but no conclusions about the bias of the algorithms could be drawn as
there were systematic errors present in the measurements.
ii
Sammanfattning
BilsäkerhetsföretagetVeoneerärutvecklareavavanceradekamerasysteminomförarassis-
tans, men kunskapen om den absoluta noggrannheten i deras dynamiska kalibreringsal-
goritmer som skattar fordonets orientering är begränsad.
I denna avhandling utvecklas och testas ett nytt mätsystem för att samla in referensdata
avettfordonsorienteringnärdetärirörelse,merspecifiktdesspitchvinkelochrollvinkel.
Fokus har legat på att skatta hur osäkerheten i mätsystemet påverkas av fel som intro-
ducerats vid dess konstruktion, samt att utreda dess potential när det kommer till att
vara ett gångbart alternativ för att samla in referensdata för evaluering av prestandan
hos algoritmerna.
Systemetbestodavtrelaseravståndssensorermonteradepåfordonetskaross. Enradmät-
försök utfördes med olika störningar introducerade genom att köra längs en vägsträcka
i Linköping med vikter lastade i fordonet. Det insamlade referensdatat jämfördes med
data från kamerasystemet där bias hos de framräknade vinklarna skattades, samt att de
dynamiskaegenskapernakamerasystemetsalgoritmerutvärderades. Resultatenfrånmät-
försökenvisadepåattnoggrannhetenimätsystemetöversteg0.1◦ förbådepitchvinklarna
och rollvinklarna, men några slutsatser kring eventuell bias hos algoritmerna kunde ej
dras då systematiska fel uppstått i mätresultaten.
iii
Acknowledgements
I would like to thank first and foremost Veoneer for the chance to prove my worth as
an engineer at their expense, it has been a good learning experience. I would also like
to thank Simon Blomstrand for all the constructive comments and help throughout the
entire process. A huge thanks to the statistics master Xijia Liu of Umeå University for
his insight on the statistical part of the error analysis. A big thank you to friends and
family who took their time to glance through the thesis in search for any mishaps and
shenanigans.
I would also like to thank Ove Andersson for his kind support and insightful comments
on the report during the writing process, and also Alexandr Talyzin for committing to
examine the project.
iv
CONTENTS
Contents
1 Introduction 1
1.1 Problem statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Project goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2.1 Car orientation measurement . . . . . . . . . . . . . . . . . . . . . 2
1.2.2 Post data processing . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2.3 Uncertainty estimation . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2.4 Accuracy evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.4 Nomenclature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2 Background and theory 5
2.1 Pinhole camera model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2 Camera calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2.1 Intrinsic parameters . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2.2 Extrinsic parameters . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.2.3 SECAL - Static Extrinsic Calibration . . . . . . . . . . . . . . . . 9
2.2.4 DECAL - Dynamic Extrinsic Calibration . . . . . . . . . . . . . . 10
2.3 Stereo camera setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.4 Coordinate systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.5 Uncertainty estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.5.1 Monte-Carlo simulations . . . . . . . . . . . . . . . . . . . . . . . . 14
2.5.2 Directional statistics . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.5.3 Resolution uncertainty . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.6 Signal filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.6.1 Low pass filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.6.2 Band stop/notch filter . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.7 Cross correlation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.8 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3 Method 23
3.1 Experimental setup and general procedure . . . . . . . . . . . . . . . . . . 23
3.1.1 Laser measurement rig . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.1.2 Car and camera system . . . . . . . . . . . . . . . . . . . . . . . . 25
3.1.3 Mathematical models. . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.2 System calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.3 Data collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.4 Data post-processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.5 Laser data filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.6 Uncertainty estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.7 Data analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
v
CONTENTS
4 Results 40
4.1 System simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
4.2 Calibration results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
4.3 SECAL - Static Extrinsic Calibration . . . . . . . . . . . . . . . . . . . . . 44
4.4 Road measurements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
5 Discussion 59
5.1 Simulation results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
5.2 Laser system calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
5.3 Experimental results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
6 Concluding remarks 63
6.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
6.2 Further work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
6.2.1 Relative orientation measurement . . . . . . . . . . . . . . . . . . . 63
6.2.2 System construction . . . . . . . . . . . . . . . . . . . . . . . . . . 64
6.2.3 Additional sensors . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
References 65
A Additional results A1
vi
1. INTRODUCTION
1 Introduction
Using computer vision to solve complex problems has become common practice in many
fields, where the goal often is to identify objects in the image or calculate the position
and orientation of a system attached to the camera generating the images. In the mid
sixties, it was thought that a short summer project would be sufficient to solve this type
of problem [1], but researchers are yet still today struggling with the same problems,
however more successfully than ever. In 2015, a team of computer scientists managed to
correctly identify 96.5% of the contents of images in a competition using their software
based on deep neural networks [2], and the accuracy achievable when calculating posi-
tion and orientation is indeed very high - under certain circumstances it is possible to
determine the translation of a camera with an accuracy exceeding 5 mm [3].
Veoneer is a company based in Linköping that is operating at the very edge of this
field, and is where this thesis has taken place. Their camera systems are implemented
in several large automotive company safety systems where the camera aids in issues like
object detection, emergency braking and lane keeping. The long term goal is to produce
systems which are fully autonomous. The performance of these systems depends heavily
on the data that is used to make the decisions, especially when it comes to altering the
velocity of a vehicle moving at 90 km/h, where faulty decisions may have catastrophic
results. Accurate calibration of the camera systems is therefore essential, and gathering
information about their performance in turn to increase it presents a challenge.
1.1 Problem statement
Today’s camera algorithms developed at Veoneer which estimate the car’s orientation
(roll, pitch and yaw) relative to the road only have known relative performance, in other
words the precision and repeatability of the algorithms are well known and understood.
Thereishowevernowelldefinedreferencemethodwhichenablesestablishmentofground
truth information for absolute accuracy estimations.
This Master thesis aims to establish a reference method for measuring ground truth
informationabouttheroadgeometry,wherethegoalistobeabletocorrelatethedynamic
calibration results from the algorithms to the reference method such that any bias can
be estimated and removed.
1
1. INTRODUCTION
1.2 Project goals
The goals of this Master’s thesis can be stated as follows:
• Establish, implement and evaluate a method for measuring the orientation of a
moving vehicle with high accuracy. The pitch and roll angles are to be measured
with an accuracy higher than 0.1 degrees.
• Use this measurement method to gather ground truth data of the vehicle’s orien-
tation, which in turn is used to evaluate the performance of the following dynamic
calibration algorithms:
– The previously deployed algorithm which estimates the pitch angle of the
camera system using Visual Horizon Recognition.
– Theconceptalgorithmcurrentlyunderdevelopmentwhichutilizesvisualodom-
etry to estimate the pitch and roll angles of the camera system.
Below follows a more detailed description of the problem that is to be solved, and the
general approach to reach the goals set for this thesis.
1.2.1 Car orientation measurement
The orientation of the vehicle is to be measured, and this constitutes quite a challenge
since the aim is an accuracy better than 0.1 degrees for both the pitch and roll angles. A
car is a quite large and bulky object, with almost no flat reference surfaces, and adding
to this challenge the measurements are to be done as the vehicle is propagating the long
winding roads of Linköping. The measurement system first has to be calibrated with the
vehicle standing still to validate its accuracy, and then a sequence of measurements in
differing situation will be performed to evaluate how it behaves, and if the gathered data
is meaningful. Parallel with the car orientation reference measurements, data will be
gathered using Veoneers camera systems and stored for later comparison and analysis.
1.2.2 Post data processing
After the measurements have been done, the reference system data will be in raw form
and needs to be processed in different ways before it is possible to analyze it. The data
will be registered using the on-board computer systems of the vehicle with all its specific
formats. The data will have to be transformed from raw data to a MATLAB compatible
format as it is the software that will be used to perform the analysis.
2
1. INTRODUCTION
1.2.3 Uncertainty estimation
A metric has to have an accuracy measurement attached to it for it to be meaningful,
and the error estimation of the measurements to be made poses quite a challenge as well.
As the setup to be used is experimental there are presently no well established methods
for estimating the error for this particular setup. The sources of error are many, such
as misalignment, mistakes made in measuring the dimensions and vibrations, and an
analysis by simulating the measurement system while introducing errors will be done for
it to be feasible.
1.2.4 Accuracy evaluation
After gathering information about the orientation of the car in different situations using
the measurement system, the results from the measurements are to be compared to the
corresponding results from the algorithms. These algorithms calculate the orientation of
the car using the video stream from the on-board vehicle camera. Statistical tests will
be performed to evaluate if any significant differences can be found, as well as qualitative
analysis of the distributions of the angles.
1.3 Limitations
Using the results of the measurements to evaluate algorithms that estimate the ego
motion of the moving vehicle, i.e high dynamic orientation estimation, is outside of the
scope of thisthesis, althoughit could most certainlybe usedto gatherinformation about
this. The results will also only be used to evaluate the two mentioned algorithms in their
present state. The number of measurement runs will also have to be limited, as data
collection is a time consuming activity.
3