Imaging and Sensing for Unmanned Aircraft Systems: Control and Performance (Control, Robotics and Sensors)

362 Pages·2020·26.736 MB·English

by  Vania V. Estrela (editor), Jude Hemanth (editor), Osamu Saotome (editor), George Nikolakopoulos (editor), Roberto Sabatini (editor)

IETCONTROL,ROBOTICS,AND SENSORS SERIES 120
Imaging and Sensing for
Unmanned Aircraft Systems
IETInternationalBookSeriesonSensing—CallforAuthors
Theuseofsensorshasincreaseddramaticallyinallindustries.Theyarefundamentalinawide
rangeofapplicationsfromcommunicationtomonitoring,remoteoperation,processcontrol,
precisionandsafety,androboticsandautomation.Thesedevelopmentshavebroughtnew
challengessuchasdemandsforrobustnessandreliabilityinnetworks,securityinthe
communicationsinterface,andclosemanagementofenergyconsumption.ThisBookSeries
coverstheresearchandapplicationsofsensortechnologiesinthefieldsofICTs,security,
tracking,detection,monitoring,controlandautomation,robotics,machinelearning,smart
technologies,productionandmanufacturing,photonics,environment,energy,andtransport.
BookSeriesEditorialBoard
● Dr.HartmutBrauer,TechnischeUniversita¨tIlmenau,Germany
● Prof.NathanIda,UniversityofAkron,USA
● Prof.EdwardSazonov,UniversityofAlabama,USA
● ProfDesineni“Subbaram”Naidu,UniversityofMinnesotaDuluth,USA
● Prof.WuqiangYang,UniversityofManchester,UK
● Prof.SheraliZeadally,UniversityofKentucky,USA
Proposalsforcoherentlyintegratedinternationalmulti-authorededitedorco-authored
handbooksandresearchmonographswillbeconsideredforthisBookSeries.Eachproposal
willbereviewedbytheIETBookSeriesEditorialBoardmemberswithadditionalexternal
reviewsfromindependentreviewers.Pleaseemailyourbookproposalto:[email protected]
[email protected].
Imaging and Sensing for
Unmanned Aircraft Systems
Volume 1: Control and Performance
Edited by
Vania V. Estrela, Jude Hemanth, Osamu Saotome,
George Nikolakopoulos and Roberto Sabatini
The Institution of Engineering andTechnology
PublishedbyTheInstitutionofEngineeringandTechnology,London,UnitedKingdom
TheInstitutionofEngineeringandTechnologyisregisteredasaCharityinEngland&
Wales(no.211014)andScotland(no.SC038698).
†TheInstitutionofEngineeringandTechnology2020
Firstpublished2020
ThispublicationiscopyrightundertheBerneConventionandtheUniversalCopyright
Convention.Allrightsreserved.Apartfromanyfairdealingforthepurposesofresearch
orprivatestudy,orcriticismorreview,aspermittedundertheCopyright,Designsand
PatentsAct1988,thispublicationmaybereproduced,storedortransmitted,inany
formorbyanymeans,onlywiththepriorpermissioninwritingofthepublishers,orin
thecaseofreprographicreproductioninaccordancewiththetermsoflicencesissued
bytheCopyrightLicensingAgency.Enquiriesconcerningreproductionoutsidethose
termsshouldbesenttothepublisherattheundermentionedaddress:
TheInstitutionofEngineeringandTechnology
MichaelFaradayHouse
SixHillsWay,Stevenage
Herts,SG12AY,UnitedKingdom
www.theiet.org
Whiletheauthorsandpublisherbelievethattheinformationandguidancegiveninthis
workarecorrect,allpartiesmustrelyupontheirownskillandjudgementwhenmaking
useofthem.Neithertheauthorsnorpublisherassumesanyliabilitytoanyoneforany
lossordamagecausedbyanyerrororomissioninthework,whethersuchanerroror
omissionistheresultofnegligenceoranyothercause.Anyandallsuchliabilityis
disclaimed.
Themoralrightsoftheauthorstobeidentifiedasauthorsofthisworkhavebeen
assertedbytheminaccordancewiththeCopyright,DesignsandPatentsAct1988.
BritishLibraryCataloguinginPublicationData
AcataloguerecordforthisproductisavailablefromtheBritishLibrary
ISBN978-1-78561-642-6(HardbackVolume1)
ISBN978-1-78561-643-3(PDFVolume1)
ISBN978-1-78561-644-0(HardbackVolume2)
ISBN978-1-78561-645-7(PDFVolume2)
ISBN978-1-78561-679-2(HardbackVolumes1and2)
TypesetinIndiabyMPSLimited
PrintedintheUKbyCPIGroup(UK)Ltd,Croydon
Contents
Abouttheeditors xi
Preface xiii
1 Introductionto advancesinUAVavionics for imaging andsensing 1
Vania V. Estrela, Jude Hemanth, OsamuSaotome,
GeorgeNikolakopoulos andRoberto Sabatini
1.1 Basic concepts 1
1.2 Navigation and intelligence 4
1.3 Communications 6
1.4 Sensors 7
1.5 Computational aspects: image/video processing, computer
graphics, modelling, and visualisation 9
1.6 Security, health, and standards 11
1.7 Applications 12
1.8 Bookorganization 13
References 17
2 Computer visionanddatastorage inUAVs 23
Vania V. Estrela, Jude Hemanth, HermesJ. Loschi,
DouglasA. Nascimento, YuzoIanoand Navid Razmjooy
2.1 Introduction 23
2.1.1 Requirements 25
2.1.2 Root file system 26
2.1.3 Data logging 26
2.1.4 Cloud support and virtualisation 27
2.2 The architecture of the cloud-based UAVcyber-physical system 27
2.3 UAVneeds versus memory use 30
2.3.1 Limitations of OVP 31
2.3.2 General solutions and their viability analysis 32
2.4 UAVdata logging 32
2.5 Types of data logging 34
2.5.1 Requirements and recommended solutions 36
2.5.2 Internal RAMwithSD 36
2.5.3 External RAMwith SD 37
2.5.4 External flash memory 37
vi Imaging and sensingfor unmanned aircraft systems, volume 1
2.6 Discussionand future trends 37
2.6.1 UAV-baseddata storage 37
2.6.2 UAV-baseddata processing 38
2.6.3 Distributed versuscentralised control 38
2.6.4 Impact of big data in UAV-CPSs 38
2.6.5 Challenges related to privacy and the protection
of personal information 40
2.6.6 Organisational and cultural barriers 40
2.7 Conclusions 41
References 42
3 Integrated optical flow for situationawareness, detection
andavoidance systemsinUAVsystems 47
William Sanchez Farfan, OsamuSaotome, Vania V. Estrela
andNavid Razmjooy
3.1 Introduction 47
3.2 Computer vision 49
3.2.1 Optical Flow 50
3.3 Optical flow and remote sensing 55
3.3.1 Aerial Triangulation 56
3.4 Optical flow and situational awareness 57
3.4.1 Detect and avoidance system 58
3.5 Optical flow and navigation byimages 60
3.5.1 Egomotion 61
3.6 Case study: INSusing FPGA 63
3.6.1 Architectural proposals 65
3.6.2 Integration INS/GPS/OFusinga Kalman filter 67
3.7 Future trends and discussion 68
3.7.1 3Doptical flow 68
3.7.2 Multispectral and hyperspectral images 69
3.8 Conclusion 70
References 71
4 Introductionto navigation andintelligence for UAVsrelying
oncomputer vision 75
SurajBijjahalli and Roberto Sabatini
4.1 Introduction 75
4.2 Basic terminology 77
4.2.1 Visual servoing 79
4.2.2 Visual odometry 84
4.2.3 Terrain-referenced visual navigation 90
4.3 Future trends and discussion 93
4.4 Conclusions 94
References 94
Contents vii
5 Modelling andsimulation of UAVsystems 101
NarendranMuraleedharan and Daniel S.Cohen
5.1 Need for modelling and simulation 101
5.1.1 Control systems design 101
5.1.2 Operator training 102
5.1.3 Sub-system development and testing 102
5.2 Historyand adoption 102
5.2.1 Early aviation 103
5.2.2 First computerised simulations 103
5.2.3 Entry of UAVsinto service 104
5.2.4 Commercial and consumer drones 104
5.3 Modelling of UAVdynamics 104
5.3.1 Model representation methods 105
5.3.2 Common reference frames 106
5.3.3 Representationof state variables 107
5.3.4 Deriving the system equations of motion 111
5.3.5 Flight physicsmodels 115
5.4 Flight dynamics simulation 116
5.4.1 Integration of the equations of motion 116
5.5 Conclusion 119
References 119
6 Multisensordatafusionfor vision-basedUAVnavigation
andguidance 123
Suraj Bijjahalli and Roberto Sabatini
6.1 Introduction 123
6.2 Data-fusionalgorithms 124
6.2.1 Extended Kalman filter 124
6.2.2 Unscented Kalman filter 127
6.2.3 Integration architectures 129
6.3 Fusionof visual sensors 131
References 142
7 Vision-basedUAVposeestimation 145
Paulo Silva Filho, Elcio Hideiti Shiguemori, OsamuSaotome,
and Jairo Panetta
7.1 Introduction 145
7.2 INS–GNSSdrawbacks 146
7.2.1 Inertial navigation systems 146
7.2.2 Global navigation satellites systems 147
7.3 Visual navigation: Aviable alternative 149
7.4 Visual navigation strategies 152
7.4.1 Photogrammetry: Extracting poseinformationfrom images 152
viii Imaging and sensing forunmanned aircraft systems, volume 1
7.4.2 Template matching 156
7.4.3 Landmark recognition 160
7.4.4 Visual odometry 162
7.4.5 Combinationof methods 164
7.5 Future developments on visual navigation systems 165
7.6 Conclusion 166
References 167
8 Visioninmicro-aerial vehicles 173
Navid Razmjooy, Vania V. Estrela and Roberto Sabatini
8.1 Introduction 174
8.1.1 Fixed-wing MAVs 174
8.1.2 Rotary-wing MAVs 177
8.1.3 Flapping-wing or biomimetic MAVs 178
8.1.4 Hybrid MAVs 182
8.2 Computer vision as a biological inspiration 183
8.3 The role of sensingin MAVs 185
8.3.1 Pose-estimation sensors 186
8.3.2 Environmental awareness sensors 187
8.3.3 Sonar ranging sensor 187
8.3.4 Infrared-range sensors 188
8.3.5 Thermal imaging 189
8.3.6 LIDAR 189
8.3.7 Cameras 190
8.4 Illumination 190
8.5 Navigation, pathfinding, and orientation 191
8.6 Communicationand polarisation-inspired machine vision
applications 194
8.6.1 Robot orientationand navigation 194
8.6.2 Polarisation-opponent sensors 195
8.7 CCDcameras and applications in machine vision 197
8.8 Error modelling of environments with uncertainties 201
8.9 Further work and future trends 201
8.9.1 MAVchallenges 202
8.9.2 Proposed solutions for MAVdesign challenges 202
8.9.3 New frontiers in sensors 204
8.10 Conclusion 204
References 205
9 Computer visioninUAVusingROS 217
Gustavo de Carvalho Bertoli, OsamuSaotome andVania V. Estrela
9.1 Introduction 217
9.2 Computer vision on ROS 218
9.3 Applications 218
Contents ix
9.3.1 OpenCV in ROS 218
9.3.2 Visual navigation 229
9.3.3 Setting the drone state estimation node 234
9.4 Future developmentsand trends in ROS 237
9.5 Conclusion 238
References 238
10 Security aspects of UAVandrobot operating system 243
Gustavo de Carvalho Bertoli and OsamuSaotome
10.1 Introduction 243
10.2 Unmanned aerial vehicles 244
10.3 ROSbasic concepts 245
10.4 SecurityUAVreview 248
10.5 SecurityROSreview 249
10.6 UAVsecurity scenarios 250
10.7 Securityassessment onconsumer UAVoperation with ROS 251
10.8 Future trends 255
10.9 Conclusion 255
References 258
11 Visionin indoor andoutdoordrones 261
Maik Basso andEdison Pignaton de Freitas
11.1 Computer vision in unmanned aerial vehicles 261
11.1.1 Indoorenvironments 264
11.1.2 Outdoor environments 269
11.2 Otherapproacheshandlingbothindoorandoutdoorenvironments 273
11.3 Conclusion 275
References 276
12 Sensorsandcomputer vision asameansto monitor andmaintain
aUAVstructural health 281
HelosmanValente de Figueiredo,OsamuSaotome
and Roberto Gil Annes da Silva
12.1 Introduction 282
12.1.1 Case study: aeroelastic instability flutter phenomenon 282
12.2 Related work 284
12.2.1 Structural health monitoring 284
12.2.2 Computer vision forstructural health 285
12.2.3 Flutter certification 285
12.2.4 Computer vision and in in-flight measurements:
future trends 286
12.3 Signal processing on flutter certification 286