Table Of ContentBestMasters
Springer awards „BestMasters“ to the best master’s theses which have been com-
pleted at renowned universities in Germany, Austria, and Switzerland.
Th e studies received highest marks and were recommended for publication by su-
pervisors. Th ey address current issues from various fi elds of research in natural
sciences, psychology, technology, and economics.
Th e series addresses practitioners as well as scientists and, in particular, off ers guid-
ance for early stage researchers.
Sven Painer
Variation Based Dense
3D Reconstruction
Application on Monocular
Mini-Laparoscopic Sequences
With a preface by Prof. Dr.-Ing. Rolf-Rainer Grigat
Sven Painer
Hamburg, Germany
Awarded with the Fokusfi nder Preis 2015 of Initiative Bilderverarbeitung e.V. and the
Karl H. Ditze Preis 2015 of Hamburg University of Technology.
OnlinePlus material to this book can be available on
http://www.springer-vieweg.de/978-3-658-12697-1
BestMasters
ISBN 978-3-658-12697-1 ISBN 978-3-658-12698-8 (eBook)
DOI 10.1007/978-3-658-12698-8
Library of Congress Control Number: 2016934715
Springer Vieweg
© Springer Fachmedien Wiesbaden 2016
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or
part of the material is concerned, specifi cally the rights of translation, reprinting, reuse of illus-
trations, recitation, broadcasting, reproduction on microfi lms or in any other physical way, and
transmission or information storage and retrieval, electronic adaptation, computer software, or by
similar or dissimilar methodology now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publication does not imply, even in the absence of a specifi c statement, that such names are
exempt from the relevant protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authors or the editors give a warranty, express or implied, with respect to the material contained
herein or for any errors or omissions that may have been made.
Printed on acid-free paper
This Springer Vieweg imprint is published by Springer Nature
The registered company is Springer Fachmedien Wiesbaden GmbH
Institute Profile
TheresarchfocusofTUHHVisionSystems,aresearchgroupofHamburgUniversityofTech-
nology,isonimageanalysisandpatternrecognition,bothinbasicresearchanditstransferto
industrialapplications.
Examples of our research expertise are three-dimensional geometric reconstruction from
videosequences(3DComputerVision),colormanagementandmultispectralmodeling,camera
calibrationandcameracharacterization,medicalimageanalysis,andmethodsforclassification,
detectionandparameterestimation.Methodsarerelatedtomathematics,physics,communica-
tionsengineering,microelectronicsandcomputerscience.
Selectedapplicationfieldsare3Dmodelingofsurfaces,self-calibrationofcameras,recon-
structionofindoorscenesfeaturingonlysparseornotexturing,inverseproblemsinmedicine,
jointsegmentationinx-rayimages, optimalcolorreproduction, andlow-levelalgorithmsfor
mobilephonecameras.
Preface
3Dmodelingofcirrhoticliversurfacesbasedonlaparoscopicvideoswillrenderpossiblethe
quantificationoffibrosisinalmostreal-time. Uptonow,qualitativeandverbaldescriptionsas
wellasvideodocumentationshadtobeused.
OriginwastheresearchofDr.-Ing. JanMarekMarcinczak,whohasbeensupervisingand
supportingMr.Painer’sMasterThesis.DuringhisPhDresearchMr.Marcinczakhasestimated
thecameramotionandthegeometryoftheorgansurfacebasedoncompletevideosequences
[13].Hismethodsworkoff-lineandnon-causal.Basedonthiswork,Mr.Painerhasdeveloped
acausalmethod,whichbasicallyworkson-lineandinreal-time.
Mr. Painerparallelizedthealgorithmsandprovedtheperformanceofhisimplementation.
Headjustedthemethodsforthecompletereconstruction,consistingofasparsereconstruction
viaStructurefromMotionandadensereconstructionviaavariationalapproach,andembedded
theseintoasoftwareframeworkdesignedentirelybyhimself.
The computationally intensive dense reconstruction has been parallelized by Mr. Painer
usingCUDAandaself-designedclasslibrarytobeexecutedonthegraphicscard. Thisclass
librarytakescareofclearingresourcesafterusagetopreventresourceleaksinthecontextof
CUDA.Theimplementationtakesonly55secondsforthereconstructionof30framesconsist-
ingofsparseanddensereconstruction. Previous,non-causalalgorithmsrequired30minutes
forthereconstruction.
Mr. Painer’s Master Thesis has been awarded the Fokusfinder Preis 2015 of Initiative
Bilderverarbeitunge.V.andtheKarlH.DitzePreis2015ofHamburgUniversityofTechnology.
ThetopicwasmotivatedbyProf.Dr.AnsgarW.Lohse,MedicalDirectorofI.Department
ofInternalMedicineofUniversitätsklinikumHamburg-Eppendorf(UKE).Thecooperationhas
beenaccompaniedbyPDDr. UlrikeDenzer,Dr. JulianHolzhüterandDr. TobiasWernerof
UKE.Wecordiallyappreciatetheircooperation.
Mr.PainerhasskillfullysolvedallmathematicallysophisticateddifficultiesduringhisMas-
terThesisandprovidedascalableconcurrentimplementation,whichcanbasicallybeexecuted
inreal-timeonanadequatehardwareplatform. Theresultpavesthewayforsupportingthe
cliniciansbothon-lineduringvideocapturingandforquantitativedocumentation. Thefeed-
backduringtheinterventioncanhelpimprovingthequalityofthevideosandthesubsequent
reconstructions.Ihopeforapositiveresponseandabroadaudienceofthiswork.
Prof.Dr.-Ing.Rolf-RainerGrigat
TUHHVisionSystems
HamburgUniversityofTechnology
Acknowledgements
IwouldliketothankmysupervisorDr.-Ing.JanMarekMarcinczakforguidingmethroughmy
work. Iwouldnothavecomesofarwithoutyourfriendlyandcompetenthelp. Youwillbean
exampleformeonhowtosupervisepeople.
IwouldalsoliketothankProf. Dr.-Ing. Rolf-RainerGrigatforgivingmethechanceof
writingthisthesisandforalltheexperiencesIwasabletogainthroughoutthetime.
AspecialthankgoestoallpeopleoftheTUHHVisionSystemsinstitutefortheniceworking
atmosphere.Itwasapleasurespendingthetimewithyou.
Ifthereareonlyafewmistakesinthisthesis,itisbecauseofthegreatsupportbyPiaBeilfuß
andMichaelGarben.Thankyouforproofreadingthisthesis.
Lastbutnotleast,Iwouldalsoliketothankmyfamilyforthegreatsupport. Withoutyou
IwouldnotbewhereIamtodayandwithoutyouIwouldnothavewrittenthisthesis. Thank
youverymuch!
Contents
ListofFigures XIII
ListofTables XV
1 Introduction 1
1.1 Motivation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 TaskDefinition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2 TheoreticalBackground 3
2.1 Legendre-FenchelTransformation . . . . . . . . . . . . . . . . . . . . . . . . 3
2.2 PhotometricInvariants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.3 SparseReconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.3.1 Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.3.2 KeyframeSelectionandPoseEstimation . . . . . . . . . . . . . . . . 7
2.3.3 Triangulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.3.4 BundleAdjustment . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.4 DenseReconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.4.1 VariationalApproach . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.4.2 CostVolume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.4.3 CouplingtoSparseReconstruction. . . . . . . . . . . . . . . . . . . . 12
2.4.4 Minimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3 Implementation 17
3.1 Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.2 CUDAFramework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.3 SparseReconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.3.1 StructurefromMotion . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.3.2 Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.3.3 PoseEstimation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.3.4 Triangulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.3.5 BundleAdjustment . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.4 DenseReconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
4 Evaluation 27
4.1 GroundTruthData . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.2 SparseReconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
XII Contents
4.3 DenseReconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4.4 Performance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
5 Conclusion 35
AppendixA ClassDiagramsofCUDAFramework 37
AppendixB ClassDiagramsofSparseReconstruction 41
AppendixC EvaluationResultsofSparseReconstruction 47
Bibliography 77
