Splitting Algorithms, Modern Operator Theory, and Applications

500 Pages·2019·9.499 MB·English

by  Heinz H. Bauschke, Regina S. Burachik, D. Russell Luke

Heinz H. Bauschke
Regina S. Burachik
D. Russell Luke Editors
Splitting Algorithms,
Modern Operator
Theory, and
Applications
Splitting Algorithms, Modern Operator Theory,
and Applications
Heinz H. Bauschke • Regina S. Burachik
D. Russell Luke
Editors
Splitting Algorithms,
Modern Operator Theory,
and Applications
123
Editors
HeinzH.Bauschke ReginaS.Burachik
DepartmentofMathematics SchoolofIT&MathematicalSciences
UniversityofBritishColumbia UniversityofSouthAustralia
Kelowna,BC,Canada MawsonLakes,SA,Australia
D.RussellLuke
Inst.Numerische&Angewandte
Mathematik
Universita¨tGo¨ttingen
Go¨ttingen,Niedersachsen,Germany
ISBN978-3-030-25938-9 ISBN978-3-030-25939-6 (eBook)
https://doi.org/10.1007/978-3-030-25939-6
©SpringerNatureSwitzerlandAG2019
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Thepublisher,theauthors,andtheeditorsaresafetoassumethattheadviceandinformationinthisbook
arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor
theeditorsgiveawarranty,expressorimplied,withrespecttothematerialcontainedhereinorforany
errorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregardtojurisdictional
claimsinpublishedmapsandinstitutionalaffiliations.
ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSwitzerlandAG.
Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland
Thisbookisdedicatedtothememoryof
JonathanMichaelBorwein,whowasagreat
visionary,researcher,teacher,andmentorto
manyofus.
Preface
This book brings together several carefully reviewed papers in the broad areas
of optimization and numerical analysis, with a particular emphasis on algorithms.
Thevolumeisacompendiumoftopicspresentedattheinterdisciplinaryworkshop
on Splitting Algorithms, Modern Operator Theory, and Applications, held at Casa
Matemática Oaxaca (CMO) in Mexico, September 17–22, 2017. The participants
came from Australia, Austria, Belgium, Brazil, Canada, Chile, France, Germany,
HongKong,Japan,theNetherlands,Poland,Russia,Spain,Sweden,andtheUnited
States.Mostpapersinthisvolumegrewoutoftalksdeliveredatthisworkshop.We
believethatthereaderwillfindthisvolumetobeavaluablesnapshotofthestateof
theart.WethankElizabethLoewfromSpringerforherhelpguidingthisvolumeto
completion and through production. Also,thanks to Arian Berdellima who helped
withthetechnicalcompilationofthebook.
TheeditorsalsothankCMOandBIRSfortheirsupportinhostingtheworkshop,
inparticularClaudiaAriasCaoRomeroandMiguelRicardoAltamiranoIbarrafor
theirlocalsupport.Wealsothanktheverydedicatedrefereeswhowereinstrumental
inensuringthehighqualityofcontributions.
Lastbutnotleast,wededicatethisvolumetothememoryofJonathanMichael
Borwein who was an early and enthusiastic supporter of this workshop. He was
a great collaborator, teacher, mentor, and friend to us and to the many attendants
of the workshop. (See also https://jonborwein.org for more on Jon’s incomparable
contributions.)
Kelowna,BC,Canada HeinzH.Bauschke
Adelaide,SA,Australia ReginaS.Burachik
Göttingen,Germany D.RussellLuke
May2019
vii
Contents
1 ConvergenceRateofProximalInertialAlgorithmsAssociated
withMoreauEnvelopesofConvexFunctions ........................... 1
HedyAttouchandJuanPeypouquet
1.1 Introduction,PreliminaryResults ................................... 2
1.1.1 Introducing(RIPA)fromaDynamicPerspective......... 2
1.1.2 TheSequence(t ) .......................................... 5
k
1.1.3 AModelResult............................................. 6
1.1.4 OrganizationofthePaper .................................. 7
1.2 ARegularizedInertialProximalAlgorithm......................... 7
1.2.1 PreliminaryLemmas ....................................... 8
1.2.2 FastConvergenceoftheValues............................ 13
1.2.3 FasterConvergence......................................... 15
1.2.4 ConvergenceoftheIterates ................................ 17
1.3 ComparisonoftheVariousApproaches............................. 18
1.3.1 TheCaseρ ≡1............................................ 18
k
1.3.2 ProximalVersusGradientApproach ...................... 18
1.3.3 LinkwiththeGeneralMaximallyMonotoneCase ....... 19
1.4 TheImpactofGeometryontheRatesofConvergence ............ 20
1.5 StabilitywithRespecttoPerturbations,Errors ..................... 27
1.6 ARegularizedInertialProximal-GradientAlgorithm.............. 34
Appendix..................................................................... 39
References.................................................................... 43
2 Constraint Splitting and Projection Methods for Optimal
ControlofDoubleIntegrator.............................................. 45
HeinzH.Bauschke,ReginaS.Burachik,andC.YalçınKaya
2.1 Introduction........................................................... 46
2.2 Minimum-EnergyControlofDoubleIntegrator.................... 47
2.3 Projections............................................................ 52
2.4 BestApproximationAlgorithms .................................... 55
2.4.1 Dykstra’sAlgorithm........................................ 56
ix
x Contents
2.4.2 Douglas–RachfordAlgorithm.............................. 56
2.4.3 AragónArtacho–CampoyAlgorithm...................... 57
2.5 NumericalImplementation .......................................... 58
2.5.1 TheAlgorithms............................................. 58
2.5.2 NumericalExperiments .................................... 59
2.6 ConclusionandOpenProblems ..................................... 66
Appendix..................................................................... 67
References.................................................................... 67
3 NumericalExplorationsofFeasibilityAlgorithmsforFinding
PointsintheIntersectionofFiniteSets .................................. 69
HeinzH.Bauschke,SylvainGretchko,andWalaaM.Moursi
3.1 Introduction........................................................... 70
3.2 TheFourConstellations.............................................. 71
3.3 TheFourFeasibilityAlgorithms .................................... 73
3.4 SettingUptheNumericalExplorations............................. 74
3.5 Determiningthe“best”Parameterλ ............................. 76
best
3.6 TrackingOrbits....................................................... 77
3.6.1 FewSetswithFewPoints.................................. 78
3.6.2 FewSetswithManyPoints ................................ 79
3.6.3 ManySetswithFewPoints ................................ 80
3.6.4 ManySetswithManyPoints............................... 81
3.6.5 Discussion .................................................. 82
3.7 LocalandGlobalBehaviour......................................... 82
3.7.1 FewSetswithFewPoints.................................. 83
3.7.2 FewSetswithManyPoints ................................ 84
3.7.3 ManySetswithFewPoints ................................ 85
3.7.4 ManySetswithManyPoints............................... 86
3.7.5 Discussion .................................................. 87
3.8 Divertissements....................................................... 87
3.9 ConcludingRemarks................................................. 87
References.................................................................... 89
4 VariableMetricADMMforSolvingVariationalInequalities
withMonotoneOperatorsoverAffineSets.............................. 91
RaduIoanBo¸t,ErnöRobertCsetnek,andDennisMeier
4.1 Introduction........................................................... 91
4.2 NotationandPreliminaries........................................... 93
4.3 AVariableMetricADMMforMonotoneOperators............... 95
4.3.1 ProblemFormulationandAlgorithm...................... 95
4.3.2 ConvergenceAnalysis...................................... 99
4.4 ConvergenceRatesintheCaseWhenA+C IsStrongly
Monotone............................................................. 109
References.................................................................... 111
Contents xi
5 RegularizationofIll-PosedProblemswithNon-negativeSolutions... 113
ChristianClason,BarbaraKaltenbacher,andElenaResmerita
5.1 Introduction........................................................... 113
5.2 Preliminaries.......................................................... 115
5.3 VariationalMethods.................................................. 117
5.3.1 Morozov-EntropyRegularization.......................... 117
5.3.2 Tikhonov-EntropyRegularization ......................... 118
5.3.3 Tikhonov–Kullback–LeiblerRegularization .............. 120
5.3.4 NonquadraticDataMisfit .................................. 121
5.3.5 MeasureSpaceSolutions................................... 122
5.3.6 NonlinearProblems ........................................ 123
5.3.7 IvanovRegularization...................................... 123
5.4 IterativeMethods..................................................... 124
5.4.1 ProjectedLandweberMethodforNon-negative
SolutionsofLinearIll-PosedEquations................... 125
5.4.2 EM Method for Integral Equations with
Non-negativeDataandKernel............................. 127
5.4.3 ModifiedEMAlgorithms .................................. 130
References.................................................................... 133
6 CharacterizationsofSuper-RegularityandItsVariants............... 137
ArisDanillidis,D.RussellLuke,andMatthewTam
6.1 Introduction........................................................... 138
6.2 NormalConesandClarkeRegularity ............................... 138
6.3 Super-RegularityandSubsmoothness............................... 142
6.4 RegularityofFunctions.............................................. 145
6.4.1 LipschitzContinuousFunctions ........................... 147
6.4.2 Non-LipschitzianFunctions................................ 148
References.................................................................... 151
7 TheInverseFunctionTheoremsofL.M.Graves....................... 153
AsenL.Dontchev
7.1 Introduction........................................................... 153
7.2 Hildebrand–GravesTheorem........................................ 156
7.3 TheLyusternik-GravesTheorem.................................... 159
7.4 Bartle–GravesTheorem.............................................. 161
References.................................................................... 163
8 Block-Wise Alternating Direction Method of Multipliers
withGaussianBackSubstitutionforMultiple-BlockConvex
Programming ............................................................... 165
XiaolingFu,BingshengHe,XiangfengWang,andXiaomingYuan
8.1 Introduction........................................................... 166
8.2 Preliminaries.......................................................... 173
8.2.1 VariationalInequalityCharacterization.................... 173
8.2.2 SomeProperties ............................................ 175
xii Contents
8.3 TheBlock-WiseADMMwithGaussianBackSubstitution........ 176
8.3.1 TheNewAlgorithm ........................................ 176
8.3.2 SomeRemarks.............................................. 178
8.4 Convergence.......................................................... 179
8.4.1 SomeMatrices.............................................. 179
8.4.2 APrediction-CorrectionReformulationof(8.3.3)........ 183
8.4.3 AnIllustrativeExampleofLemma8.4.3.................. 187
8.4.4 ConvergenceProof ......................................... 189
8.5 ConvergenceRate.................................................... 191
8.5.1 ConvergenceRateintheErgodicSense................... 191
8.5.2 ConvergenceRateinaNonergodicSense................. 193
8.6 SomeSpecialCases.................................................. 196
8.6.1 TheADMM-GBSin[15] .................................. 199
8.6.2 TheSplittingMethodin[17]............................... 200
8.7 ARefinedVersionofAlgorithm1withCalculatedStepSizes .... 204
8.8 ALinearizedSplittingBlock-WiseADMMwithGaussian
BackSubstitution..................................................... 206
8.8.1 Algorithm................................................... 207
8.8.2 ConvergenceAnalysis...................................... 208
8.9 NumericalExperiments.............................................. 213
8.9.1 ConvergenceofAlgorithm1............................... 213
8.9.2 Convergence of Algorithm 1 with Iteratively
CalculatedStepSizes....................................... 217
8.9.3 ConvergenceofAlgorithm2............................... 218
8.10 Conclusions........................................................... 222
References.................................................................... 224
9 VariableMetricAlgorithmsDrivenbyAveragedOperators.......... 227
LilianE.Glaudin
9.1 Introduction........................................................... 227
9.2 NotationandBackground............................................ 228
9.3 MainConvergenceResult............................................ 230
9.4 ApplicationstotheForward-BackwardAlgorithm................. 234
9.5 ACompositeMonotoneInclusionProblem ........................ 239
References.................................................................... 242
10 A Glimpse at Pointwise Asymptotic Stability for
Continuous-TimeandDiscrete-TimeDynamics ........................ 243
RafalGoebel
10.1 Introduction........................................................... 243
10.2 Dynamics............................................................. 244
10.2.1 WhyBasicAssumptions?.................................. 246
10.3 AsymptoticStability ................................................. 247
10.4 PointwiseAsymptoticStability:SomeExamples .................. 250
10.4.1 FejérMonotonicity ......................................... 251
10.4.2 SteepestDescentforConvexFunctions,andBeyond..... 251
10.4.3 ConsensusAlgorithms ..................................... 256