Table Of ContentSPRINGER BRIEFS IN MATHEMATICS
Jingrui Sun
Jiongmin Yong
Stochastic
Linear-Quadratic
Optimal Control
Theory: Open-Loop
and Closed-Loop
Solutions
SpringerBriefs in Mathematics
Series Editors
Nicola Bellomo, Torino, Italy
Michele Benzi, Pisa, Italy
Palle Jorgensen, Iowa City, USA
Tatsien Li, Shanghai, China
Roderick Melnik, Waterloo, Canada
Otmar Scherzer, Linz, Austria
Benjamin Steinberg, New York City, USA
Lothar Reichel, Kent, USA
Yuri Tschinkel, New York City, USA
George Yin, Detroit, USA
Ping Zhang, Kalamazoo, USA
SpringerBriefsinMathematicsshowcasesexpositionsinallareasofmathematics
andappliedmathematics.Manuscriptspresentingnewresultsorasinglenewresult
inaclassicalfield,newfield,oranemergingtopic,applications,orbridgesbetween
newresultsandalreadypublishedworks,areencouraged.Theseriesisintendedfor
mathematicians and applied mathematicians.
BCAM SpringerBriefs
Editorial Board
EnriqueZuazua
DeustoTech
UniversidaddeDeusto
Bilbao,Spain
and
DepartamentodeMatemáticas
UniversidadAutónomadeMadrid
Cantoblanco,Madrid,Spain
IreneFonseca
CenterforNonlinearAnalysis
DepartmentofMathematicalSciences
CarnegieMellonUniversity
Pittsburgh,USA
JuanJ.Manfredi
DepartmentofMathematics
UniversityofPittsburgh
Pittsburgh,USA
EmmanuelTrélat
LaboratoireJacques-LouisLions
InstitutUniversitairedeFrance
UniversitéPierreetMarieCurie
CNRS,UMR,Paris
XuZhang
SchoolofMathematics
SichuanUniversity
Chengdu,China
BCAM SpringerBriefs aims to publish contributions in the following disciplines: Applied
Mathematics,Finance,StatisticsandComputerScience.BCAMhasappointedanEditorialBoard,
whoevaluateandreviewproposals.
Typicaltopicsinclude:atimelyreportofstate-of-the-artanalyticaltechniques,bridgebetweennew
researchresultspublishedinjournalarticlesandacontextualliteraturereview,asnapshotofahot
oremergingtopic,apresentationofcoreconceptsthatstudentsmustunderstandinordertomake
independentcontributions.
Please submit your proposal to the Editorial Board or to Francesca Bonadei, Executive Editor
Mathematics,Statistics,andEngineering:[email protected].
Moreinformationaboutthisseriesathttp://www.springer.com/series/10030
Jingrui Sun Jiongmin Yong
(cid:129)
Stochastic Linear-Quadratic
Optimal Control Theory:
Open-Loop and Closed-Loop
Solutions
123
Jingrui Sun JiongminYong
Department ofMathematics Department ofMathematics
SouthernUniversity ofScience University of Central Florida
andTechnology Orlando, FL,USA
Shenzhen,Guangdong, China
ISSN 2191-8198 ISSN 2191-8201 (electronic)
SpringerBriefs inMathematics
ISBN978-3-030-20921-6 ISBN978-3-030-20922-3 (eBook)
https://doi.org/10.1007/978-3-030-20922-3
©TheAuthor(s),underexclusivelicensetoSpringerNatureSwitzerlandAG2020
Thisworkissubjecttocopyright.AllrightsaresolelyandexclusivelylicensedbythePublisher,whether
thewholeorpartofthematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseof
illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and
transmissionorinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilar
ordissimilarmethodologynowknownorhereafterdeveloped.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfrom
therelevantprotectivelawsandregulationsandthereforefreeforgeneraluse.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authorsortheeditorsgiveawarranty,expressorimplied,withrespecttothematerialcontainedhereinor
for any errors or omissions that may have been made. The publisher remains neutral with regard to
jurisdictionalclaimsinpublishedmapsandinstitutionalaffiliations.
ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSwitzerlandAG
Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland
To Our Parents
Yuqi Sun and Xiuying Ma
Wenyao Yong and Xiangxia Chen
Preface
Linear-quadratic optimal control theory (LQ theory, for short) has a long history,
andthegeneralconsensusisthatLQtheoryisquitemature.Itchieflyinvolvesthree
well-known and relevant issues: the existence of optimal controls, the solvability
of the optimality system (which is a two-point boundary value problem), and the
solvability of the associated Riccati equation. Broadly speaking, these three issues
aresomehowequivalent.Forthepastfewyearswe,togetherwithourcollaborators,
have been reinvestigating LQ theory for stochastic systems with deterministic
coefficients. In this context, we have identified a number of interesting issues,
including
(cid:129) For finite-horizon LQ problems, open-loop optimal controls may not have a
closed-loop representation.
(cid:129) For finite-horizon LQ problems, a distinction should be made between
open-loop optimal controls and closed-loop optimal strategies. The existence
of the latter implies the existence of the former, but not vice versa.
(cid:129) For infinite-horizon LQ problems (with constant coefficients), under proper
conditions, the open-loop and the closed-loop solvability are equivalent.
Moreover, our investigations have revealed some previously unknown aspects;
these include but are not limited to the following:
(cid:129) For finite-horizon LQ problems, the open-loop solvability is equivalent to the
solvability of the optimality system, which is a forward–backward stochastic
differential equation (FBSDE), together with the convexity of the cost
functional.
(cid:129) For finite-horizon LQ problems, the closed-loop solvability is equivalent to the
existence of a regular solution to the Riccati differential equation.
(cid:129) For infinite-horizon LQ problems (with constant coefficients), both the
open-loopandtheclosed-loopsolvabilityareequivalenttothesolvabilityofan
algebraic Riccati equation.
vii
viii Preface
The purpose of this book is to systematically present the above-mentioned
results and many other relevant ones. We assume that readers are familiar with
basic stochastic analysis and stochastic control theory.
This work was supported in part by NSFC Grant 11901280 and NSF Grants
DMS-1406776 and DMS-1812921.
Theauthorswouldalsoliketoexpresstheirgratitudetotheanonymousreferees
for their constructive comments, which led to this improved version.
Shenzhen, China Jingrui Sun
Orlando, USA Jiongmin Yong
March 2020
Contents
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Why Linear-Quadratic Problems? . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Standard Results for Deterministic LQ Problems . . . . . . . . . . . . . 4
1.3 Quadratic Functionals in a Hilbert Space . . . . . . . . . . . . . . . . . . . 6
2 Linear-Quadratic Optimal Controls in Finite Horizons . . . . . . . . . . 11
2.1 Formulation of the Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2 Representation of the Cost Functional . . . . . . . . . . . . . . . . . . . . . 18
2.3 Open-Loop Solvability and FBSDEs . . . . . . . . . . . . . . . . . . . . . . 26
2.4 Closed-Loop Solvability and Riccati Equation . . . . . . . . . . . . . . . 28
2.5 Uniform Convexity of the Cost Functional. . . . . . . . . . . . . . . . . . 36
2.6 Finiteness and Solvability Under Other Conditions. . . . . . . . . . . . 49
2.7 An Example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
3 Linear-Quadratic Optimal Controls in Infinite Horizons . . . . . . . . . 61
3.1 Formulation of the Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
3.2 Stability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
3.3 Stabilizability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
3.3.1 Definition and Characterization. . . . . . . . . . . . . . . . . . . . . 68
3.3.2 The Case of One-Dimensional State . . . . . . . . . . . . . . . . . 73
3.4 Solvability and the Algebraic Riccati Equation. . . . . . . . . . . . . . . 75
3.5 A Study of Problem (SLQ)01. . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
3.5.1 A Finite Horizon Approach . . . . . . . . . . . . . . . . . . . . . . . 80
3.5.2 Open-Loop and Closed-Loop Solvability. . . . . . . . . . . . . . 84
3.6 Nonhomogeneous Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
3.7 The One-Dimensional Case. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
ix
x Contents
Appendix: Linear Algebra and BSDEs.. .... .... .... .... ..... .... 105
References.... .... .... .... ..... .... .... .... .... .... ..... .... 115
Index .... .... .... .... .... ..... .... .... .... .... .... ..... .... 119
