Sequence-Based Anytime Control

14 Pages·2013·3.85 MB·English

IEEETRANSACTIONSONAUTOMATICCONTROL,VOL.58,NO.2,FEBRUARY2013 377
Sequence-Based Anytime Control
Daniel E. Quevedo, Member, IEEE, and Vijay Gupta, Member, IEEE
Abstract—Wepresent two related anytimealgorithms forcon- gorithms, if the processor has constant, but limited, computa-
trolofnonlinearsystemswhentheprocessingresourcesavailable tional resources.Henriksson etal.[16],[17] studiedtheeffect
are time-varying. The basic idea is to calculate tentative control ofnotupdatingthecontrolinputincontinuoustimesystemsfor
inputsequencesforasmanytimestepsintothefutureasallowed
the duration of the computational delay for optimization algo-
bytheavailableprocessingresourcesateverytimestep.Thisserves
rithmsbasedon activeset methods.Alsorelatedareworkson
tocompensateforthetimestepswhentheprocessorisnotavailable
toperformanycontrolcalculations.UsingastochasticLyapunov event-triggered and self-triggered control systems, and online
function-basedapproach,weanalyzethestabilityoftheresulting sampling,e.g.,[8],[43],[45],[46],whereacontrolinputiscal-
closed-loopsystemforthecaseswhentheprocessoravailabilitycan culated aperiodically, but on demand, depending on the plant
bemodeledasanindependentandidenticallydistributedsequence state.Inaddition,wewouldliketomentionworkonscheduling
andviaanunderlyingMarkovchain.Numericalsimulationsindi-
of control tasks [6],[7], [41] that looks at the problem of pro-
cate that the increase in performance due to the proposed algo-
cessorqueuescheduling,whencontrolcalculationismerelyone
rithmscanbesignificant.
ofthetasksinthequeue.
Index Terms— Anytime algorithms, embedded control, non-
Analternativeapproachtoachievesystemrobustnessinthe
linearcontrolsystems,stabilityanalysis,stochasticsystems.
presence of time-varying processing resources is to develop
anytime algorithms. The main purpose of anytime algorithms
I. INTRODUCTION istoprovide solutionevenwithlimitedprocessingresources,
andtorefinethesolutionasmoreresourcesbecomeavailable.
A lotofrecentattentionhasfocusedonnetworkedandem-
Anytimealgorithmsseektomakeefficientuseofresourcesand
beddedcontrol(see,e.g.,thespecialissue[1]andtheref-
are, thus, popular in the context of real-time systems. In con-
erences therein). One issue which plays an important role, es-
trol, however, there are few methods available for developing
peciallyinembeddedsystems,isthatoftime-varyingandlim-
anytime controllers. A notable work is that of Bhattacharya et
itedprocessingpower.Asmoreandmoreobjectsareequipped
al.[4]whofocusedonlinearsystems,andpresentedacontrol
withmicroprocessorsthatareresponsibleformultiplefunctions
algorithm that updated a different number of states depending
such as control, communication, data fusion, system mainte-
on the available computational time. However, the available
nanceandsoon,theimplicitassumptiontraditionallymadein
computationaltimewasrequiredtobeknowntothecontrollera
controldesignabouttheprocessorbeingabletoexecutethede-
priori.AnotherimportantworkisthatofGrecoetal.[11],who
siredcontrolalgorithmatanytimewillbreakdown.Similarly,if
proposed switching among a pre-designed set of controllers
aremotecontrollerisinchargeofmanydevices,multiplecon-
that may require different execution times. Although the idea
troltaskswillcompeteforsharedprocessorresources,leading
can be generalized to nonlinear processes, the analysis in the
to constrained availability of processing resources for the in-
paperreliedonMarkovianjumplinearsystemtheory.InGupta
dividual control loops. It is, thus, of interest to study control
and Luo [12], an anytime algorithm for systems with multiple
algorithms that can function despite limited and time-varying
inputswaspresented.The main idea was based on calculating
availabilityofprocessingpower.
thecomponentsofthecontrolvectorsequentially,andrefining
Thereisagrowingnumberofworksthatdealwiththisissue.
theprocessmodelasmoreprocessingtimebecomesavailable.
The impact of finite computational power has been looked at
Since the algorithm is based on identifying the modes of the
most closely for techniques such as model predictive control.
process that require more urgent control, it is, thus, again
McGovernandFeron[27],[28]presentedboundsoncomputa-
largelylimitedtolinearprocesses.
tional time for achieving stability for specific optimization al-
In the present work, we present two anytime control algo-
rithms for nonlinear plants described in state-space form that
are based on using extra processor availability to calculate se-
ManuscriptreceivedDecember19,2010;revisedOctober10,2011,May03, quenceswhichhavethepotentialtobeimplementedattheplant
2012,andJuly01,2012;acceptedJuly16,2012.DateofpublicationJuly24,
inputatfuturetimes.Thissafeguardsperformanceatthosetime
2012;dateofcurrentversionJanuary19,2013.TheworkofD.E.Quevedo
wassupportedbytheAustralianResearchCouncil’sDiscoveryProjectsfunding steps where the processor is entirely unavailable for control.
scheme(projectnumberDP0988601).TheworkofV.Guptawassupportedby Availability of processor time for control calculations deter-
NSFawards0846631and0834771.Apreliminaryversionofpartsofthiswork
mines the length of the sequences calculated and, thereby, af-
waspresentedatthe49thIEEEConferenceonDecisionandControl,see[15].
RecommendedbyAssociateEditorH.Ishii. fectsthequalityoftheresult.Adistinguishingfeatureoftheal-
D.E.QuevedoiswiththeSchoolofElectricalEngineeringandComputer gorithmspresentedisthatprocessoravailabilityisallowedtobe
Science,TheUniversityofNewcastle,Callaghan,Newcastle2308,Australia
random,with unknowndistribution.Moreover,ouralgorithms
(e-mail:[email protected]).
V. Gupta is with the Department of Electrical Engineering, University of areoneofthefirstthataresuitablefornonlinearplants.Forcases
NotreDame,NotreDame,IN46556USA(e-mail:[email protected]). whereprocessoravailabilityisgovernedbyasuitableMarkov
Colorversionsofoneormoreofthefiguresinthispaperareavailableonline Chain,weuseLyapunovfunctionstoestablishsufficientcondi-
athttp://ieeexplore.ieee.org.
tionsforstochasticstabilityoftheclosedloop.Numericalsim-
DigitalObjectIdentifier10.1109/TAC.2012.2209977
0018-9286/$31.00©2012IEEE
378 IEEETRANSACTIONSONAUTOMATICCONTROL,VOL.58,NO.2,FEBRUARY2013
ulationsillustratethatperformancegainsachievedwiththeal- Whenimplementingdiscrete-timecontrolsystemsitisgen-
gorithmsproposedcanbesignificant. erally assumed that the processing resources available to the
Itisworthemphasizingthatinthealgorithmspresented,the controller are such that the control law can always be evalu-
potential control values are calculated sequentially, reutilizing ated within a fixed (and small) time-delay, say .1
the already computed values for the next computation. This is However, in practical networked and embedded systems, the
computationallyattractive,especiallysincethelengthofthese- processingresources(e.g.,processorexecutiontimes)available
quencetobecalculatedistime-varyingandnotknownapriori. for controlcalculationsmayvary and,at times,beinsufficient
Thus,ourapproachdifferssignificantlyfromthemethodsused to generate a control input within the prescribed time-delay .
in packetized predictive control, e.g., in [10], [29], [33]–[37], Onepossibleremedyforthisissuewouldtoredesignthecontrol
and [44]. In the latter works calculation of control sequences systemforaworstcasebychoosinglargervaluesof ,andpos-
requiressolvingoptimizationproblemsoverafinitehorizonof
sibly, .Clearly,suchanapproachwill,ingeneral,leadtoun-
lengthdeterminedbythecontrolleritself.
necessaryconservativenessandassociatedpoorperformance.In
Theremainderofthismanuscriptisorganizedasfollows:In
thepresentworkweadoptananytimecontrolparadigmtoseek
Section II, we formulate the anytime control design problem
favorabletrade-offsbetweenprocessoravailabilityandcontrol
studied. Section III, presents the proposed algorithms. Sto-
performance.
chastic stability analysis is carried out in Sections IV–VII.
Beforeproceedingwenotethatadirectimplementationofthe
Numerical simulations are documented in Section VIII.
controlpolicyusedinAssumption1,whenprocessingresources
SectionIXdrawsthefinalconclusions.
are time varying, results in a baseline algorithm, which gives
Notation: We write for and for .
risetotheplantinput
represents the real numbers and . The
ifsufficientcomputationalresourcesto
identity matrix is denoted via , is the all-zeroes
evaluate areavailablebetween
matrix, ,and .Thenotation stands
and
for ,where .Weadopttheconventions
otherwise
and ,if andirrespective (4)
of .Thesuperscript referstotranspose.TheEuclidean where the symbol with denotes the plant input
normofavector isdenotedvia .Afunction whichisappliedduringtheinterval2 .
isofclass- ,ifitiscontinuous,zero Whilethebaselinealgorithm(4)isintuitiveandsimple,itisby
at zero, strictly increasing, and unbounded. The probability of
no means clear that it cannot be outperformed by more elab-
anevent isdenotedby andtheconditionalprobability
orated control formulations. In the following section, we will
of given by . The expected value of a random
present two related anytime control algorithms for the plant
variable given isdenotedby while refersto
model (1). The aim is to make more efficient use of the pro-
theunconditionalexpectation.
cessing resources available for control, when compared to the
baselinealgorithm(4).
II. PROBLEMFORMULATION
Weconsidernonlinear(andpossiblyunstable)plantssampled III. ANYTIME CONTROL THROUGH CALCULATION
periodically with sampling interval and described in OFCONTROLSEQUENCES
discrete-time via Throughout this work, we will assume that the controller
needs processor time to carry out mathematical computations,
(1)
such as evaluating functions. However, simple operations at
a bit level, such as writing data into buffers, shifting buffer
In(1), istheplantstate, istheplantinput,and
contents and setting values to zero, do not require processor
isanunmeasureddisturbance.Themodel(1)satisfies
time. Similarly, input–output operations, i.e., A/D and D/A
and the initial state, , is arbitrarily
conversion are triggered by external asynchronous loops with
distributed(withpossiblyunboundedsupport).
a real-time clock and do not require that the processor be
Throughout this work, we will assume that the unperturbed
availableforcontrol.Asinregulardiscrete-timecontrol,these
plant model
external loops ensure that state measurements are available
at the instants and that the controller outputs are
(2)
passedontotheplantactuatorsattimes ,where
is globally stabilizable via state feedback. More precisely, we isfixed;see,e.g.,[2].
makethefollowingassumption: A standing assumption is that if the processor were fully
Assumption 1: There exist functions , available for control, then calculating the desired plant input
, , and a constant , such
1Recallthatfixeddelayscanbeeasilyincorporatedintothemodel(1)by
thatforall aggregatingthepreviousplantinputtotheplantstate,seealso[32].Foreaseof
exposition,throughoutthiswork,wewillusethestandarddiscrete-timenotation
asin(1).
2Ifsufficientcomputationalresourcesarenotavailable,thenonecouldalter-
(3)
nativelyholdthepreviouscontrolvalueandset .Thesitu-
ationmirrorsthatencounteredwhenthecontrolinputisaffectedbydropouts;
see,e.g.,[38].
QUEVEDOANDGUPTA:SEQUENCE-BASEDANYTIMECONTROL 379
Fig.1. Anytimecontrolstructurewithinternalbufferstate .
foragivenplantstate wouldbepossiblewithinthe
pre-allocatedtime-frame .Issuesarisewhen,
at times, processor availability does not permit the desired
plant input to be calculated. To take care of the associated
performanceloss,inthepresentworkweproposetouseoneof
thetwoanytimecontrolalgorithmspresentedbelow.
A. AlgorithmDescriptions
Both algorithms are based on the following basic idea: At
time intervals when the controller is provided with more pro-
cessing resources than are needed to evaluate the current con-
trolinput,thealgorithmcalculatesasequenceoftentativefuture
plantinputs,say .Thesequenceisstoredinalocalbuffer
andmaybeusedwhen,atsomefuturetimesteps,theprocessor
availabilityprecludesanycontrolcalculations,seeFig.1.
Forfurtherreference,wedenotethebufferstatesvia ,
where
. (5)
.
.
Fig.2. Algorithm .
for a given value and where each ,
. We also introduce the shift matrix and the unit
andsuggestedplantinputsatfuturetimesteps.Atthetimesteps
vector via
whenmoreprocessortimeisavailable,alongersuggestedtra-
jectoryofcontrolinputsiscalculatedandstoredinthebuffer.3
Ifthebufferrunsoutoftentativeplantinputs(ascalculatedin
Step3),thentheactuatorvaluesaresettozero.WithAlgorithm
... ... ... ... ... it th,raoswssooanwaasytthheeprreomcaeisnsionrgceallecmuleantetssaincothnetrboulfifnepr,utsee line,
“ ”inStep3.
Algorithm is almost identical to the first algorithm, .
The only difference is that, in Step 3, the buffer contents are
neverresettozero,i.e.,theline“ ”iseliminated,see
Fig.3.Thus,ifAlgorithm isused,thenbufferelementsmay
.. (6) stemfromcalculationscarriedoutatdifferenttimeinstants.By
.
notdeletingtheentirebuffer,butonlyreplacingtheappropriate
entries,whenusing thebufferwillrunoutofdatalessoften
thanwhenusing .
Algorithm is presented in Fig. 2. It can be seen that the
Itisworthnotingthatneitheralgorithmrequirespriorknowl-
algorithmproposedamountstoadynamicstatefeedbackpolicy
edgeoffutureprocessoravailabilityforcontrol.Thisopensthe
withinternalstatevariable .Thelatterprovides
possibility to employ the algorithms in shared systems, where
(8) 3Notethat,byAssumption1,inStep3,onecouldsimplyset .
380 IEEETRANSACTIONSONAUTOMATICCONTROL,VOL.58,NO.2,FEBRUARY2013
Intermsofthenotationintroducedaboveandin(6),ifAlgo-
rithm isused,thenthebuffer obeystherecursion
if
(10)
if .
Ontheotherhand,ifAlgorithm isused,thenwehave
if
if
(11)
where
(12)
with
if
(13)
if .
Fig.3. Step3ofalgorithm .
Inadditiontostudyingthelengthofthetentativecontrolse-
quencesprovidedbythealgorithms,namely ,itiscon-
the controller task can be preempted by other computational
venient to investigate how many values which stem from the
tasks carried out by the processor,see also [5], [25], and [48].
tentativecontrolsequences , arecontainedin
As in other anytime algorithms, there exists a compromise
thebufferstate .Wewillrefertothisvalueastheeffective
between resultant closed loop performance and the processor
bufferlength(attime ),anddenoteitas
availability.Understandingthistradeoffformsthebulkofthis
work.
(14)
with .Itiseasytoseethat,ifAlgorithm isused,
then isgovernedby
B. BasicProperties
if
With the algorithms presented in Section III-A, extra pro- (15)
if ,
cessingtimeisusedtocalculateadditionalelementsoftheten-
tative plant input sequences, thus providing higher quality re- whereas,withAlgorithm ,wehave
sults,i.e.,sequences whichbettersafeguardagainstperfor-
mancelossatfuturetimeinstanceswhereprocessoravailability
(16)
maybeinsufficient.Tofurtherelucidatethesituation,wenote
thatinbothalgorithms,duringeachiterationofthewhile-loop
The following example illustrates the quantities introduced
in Step 3, the state value is used to calculate a tentative
above:
control, namely . In the sequel, we will denote by
Example1: Supposethat andthattheprocessoravail-
the total number of iterations ofthe while-loop which are car-
abilityforcontrolissuchthat , , ,
riedoutduringtheinterval andnotethat
.IfAlgorithm isused,thenthebufferstateattimes
.Thus,if ,then the entire se-
becomes
quenceoftentativecontrolsis
. (9)
.
.
If ,thentheprocessorwasnotavailableforcontrol,
and(witheitherofthealgorithms)theactuatorvaluesaretaken
as the first elements of the shifted state ,
see (6).
QUEVEDOANDGUPTA:SEQUENCE-BASEDANYTIMECONTROL 381
whichgives , , , ,andplant InSectionVII,wewillshowhowtoextendthisanalysisfor
inputs , , , . thecasewhentheprocessoravailabilitycanbedescribedbya
Ontheotherhand,ifAlgorithm isused,thenwehave Markovchain,andthushasmemory.
Assumption 3, stated below, bounds the rate of increase of
the Lyapunov function in (3), when the nominal system (2)
is run in open-loop. It also imposes a (mild) restriction on the
distributionoftheinitialplantstate.
Assumption3: Thereexists suchthat
(20)
Theinitialplantstatesatisfies
(21)
where isasin(3).
and , , , , , ItisworthemphasizingthatthefactthatAssumptions1and3
, , . Note that with areglobalandstatedintermsofacommonLyapunovfunction
Algorithm , and therefore the plant input at time limitstheclassofplantsandcontrolpoliciesconsideredinour
issettozero;withAlgorithm ,thevaluecalculatedat subsequent analysis. One case where (20) is satisfied is when
time isused. and are globally Lipschitz continuous, or more precisely,
whenthereexist suchthat
IV. STOCHASTIC STABILITY OF ANYTIME
CONTROLALGORITHMS
Since the processor availability for control calculations is
random, the plant input is random, and thus the system (1)
Inthiscase,andsince ,wehave
evolves stochastically. Various stability notions for stochastic
systemshavebeenstudiedintheliterature(e.g.,[20],[22]).In
thepresentwork,weareinterestedinthefollowingnotion:
Definition1(StochasticStability): Adynamicalsystemwith
statetrajectory issaidtobestochasticallystable,if
for and (20) will hold provided
(17) .
Example 2: Consider an open-loop unstable constrained
plantmodeloftheform(1)with
forsome .
Remark1: Itfollowsdirectlyfrom (17),thatstochasticsta-
bilityimpliesthatthereexists ,suchthat:
(18) with
if
In the particular case where , (17) reduces to
if
,and(18)to ;
if
seealso[9],[20].
see [21, Example 2.3] and [34]. The second component of the
A. Assumptions plant input is constrained via , . If we
Our subsequent stability analysis considers the unperturbed choose and policy
system (2), i.e., where , for all . For peda- ,thendirectcalculationsprovidethat
gogicalease,wealsobeginbypresentingtheanalysiswiththe
additionalassumptionthattheprocessoravailabilityforcontrol
isindependent andidenticallydistributed(i.i.d.).Thus,forthe
analysisinSectionsVandVI,wemakethefollowingassump-
tion:
Thus,Assumption1holdswith ,and
Assumption 2: The process introduced in
.Furthermore,byproceedingasin[21,p.73],itcanbeshown
SectionIII-Bisi.i.d.,withprobabilitydistribution
that(20)holdswith .Thus,providedthat(21)holds
(19) and ,Assumption3isalsosatisfied.
The following example illustrates that, at times, it may be
where andwith . convenient to first find a Lyapunov function which satisfies
382 IEEETRANSACTIONSONAUTOMATICCONTROL,VOL.58,NO.2,FEBRUARY2013
(20)andthenseekacontrolpolicywhichensuresthatAssump- Proof: First we note that, by (22), the process is
tion 1 holds. Markovian. Thus, stability can be examined by using a sto-
Example3: Considerascalarunconstrainedandunperturbed chasticLyapunovfunctionapproach;see,e.g.,[22].Thelawof
open-loop unstable nonlinear plant where totalexpectation,whenappliedto ,with as
, with . A stabilizing control policy which satisfies in (3), gives
Assumption1for isgivenby ,
with .However, isnotsuitableforuse
in Assumption 3, since as
.
(24)
Incontrast,ifwechoose as ,
forall ,then
where we have used (3), (20) and (23). Theorem 2 of [22,
Ch. 8.4.2] implies that there exists such that
. Thus, by using
(3) and taking expectation with respect to the distribution of
,weobtain
and (20) holds with open-loop rate of growth bound con-
stant . The associated control policy
,where ,gives
where the last inequality follows from (21). Since ,
Weconcludethatif andtheinitialplantstateissuit- stochasticstabilityfollows.
ablydistributed,thenAssumptions1and3willhold. For the proposed anytime algorithms, stability analysis is
more subtle than for the baseline algorithm. The main reason
B. StochasticStabilityWiththeBaselineAlgorithm isthat,duetobuffering,theplantstate will,ingeneral,
not be Markovian and simple conditioning as in (24) is not
We will next present sufficient conditions under which
possible.4
the baseline algorithm (4) achieves stochastic stability of the
closed-loop system. As in (19), we denote via the proba-
bilitythatthecontrollerisunabletocalculateanycontrolinput. V. STABILITYWITHALGORITHM
Thus, if the baseline algorithm (4) is used and Assumption 2
To derive sufficient conditions for stochastic stability when
holds, then (in the disturbance-free case) the closed loop is
Algorithm is used, we will employ a technique which is
characterized by
roughlybasedontheapproachesusedin[23],[34]–[36],[47].
As will become apparent, randomness of the sequence length
process ,see(9),makestheanalysisoftheanytimeal-
if
(22) gorithms studied significantly more involved than the analysis
if .
ofthepredictivenetworkedcontrolformulationsof[34]–[36].
Itcanbeseenthattheplantstatetrajectoryissimilartothatof
A. PlantModelatTimes
anetworkedcontrolsysteminwhichthecontrollerisunableto
communicatewiththeactuatorwithprobability atanytime Foreaseofexposition,inthesequelweassumethat
step. Stability conditions for such systems have been derived and denotethetimestepsat whichatleast onecontrolinput
bothforlinearsystems[13],[18]andnonlinearsystems[34].In iscalculatedvia ,where and
particular,forascalarlinearplantmodelwithascalarinput
(25)
Itisconvenienttointroducetheiteratedmappingswithinput
and quadratic Lyapunov function, , the condition , :5
has been shown to be necessary and sufficient for
stabilizabilityin[13].Thus,theconstant needstosatisfy
forstabilitywiththebaselinealgorithm.Moregener- if
ally, we have the following sufficient condition for stochastic if
stabilitywhenthebaselinealgorithmisused: (26)
Theorem1: SupposethatAssumptions1to3hold.If
4NotethatsomeoftheresultsincludedinSectionIVof[15]areincorrect.
(23) 5For example, we have and
.Notethat,byStep3in
thealgorithmdescription,thevalues ,
then(22)isstochasticallystable. aredeterminedby .
QUEVEDOANDGUPTA:SEQUENCE-BASEDANYTIMECONTROL 383
and the related mappings which describe the nominal plant are i.i.d. Thus, we can condition upon to
modelwhentheinputissettozero: obtain
if
(27)
if .
We also denote the time betweentwo consecutiveelementsof
via
(28)
and notethat,byAssumption2,theprocess isi.i.d.
withgeometricdistribution
B. MainResults
(29) As a consequence of (31) and (32), and since is de-
terminedby ,ifAlgorithm isused,thentheplantstate
see [47]. In terms of the quantities introduced above, the state ,with ,isMarkovian.Stabilityoftheclosedloop
of the nominal plant (2) when Algorithm is used satisfies can be analyzed by using a stochastic Lyapunov function ap-
(30),shownatthebottomofthepage,forall .Itisworth proachwhich,tosomeextent,parallelsthatusedtoproveThe-
emphasizingthat(30)describestheplantstatetrajectory orem1.Tostateourresult,wefirstgivethefollowinglemma:
for all . Lemma2: Consider(31)andsupposethatAssumptions2and
By setting in (30), we obtain that the state in (2) 3hold.Then and ,wehave
whenAlgorithm isemployedcanbedescribedattheinstants
via
(33)
where
if
if
(34)
(31) Proof: Weusethetotalprobabilityformulatwice.First,we
conditiononthelengthofthetentativecontrolsequencecalcu-
where . latedduring
Itisworthnotingthatin(31),thenumberofpossiblevalues
for given is countably infinite, whereas if the
baselinealgorithmisused,thereareonlytwopossibilities,see
(22). The terms can be easily evaluated as
perthefollowinglemma:
Lemma1: SupposethatAssumption2holds,then
(35)
(32) We note that in Algorithm previously calculated
Proof: By (28), the random variables and are control values are erased at the instant and, thus,
independent. Furthermore, the two processes and (31) holds. Consequently, the conditional expectation
if
(30)
if and
384 IEEETRANSACTIONSONAUTOMATICCONTROL,VOL.58,NO.2,FEBRUARY2013
can be evaluated by condi-
(40)
tioningfurtheron
Using(40)and(37),weobtain
(36)
Hence, (38) is equivalent to (39). As a consequence, if (38)
wherewehaveused(29).Now,usingAssumption3and(7),we holds, then [22, Ch. 8.4.2, Th. 2] implies exponential stability
obtain thebound attheinstants ,i.e.,wehave
if
if . (41)
Now for the time instants , i.e., at those time steps
Thus, (36) gives whennocontrolinputiscalculated,weproceedasintheproof
ofLemma2,toobtainthat
since, by Assumption 3, we have . Substitution into
(35)establishes(33).
DespitethefactthatLemma2considersonlythetimeinstants
where ,see(25),theboundin(33)canbeused
toconcludeaboutstochasticstability(forall ).
Theorem2: SupposethatAssumptions1–3holdanddefine
Since ,wecanbound
(37)
If
(38)
thenthesystem(30)(withstatetrajectory )isstochasti-
cally stable.
Proof: ByLemma2andsince isMarkovian,wehave
so that
that if
(39)
where aredefinedin(34),then
in turn yielding
Since,byAssumption1, ,weconcludethat
isastochasticLyapunovfunctionfor(31);cf.[22],[26].
Directcalculationsyieldthat
(42)
QUEVEDOANDGUPTA:SEQUENCE-BASEDANYTIMECONTROL 385
where . The expectation on the left- Theorem1.Thisobservationfollowsuponnotingthat canbe
hand side of (42) is taken with respect to the distributions of rewritten as
and .Since isMarkovianand and are
independent, we can take conditional expectation
onbothsidesof(42)toobtain
Thus, if for some , then
and . This suggests
that Algorithm has better stabilizing properties than the
baselinealgorithm.
VI. STABILITYWITHALGORITHM
Wefirstnotethatfor ,Algorithm isequivalent
toAlgorithm .Henceforth,wefocusoncaseswhere .
It follows directly from (11) and (16) that with Algorithm
if and , then
and the plant input at times
willstem
frombuffercontentsattime ,seealsoExample1.Thus,
with Algorithm , and are not Markovian and
the analysis carried out for Algorithm does not carry over
directly.
TorecoveraMarkovianstructure,considertheoverallsystem
where we have used the bound in (41). Since we assume that
state definedvia
,thisgives
(43)
Intermsof ,(11)givesthatatalltimeswhere ,
theplantinputisgivenby
(44)
Furthermore, andtherebyalso areMarkovianpro-
cesses.Themapping6
if
Thus,byletting ,itfollowsthatthereexists
(45)
such that if
where
The remainder of the proof now follows as in the proof of
Theorem1. allowsonetocharacterizethenominalsystembehaviorattimes
Theorem2establishessufficientconditionsforstochasticsta- wherecomputationalresourcesareinsufficienttocalculatecon-
bilityoftheclosedloopwhenprocessoravailabilityisi.i.d.and trol inputs, so that buffered plant inputs are used. More pre-
cisely,thenominalplantstatewhenAlgorithm isusedcan
Algorithm isused.Thequantityintroducedin(37)involves
be stated in terms of a random mapping with inputs as
thedistributionof ,thecontractionfactorofthebaseline
follows:
controller ,see(3),andtheboundontherateofincreaseofthe
plantstatewhenleftinopenloop,see(20).
Asaparticularcase,supposethatthedistributionof
if
satisfies , i.e., the processor time availability is
if and
suchthattheAlgorithm providesatmostonecontrolinput.
and
Inthiscase,expression(37)givesthat and,notsurpris-
if
ingly,werecoverthesufficientconditionforstochasticstability
and
establishedforthebaselinealgorithm(4)in(23).
(46)
Moregenerally,iftheprobabilitythatAlgorithm provides
morethanonecontrolvalueisnonzero,thenTheorem2estab- 6Forexample,for wehave ,see
lishesstochasticstabilityforalargerclassofplantmodelsthan also(5).
386 IEEETRANSACTIONSONAUTOMATICCONTROL,VOL.58,NO.2,FEBRUARY2013
where and are defined in (26) and (27), respectively, Ontheotherhand,since isafunctionof and
,andwith ,we have
(47)
if
if
At the instants , the nominal plant state in (2) when
Algorithm isusedcanthusbedescribedvia
where we have used the bounds in (7), (20) and where
with denotingtheindexofthelastnonzero
entryin ,see(16).Substitutioninto(50)yieldsthat
if
if
if
(48)
where
(49) where isdefinedin(39)andwherewehaveusedthefactthat
and .
Since isMarkovian,itfollowsfrom[22,Ch.8.4.2,Th.
Note that, as shown in Lemma 1, the probabilities
2] that
usedin(48)arei.i.d.Nevertheless,itiseasytoseethat
Theremainderoftheproofnowfollows,mutatismutandis,that
ofTheorem2.
VII. MARKOVIANPROCESSORSTATEMODEL
Sofarwehaveassumedthattheprocess isi.i.d.Insit-
uationswherethecontrolloopissharedwithotherapplications
having time-varying and correlated processing demands, it is
likelythatAssumption2willnotbesatisfied.Wewillnextout-
linehowtheanalysispresentedcanbeextendedtoencompass
cases where the processor availability for control, henceforth
expression which depends upon and therefore on
modeledviatheprocessorstateprocess ,iscorrelated.
.
Assumption 4: The processor state process is an ir-
Thefollowingstochasticstabilityresultisakintotheonede-
reducible aperiodic finite Markov Chain (see, e.g., [24]) with
velopedinSectionV-BforAlgorithm .Itshowsthatthesuf-
valuesinthefiniteset , .Itstransitionma-
ficientconditiondevelopedforAlgorithm isalsosufficient
trix isgiven by
toguaranteestochasticstabilitywhenAlgorithm isused.
Theorem 3: Suppose that Assumptions 1 to 3 hold and that
Algorithm isused.If(38) issatisfied,thentheclosed-loop
system(withstatetrajectory )isstochasticallystable. ... ... ... ... (51)
Proof: Itfollowsfrom(48),(45)andbyproceedingasin
theproofofLemma2that
where ,
. Given any processor state ,
, the conditional
distributionoftheprocess satisfies
(52)
(50) withgivenprobabilities .