Distributed Subgradient-based Multi-agent Optimization with More General Step Sizes

0.13 MB·

by  Peng Wang

1
Distributed Subgradient-based Multi-agent Optimization with
More General Step Sizes
Peng Wang, and Wei Ren
Abstract—Awiderselectionofstepsizesisexploredforthedistributed shown to converge faster than the distributed subgradient algorithm
subgradient algorithm for multi-agent optimization problems, for both in [5]. A zero-gradient-sum algorithm is developed in [13], in
time-invariant and time-varying communication topologies. The square
6 which each agent starts from its local minimizer and the sum of
summable requirement of the step sizes commonly adopted in the
1 literature is removed. The step sizes are only required to be positive, the gradients is kept at zero. On the other hand, some dual or
0 vanishingandnon-summable.Itisprovedthatinbothunconstrainedand primal-dual subgradient algorithms are developed for distributed
2 constrainedoptimizationproblems,theagents’estimatesreachconsensus optimization problems with equality and inequality constraints. Ref.
n andconvergetotheoptimalsolutionwiththemoregeneralchoiceofstep [10]proposesadistributedprimal-dualsubgradientalgorithmtodeal
sizes.Theideaistoshowthataweightedaverageoftheagents’estimates
a approaches the optimal solution, but with different approaches. In the with identical affine equality and convex inequality constraints. A
J unconstrained case, the optimal convergence of the weighted average projected subgradient method isdesigned tofind thesaddle point of

0 of the agents’ estimates is proved by analyzing the distance change theLagrangianoftheprimalproblem.Thenin[11],asimilarideais
3 from the weighted average to the optimal solution and showing that adoptedtodevelopadistributeddualsubgradientalgorithmtosolvea
the weighted average is arbitrarily close to the optimal solution. In the
non-convex problem approximately, with the consensus requirement
constrained case,thisisachieved byanalyzingthedistancechangefrom
] relaxed.
C theagents’estimatestotheoptimalsolutionandutilizingtheboundedness
oftheconstraints.Thentheoptimalconvergence oftheagents’estimates Intheabovepapers onsubgradient-related distributedsolutionsto
O followsbecauseconsensusisreachedinbothcases.Theseresultsarevalid theoptimizationproblem[5],[6],[8],[10],[11],thestepsizesforthe
. for both a strongly connected time-invariant graph and time-varying subgradient should be positive, vanishing, non-summable but square
h balanced graphs thatare jointly strongly connected.
summable. Intuitively,thepositiveness makesthealgorithmtravelin
t
a thedescentdirection,andthenon-summablitymakesthesubgradient
m
I. INTRODUCTION a persistent factor in the optimization process finally leading to the
[ optimal solution. But there seems to be no obvious meaning for the
With the emergence of large scale networks and complex large
square summability of the step size.
1 systems, distributed optimization arises in many areas such as dis-
In this paper, we will show that the square summability is not
v tributed model predictive control [1], distributed signal processing
necessaryforthedistributedsubgradientmethod.Wewillprovethata
8 [2], optimal network flow [3] and network utility maximization [4]
positive,vanishingandnon-summablestepsizecanmaketheagents’
4 and has attracted significant attention. The distributed optimization
0 estimatesconverge to theoptimal solution inboth the unconstrained
problems can be roughly classified into two categories. In the first
0 and constrained distributed optimization problems. This step size
category, each agent has a local objective function and sometimes a
0 selection is actually the same as that required by the centralized
local constraint, both unknown to others, but different agents share
. subgradient method [14]. Our results are valid for both the time-
2 the same optimization variable. This means that different agents’
varyingbalancedandtime-invariantunbalancednetworks.Itisworth
0 estimates of the optimizer should be the same at last [5]–[11]. The
6 mentioningthat[15]solvesthedistributedoptimizationproblemwith
problems in this category can be regarded as a distributed potential
1 a continuous-time algorithm, where a feedback term instead of the
problem. In the second category, every agent has a local objective
: projection operator is used to drive the agents’ estimates to the
v functionunknowntoothers,theconstraintsoftheagentsarecoupled,
constraint set. In [15], the step size is required to be positive and
i and every agent knows only a part of the coupled constraints [1],
X vanishing and to have infinite integral. While the results in [15]
[3], [4], [12]. The problems in this category can be regarded as a
are interesting and relax the step size requirement, our results are
r
distributednetworkflowproblem. Inthispaper wewillfocusonthe
a differentfromandcomplementthosein[15]inthefollowingaspects:
problems in the first category.
first, the results in this paper are valid for the problem with non-
Various algorithms have been developed to solve the problems
identical constraints, while those in [15] only deal with that with
in the first category. In [5], a distributed subgradient algorithm
identical constraints; second, bothtime-varying balanced graphs and
is designed for an unconstrained distributed optimization problem,
fixed unbalanced graphs are considered in this paper, while only a
with the assumption of uniformly bounded subgradients, and a non-
fixed undirected graph is taken into account in [15]; third, the local
degenerate, time-varying and balanced communication topology. In
objective functions are only required to be convex in this paper,
[6],adistributedoptimizationproblemwithidenticallocalconstraints
while they are required to be strictly convex and differentiable in
ornon-identicallocalconstraintsinthecontextofacompletegraphis
[15]; fourth, the algorithms are different (discrete-time algorithm in
consideredthroughaprojecteddistributedsubgradientalgorithm.Ref.
this paper versus continuous-time algorithm in [15]) and so are the
[7] considers non-identical local constraints for balanced and state-
analysisapproaches.Thediscrete-timealgorithmisprojectionbased,
dependent switchinggraphs. Then[8] provestheconvergence of the
ensuring that the agents stay in their constraint sets at each time
distributedsubgradient algorithmwithnon-identicallocalconstraints
instant while the continuous-time algorithm only ensures that the
under time-varying balanced and fixed unbalanced graphs. Some
agents approach their constraint sets eventually.
accelerated algorithms are proposed in [9], in which two distributed
Inthispaper,weshowthatwiththemoregeneralselectionofstep
Nesterov gradient methods are designed and these algorithms are
sizes, the agents’ estimates can still reach a consensus and arrive
at an optimal solution using the distributed subgradient method. For
Peng Wang and Wei Ren are with the Department of Electrical and
Computer Engineering, University of California, Riverside, Riverside, CA, the unconstrained optimization problem, we first show the optimal
92521.Emails:[email protected] [email protected] convergenceofasub-sequenceofaweightedaverageoftheestimates
2
of different agents by investigating the distance change from the An optimization problem
weightedaveragetotheoptimalset.Thenweshowthatastimegoes
minimize f(x) subject to x∈X
by, the weighted average stays in the neighborhood, vanishing with
step size, of arbitrary sublevel sets of the global objective function. is a convex optimization problem if the objective function f(x) is
Next, with consensus, we prove that the estimates of all agents convex and the constraint set X is also convex.
approach the optimal solution. For the constrained optimization A vector g is a subgradient of a function f at the point x if for
0
problem, we first prove the optimal convergence of a sub-sequence all x in the domain of f,
of the weighted average of the agents’ estimates by studying the
distancechangefromtheestimatesofdifferentagentstotheoptimal f(x)−f(x0)≥gT(x−x0). (1)
solution. Then the convergence of the corresponding sub-sequence The set of subgradients of f at x is called subdifferential, denoted
0
of the agents’ estimates follows from consensus. Next, we show by¶ f(x0).Theconcept ofsubgradients(orsubdifferential)isagen-
the convergence to the optimal solution of the estimates of different eralizationofthatofgradients.Whenthefunction f isdifferentiable
agentswiththeboundednessoftheconstraints.Theaboveresultshold at x , the gradient of f at x is the subgradient.
0 0
forbothastronglyconnected fixedgraphandtime-varyingbalanced For a projection operator onto a closed convex set, we have the
graphs that are jointly strongly connected. following non-expansiveness property.
a) Notations: WeuseRforthesetofrealnumbers, Rn forthe
set of n×1 real vectors and Rn×n for the set of n×n real matrices. Lemma 1. [6]LetX⊂Rm beaclosedconvex set.Forany pairof
The symbol N+ represents the set of positive integers, i.e., N+= points x and y in Rm, we have kPX(x)−PX(y)k≤kx−yk.
{1,2,3,···},andthesymbolNrepresentsthesetofnaturalnumbers,
i.e. N={0} N+. A sequence of real numbers or vectors x(k), k= III. PROBLEMSTATEMENT
S
1,2,···,isrepresentedby{x(k)}.Thedistancebetweenapointxand For a multi-agent system with n agents, we regard each agent
some set X is d(x,X)= infkx−pk, and the distance between two as a vertex. There is an edge (j,i) if agent i receives information
sets X andY is defined aps∈dX(X,Y)= inf kx−yk. The transpose fromagent j.Thecorresponding entryaij intheadjacencymatrixA
x∈X,y∈Y denotes the weight assigned by agent i to the received information
of a vector a is represented by aT. We let 1n be the n×1 vector of from agent j.
all ones. We use PX(x) to denote the projection of a point x onto a Wewillfocusonthefirstkindofdistributedoptimizationproblems
closed convex set X: PX(x)=argminkx−pk. The convex hull of a described in Section I. Each agent has a private local objective
p∈X
set X is denoted by conv(X). function unknown to the other agents, and shares the same variable
withtheotheragents.Alsoithasitsprivatelocalconstraint.Thegoal
ofthemulti-agentsystemistocooperativelyfigureouttheminimizer
II. PRELIMINARIES
of the weighted sum of all local objective functions in the common
In this section, we introduce some preliminary results on graph part of all local constraints:
theory and convex optimization.
n n
(cid:229)
minimize f(x)= qifi(x) subject to x∈X=\Xi, (2)
A. Graph Theory i=1 i=1
Annthorderdirectedgraph,denotedbyG(V,E,A),iscomposedof where x is the variable of the multi-agent system, fi, i∈V, are the
avertexsetV ={1,···,n},anedgesetE⊆V×V andanadjacency localobjectivefunctions,Xi⊆Rm,i∈V,arethelocalconstraints,and
matrix A. We use the pair (j,i) to denote the edge from vertex j the positive weights qi, i∈V, are to be specified later. The problem
(2) is equivalent to the following problem
to vertexi. Wesuppose that (i,i)∈E,∀i∈V. The adjacency matrix
A=(aij)n×n∈Rn×n associatedwiththegraphG isdefinedsuchthat (cid:229)n
aij ispositive if (j,i)∈E, and aij=0otherwise. Weassume that A minimize qifi(xi)subject to xi∈Xi, ∀i∈V,xi=xj,∀i,j∈V,
n i=1
is row stochastic, i.e., j(cid:229)=1aij=1, ∀i∈V. The graph G is balanced where xi ∈ Rm is the variable of agent i. For an unconstrained
if (cid:229)n aij = (cid:229)n aji, ∀i∈V. The neighbor set of vertex i is defined opfroobpletmim,iwzaetiolentXpiro=blRemm,,ib∈ecVa.usCeonthseenvsuasriaisblneesceosfsadriyffeforerntthiasgkeinntds
j=1 j=1
as Ni={j:(j,i)∈E}. A directed path from i to j is a sequence of shouldbethesameanddifferentagentsshouldfigureoutacommon
edges (i,i1),(i1,i2),···,(ip,j), starting from vertex i and sinking at solution of the problem (2).
vertex j. The directed graph G is strongly connected if for any pair For the multi-agent network, we have some assumptions on its
of vertices i and j, there is a directed path from i to j. Intuitively connectivity and the weights in the adjacency matrices.
speaking, every vertexina strongly connected graph can havesome
Assumption 1. There exists an infinite sequence k0,k1,···,kp,···
influenceon thewholenetwork. Theunion of acollectionof graphs kp+1−1
isagraphwiththevertexandedgesetsbeingtheunionsofthevertex with 0<kp+1−kp≤B, B∈N+, such that the union S G(k) is
and edge sets of the graphs in the collection. k=kp
strongly connected, for all p∈N.
The essence behind Assumption 1 is that the emerging edges
B. Convex Optimization
should formastrongly connected graph andtheseedges should also
A set C is convex if ∀x, y∈C, a x+(1−a )y∈C, ∀a ∈[0,1]. appear sufficiently often to guarantee consensus and convergence to
Thatis,thelinesegmentisinthesetC ifthetwoendpointsare.The
the optimizer.
convex hull of a set D, denoted by conv(D) is the smallest convex
set that contains D, i.e., a) conv(D) isconvex, b) D⊂conv(D), and Assumption 2. The adjacency matrices A(k), k=1,2,···, share a
c)forarbitraryconvexsetCthatcontainsD,conv(D)⊂C.Afunction common positive left eigenvector associated with eigenvalue 1. That
f isconvexifitsdomainisconvexandforallxandyinitsdomain, is, there exists a constant stochastic vector q=(q1,···,qn)T with
f(a x+(1−a )y)≤a f(x)+(1−a )f(y), ∀a ∈[0,1]. qi>0, i∈V, and 1Tnq=1, such that for all k, qTA(k)=qT.
3
Remark 1. Under Assumption 1, when G(k) is fixed and hence Remark 2. Assumption 8 allows a wider selection of the step
stronglyconnected,qisthepositivelefteigenvectoroftheadjacency sizes for the distributed subgradient algorithm (3), by removing the
¥
matrix A associated with eigenvalue 1 satisfying 1Tnq=1. When the requirementof (cid:229) a (k)2<¥ commonlyadoptedintheliterature[5],
time-varying graph G(k) isbalanced, q= n11n. [6], [8], [10],k[=111]. Also, Assumption 8 is the same as that for the
Assumption 3. The graph is non-degenerate. That is, there exists centralized subgradient method [14], which might imply that this is
h >0, such that for all k∈N, if aij(k)>0, then aij(k)>h , and among the widest range of step sizes for the distributed subgradient
aij(k)=0 otherwise. algorithm.
This assumption shows that if agent i receives information from
agent j, then the edge weight aij is uniformly bounded away from IV. MAINRESULTS
zero.Thisassumptionensuresthattheinfluenceofanindividualagent
Inthissection,weprovethatallagents’estimatesoftheminimizer
on the network, if there is any, is persistent and does not vanish as
of the convex optimization problem (2) generated by the distributed
time goes by.
subgradient algorithm (3) converge to the optimal solution of (2),
For the optimization problem (2), we have the following assump- ¥
withoutrequiring (cid:229) a (k)2<¥ .Evenwithoutthesquaresummable
tions:
k=1
assumption, the existing results can still ensure that the agents’
Assumption 4. The problem (2) has a bounded nonempty set of
estimatesreachaconsensusinboththeunconstrainedandconstrained
optimal points, denoted by X⋆.
cases, as summarized in the next lemma.
Assumption5. Eachlocalobjectivefunction fi,i∈V,isconvexand Lemma 2. [6], [8] For a graph sequence G(k), k=0,1,2,···,
continuous in its local constraint set X.
i satisfying Assumptions 1, 2 and 3 and the optimization problem (2)
From [14] we know that a convex function is continuous in the satisfyingAssumptions5,7,8witheitherXi=Rm,i∈V,orAssump-
interior of its domain, but Assumption 5 only requires fi to be tion 6, the agent’s estimates xi, i∈V, in the distributed subgradient
continuous in its local constraint set Xi. algorithm(3)reachaconsensus,i.e., limkxi(k)−xj(k)k=0,∀i,j∈
k→¥
V.
Assumption6. EachlocalconstraintsetXi,i∈V,isbounded,closed
and convex if Xi6=Rm. However,itisnotclearwhethertheagents’estimateswillconverge
totheoptimalsolution.Next,wewillprovetheconvergenceof(3)to
Asthesumofconvexfunctionsisalsoconvex,theglobalobjective
theoptimalsolutionof(2)inboththeunconstrainedandconstrained
function f is convex from Assumption 5. With Assumption 6, the
n cases.
constraintsetXi isconvexandsoistheintersectionX= TXi.Then
i=1
the problem (2) is a convex optimization problem.
A. Unconstrained Case
One of the distributed ways to solve the convex optimization
problem (2) is to use the distributed subgradient method [5]–[8] In this section, we will prove that the global weighted average of
n the agents’ estimates converges to the global optimal set with the
xi(k+1)=PXi((cid:229) aij(k)xj(k)−a (k)gi(k)), (3) step size in Assumption 8 by analyzing the distance change from
j=1 the weighted average to the optimal solution. Then as consensus is
where xi(k) is agent i’s estimate of the minimizer of the global shown in Lemma 2, all agents reach a common minimizer for the
objective function f at the kth iteration, aij(k) is the (i,j)th entry problem (2). The rigorous statement is as follows:
of the adjacency matrix A(k) at the kth iteration, a (k) is the step Theorem 1. For a graph sequence G(k), k=0,1,2,···, satisfying
size, gi(k) is the subgradient of the local objective function fi at Assumptions 1, 2 and 3 and the optimization problem (2) satisfying
n
(cid:229) aij(k)xj(k), and PXi is the projection operator onto Xi. Assumptions4,5,7,and8withXi=Rm,i∈V,theagents’estimates
j=1 n xi, i∈V, in the distributed subgradient algorithm (3) converge to a
In (3), the row stochastic property, i.e., (cid:229) aij =1, makes each common point in the optimal set X⋆ of (2).
j=1
agent reach a consensus and converge to a point minimizing a Proof: Letx⋆ besomepointintheoptimalsetX⋆.Alsolety(k)=
weighted sum of the locanl objective functions [8], while the column (cid:229)n qixi(k) be the global weighted average of the estimates of all
stochastic property, i.e, (cid:229) aji=1, makes all agents converge tothe ai=g1ents and
j=1
optimizer of the sum of the local objective functions [6], [8].
n
(cid:229)
Assumption7. Thesubgradientsof fi,i∈V,areuniformlybounded, vi(k)= aij(k)xj(k) (4)
i.e., there exists G>0 such that for all g∈¶ fi(x), kgk≤G, ∀x∈ j=1
Xi, ∀i∈V. bethelocalweightedaverageoftheesntimatesofagenti’sneighbors.
The assumption of uniformly bounded subgradients can be found Then note that qTA(k) =qT, i.e. (cid:229) qiaij(k) = qj, ∀j ∈V with
i=1
in many references [5], [6], [8]–[11], and plays an important role in Assumption 2, we have
theconsensusandconvergenceofthedistributedsubgradientmethod.
n n n
But with Assumption 6 when the local constraint sets Xi, ∀i∈V, y(k+1)=(cid:229) qixi(k+1)=(cid:229) qi((cid:229) aij(k)xj(k)−a (k)gi(k))
are compact, i.e., closed and bounded, Assumption 7 is redundant i=1 i=1 j=1
btheecacuosmeptahcetnbeosusnodfedthneescsonosftrtahientsusebtgsr.adients can be deduced from = (cid:229)n ((cid:229)n qiaij(k))xj(k)−a (k)(cid:229)n qigi(k)
j=1 i=1 i=1
sAusmsummabpltei,oni.e8.,.aT(hke)>ste0p, sliimzeaa((kk))=is0paonsditiv(cid:229)¥e,ava(nki)s=hin¥g.and non- = (cid:229)n qjxj(k)−a (k)(cid:229)n qigi(k)=y(k)−a (k)(cid:229)n qigi(k).
k→¥ k=0 j=1 i=1 i=1
4
Then the distance between the global weighted average y(k) DenoteK0=max{Ke ,Kc,Ka }.Wehaveky(K0+m)−x⋆k2≤ky(K0+
and the point x⋆ in the optimnal set evolves as follows 1)−x⋆k2−e K0+(cid:229)m−1a (t). As (cid:229)¥ a (k)=¥ , we have ky(K0+m)−
ky(k + 1) − x⋆k2 = ky(k) − a (k) (cid:229) qjgj(k) − x⋆k2 = ky(k) − t=K0+1 k=1
x⋆k2+a (k)2k (cid:229)n qjgj(k)k2−2a (k)j=(cid:229)n1qjgj(k)(y(k)−x⋆). Accord- x⋆k2≤ky(K0+1)−x⋆k2−e Kt=0+K(cid:229)m0+−11a (t)<0 when m is sufficiently
j=1 j=1 large. This contradicts with the fact that ky(K0+m)−x⋆k2≥0. It
ing to Assumption 7, the subgradient gj(k) ≤ G. Note that follows that
gj(k)(y(k)−vj(k))≥−kgj(k)kky(k)−vj(k)k≥−Gky(k)−vj(k)k
oanfdsufbjg(vrajd(kie)n)t−s ifnj((y1()k,))w≥e ghTajv(ey(k))(vj(k)−y(k))fromthedefinition likm→i¥nf(cid:229)n (fj(y(k))−fj(x⋆))≤0. (7)
j=1
n
(cid:229) qjgj(k)(y(k)−x⋆)
j=1 Next we show the optimal convergence of the agents’ estimates.
=(cid:229)n qjgj(k)(y(k)−vj(k))+(cid:229)n qjgj(k)(vj(k)−x⋆) Notethat j(cid:229)=n1qj(fj(y(k))−fj(x⋆))≥0becausex⋆isintheoptimalset
j=1 j=1 n
n n X⋆.Combingwith(7),wehavethatliminf (cid:229) (fj(y(k))−fj(x⋆))=0.
≥−(cid:229) Gqjky(k)−vj(k)k+(cid:229) qj(fj(vj(k))−fj(x⋆)) k→¥ j=1
j=1 j=1 Then there exists a sub-sequence {y(kp)} of {y(k)}, such that
n
(cid:229)n (cid:229)n lim y(kp)=x⋆ and lim f(y(kp))= f(x⋆), where f = (cid:229) qifi as in
=− Gqjky(k)−vj(k)k+ qj(fj(vj(k))−fj(y(k))) kp→¥ kp→¥ i=1
j=1 j=1 (2).Itfollowsthatforalld >0,thereexistsKd ∈N+,suchthatforall
+(cid:229)n qj(fj(y(k))−fj(x⋆)) aksp>theKdl,evfe(ly(ckupr)v)e−off(xt⋆h)e≤gdlo.bDalefionbejeUctdiv=e {fuyn:ctfi(oyn).−Lfe(tx⋆d)(d=)d=}
j=1
maxminky−pk be the maximum distance from the level curve
(cid:229)n (cid:229)n y∈Ud p∈X⋆
≥− Gqjky(k)−vj(k)k+ qjgj(y(k))(vj(k)−y(k)) Ud to the optimal set X⋆. From a (k) →0 and (5), there exists
+j(cid:229)=n1qj(fj(y(k))−fj(x⋆))j=1 Kfoa′r ∈alNl +k >andKc′K,c′ ∈(cid:229)n Nq+jk,ys(uk)ch−tvhaj(tk)fokr≤all8dGk.>IfKa′f,(ya((kk)))≤≤2fGd(2x⋆a)n+d
j=1 j=1
n n d , then minky(k)−pk ≤ d(d ). We have minky(k+1)−pk ≤
≥−2G(cid:229) qjky(k)−vj(k)k+(cid:229) qj(fj(y(k))−fj(x⋆)). p∈X⋆ n p∈X⋆
j=1 j=1 d(d )+a (k)k (cid:229) qjgj(k)k≤d(d )+a (k)G. On the other hand, if
j=1
Combining with the fact that k (cid:229)n qjgj(k)k2 ≤ (cid:229)n qjkgj(k)k2 ≤ f(y(k))> f(x⋆)+d ,itfollowsfrom(6)thatwhenk>max{Ka′ ,Kc′},
j=1 j=1 ky(k+1)−x⋆k2 ≤ ky(k)−x⋆k2−a (k)d ≤ ky(k)−x⋆k2. Taking
4Ga2,(k)wj(cid:229)=ne1Ghqavjkey(kk)y−(kv+j(k1))k−−2xa⋆k(k2)≤j=(cid:229)n1kqyj((kf)j(−vj(xk⋆)k)2−+fja(x(⋆k)))2.G2 + ipmn∈tiXon⋆kcoy(nksi+de1ra)t−ionpko≤f bpmo∈tiXhn⋆kcya(skes),−wpekh+avmekaxw{haen(k)k}>Gm≤adx({dK)a′+,K2cd′G}.,
n Asd isarbitraryandd(d )→0whend →0,weget minky(k)−pk→
Next we prove that liminf (cid:229) qj(fj(y(k))−fj(x⋆))≤0 by con- p∈X⋆
k→¥ j=1 0, which means that the global weighted average of all agents’
tradiction. If not, there exist e >0 and Ke ∈N+, such that for estimatesconvergestosomepointintheoptimalsetX⋆.Finallywith
n
all k>Ke, j(cid:229)=1qj(fj(y(k))−fj(x⋆))>e . Then ky(k+1)−x⋆k2≤ Lemma 2, we obtain that kl→im¥ pm∈iXn⋆kxi(k)−pk≤kl→im¥ pm∈iXn⋆(ky(k)−
ky(k)−x⋆k2+a (k)2G2+4G (cid:229)n qjky(k)−vj(k)ka (k)−2a (k)e = cpokn+vekrxgie(kt)o−thye(ko)pkt)im=a0l,sweth.i(cid:3)chmeansthattheestimatesofallagents
j=1
n
4G(cid:229) qjky(k)−vj(k)k−e Remark 3. A similar analysis can be applied to the push-sum sub-
ky(k)−x⋆k2−a (k)e +G2(a (k)2+ j=1 a (k)).From
G2 gradient algorithm in [16] to prove that the more general step sizes
Lemma 2, kl→im¥ kxi(k)−xj(k)k=0. We have that are positive, vanishing and non-summable can also guarantee
the optimal convergence of the push-sum subgradient algorithm. As
kl→im¥ kvi(k)−y(k)k thereisnosignificantdifferenceintheproof,weomititinthispaper.
n n
(cid:229) (cid:229)
=limk aij(k)xj(k)−y(k)k≤ lim aij(k)kxj(k)−y(k)k
k→¥ k→¥
j=1 j=1
n n n n
(cid:229) (cid:229) (cid:229) (cid:229)
=lim aij(k)kxj(k)− qixi(k)k≤ limkxi(k)−xj(k)k B. Constrained Case
k→¥ k→¥
j=1 i=1 j=1i=1
=0. Inthissection,weprovethatthedistributedsubgradientalgorithm
(5) (3) under Assumption 8 without the square summable requirement
Then it follows that there exists Kc ∈N+, such that for all k> can drive every agent to the optimal solution of the optimization
Kc, (cid:229)n qjky(k)−vj(k)k≤ 8eG. Then we have ky(k+1)−x⋆k2 ≤ problem (2) with constraints.
ky(k)j=−1x⋆k2−a (k)e +G2(a (k)2− e a (k)).Asa (k)vanishesfrom Theorem 2. For a graph sequence G(k), k=0,1,2,···, satisfying
Assumption 8, there exists Ka ∈2NG+2 , such that for all k >Ka , Assumptions 1, 2 and 3 and the optimization problem (2) satisfying
a (k)≤ e . Then it follows that a (k)2− e a (k)<0 and Assumptions 4, 5, 6, 7, 8, the agent estimates xi, i∈V, in the dis-
2G2 2G2 tributedsubgradient algorithm (3) converge toa common minimizer
ky(k+1)−x⋆k2≤ky(k)−x⋆k2−a (k)e . (6) of (2).
5
Proof: Letx⋆ besomepointintheoptimalsetX⋆ oftheproblem limd(xj(k),Xi)≤ limkxi(k)−xj(k)k=0, ∀i∈V. Then it follows
(2). Let vi(k) be defined in (4). Then we have tkh→a¥t limkxj(k)−PkX→(x¥ j(k))k= limd(xj(k),X)=0. Hence we have
k→¥ k→¥
n
(cid:229) qikxi(k+1)−x⋆k2 limkxi(k)−y(k)k=0 (9)
i=1 k→¥
=i(cid:229)=n1qikPXi(j(cid:229)=n1aij(k)xj(k)−a (k)gi(k))−x⋆k2 and kl→im¥ kvi(k) − y(k)k = kl→im¥ kj(cid:229)=n1aij(k)xj(k) − y(k)k ≤
n
≤(cid:229)n qik(cid:229)n aij(k)xj(k)−a (k)gi(k)−x⋆k2 j(cid:229)=1kl→im¥ kxj(k) −y(k)k = 0. As fi, i ∈ V, are convex, and
i=1 j=1 continuous in the constraints under Assumption 5, we have
=(cid:229)n qik(cid:229)n aij(k)xj(k)−x⋆k2+a (k)2(cid:229)n qikgi(k)k2 likm→i¥nfi=(cid:229)n1qi(fi(y(k))−fi(x⋆))=likm→i¥nfi=(cid:229)n1qi(fi(vi(k))−fi(x⋆))≤0.
i=1 j=1 i=1 n
n Also as y(k)∈X, it follows that (cid:229) qi(fi(y(k))−fi(x⋆))≥0. Then
−2a (k)(cid:229) qjgTj(k)(vj(k)−x⋆), n i=1
j=1 wehaveliminf (cid:229) qi(fi(y(k))−fi(x⋆))=0. Therefore, thereexistsa
k→¥ i=1
where the inequality is obtained from Lemma 1. As k·k2 is sub-sequence {y(kp)} of {y(k)}, such that
n
convex and qTA(k)=qT, i.e., i=(cid:229) 1qiaij(k)=qj, under Assumption lim (cid:229)n qi(fi(y(kp))−fi(x⋆))=liminf(cid:229)n qi(fi(y(k))−fi(x⋆))
2, we have (cid:229)n qik (cid:229)n aij(k)xj(k)−x⋆k2 ≤ (cid:229)n qi (cid:229)n aij(k)kxj(k)− kp→¥ i=1 k→¥ i=1
i=1 j=1 i=1 j=1 =0.
n n n
x⋆k2= (cid:229) ((cid:229) qiaij(k))kxj(k)−x⋆k2= (cid:229) qjkxj(k)−x⋆k2.Because (10)
j=1 i=1 j=1
kgi(k)k≤G under Assumption 7, it follows that As {y(kp)}∈X is uniformly bounded from Assumption 6, {y(kp)}
has a convergent sub-sequence. Without loss of generality, suppose
n n
i(cid:229)=1qikxi(k+1)−x⋆k2≤i(cid:229)=1qikxj(k)−x⋆k2+a (k)2G2 ltihmatitthpeoicnot.nvWeregeanlstosukbn-oswequtheantcey¥is∈{yX(⋆kpf)r}omits(e1lf0,).wWithithy¥oubtelionsgsiotsf
n (8) generality, let x⋆=y¥ . Then we get from (9) that
−2a (k)j(cid:229)=1qjgTj(k)(vj(k)−x⋆). klpi→m¥ kxi(kp)−x⋆k=klpi→m¥ kxi(kp)−y¥ k
As fj, j ∈V, are convex, fj(vj(k))− f(x⋆) ≤gTj(k)(vj(k)−x⋆). ≤ lim (kxi(kp)−y(kp)k+ky(kp)−y¥ k) (11)
As a result, we have (cid:229)n qikxi(k+1)−x⋆k2≤ (cid:229)n qikxi(k)−x⋆k2+ =0kp→¥
i=1 i=1
n
a (k)2G2−2a (k)j(cid:229)=1qj(fj(vj(k))−n fj(x⋆)). and klpi→m¥ i=(cid:229)n1qifi(xi(kp))= f(x⋆).
Next, we prove that liminf (cid:229) qj(fj(vj(k))− fj(x⋆)) ≤ 0 by We then prove the convergence of the estimates {xi(k)},∀i ∈
contradiction. If not, therek→ex¥ istj=e1 > 0 and Ke ∈ N+, such that V to x⋆. For the last term in (8), we have (cid:229)n qigTi (k)(vi(k)−
n n i=1
∀k>Ke, j(cid:229)=1qj(fj(vj(k))−fj(x⋆))>e . Then we have i=(cid:229) 1qikxi(k+ x⋆) ≤ (cid:229)n qikgi(k)kkvi(k)−x⋆k ≤ G (cid:229)n qik (cid:229)n aij(k)xj(k)−x⋆k ≤
n n i=1 i=1 j=1
1)−x⋆k2≤i=(cid:229) 1qikxi(k)−x⋆k2+a (k)2G2−2a (k)e =i=(cid:229) 1qikxi(k)− G (cid:229)n (cid:229)n qiaij(k)kxj(k)−x⋆k=G (cid:229)n qikxi(k)−x⋆k, where the last
x⋆k2−a (k)e +(a (k)2G2−a (k)e ). As lima (k) = 0, there ex- j=1i=1 i=1
k→¥ equality is obtained from the fact that qTA(k) = qT under As-
ipsltisesKtaha∈t aN(+k,)2sGu2ch−taha(kt)e∀k<>0.KaH,en0ce<∀ak(>k)K<=Ge2m, awx(hKiceh,Kiam)-, sumption 2. Then (8) can be transformed into i=(cid:229)n1qikxi(k+1)−
n n n n
we have (cid:229) qikxi(k+1)−x⋆k2 ≤ (cid:229) qikxi(k)−x⋆k2−a (k)e . Be- x⋆k2 ≤ (cid:229) qikxi(k)−x⋆k2+a (k)2nG2+2a (k) (cid:229) Gqikxi(k))−x⋆k.
cause (cid:229)¥ i=a 1(k) = ¥ , it follows thia=t1when k is sufficiently large As kxi(ki=)1)−x⋆k≤kxi(k)k+kx⋆k, i∈V, and bi=o1th the optimal set
k=1 X⋆ andtheconstraintsetsXi,i∈V,arebounded under Assumptions
(cid:229)n qikxi(k+1)−x⋆k2 ≤ (cid:229)n qikxi(K+1)−x⋆k2− (cid:229)k a (t)e <0. 4 and 6, kxi(k))−x⋆k, i∈V, are also bounded. With kl→im¥ a (k)=
tTi=hha1itslcikmo→ni¥tnrfadj(cid:229)=nic1tqsjw(fitjh(vkj(xki()ki)=+−11f)j(−x⋆x)⋆)k≤2≥0.0.Itcantht=usK+b1econcluded 0x⋆,kw2+eah(akv)e2nlGimk2→+s¥u2pai=(cid:229)n(1k)qii=(cid:229)knx1iG(kqi+kx1i()k−))x−⋆xk⋆2k≤)=lilkmi→km→i¥ni¥nf(fii=(cid:229)=(cid:229)nn11qqiikkxxii((kk))−−
Next we show the optimal convergence of the agents’ estimates. n
WDeefihnaevye(k)=1ni=(cid:229)n1PX(xi(k)).Notethaty(k)∈X becauseX isconvex. xl⋆imk2.(cid:229)nSqoikxi=(cid:229)i(1kq)i−kxxi(⋆kk)2−=xl⋆ikm2 i(cid:229)ns qcioknxvi(ekrgp)en−t.x⋆Fkr2o=m0.(1A1s)qwi>e0h,aiv∈e
kxi(k)−y(k)k=kxi(k)−1 (cid:229)n PX(xj(k))k≤ 1 (cid:229)n kxi(k)−PX(xj(k))k Vk→,¥uni=d1er Assumptions 1kapn→d¥ 2i=, 1it follows that kl→im¥ kxi(k)−x⋆k=0
n n n
j=1 j=1 and lim (cid:229) qifi(xi(k))= f(x⋆). (cid:3)
1 (cid:229)n k→¥ i=1
≤ ( kxi(k)−xj(k)k+kxj(k)−PX(xj(k))k).
n Remark 4. We can see from Theorem 1 and 2 that for both the
j=1
unconstrainedandconstraineddistributedoptimizationproblems(2),
From Lemma 2, we know that limkxi(k)−xj(k)k=0. So we have the positive, vanishing and non-summable step sizes can guarantee
k→¥
6
tThheeospqtuimaraelscuomnvmeargbeilnitcye(cid:229)o¥ ftahe(kd)i2s<tri¥buitsednostunbegcraesdsiaenryt.aWlghoernithGm(k()3i)s. Kan1d∈N(cid:229)n+q,idsuic≤h t4hGeaHt.foTrhaelnl kit>foKll1o,wwsethhaavte(cid:229)qnidqi2ik+xi2(kqi+di1<)−1,x∀⋆ik∈2V=,
k=1 i=1 i=1
bfuanlactnicoends,. wWehceannGg(ekt)thiesmuninbiamlaiznecredofathnedsfiuxmedo,fwtheeclaoncaolbotbajiencttihvee (cid:229)n qikxi(k)−x⋆k2−ea (k)+2nG2a (k)2−e2a (k).Asa (k)→0,there
minimizer of a weighted average of the local objective functions, ie=x1istsK2∈N+,suchthatforallk>K2,wehavea (k)≤ 4neG2.Then
with the weights being the elements in the positive left eigenvector n n
of the adjacency matrix associated with eigenvalue 1. it follows that (cid:229) kxi(k+1)−x⋆k2= (cid:229) kxi(k)−x⋆k2−ea (k). The
i=1 i=1
rest part is very similar to the proof of Theorem 2 and is omitted.
Remark 5. Sometimes we can transform an unconstrained dis-
tributedoptimizationproblemwithunbounded subgradients intoone V. CONCLUSIONS
with compact constraints, when the graph is balanced. Suppose We proved the convergence to a common optimal solution of the
every local objective function fi is bounded below, i.e., there exists distributedsubgradient method for adistributedconvex optimization
B ∈ R, such that fi(x) ≥ B. Without loss of generality, suppose problemforboththeunconstrainedandconstrainedcases.Werelaxed
that fi(x)≥0 (optimizing fi(x) and fi(x)−B is the same). Also the requirement on the step size by removing the square summable
assume that the sublevel set of each local objective function is requirement, and showed the positive, vanishing and non-summable
compact. Then the transformation can be achieved in the following step sizes were sufficient for the optimal convergence of the dis-
steps. First, initialize all agents’ estimates at the same value, i.e., tributed subgradient algorithm, when the topology is fixed or time-
xi(0) =xj(0),∀i,j ∈V. Then each agent runs a consensus algo- varying but balanced.
n
rithm to compute f(x(0))= (cid:229) fi(xi(0)). Next, denote the sublevel REFERENCES
i=1
set Si(a) = {x : fi(x) ≤ f(x(0))+a, ∀a ≥ 0}. Because fi(x⋆) ≤ [1] J.Mota,J.Xavier,P.Aguiar,andM.Puschel,“Distributedoptimization
n n
(cid:229) fi(x⋆)≤ (cid:229) fi(xi(0))= f(x(0))≤ f(x(0))+a,∀a>0, it follows withlocaldomains:Applicationsinmpcandnetworkflows,”Automatic
i=1 i=1 Control,IEEETransactionson,vol.60,no.7,pp.2004–2009,July2015.
that x⋆ ∈Si(a). Thus, the unconstrained optimization problem be- [2] A.D’Amico, L.Sanguinetti, andD.Palomar, “Convex separable prob-
comes a constrained one. lemswithlinear constraints insignal processing andcommunications,”
Signal Processing, IEEE Transactions on, vol. 62, no. 22, pp. 6045–
Remark 6. For the constrained case, if the step sizes are selected 6058,Nov2014.
in a non-uniform way as a i(k)=a (k)(1+di(k)), where a i(k) is [3] M. Zargham, A. Ribeiro, A. Ozdaglar, and A. Jadbabaie, “Accelerated
the step size of agent i at iteration step k, a (k) satisfies a (k)>0, dual descent for network flow optimization,” Automatic Control, IEEE
¥ Transactions on,vol.59,no.4,pp.905–920,April2014.
lima (k)=0and (cid:229) a (k)=¥ ,and limdi(k)=0,thenwecanalso [4] E. Wei, A. Ozdaglar, and A. Jadbabaie, “A distributed newton method
k→¥ k=1 k→¥ fornetworkutilitymaximizationi:Algorithm,”AutomaticControl,IEEE
prove the convergence to the optimal point of the estimates of the
Transactions on,vol.58,no.9,pp.2162–2175, Sept2013.
agents generated from the distributed subgradient algorithm with a [5] A.NedicandA.Ozdaglar, “Distributedsubgradientmethodsformulti-
similar analysis process. First we analyze the distance change from agentoptimization,” AutomaticControl,IEEETransactions on,vol.54,
no.1,pp.48–61,Jan2009.
the agents’ estimates to the optimal solution as:
[6] A. Nedic, A. Ozdaglar, and P. Parrilo, “Constrained consensus and
n optimization inmulti-agent networks,”AutomaticControl,IEEETrans-
(cid:229) qikxi(k+1)−x⋆k2 actions on,vol.55,no.4,pp.922–938, April2010.
i=1 [7] I. Lobel, A. Ozdaglar, and D. Feijer, “Distributed multi-agent
n n optimization with state-dependent communication,” Math. Program.,
≤(cid:229) qikxi(k)−x⋆k2+a (k)2(cid:229) qikgi(k)k2 vol. 129, no. 2, pp. 255–284, Oct. 2011. [Online]. Available:
i=1 j=1 http://dx.doi.org/10.1007/s10107-011-0467-x
n n [8] P. Lin and W. Ren, “Distributed subgradient projection algorithm for
+a (k)2(cid:229) qikdi(k)gi(k)k2+2a (k)2(cid:229) qidi(k)gTi (k)gi(k) (12) multi-agent optimization with nonidentical constraints and switching
topologies,” in Decision and Control (CDC), 2012 IEEE 51st Annual
i=1 i=1
n Conference on,Dec2012,pp.6813–6818.
−2a (k)(cid:229) qj(fj(vj(k))−f(x⋆)) [9] D. Jakovetic, J.Xavier, and J. Moura, “Fast distributed gradient meth-
ods,”AutomaticControl,IEEETransactionson,vol.59,no.5,pp.1131–
j=1
1146,May2014.
n
+2a (k)(cid:229) qidi(k)kgi(k)kk(vi(k)−x⋆)k. [10] M.ZhuandS.Martinez,“Ondistributedconvexoptimizationunderin-
equalityandequalityconstraints,”AutomaticControl,IEEETransactions
i=1
on,vol.57,no.1,pp.151–164, Jan2012.
n [11] ——,“Anapproximatedualsubgradientalgorithmformulti-agentnon-
Then we can prove that liminf (cid:229) qj(fj(vj(k))− fj(x⋆)) ≤ 0 by convexoptimization,”AutomaticControl,IEEETransactionson,vol.58,
k→¥ j=1 no.6,pp.1534–1539, June2013.
contradiction. Suppose not. Then there exist e >0 and K ∈N+, [12] T.-H. Chang, A. Nedic, and A. Scaglione, “Distributed constrained
n
such that for all k>K, (cid:229) qj(fj(vj(k))−fj(x⋆))>e . Because the optimizationbyconsensus-basedprimal-dualperturbationmethod,”Au-
j=1 tomatic Control, IEEETransactions on,vol.59,no.6,pp.1524–1538,
local constraint set Xi,i∈V, is bounded from Assumption 6, we June2014.
have that vi(k)∈conv(Sn Xi) and x⋆∈ Tn Xi are bounded. Then it [13] cJ.onLvuexanodptiCm.izYa.tioTna:ngT,h“eZecroon-tgirnaudoiuens-tt-ismume caalsgeo,r”ithAmutsomfoarticdisCtroibnutrtoedl,
i=1 i=1
followsthat vi(k)−x⋆ isalso bounded, i.e.,there existsH>0, such IEEETransactions on,vol.57,no.9,pp.2348–2354, Sept2012.
n [14] N. Z. Shor, K. C. Kiwiel, and A. Ruszcayn`ski, Minimization Methods
that kvi(k)−x⋆k≤H. Then we have from (12) that (cid:229) qikxi(k+ forNon-differentiableFunctions. NewYork,NY,USA:Springer-Verlag
i=1 NewYork,Inc.,1985.
n n
1)−x⋆k2 ≤ (cid:229) qikxi(k)−x⋆k2+G2((cid:229) (1+qidi2+2qidi))a (k)2+ [15] S. Liu, Z. Qiu, and L. Xie, “Continuous-time distributed convex opti-
i=1 i=1 mization withsetconstraints,” inIFACWorldCongress,vol.19,no.1,
2(GH (cid:229)n qidi−e )a (k) = (cid:229)n qikxi(k)−x⋆k2−ea (k)+G2((cid:229)n (1+ [16] A20.1N4e,dpipc.a9n7d6A2–.9O7l6s7h.evsky,“Distributedoptimizationovertime-varying
i=1 i=1 i=1
n directed graphs,” Automatic Control, IEEE Transactions on, vol. 60,
qidi2+2qidi))a (k)2+(2GH (cid:229) qidi−e )a (k),Asdi→0,thereexists no.3,pp.601–615, March2015.
i=1