Table Of ContentON THE OPTIMALITY AND SHARPNESS OF LAGUERRE’S
LOWER BOUND ON THE SMALLEST EIGENVALUE OF A
7
1 SYMMETRIC POSITIVE DEFINITE MATRIX
0
2
Yusaku Yamamoto,Chofu, Tokyo, 182-8585, Japan
b
e
F (Received January 31, 2017)
1
]
A
Abstract. Lower bounds on the smallest eigenvalue of a symmetric positive definite
N matricesA∈Rm×m playanimportantroleinconditionnumberestimationandiniterative
. methods for singular value computation. In particular, the bounds based on Tr(A−1) and
cs Tr(A−2) attract attention recently because they can be computed in O(m) work when A
[ is tridiagonal. In this paper, we focus on these bounds and investigate their properties in
detail. First, we consider the problem of finding theoptimal bound that can be computed
1 solely from Tr(A−1) and Tr(A−2) and show that so called Laguerre’s lower bound is the
v
optimal one in terms of sharpness. Next, we study the gap between the Laguerre bound
8
0 andthesmallesteigenvalue. Wecharacterizethesituationinwhichthegapbecomeslargest
1 in terms of the eigenvalue distribution of A and show that thegap becomes smallest when
0 Tr(A−2)/{Tr(A−1)}2 approaches 1 or m1. These results will be useful, for example, in
0 designing efficient shift strategies for singular valuecomputation algorithms.
.
2
Keywords: eigenvalue bounds, symmetric positive definite matrix, Laguerre bound, sin-
0
gular valuecomputation, dqdsalgorithm
7
1 MSC 2010: 15A18, 15A42
:
v
i
X
r
a
1. Introduction
Let A Rm×m be a symmetric positive definite matrix and denote the smallest
∈
eigenvalue of A by λ (A). In this paper, we are interested in a lower bound on
m
λ (A). If the Cholesky factorization of A is A = BB , where B Rm m is a
m ⊤ ×
∈
nonsingularlower triangularmatrix, the smallest singular value of B can be written
ThisstudyhasbeensupportedbyJSPSKAKENHIGrantNumbersJP26286087, JP15H02708,
JP15H02709andJP16KT0016.
1
as σ (B) = λ (A). Hence, finding a lower bound on λ (A) is equivalent to
m m m
finding a lower bound on σ (B).
p m
A lower bound on λ (A) or σ (B) plays an important role in various scientific
m m
computations. For example, when combined with an upper bound on A , a lower
2
k k
bound on λ (A) can be used to give an upper bound on the condition number of
m
A. In singular value computation algorithms such as the dqds algorithm [3], the
orthogonalqd algorithm[10]andthe mdLVs algorithm[6], a lowerbound onσ (B)
m
is used as a shift to accelerate the convergence. In the latter case, the matrix B is
usually a lower bidiagonal matrix as a result of preprocessing by the Householder
method [4].
Several types of lower bounds on λ (A) or σ (B) have been proposed so far.
m m
Therearebounds basedoneigenvalueinclusiontheoremssuchasGershgorin’scircle
theorem [7] or Brauer’s oval of Cassini [8]. The norm of the inverse, A 1 , can
−
k k∞
also be used to bound the maximum eigenvalue of A 1 from above, and therefore
−
to bound λ (A) from below. There are also bounds based on the traces of the in-
m
verses, namely, Tr(A 1) and Tr(A 2). Among them, the last class of bounds are
− −
attractive in the context of singular value computation, because they always give a
valid (positive) lower bound, as opposed to the bounds based on the eigenvalue in-
clusiontheorems,andtheycanbecomputedinO(m)workusingefficientalgorithms
[9, 11, 13]. Examples of lower bounds of this type include the Newton bound [10],
the generalized Newton bound [9, 1] and the Laguerre bound [10].
In this paper, we focus on the lower bounds of λ (A) derived from Tr(A 1) and
m −
Tr(A 2)andinvestigatetheirproperties. Inparticular,wewilladdressthefollowing
−
two questions. The first is to identify an optimal formula for a lower bound on
λ (A) that is based solely on Tr(A 1) and Tr(A 2). Here, the word ”optimal”
m − −
means that the formula always gives a sharper (that is, larger) bound than any
other formulas using only Tr(A 1) and Tr(A 2). As a result of our analysis, we
− −
showthatthe Laguerreboundmentionedaboveis the optimalformulainthis sense.
The secondquestion is to evaluate the gap between the Laguerrebound andλ (A).
m
Unlike the Laguerre bound, λ (A) is not determined uniquely only from Tr(A 1)
m −
and Tr(A 2). Hence, for some of the matrices, there must be a gap between the
−
bound and λ (A). Our problem is to quantify the maximum possible gap and
m
identify the conditions under which the maximum gap is attained. These results
will be useful, for example, in designing an efficient shift strategy for singular value
computation algorithms, which combines the Laguerre bound with other bounds
with complementary characteristics [12].
Therestofthispaperisstructuredasfollows. InSection2,weinvestigatethelower
bounds on λ (A) derived from Tr(A 1) and Tr(A 2) and show that the Laguerre
m − −
boundisanoptimaloneintermsofsharpness. Section3dealswiththe gapbetween
2
theLaguerreboundandλ (A). Inparticular,wecharacterizethesituationinwhich
m
the gapbecomes largestinterms ofthe eigenvalue distributionof A. Section 4 gives
some concluding remarks.
2. An optimal lower bound based on Tr(A 1) and Tr(A 2)
− −
2.1. Lower bounds based on Tr(A 1) and Tr(A 2). Let A be an m m real
− −
×
symmetric positive matrix. We denote the kth largest eigenvalue of A by λ (A),
k
or λ for short. Let f(λ) = det(λI A) be the characteristic polynomial of A. To
i
−
find a lower bound onthe smallesteigenvalue λ, we consider applying a rootfinding
methodforanalgebraicequationtof(λ)=0startingfromtheinitialvalueλ(0) =0.
There are several root finding methods, such as the Bailey’s (Halley’s) method [2],
Householder’s method [5] and Laguerre’s method [14, 10], for which the iteration
formulas can be written as follows:
f(λ(n)) 1
(2.1) λ(n+1) = λ(n) ,
B − f′(λ(n)) · 1− f(λ2(fn′)()λf(′n′()λ)2(n))
f(λ(n)) f(λ(n))f (λ(n))
(2.2) λ(n+1) = λ(n) 1+ ′′ ,
H − f (λ(n)) 2f (λ(n))2
′ (cid:26) ′ (cid:27)
f(λ(n))
λ(n+1) = λ(n)
L − f (λ(n))
′
m
(2.3)
×1+ (m−1) m· f′(λ(n))2f−′f(λ(λ(n()n))2)f′′(λ(n)) −1
r
n o
Eqs.(2.1),(2.2)and(2.3)representtheiterationformulasofBailey’smethod,House-
holder’s method and Laguerre’s method, respectively. When applied to f(λ) =
det(λI A)startingfromλ(0) =0,theseformulasproduceasequencethatincreases
−
(1) (1) (1)
monotonically and converges to λ . Hence, all of λ , λ and λ can be used as
m B H L
a lower bound on λ .
m
m
Noting that f(λ)= (λ λ ), we have
k=1 − k
Qm
f (λ) = (λ λ)
′ j
− −
k=1j=k
XY6
m m
1
(2.4) = (λj λ) = f(λ)Tr (A λI)−1 ,
− − λ λ − −
k
jY=1 kX=1 − (cid:16) (cid:17)
m
1
f′′(λ) = f′(λ)Tr (A λI)−1 f(λ)
− − − (λ λ)2
k
(cid:16) (cid:17) kX=1 −
(2.5) = f′(λ)Tr (A λI)−1 f(λ)Tr (A λI)−2 .
− − − −
(cid:16) (cid:17) (cid:16) (cid:17)
3
Hence,
f(λ) 1
(2.6) = ,
f′(λ) −Tr (A λI)−1
−
(cid:16) (cid:17)
f(λ)f (λ) Tr (A λI)−2
′′ −
(2.7) = 1 .
f′(λ)2 − Tr (cid:16)(A λI)−1(cid:17) 2
−
n (cid:16) (cid:17)o
Insertingthese into Eqs.(2.1), (2.2) and(2.3) withλ(0) =0, we obtainthe following
lower bounds on λ (A):
m
2Tr A 1
−
(2.8) L (A) = ,
B Tr(A 1) 2+Tr(A 2)
{ − }(cid:0) (cid:1) −
1 3 1 Tr A−2
(2.9) L (A) = ,
H Tr(A−1)"2 − 2 · {Tr((cid:0)A−1)(cid:1)}2#
1 m
(2.10) L (A) = .
L Tr(A−1) · 1+ (m 1) m Tr(A−2) 1
− · Tr(A−1) 2 −
r { }
h i
We call L (A), L (A) and L (A) the Bailey bound, the Householder bound and
B H L
theLaguerrebound,respectively. Inadditionto these,wealsohaveasimplebound:
1
m 1 −2
(2.11) LN(A)={Tr(A−2)}−12 ≤ k=1λ2k! <λm,
X
which is called the Newton bound of order 2 [10, 9, 1]. In the case where A is a
tridiagonalmatrix,bothTr(A 1)andTr(A 2)canbecomputedinO(m)workfrom
− −
its Cholesky factor B [9, 11, 13]. Accordingly, any of these bounds can be employed
inapracticalshift strategyfor singularvalue computationalgorithms. The problem
then is which of the four lower bounds, or possibly another bound derived from
Tr(A 1) and Tr(A 2), is optimal in terms of sharpness.
− −
2.2. The optimal lowerbound. Toanswerthequestion,wereformulatetheprob-
lem as follows. Assume that Tr(A 1) and Tr(A 2) are specified for a symmetric
− −
positive definite matrix A. Then, how small can the smallest eigenvalue λ (A) be?
m
Ifthis boundcanbe obtainedexplicitly asafunctionofTr(A 1)andTr(A 2),then,
− −
it will be the optimal formula for the lower bound of λ (A).
m
Now, let a Tr(A 1), b Tr(A 2) and x 1/λ (k = 1,2,...,m). Then, the
− − k k
≡ ≡ ≡
upper bound on x (the reciprocal of the lower bound on λ ) can be obtained by
m m
4
solving the following constrained optimization problem:
(2.12) maximize x
m
m
(2.13) s.t. x =a,
k
k=1
X
m
(2.14) x2 =b,
k
k=1
X
(2.15) x >0 (k =1,2,...,m),
k
(2.16) x x x .
1 2 m
≤ ≤···≤
Actually, the constraint (2.16) is redundant, because if (x ,x ,...,x ) is a solu-
∗1 ∗2 ∗m
tion of the optimization problem without constraint (2.16), then, from symmetry,
(x ,x ,...,x )isalsoasolutionforanypermutationσ of 1,2,...,m ,and
∗σ(1) ∗σ(2) ∗σ(m) { }
therefore we can choose a solution that satisfies (2.16). Hence we omit (2.16) in the
following.
To solve the optimization problem (2.12)–(2.15), we remove the constraint (2.15)
and consider a relaxed problem described by (2.12)–(2.14). By introducing the La-
grange multipliers µ and ν, we can write the Lagrangianas
m m
(2.17) L=x µ x a ν x2 b .
m− k− !− k− !
k=1 k=1
X X
Then the solution to (2.12)–(2.14) must satisfy
∂L
(2.18) = 1 µ 2νx =0,
m
∂x − −
m
∂L
(2.19) = µ 2νx =0 (k =1,2,...,m 1),
k
∂x − − −
k
m
∂L
(2.20) = x a=0,
k
∂λ −
k=1
X
m
∂L
(2.21) = x2 b=0.
∂µ k−
k=1
X
From (2.19), we have either ν = 0 or x =x = =x . However, when ν =0,
1 2 m 1
··· −
we have µ = 0 from (2.19) and µ = 1 from (2.18), which is a contradiction. Thus
x =x = =x must hold. Inserting this into (2.20) and (2.21) leads to
1 2 m 1
··· −
(2.22) x +(m 1)x a=0,
m 1
− −
(2.23) x2 +(m 1)x2 b=0.
m − 1−
5
Solving these simultaneous equations with respect to x gives
m
a m(m 1)b (m 1)a2
(2.24) x±m = ± −m − − .
p
Note that the x given by (2.24) is real, since
m
m m 2
m(m 1)b (m 1)a2 = (m 1) m x2 x
− − − − k− k!
Xk=1 Xk=1
m k 1
(2.25) = (m 1) − (x x )2 0.
k l
− − ≥
k=1l=1
XX
Now wereturnto the relaxedoptimizationproblem(2.12)–(2.14). Since the feasi-
blesetofthisproblemiscompactandboththeobjectivefunctionandtheconstraints
are differentiable, it must have a minimum and a maximum at a point where the
gradient of the Lagrangian is zero. Furthermore, since the objective function is x
m
itself, the maximum is attained when x =x+. Then, from Eq. (2.22), we have
m m
(m 1)a m(m 1)b (m 1)a2
(2.26) x =x = =x = − − − − − .
1 2 m 1
··· − pm(m−1)
Hence,Eq.(2.26)andx =x+ arethesolutionoftherelaxedoptimizationproblem.
m m
Finally, we consider the positivity constraint (2.15). It is clear from (2.24) that
x+ >0. To investigate the positivity of the other variables, note that
m
m 2 m m k 1
1 1 − 1 1
(2.27) a2 b= =2 >0,
− k=1λk! −k=1λ2k k=1l=1 λk · λl
X X XX
where we used the fact that a and b are the traces of the inverse of a matrix with
positive eigenvalues. Then (2.26) can be rewritten as
(m 1)2a2 m(m 1)b (m 1)a2
x =x = =x = − −{ − − − }
1 2 m 1
··· − m(m 1) (m 1)a+ m(m 1)b (m 1)a2
− { − − − − }
m(m 1)(a2 b)
p
(2.28) = − − >0.
m(m 1) (m 1)a+ m(m 1)b (m 1)a2
− { − − − − }
p
This shows that the solution to the relaxed problem (2.12)–(2.14) automatically
satisfies the constraint (2.15). Hence it is also a solution to the original problem
6
(2.12)–(2.15). Returning to the original variables λ = 1/x , we know that the
k k
smallest value that λ can take is
m
1 m
(2.29) .
Tr(A−1) · 1+ (m 1) m Tr(A−2) 1
− · Tr(A−1) 2 −
r { }
h i
This gives the optimal lower bound on λ (A) in terms of Tr(A 1) and Tr(A 2).
m − −
Since Eq. (2.29) is exactly the Laguerre bound (2.10), we arrive at the following
theorem.
Theorem 2.1. Among the lower bounds on λ (A) computed from Tr(A 1) and
m −
Tr(A 2), the Laguerre bound (2.10) is optimal in terms of sharpness.
−
3. The gap between the Laguerre bound and the smallest eigenvalue
NowthatwehaveestablishedthattheLaguereboundistheoptimallowerbound,
we next study the gap between the bound and the minimum eigenvalue. We begin
with a lemma that holds for a 3 3 matrix and then proceed to the general case.
×
In the course of discussion,we also allowinfinite eigenvalues to make the arguments
simpler.
Assume that A R3 3 is a symmetric positive definite matrix with Tr(A 1)=a
× −
∈
and Tr(A 2) = b. Let the eigenvalues of A be λ λ λ > 0. To evaluate the
− 1 2 3
≥ ≥
gap, we consider how large λ can be under the fixed values of a and b. First, we
3
show the following lemma.
Lemma 3.1. For fixed a = Tr(A 1) and b = Tr(A 2), λ can take a maximum
− − 3
only when λ =λ or λ =+ .
2 3 1
∞
Proof. Let x = 1/λ , y = 1/λ and z = 1/λ . Since we allow infinite eigenvalues,
3 2 1
the point (x,y,z) lies in a region D of the xyz space specified by x+y +z = a,
x2 +y2 +z2 = b, and x y z 0. Since D is a compact set, the continuous
≥ ≥ ≥
function x attains a minimum somewhere in D. Hence, if we can show that x does
not attain a minimum when x > y and z > 0, it means that x attains a minimum
when x=y or z =0.
Assume that the point (x,y,z) is in D and both x>y and z >0 hold. Then, let
ǫ>0 be some small quantity and t R and consider changing (x,y,z) to (x,y ,z )
′ ′ ′
∈
as follows:
(3.1) x′ = x ǫ,
−
(3.2) y = y+tǫ,
′
(3.3) z = z+(1 t)ǫ.
′
−
7
Clearly, the new point (x,y ,z ) lies on the plane x+y+z =a. We determine t so
′ ′ ′
that it is also on the sphere x2+y2+z2 =b. The condition can be written as
(3.4) (x ǫ)2+(y+tǫ)2+ z+(1 t)ǫ 2 =x2+y2+z2,
− { − }
or
(3.5) ǫt2+(y z ǫ)t+( x+z+ǫ)=0.
− − −
Solving this with respect to t gives
(y z ǫ) (y z ǫ)2+4ǫ(x z ǫ)
(3.6) t = − − − ± − − − − .
± p 2ǫ
In the following, we adopt the solution t = t . Now we consider two cases. First,
+
consider the case of y =z. Then, we have from Eq. (3.6),
ǫ+ ǫ2+4ǫ(x z ǫ)
(3.7) t ǫ= − − =O(√ǫ).
+
2
p
Inserting this into (3.1) through (3.3), we know that the changes in x, y and z are
at most O(√ǫ) when ǫ is small.
Next, consider the case of y >z. In this case, we can rewrite (3.6) as
2(x z ǫ)
(3.8) t = − − .
+
(y z ǫ)+ (y z ǫ)2+4ǫ(x z ǫ)
− − − − − −
p
Since x z ǫ>0 and y z ǫ>0 for a sufficiently small ǫ, we have
− − − −
x z ǫ
(3.9) 0<t < − − .
+
y z ǫ
− −
Hence, t ǫ = O(ǫ) when ǫ is small and therefore the changes in x, y and z are at
+
most O(ǫ) in this case.
In summary, in both cases, the changes of x, y and z can be made arbitrarily
small. Thus, by choosing ǫ sufficiently small, we can make x smaller than x while
′
keeping the relation x > y > 0 and x >z > 0 (Fig. 1). The relation y z may
′ ′ ′ ′ ′ ′
≥
not hold, but in that case, we can interchange y and z . In this way, we can obtain
′ ′
another point (x,y ,z ) D which attains a smaller value of x. Hence x cannot
′ ′ ′
∈
attain a minimum when both x>y and z >0 hold and the lemma is proved. (cid:3)
Using this lemma, we can prove the following theorem.
8
Figure 1. The values of x, y and z before and after the perturbation.
Theorem 3.2. Let a = Tr(A 1) and b = Tr(A 2) be fixed and q be an integer
− −
satisfying q < a2/b q+1. Then, λ (A) takes a maximum when λ (A) = =
m 1
≤ ···
λ (A)=+ and λ = =λ (A). The maximum is given as
m q 1 m q+1 m
− − ∞ − ···
1 q(q+1)
(3.10) λ (A)= .
∗m Tr(A−1) · q+ q (q+1) Tr(A−2) 1
· Tr(A−1) 2 −
r { }
h i
Proof. Let x = 1/λ . First, assume that there are two or more eigenvalues which
k k
are neither an infinite eigenvalue nor equal to λ (A). In this case, as we will show
m
in the following, we can make λ smaller by adding appropriate perturbations. We
m
divide the cases depending on the multiplicity q of the smallest eigenvalue.
When q = 1, from the assumption, both λ and λ are neither an infinite
m 2 m 1
− −
eigenvalue nor equal to λ (A). Thus, we have 0 < x x < x . Then, by
m m 2 m 1 m
− ≤ −
picking up these three variables and adding the same perturbations as in Lemma
3.1, we can make x smaller while keeping the condition 0 < x x <
m m 2 m 1
− ≤ −
x (Fig. 2). Clearly, the values of Tr(A 1) and Tr(A 2) are unchanged by this
m − −
perturbation. Hence, x cannot take a minimum in this case.
m
Figure 2. The values of x , x and x before and after the perturbation.
m m 1 m 2
− −
When q > 1, 0 < x x < x = = x holds from the as-
m q 1 m q m q+1 m
− − ≤ − − ···
sumption. Then, by picking up the three variables x , x and x and
m q 1 m q m q+1
− − − −
addingthe perturbationsasinLemma3.1, wecanmakex smallerwhile keep-
m q+1
−
ing 0 < x x < x . This does not change the smallest eigenvalue,
m q 1 m q m q+1
− − ≤ − −
9
but reduces its multiplicity from q to q 1 (Fig. 3). Moreover, the condition that
−
there are two or more eigenvalues which are neither an infinite eigenvalue nor equal
to λ (A) still holds. Hence, we can repeat this procedure and reduce q to 1, while
m
keeping the value of the smallest eigenvalue unchanged. But in this last situation,
x cannot take a minimum, as concluded in the analysis of the q =1 case.
m
(cid:127)
(cid:127)
(cid:127)
(cid:127)
(cid:127)
(cid:127)
(cid:127)
Figure 3. The values of x ,x ,...,x before and after the perturbation.
m q 1 m q m
− − −
From the above analysis, we can conclude that x cannot take a minimum when
m
there are two or more eigenvalues which are neither an infinite eigenvalue nor equal
to λ (A). Thus, the only possible case is when x = = x = 0 and
m 1 m q 1
··· − −
x = =x holds for some q. In this case, we have
m q+1 m
− ···
(3.11) x +qx = a,
m q m
−
(3.12) x2 +qx2 = b,
m q m
−
or
aq q (q+1)b a2
(3.13) x±m = ± q{(q+1) − },
p
a q (q+1)b a2
(3.14) x±m−q = ∓p {q+1 − }.
For x and x to be real, q must satisfy q+1 a2/b. Then, for x x to
m m q m q m
− ≥ − ≤
hold, we have to choose x+ and x+ . In addition, for x+ > 0 to hold, we must
m m q m q
haveq <a2/b. Fromtheconditionq−<a2/b q+1,qisdete−rmineduniquely. Hence,
≤
there is only one set of q, x and x that satisfy the condition for minimum x .
m m q m
Since the feasible region of (x ,x ,...−,x ), specified by m x =a, m x2 =b
1 2 m k=1 k k=1 k
and 0 x x x , is compact, x must have a minimum somewhere in
≤ 1 ≤ 2 ≤ ··· ≤ m m P P
this region. Accordingly, we conclude that x takes a minimum when q < a2/b
m
≤
q + 1, x = = x = 0, x = x+ and x = = x = x+.
Eq. (3.101) is o·b··tainedmf−roq−m1λ =1/mx−q. m−q m−q+1 ··· m m(cid:3)
m m
10
