Table Of ContentBernoulli 22(1), 2016, 563–588
DOI: 10.3150/14-BEJ668
Unitary transformations, empirical processes
6
1 and distribution free testing
0
2
n ESTATE KHMALADZE
a
J School of Mathematics, Statistics and Operations Research, Victoria University of Wellington,
Wellington, New Zealand. E-mail: [email protected]
7
] Themain message inthispaperisthattherearesurprisingly manydifferentBrownian bridges,
T
someofthem–familiar, someofthem–lessfamiliar. ManyoftheseBrownian bridgesarevery
S
close to Brownian motions. Somewhat loosely speaking, we show that all the bridges can be
.
h convenientlymapped ontoeach other, and hence,to one “standard” bridge.
t The paper shows that, a consequence of this, we obtain a unified theory of distribution free
a testinginRd,bothfordiscreteandcontinuouscases,andforsimpleandparametrichypothesis.
m
[ Keywords: Brownian bridge; empirical processes; goodness of fit tests in Rd; g-projected
Brownian motions; parametric hypothesis; unitary operators
1
v
6 1. Introduction
2
4
1 Let X ,X ,...,X be a sequence of i.i.d. random variables in Rd with distribution F,
1 2 n
0 and consider an empirical process based on this sequence:
.
1
0 vnF(B)=√n[Fn(B) F(B)],
−
6
1 where B is a Borel subset of Rd and
:
v n
1
Xi Fn(B)= n I{Xi∈B}
i=1
r X
a
is an empirical distribution. If sets B are chosen as unbounded rectangles ( ,x]=
−∞
( ,x ] ( ,x ], then we obtain more common form of empirical processes
1 d
in−d∞exedb×y p··o·in×ts−x∞Rd anddenoted v (x), but mostof the time we will be using the
nF
∈
function-parametric version of empirical process,
n
1
v (φ)= φ(x)v (dx)= [φ(X ) Eφ(X )], φ L (F).
nF nF i i 2
Rd √n − ∈
Z i=1
X
This is an electronic reprint of theoriginal article published by theISI/BS in Bernoulli,
2016, Vol. 22, No. 1, 563–588. This reprint differsfrom theoriginal in pagination and
typographic detail.
1350-7265 (cid:13)c 2016ISI/BS
2 E. Khmaladze
As we know (see, e.g., [30], Chapter 2), on properly restricted class of functions φ Φ,
∈
the empirical processes v converge to function-parametric Brownian bridge v . If the
nF F
distributionF isuniformon[0,1]d,thenv becomesastandardBrownianbridge,which
F
we denote u. We recall exact definitions in the next section.
In this paper, we show that from a certain type of transformation of v a unified
nF
approachtodistributionfreetestingofhypothesisaboutF isemerging.Theapproachcan
beusedregardlessofwhetherthehypothesisissimpleorparametric,orwhetherF isone-
dimensional or multidimensional, and also whether F is continuous or discrete. The last
pointisdemonstratedin[13]andalsoinCorollary5,Section3.1.Wealsobelievethatthe
approachis simple to implement:on-goingresearchshowsthat parametricfamilies with,
multidimensional parameters, as say, family with 9 parameters, in one of the examples
in [24], can be studied without noticeable numerical difficulties.
The structure of the transformation in question is the following: let K be a unitary
operator of a certain type, acting on L (F), and consider a transformed process
2
(K∗v )(φ)=v (Kφ). (1)
nF nF
TheexplicitdescriptionoftheoperatorsweproposetousewedefertoSection3,wherewe
showthat the processesso obtainedwill haveverydesirableasymptotic properties while
being one-to-one transformations of v and, therefore, containing the same amount
nF
of “statistical information”. As a preliminary illustration of one type of results of this
form, let us formulate the following proposition. It is a particular case of Theorem 2 of
Section 3.1.
Proposition 1. Suppose F is an absolutely continuous distribution on [0,1]d (different
from uniform distribution), which has a.e. positive density f. The process u= u(x),x
{ ∈
[0,1]d with the differential
}
1 1 f(x) 1
u(dx)= v (dx) − v (dy)dx (2)
F F
f(x) − 1− [0,1]pd f(y)dyZ[0,1]d f(y)
p R p p
is the standard Brownian bridge.
ForgoodnessoffittheoryonRd,thismeansthatwithhelpofasinglestochasticintegral
above,theasymptoticsituationoftestingasimplenullhypothesisF canbetransformed
into the situation of testing the uniform distribution. In other words, transformation
(2) fromempiricalprocess v possessesthe same conveniencefor asymptotic statistical
nF
inference as the uniform empirical process in [0,1]d.
As the first step toward (1), in Section 2 below we will find that there are many
more different Brownian bridges than is commonly realized. We will also see, within
the same framework, that although their distributions remain mutually singular, the
boundary between Brownian bridges and Brownian motion is somewhat blurred and
unitary operators can easily be used to transform Brownian bridges into a version of
bridges, which are “almost” Brownian motion. This is described in Section 3.2.
Unitary transformations of empirical processes 3
Letusnowbriefly outlinethe situationwithdistributionfreegoodnessoffittesting in
Rd.
If F is a continuous distribution in R, and u is the uniform empirical process, then,
n
since [19], we know that v can be transformed to u as
nF n
v (x)=u (t), t=F(x),
nF n
or, in function-parametric setting, for φ L ([0,1])
2
∈
v (Uφ)=u (φ),
nF n
where U∗φ(x)=φ(F(x)). It is good to note that this operator,from L ([0,1]) to L (F),
2 2
is also a unitary operator, that is,
(U∗φ)2(y)dF(y)= φ2(F(y))dF(y)= φ2(t)dt,
Zy∈R Zy∈R Zt∈[0,1]
although there is little tradition of using this terminology, because in this situation it
looks inconsequential.
An analog of time transformation t=F(x) exists in Rd as well and is called the
Rosenblatt transformation,[26]. In, say, three-dimensional space, in obvious notation, it
has the form t =F(x ),t =F(x x ),t =F(x x ,x ). For some reason, and maybe
1 1 2 2 1 3 3 1 2
| |
because dealing with conditional distributions is often awkward, the transformation is
rarely used. It also fails to lead to distribution free testing, when F depends on a finite-
dimensional parameter (cf. Section 3.3).
A unitary operator, very different in its nature from time transformation, was intro-
ducedfortheempiricalprocessesind-dimensionaltimein[16]and[17]and,intwo-sample
problem, in [6]. In its origin it is connected with the innovation problem for curves in
Hilbert spaces, [3], and the theory of innovation martingales;see, for example, [22], Sec-
tion7.4.Initssimplestform,itisanoperatorfrom = φ L (F): φ(x)dF(x)=0
LF { ∈ 2 Rd }
onto L (F) and the result was a one-to-one transformation from Brownianbridge v to
2 F
Brownianmotionw .ItextendstothecaseofparametrichypothesisRinRd.Wecomment
F
further on it in Section 3.2.
The approach of this paper seems to us closest to the geometric argument behind K.
Pearson’s chi-square statistic, [25]; see also retrospective historic account in [28]. The
idea itself is very simple and it is somewhat strange that it was not discovered before.
In the case of one given F, the operators involved will map in and subsequently
F G
L L
transform one Brownian bridge, v , into another Brownian bridge, v , with G of our
F G
choice. Just as Fisher [7] and [8] has extended chi-square theory to the parametric case,
our approachas we said, also extends to the case of parametric families; see Section 3.3.
Next, in Section 2, as we said, we present a somewhat broader definition of Brownian
bridges as projected Brownian motions. In Section 3, we present the main results. The
case of a simple hypothesis, which also serves as an illustration of the whole approach,
is treated in Section 3.1, the transformation to “almost” Brownian motion is shown
in Section 3.2, while the case of parametric hypotheses is considered in Section 3.3. In
4 E. Khmaladze
Section3.4,wediscusstheproblemofuniquenessoftheproposedtransformations.Inthe
lastSection4,weillustratetherateofconvergenceoftransformedempiricalprocessesto
theirdistributionfreelimitsthroughtherateofconvergenceofthetwoclassicalgoodness
offitstatisticsbasedontheseprocesses:Kolmogorov–Smirnovstatisticandomega-square
statistic.
2. Preliminaries: q-projected Brownian motions
Consider w (φ),φ L (F), a function parametric F-Brownianmotion, which is a linear
F 2
∈
functional in φ and for each φ is a Gaussian random variable with mean 0 and variance
Ew2(φ)= φ2(y)dF(y)= φ 2.
F Zy∈Rd k kF
This implies that the covariance between w (φ) and w (φ˜) is
F F
Ew (φ)w (φ˜)= φ(y)φ˜(y)dF(y)= φ,φ˜ .
F F F
Zy∈Rd h i
Asfaraswearenotconsideringtrajectoriesofw (φ)inφ,weneedonlytoknowthatφis
F
indeedsquareintegrablewithrespectto F.Forthetheoryofw (φ) aslinearfunctionals
F
on L (F) and reproducing kernel Hilbert spaces, where they live, we refer, for example,
2
to [10] or the monograph[21].
Letv (φ)denotethefunction-parametricF-Brownianbridge,definedasalineartrans-
F
formation of w :
F
v (φ)=w (φ) φ,q w (q ). (3)
F F 0 F F 0
−h i
Here, we used q for the function identically equal to 1. This transformation has a par-
0
ticular structure, which is important for what follows. Namely, we have the following
lemma.
Lemma 1. Equality (3) represents v as an orthogonal projection of w parallel to the
F F
function q .
0
Thisstatementwasinitiallyprovedasearlyas[14].Weshowitsproofhereforreaders’
convenience.
Proof of Lemma 1. Toshortennotation,denotetheright-handsideof(3)byΠw (φ),
F
so that (3) takes the form v (φ)=Πw (φ),φ L (F). Then it is easy to see that
F F 2
∈
ΠΠw =Πv =v ,
F F F
or Π2=Π, so that Π is indeed a projector. Besides, Πw (q )=v (q )=0, which, in
F 0 F 0
usualterminology(see, e.g., [9], Section 1.10,and [21]), means that the linear functional
v () and the function q are orthogonal. (cid:3)
F 0
·
Unitary transformations of empirical processes 5
Substituting the indicator function φ=I , from (3) we obtain
(−∞,x]
v (x)=w (x) F(x)w ( ), (4)
F F F
− ∞
whichrepresentstrajectoriesof v (x) as projectionoftrajectoriesof w .It alsoleadsto
F F
thedefinitionofv astheGaussianprocessinxwithmean0andvarianceF(x) F2(x)
F
−
(or covariance F(min(x,x′)) F(x)F(x′)).
−
We can now replace q with any other function q of unit L (F)-norm. This will lead
0 2
to the process
vq(φ)=w (φ) φ,q w (q), (5)
F F −h iF F
which certainly is again a projection of w parallel to q and, therefore, also could be
F
called Brownian bridge. However, it does not satisfy the second definition of a bridge.
This is more visible in point-parametric version
vq(x)=w (x) q(y)dF(y) q(y)w (dy) (6)
F F −Zy≤x Zy∈Rd F
and the variance of vq is of a different form:
F
2
E[vq(x)]2=F(x) q(y)dF(y) , (7)
F −
(cid:20)Zy≤x (cid:21)
so that if q =q , the second term is not square of the first. Therefore, even in one-
0
6
dimensionalcase,withF beingjustuniformdistributiononinterval[0,1],thedistribution
of max vq(x) is not Kolmogorovdistribution and the distribution of 1[vq(x)]2dF(x)
is not oxm| eFga-s|quare distribution unless q=q F-a.e. We call vq(x) a0sligFhtly longish
name of a q-projected F-Brownian motion. T0he processes vq aFrise nRaturally as weak
F
limits in certain statistical problems and they will be useful in this paper.
We stress again, that the definition of vq involves two objects – a distribution F and
F
a function q L (F). When F is uniform distribution on [0,1]d we call vq a q-projected
∈ 2 F
standard Brownian motion (or simply q-projected Brownian motion) and use, most of
the time, notation vq without index F. In the case of general F, we would still call vq0
F
a Brownian bridge and often omit q from notation. Obviously vq0 is just a standard
0
Brownianbridge u.We formulate the lemma below for convenienceofreference lateron.
Lemma 2. Suppose distribution F is supported on the unit cube [0,1]d and has a.e.
positive density f. Suppose w is standard Brownian motion on [0,1]d and v is defined
F
as in (3) and (4). Then
1
ξ(x)= v (dy)
F
f(y)
Zy≤x
is q-projected standard Brownian motion wipth q=√f,
ξ(x)=w(x) f(y)dy f(y)w(dy), (8)
−Zy≤x Zy∈[0,1]d
p p
6 E. Khmaladze
or, for ψ L ([0,1]d),
2
∈
ξ(ψ)=w(ψ) ψ, f w( f). (9)
−h i
Conversely, if ξ is q-projected standard Brownipan motpion, then
v (x)= q(y)ξ(dy)
F
Zy≤x
is F-Brownian bridge, as defined in (4), with F(x)= q2(y)dy.
y≤x
R
Proof. The first statement of the lemma follows from the connection (4) between v
F
and w . Indeed, substitute the normalized differential of v ,
F F
1 1
v (dy)= w (dy) f(y)w (Rd)dy,
F F F
√f(y) √f(y) −
p
in the definition of ξ(x) to obtain
1
ξ(x)= w (dy) f(y)dyw ( )
F F
f(y) − ∞
Zy≤x Zy≤x
p
p
and note that
1
w(x)= w (dy)
F
√f(y)
Zy≤x
is the standard Brownian motion – it obviously is 0-mean Gaussian process with inde-
pendent increments and
2
1 1
E w (dy) = F(dy)=x.
F
√f(y) f(y)
(cid:20)Zy≤x (cid:21) Zy≤x
Note also that we can write w ( ) as
F
∞
f(y)w(dy).
(cid:3)
Zy∈[0,1]d
p
Remark. Note, that the normalization v (dy) by √f(y) does not help to standardize
F
ξ(dx) –in(9)westillhavelinearfunctional ψ,√f ,andthus,thedependence onF inξ
h i
is stillpresent.This waswellunderstoodfor a verylong time,andit is quite unexpected
thatusingoneextrastochasticintegral(seeProposition1),thestandardizationbecomes
possible.
Thenormalizationby1/√f usedinthelemmaisaparticularformofthemoregeneral
mapping. Namely, let G be another distribution on Rd, which is absolutely continuous
Unitary transformations of empirical processes 7
with respect to F. Then the function
dG
l(x)= (x) (10)
dF
r
belongs to L (F). and moreover, if ψ L (G), then lψ L (F) and ψ = lψ . If
2 2 2 G F
∈ ∈ k k k k
distributions G and F are equivalent (mutually absolutely continuous), then the inverse
is also true: if φ L (F) then φ/l L (G), and the norm is preserved. This, in par-
2 2
∈ ∈
ticular, means that re-normalization of F-Brownian motion into G-Brownian motion is
straightforward:if w (φ) is an F-Brownian motion in φ L (F), then w (lψ)=w (ψ)
F 2 F G
is a G-Brownianmotion in ψ L (G). This, we repeat, d∈oes not extend to vq and v –
∈ 2 F F
thedistributionof,say,v (lψ)depends onbothF andG.ThefirsttheoreminSection3
F
below shows, however, that a simple isomorphism exists.
To describe one more object we consider in this paper, complement q by a sequence
0
of orthonormal functions q ,...,q , which are also orthogonal to q , and consider the
1 κ 0
process
κ
vˆ (φ)=w (φ) q ,φ w (q ).
F F i F F i
− h i
i=0
X
Similar to what we said about v , the process vˆ is the orthogonal projection of w
F F F
parallelto the functions q ,...,q . We still call vˆ a q-projected F-Brownianmotion. It
0 κ F
maybe thatnotation vˆq is usedagain,but when q is avectorfunction,there is noother
F
“more traditional” notion to be confused with vˆ ; so we skip q as an upper index.
F
The role of the process vˆ becomes clear when we examine asymptotic behavior of
F
the parametricempiricalprocess.Considerthe problemoftesting parametrichypothesis
that the distribution function of X s belongs to a given family of distribution functions
i
F (x), depending on a finite-dimensional parameter θ. The value of this parameter is
θ
not prescribed by the hypothesis and has to be estimated using the sample X ,...,X .
1 n
Denote
v (B,θˆ )=√n[F (B) F (B)]
n n n − θˆn
the parametric empirical process (indexed by sets). (Note that, in presence of θ and θˆ,
one can skip index F in notation.) As has been known since Kac et al. [12] and later
Durbin[5]andotherwork,theasymptoticbehaviorofempiricalprocesseswithestimated
parametersisdifferentfromthatofv ,andinparticular,its limit distributiondepends
nF
not only on the true value of the parameter but also on the score function. However,we
can say more.
Namely,underusualandmildassumptions(see,e.g.,[2],Chapter3,andseethemodern
exposition in [29], Section 5), the MLE θˆ possesses an asymptotic representation
n
f˙ (x)
√n(θˆ θ)=Γ−1 θ v (dx,θ)+o (1), n ,
n− θ Zx∈Rd fθ(x) nF P →∞
8 E. Khmaladze
where we denote by f and f˙ the hypothetical density and the vector of its derivative
θ θ
with respect to parameter θ and denote
f˙ (y)f˙T(y)
Γ = θ θ dy
θ
Zy∈Rd fθ(y)
the Fisher information matrix. Consequently, the parametric empirical process has
asymptotic expansion
f˙T(y) f˙ (y)
v (B,θˆ )=v (B,θ) θ F (dy)Γ−1 θ v (dy,θ)+o (1),
n n n −ZB fθ(y) θ θ Zy∈Rd fθ(y) n P
(11)
=v (B,θ) βT(y)F (dy) β (y)v (dy,θ)+o (1),
n −ZB F θ Zy∈Rd F n P
where
f˙ (x)
β (x)=Γ−1/2 θ .
F θ f (x)
θ
As shown in [14] (see also [18], Section 2.2), the main part of this expansion represents
v (,θˆ ) as the orthogonal projection of v (,θ) parallel to the normalized score func-
n n n
· ·
tion β and, therefore, the limit in distribution of v (,θˆ ) can be written (in function-
F n n
·
parametric form) as
v (φ) βT,φ v (β ).
F −h F iF F F
At the same time, the score function β is orthogonal to the function q and its coor-
F 0
dinates are orthonormal and will play the role of functions q ,...,q above. Therefore,
1 κ
substituting representation (3) of v through w , we see that the limit in distribution
F F
of the process v (,θˆ ) is vˆ .
N n F
·
Itiswellknownthattheactualweakconvergencestatementinfunction-parametricset-
uprequiressomerestrictiononthe underlying classoffunctions φ,butthese restrictions
are well understood and we refer readers to [30]. For an earlier proof in Skorohodspace,
see [20] and [5], while for the proof in L (F) see [14].
2
3. The main result and its corollaries
The main geometric idea in this paper can be described as follows. When testing for
fixed distribution F, the correspondingempiricalprocesses will convergeto v , which is
F
an orthogonal projection of the Brownian motion. When testing for a different G, there
will be convergence to v , which is also an orthogonal projections of Brownian motion.
G
However,wewillseethatifGandF areequivalent;theseprojectionscanbe“rotated”to
eachother.Theunitaryoperatorsinvolvedinthisrotationformagroup,transientonthe
classofallBrownianbridgeswithall Gequivalenttothe F.Inotherwords,theproblem
of testing F can be mapped to the problem of testing G and vice versa, and these,
Unitary transformations of empirical processes 9
seemingly distinct problems are not distinct problems, but form one equivalence class.
Therefore,onerepresentativeofeachequivalenceclassissufficient,andweproposeaform
ofsuchrepresentative.Since the processes vˆ and vˆ areboth orthogonalprojectionsas
F G
well,theideaofunitarytransformationextendstotheparametricclassesofdistributions.
3.1. The case of fixed F
AlthoughthefollowingTheorem2isgeneralizedbyTheorem7,bystartingwiththecase
ofonefixedF andgivinganindependentproofwehopetomaketheoverallpresentation
more transparent.
Consider an operator on L (F)
2
2
K=I (l q ) l q , , (12)
− l q 2 − 0 h − 0 ·iF
k − 0kF
where I is identity operator and l is the function defined in (10), while the function q
0
identically equals 1. Below we will also need the linear subspace = (q ,l), generated
0
L L
by functions q and l and functions l and q , which are parts of l and q , orthogonal
0 ⊥ 0,⊥ 0
to q and l, respectively,
0
l =l l,q q , q =q l,q l.
⊥ 0 F 0 0⊥ 0 0 F
−h i −h i
It is clear that
1 1
q , l and l, q
0 0⊥
l q
(cid:18) k ⊥k (cid:19) (cid:18) k 0⊥k (cid:19)
form two orthonormalbases of .
L
The operator K has the following properties.
Lemma 3. (i) Operator K is a (self-adjoint) unitary operator on L (F), Kφ =
2 F
k k
φ , such that
F
k k
Kφ=φ, ifφ l,q and Kl=q , Kq =l , while Kq =l.
0 0 0⊥ ⊥ 0
⊥
(ii) Coordinate representation of this operator is
K=I +q l, +l q , ,
L⊥ 0 F ⊥ 0⊥ F
h ·i h ·i
where I is the projection operator on the subspace of L (F) orthogonal to .
L⊥ 2
L
The reader can easily verify the lemma. Part (i) is needed just below, part (ii) will
be useful to draw similarity with Section 3.3. below. Note that one could use a similar
unitary operator,with l q replacedby l+q . We chose the present formonly because
0 0
−
the norm l q isa well-knownobject–the Hellinger distancebetweendistributions
0 F
k − k
10 E. Khmaladze
F and G. To whatextentthe choice of K is unique is discussedinSection 3.4.Note also
that
l q 2 =2 (1 l(y))dF(y)= 2 l q ,q . (13)
k − 0kF Zy∈Rd − − h − 0 0iF
Theorem 2. Suppose distribution G is absolutely continuous with respect to distribution
F (and different from F). If v is F-Brownian bridge, then the process with differential
F
v (dx)=l(x)v (dx)
G F
(14)
1
l(y)v (dy) [l2(x) l(x)]f(x)dx
F
−Zy∈Rd 1− y∈Rdl(y)dF(y) −
R
is G-Brownian bridge.
Ifdistributions G and F are equivalent, that is, if l= dG/dF is positive F-a.e., then
(14) is one-to-one.
p
If F is an absolutely continuous distribution on the unit cube [0,1]d and its density f
is positive a.e., while G is uniform on this cube, then l(x)=1/ f(x) and we obtainthe
transformation of F-Brownian bridge into the standard Brownian bridge, already given
p
in Proposition1.
Remark. It was interesting to realize that v in (14) remains G-Brownian bridge even
G
if dG/dF can be 0 on a set of positive probability F.
Proof of Theorem 2. As we know, for any function ψ L (G), under our conditions,
2
∈
lψ L (F). Since v (l q )=v (l), the function-parametric form of (14) is
2 F 0 F
∈ −
v (ψ)=v (Kφ), with φ=lψ.
G F
We need to show that the covariance operator of v is that of G-Brownian bridge. For
G
this it is sufficient to consider the variance of v (ψ),
G
E[v (ψ)]2=E[v (Kφ)]2= Kφ 2 [ Kφ,q ]2.
G F k kF − h 0iF
However,
Kφ 2 = φ 2 = ψ 2,
k kF k kF k kG
and, using (13), we obtain
2
Kφ,q = φ,q l q ,q l q ,φ = φ,l
h 0iF h oiF − l q 2 h − 0 0iFh − 0 iF h iF
k − 0kF
= lψ,l = ψ,q .
F 0 G
h i h i
