Noise Stability is computable and low dimensional

0.43 MB·

by  Anindya De

Noise Stability is Computable and approximately
Low-Dimensional
Anindya De Elchanan Mossel Joe Neeman
7 Northwestern University MIT UTAustin
1
[email protected] [email protected] [email protected]
0
2
February 17,2017
b
e
F

5 Abstract
1
Questions of noise stability playanimportant role in hardnessof approximationin computer

] science as well as in the theory of voting. In many applications, the goal is to find an optimizer
R
of noise stability among all possible partitions of Rn for n 1 to k parts with given Gaussian
P ≥
. measuresµ1,...,µk. Wecallapartitionǫ-optimal,ifitsnoisestabilityisoptimaluptoanadditive
h ǫ.Inthispaper,wegiveanexplicit,computablefunctionn(ǫ)suchthatanǫ-optimalpartitionexists
at in Rn(ǫ). This resulthasimplications for the computabilityof certainproblemsin non-interactive
m simulation,whichareaddressedinasubsequentwork.
[

2
v
3
8
4
1
0
.
1
0
7
1
:
v
i
X
r
a
1
1 Introduction
IsoperimetricTheoryandNoiseStability. Isoperimetricproblemshavebeenstudiedinmathemat-
icssinceantiquity. ThesolutiontotheisoperimetricprobleminthetwodimensionalEuclideanplane
wasknowninancientGreece. Thestudyofisoperimetricproblemsiscentralinmodernmathematics
andtheoreticalcomputerscience. Somecentralexamplesincludethestudyofexpandersandmixing
ofMarkovchains.
Our interest in this work is in a central modern isoperimetric problem, i.e, the problem of noise
stability. This problem, originally studied by Borell in relation to the study of the heat equation in
mathematical physics [4], has emerged as a central problem in theoretical computer science [17], as
wellasincombinatoricsandinvotingtheory [15]asweelaboratebelow.
The fundamental question in this area is to find a partition of Gaussian space (with prescribed
measures)whichmaximizesthenoisestabilityofthepartition. WeequipRn withthestandardGaus-
sian measure, denoted by γ . The Ornstein-Uhlenbeck operator P is defined for t [0, ) and
n t
∈ ∞
f :Rn Rby
→
(P f)(x) = f(e t x+ 1 e 2t y)dγ (y).
t − − n
· − ·
Zy Rn
∈ p
TheresultingnotionofnoisestabilityisdefinedbyStab (f):= E[fP f],wheretheexpectationiswith
t t
respecttotheγ . Iff denotestheindicatorfunctionofaset(callit ),thenStab (f)istheprobability
n f t
A
that two e t-correlated Gaussians x,y both fall in . In the limit t 0, noise stability captures the
− f
A →
Gaussiansurfaceareaoftheset [19]. Inparticular,wehavethefollowingrelationforanyset:
f
A
√2π
GAS( ) = lim E[f (f P f)],
Af t→0 arccos(e−t) · · − t
whereGAS( )denotestheGaussiansurfaceareaoftheset .
f f
A A
Halfspacesaremoststablesets. Forbothnoisestabilityandsurfacearea,halfspacesareknowntobe
theoptimalsets[3,27,4,2]. Wewillstatethisfactinaslightlyconvolutedway,inordertomoreeasily
generalizeit. Let∆ denotetheprobabilitysimplexinRk (i.e. theconvexhullformedbythestandard
k
unit vector e ,...,e ). Any function with range [k] naturally embeds in ∆ by identifying i [k]
1 k k
{ } ∈
with e . Moreover, we extend the Ornstein-Uhlenbeck operator to act on vector valued functions
i
in the obvious way: if f = (f ,...,f ) : Rn Rk then P f = (P f ,...,P f ). Finally, say that
1 k t t 1 t k
→
f = (f ,f ) :Rn ∆ isahalfspace ifthereexista,b Rn suchthatf (x) = 1 .
1 2 2 1 x a,b 0
→ ∈ {h − i≤ }
Theorem1(Borell). Foranyg : Rn ∆ ,ahalfspace f :Rn ∆ withE[f]= E[g]satisfies
2 2
→ →
E[ f,P f ] E[ g,P g ].
t t
h i ≥ h i
Theorem 1 has many applications in computational complexity. Most famously, it can be com-
binedwithan invarianceprinciple [20]inordertoproveanumberoftighthardness-of-approximation
resultsundertheuniquegamesconjecture[16].
Moreparts? Itisstraightforwardtoextendthenotionofnoisestabilitytopartitionswithmanyparts.
Namely,apartitionofRn canbedescribedbyf :Rn [k]. Identifying[k]with e ,...,e asabove,
1 k
→ { }
we define the noise stability of such a partition by E[ f,P f ]. One may then ask for an analogue of
t
h i
Theorem1,butwhere∆ isreplacedby∆ forsomek 3.
2 k
≥
Eventhethree-partversionofTheorem1turnsouttoberatherharderthanthetwo-partone. We
will try to explain why, by analogy with isoperimetric problems. First of all, the Euclidean, isoperi-
metricanaloguefork = 3isknownasthe“doublebubble”problem. Thewell-known“doublebubble
2
conjecture” states that the minimum total surface area of two bodies separating and enclosing two
given (Lebesgue) volumes is achieved by two spheres meeting at 120 . After being open for more
◦
thanacentury,thisproblemwassettledratherrecentlyinR2,R3,andR4[12,13,26]. FortheGaussian
space, [6] showedthat for somesmall but positive constant c > 0, theGaussian surface area ofthree
partitions is minimized by the “standard simplex partition” as long as the measures of all the three
partsiswithin1/3 c.
±
Sincehalfspacesbothmaximize noisestabilityandminimizeGaussiansurfacearea,andsincethe
standardsimplexpartitionisknowntominimize multi-part Gaussiansurfaceareaincertaincases,it
seems natural to guess that the standard simplex partition also maximizes multi-part noise stability.
Thiswasexplicitlyconjecturedin[16],inthespecialcasethatallofthepartsinthepartitionhaveequal
measure. However,asomewhatsurprising(at leasttotheauthors)recentwork[10]showedthatthe
standardsimplexpartitionfailstomaximizemulti-partnoisestabilityunlessallofthepartshaveequal
measure. Ontheotherhand,thereisalsosomesupportfor theconjecturein theequal-measurecase:
Heilman[11]showedthattheconjectureistrueifthedimensionisboundedbyafunctionoft.
Approximatenoisestabilityofmultipartitions? Inlightoftheuncertaintyaboutoptimalpartition-
ing for k 3, onecan ask amore modestquestion. Given k 3, t > 0, and prescribed measuresfor
≥ ≥
the k parts, let α be the optimal noise stability that can be obtained in Rn under these constraints.
n
SincetheGaussianmeasureisaproductmeasure,α isclearlynon-increasinginn. Sinceitisbounded
n
belowbyzero,ithasalimitasn .
→ ∞
Our main result is that there is an explicitly computable n = n (k,t,ǫ) such that α α ǫ
0 0 n0 ≥ m −
for all m N. Although the bound on n that we give is not particularly good, the key point is that
0
∈
itisexplicitlycomputable. Asaconsequence,uptoerrorǫ,thenoisestablepartitionisalsoexplicitly
computable. Weconcludetheintroductionbyanopenquestion:
Question2. Doesthereexistsn suchthatα = α forn > n ?
0 n0 n 0
Ourcurrenttechniquesarenotsuitableforaddressingthequestionabove.
1.1 Acknowledgments
E.M.acknowledgesthesupportofgrantN00014-16-1-2227 fromOfficeofNavalResearchandofNSF
award CCF 1320105. A.D. acknowledgesthe supportof a start-upgrant from NorthwesternUniver-
sity. The authors started this research at the workshop on “Information theory and concentration
phenomena”attheIMA,Minneapolisin2015. Theauthorswouldliketothanktheorganizersofthis
workshopandtheIMAfortheirhospitality.
2 Main theorem and overview of proof technique
In orderto statethemain theorem,we firstneedto recall thenotion ofa polynomial thresholdfunc-
tion. A function f : Rn 0,1 is said to be a degree-dPTF if there existsa polynomial p : Rn R
→ { } →
of degree d such that f(x) = 1 if and only if p(x) > 0. We will need a k-ary generalization of this
definition. WenotethatthereareseveralpossiblewaystogeneralizethenotionofPTFstok-aryPTFs
andourparticularchoiceisdictatedbytheconvenienceofusingtherelevantresultsfrom[7].
Definition1. Afunctionf :Rn [k]issaidtobeamultivariatePTFisthereexistspolynomialsp ,...,p :
1 k
→
Rn Rsuchthat
→
j ifp (x) >0 andfori = j, p (x) 0
j i
f(x) = 6 ≤
(1 otherwise
3
Inthiscase,wedenotef = PTF(p ,...,p ). Further,f issaidtobeadegree-d multivariate PTFifp ,...,p
1 k 1 k
aredegree dpolynomials.
We now state the main theoremof this paper. We set the convention, that unless explicitly men-
tioned otherwise, the underlying distribution is γ , the standard n-dimensional Gaussian measure.
n
Likewise, given any random variable X over Rk, E[X] denotes its (vector-valued) expectation and
Var(X)denotesitscovariancematrix.
Theorem 3. Let f : Rn [k] such that E[f] = µ Rk. Then, given any t > 0,ǫ > 0, there exists an
→ ∈
explicitly computable n = n (t,k,ǫ) andd = d(t,k,ǫ) suchthat there is adegree-d PTFg : Rn0 [k]such
0 0
→
that
1. E[f] E[g] ǫ.
1
k − k ≤
2. E[ g,P g ] E[ f,P f ] ǫ.
t t
h i ≥ h i −
Notethattheabovetheoremautomaticallyimpliesthatafunctiongsatisfyingtheaboveproperties
can be explicitly computed (up to some additional error ǫ). This is because the set of degree-d PTFs
onRn0 admitsafinitesizedexplicitlyenumerableǫ-cover.
ProofSketch TheproofofTheorem3consistsofthefollowingmainsteps:
From general partitions to PTF. The first step in the proof is to show that given any f : Rn [k],
→
there is a multivariate PTF g : Rn [k] which meets the two criteria in Theorem 3 and has degree
′
→
d = d(t,k,ǫ), for some explicit function d(t,k,ǫ). In other words, g satisfies E[f] E[g ] ǫ and
′ ′ 1
k − k ≤
Stab (g ) Stab (f ) ǫ. This is done in Section 5. Note that main difference between the desired
t ′ t ′
≥ −
conclusionofTheorem3andwhatisaccomplishedinthisstepisthattheambientdimensionremains
nasopposedtoaboundeddimensionn .
0
Why is this true? The basic intuition is that if f is noise stable then it should have most of its
Hermiteexpansionweightatlowdegree. Thereforeweshouldbeabletoreplacef withthePTFwhere
thepolynomialisthetruncatedexpansionoff. Thereare anumber ofchallengesin formalizing this
intuition: 1. Wecannotrule outthatapositivefraction oftheweightoff isat highdegrees(perhaps
aslargeasn). 2. ItisnotclearthatthePTFobtainedthiswayisnoisestablenorthat3. Ithastheright
expectedvalue.
Our analysis proceeds as follows. We would like to construct g from f by “rounding” P f for
′ t
some small t. The advantage of P f over f is that P f is guaranteed to have decaying tails. The
t t
rounding of P f can be performed given some a Rn by considering the function g : Rn [k]
t a
∈ →
which takesthevalue i wheneveriis thelargest coordinate ofP f a. Itis nothard toprove that it
t
−
ispossibletochooseasuchthatE[g ] = E[f];moreover,onecanshowthatthisfunctiong hasbetter
a a
noise stability than f does. The main obstacle is that the function g is not a PTF. Unfortunately the
a
Hermite decay of P f does not translate to Hermite decay of g . Instead we use smoothed analysis
t a
toshowthatformosta’s,g hasHermitedecayand can thereforebewellapproximatedbyPTF.The
a
smoothedanalysisargumentusestheco-areaformulaandgradientboundsanddrawsonideasfrom
[21,18].
From PTF in dimension n to a small PTF of bounded degree polynomials Given the function
g : Rn [k] of degree d = d(t,k,ǫ), our next goal is show it is possible to obtain a PTF g on some
′
→
n = n (t,k,ǫ) variables such that (i) E[g] E[g ] ǫ and (ii) g,P g g ,P g ǫ. This part
0 0 ′ 1 t ′ t ′
k − k ≤ |h i−h i| ≤
builds on and extends the theory and results of [7]. The key notion introduced in [7] is that of an
eigenregular polynomial. Namely,apolynomialis saidtobe δ-eigenregularifforthecanonical tensor
4
associatedwiththepolynomial, theratioofthemaximumsingularvalue toitsFrobeniusnormis
p
A
atmostδ (thetensornotionsareexplicitlydefinedlater).
Thekeyadvantageofthisdefinitionisthatasshownin[7],whenδ 0,thedistributionofp(un-
→
derγ )convergestoanormal. Inotherwords,eigenregularpolynomialsobeyacentrallimittheorem.
n
In fact, given k polynomials p ,...,p which are δ-eigenregular, they also obey a multidimensional
1 k
centrallimittheorem.
The regularity lemma from [7] implies that the polynomials p ,...,p can be jointly expressed as
1 k
bounded(intermsoft,kandǫ)sizepolynomialsineigenregularhomogenouspolynomials In(p )
s,q,ℓ
{ }
where 1 s k, 1 q d and 1 ℓ num(s,q). In other words, we may write p =
s
≤ ≤ ≤ ≤ ≤ ≤
Out(p )( In(p ) ), wherenum(s) = d num(s,q)isboundedintermsoft,k andǫ
s { s,q,ℓ }q∈[d],ℓ∈[num(s,q)] q=1
andalltheInnerpolynomialsareδ-eigenregular.
P
[7] used the statement above to conclude that the joint distribution of p ,...,p can be approxi-
1 k
matedinaboundeddimensionaswecanreplaceeachoftheinnerpolynomialsbyaonedimensional
Gaussian. Forourapplicationthingsaremoredelicate,aswearenotonlyinterestedinthejointdistri-
butionofp ,...,p butalsointhenoisestabilityofp ,...,p . Forthisreasonitisimportantforusto
1 k 1 k
maintain the degreesof the inner polynomials (each of which is homogenous)and not replace them
withGaussians.
Asmall PTFrepresentation In thefinal stepoftheproof, wemaintain Out(p )and showhowthat
s
polynomials In(p ) canbereplacedbyacollectionofpolynomials In(r ) inboundeddimen-
s,q,ℓ s,q,ℓ
{ } { }
sions(in t,k andǫ)thuscompletingtheproof. Thefact thatacollection ofhomogenouspolynomials
can be replaced by polynomials in bounded dimensions is a tensor analogue of the fact that for any
k vectors in Rn, there exist k vectors in Rk with the same matrix of inner products. Once such poly-
nomials are found, it is not hard to construct eigenregular polynomials from them by averaging the
polynomialsoverindependentcopiesofrandomvariables.
3 Applications
GiventhewideapplicabilityofBorell’sisoperimetricresulttocombinatoricsandtheoreticalcomputer
science,webelievethatTheorem3willalsobewidelyapplicable. Wewillnowpointoutsomeappli-
cations of this theorem. First, by combining Theorem3 with the invariance principle [20], we derive
aweakk-aryanalogueof“MajorityisStablest”. TheanalogueoftheOrnstein-Uhlenbeckoperatoris
theso-calledBonami-Beckneroperator definedasfollows: forρ [ 1,1]andx 1,1 n,let (x)be
ρ
∈ − ∈ {− } D
the product distribution over 1,1 n such that for y (x), for all 1 i n, E[x y ] = ρ. Then,
ρ i i
{− } ∼ D ≤ ≤
forany f : 1,1 n Rk, T f(x) = E [f(y)]. Likewise,for anyi [n]and z 1,1 n 1,let
{− } → ρ y∼Dρ(x) ∈ ∈ {− } −
f : 1,1 Rk denotethefunction obtained by restricting all but theith coordinate toz. Then,
z, i
Var−(f {−)= E} →[ f E [f ] 2]. Definetheinfluenceoftheith coordinateonf by
z,−i z k z,−i− z z,−i k2
Inf (f)= E [Var(f )].
i z 1,1 n−1 z, i
∈{− } −
Theorem 4. Given any k N and ρ [0,1],ǫ > 0, n = n (k,ǫ,ρ),C = C(k,ǫ,ρ) (which is explicitly
0 0
∈ ∈ ∃
computable) suchthat thefollowing holds: Foranyµ = (µ ,...,µ ) ∆ andn n , there isan explicitly
1 k k 0
∈ ≥
cCo/m√pnutaanbldefgor=angyµ,fρ,ǫ::{−11,1,1}nn →[k[k]]susuchchththatatmPraxiInfi(g)[f≤(xC)/=√in]anµd|Prxǫ∈a{n−d1,1m}na0x[gI(nxf)(=f)i]−ǫµ,i|≤
x 1,1 n i i i
{− } → | ∈{− } − | ≤ ≤
E[ f,T f ] E[ g,T g ]+2kǫ.
ρ ρ
h i ≤ h i
Theproofofthetheorem,whichisomitted,isbynowastandardreductionfromaGaussiannoise
stability resulttoadiscreteone,wherein onedirectiontheinvariance principle immediatelybounds
5
thediscretestabilitybytheGaussianstability. Intheotherdirection,startingforanǫ-optimalGaussian
partition,eachGaussianisreplacedbyanormalizedsumofindependentvariablestoobtainadiscrete
partition. Thesameresultholdsforotherdomains,forexampleforf,g : [k]n [k]. Inparticular, the
→
casewhere(µ ,...,µ ) = (1/k,...,1/k)impliesthatinatiedelectionsbetweenkalternatives,wecan
1 k
findanǫ-optimallyrobustnoisestablevotingrule,wherethestabilityiswithrespectofeachcandidate
randomizingtheirvoteindependentlywithprobability1 ρ.
−
3.1 Relationship toroundingof SDPs
To the best of our knowledge our results are independent of the the results of Raghavendra and
Steurer [25] who showed that for any CSP, there is an a rounding algorithm that is optimal up to
ǫ, whose running time is polynomial in the instance size and doubly exponential in 1/ǫ. It is natu-
ral to suspect that the two results are related, since there are well-known connections between SDP
roundingandGaussiannoisestability.
However, the usual analysis relating rounding to Gaussian noise stability seems to require that
halfspaces maximize noise stability for all possible values of the noise. In our results, this property
doesnotnecessarilyhold.
In the other direction, it is tempting to try to cast the noise stability problem as an optimization
problemonGaussiangraphsandthenapplytheresultsofSteurerandRaghavendratoobtainexplicit
boundsonthedimensionwhereanalmostoptimalsolutioncan beachieved. Itishardtoimplement
thisapproachfortworeasons: first,wedonotknowtheSDPsolutionfortheGaussiangraph;second,
we are interested in the optimal solution and it is not clear what is the relation between the best
integralsolutionandtheSDPsolutionfortheGaussiangraph.
Whilewedonotseehowtoformallyrelatethetwoworks,connectingthetwo(ifpossible)would
surelyyieldimportantinsights.
3.2 Non-interactive correlation distillation
Next,wetalkaboutabasicproblemininformationtheoryandcommunicationcomplexitywhichwas
recentlyconsideredintheworkofGhazi,KamathandSudan[9]. Lettherebetwonon-communicating
playersAlice andBobwhohave accesstoindependentsamplesfromajointdistributionP = (X,Y)
ontheset . Inotherwords,Alice(resp. Bob)haveaccessto(x ,x ,...,)(resp. (y ,y ,...))such
1 2 1 2
A×B
thatx ,y andforeachi N,(x ,y )isdistributedaccordingtoP,andtherandomvariables
i i i i
∈ A ∈ B ∈
(x ,y ) n aremutuallyindependent. Letµ = (µ ,...,µ ) ∆ andν = (ν ,...,ν ) ∆ . Whatis
{ i i }i=1 1 k ∈ k 1 k ∈ k
themaximumκ [0,1] suchthatAliceandBobcannon-interactivelyjointlysampleadistributionQ
∈
on[k] [k]such thatthedistributionofthemarginals ofAlice and Bobare µand ν respectivelyand
×
theysamplethesameoutputwithprobabilityκ?
To formulate this problem more precisely, we introduce some notation. Let Xn = (X ,...,X )
1 n
and Yn = (Y ,...,Y ), where (X ,Y ) are independently drawn from P. Now, note that a non-
1 n i i
interactiveprotocolforAliceandBobisequivalenttoapair(f,g)wheref : n [k]andg : n [k]
A → B →
(for somen N). In this terminology,thequestionnowbecomesthe following: given µ, ν, dothere
∈
existnandf : n [k]andg : n [k]suchthatf(Xn) µ,g(Yn) ν,andPr(f(Xn) = g(Yn)) =
A → B → ∼ ∼
κ?
Before we state the main result of [9] and our extension, we consider a motivating example. Let
= = Randlet P = (X,Y) betwoρ-correlatedstandardGaussians. Letk = 2andµ = ν. Then,
A B
Borell’sisoperimetrictheorem(Theorem1)statesthatthemaximumachievableκisgivenby
κ = Pr[f(X) = f(Y)],
X,Y
6
where f : R ∆ is a halfspace with measure µ. Thus, in the above case, n = 1 suffices and f = g
2
→
is the halfspace whose measure is µ. We now state the main result of [9]. For the result below, for
probabilitydistributionP,welet P denotethesizeofsomestandardencodingofP.
| |
Theorem 5. [Ghazi-Kamath-Sudan] Let ( ,P) be a probability space, and let Xn and Yn be as in
A × B
Theorem 5. There is an algorithm running in time OP,δ(1) such that given µ and ν in ∆2 and a parameter
| |
κ [0,1], itdistinguishes betweenthefollowingtwocases:
∈
1. There exist n N and f : n 0,1 and g : n 0,1 such that f(Xn) µ, g(Yn) ν, and
∈ A → { } B → { } ∼ ∼
Pr(f(Xn) = g(Yn)) κ δ. Inthiscase,thereisanexplicitn = n (P,δ) suchthatwemaychoose
0 0
≥ − | |
n n . Further,thefunctionsf andg areexplicitly computable.
0
≤
2. For any n N and f : n 0,1 and g : n 0,1 , if g : n 0,1 satisfy f(Xn) µ and
∈ A → { } B → { } B → { } ∼
g(Yn) ν thenPr(f(Xn)= g(Yn)) κ 8δ.
∼ ≤ −
Inotherwords,theabovetheoremstatesthatthereisanalgorithmwhich,givenP,targetmarginals
µ,ν, andcorrelation κ,can distinguishbetweentwocases: (a)Inthefirstcase,itispossibleforAlice
and Bobtonon-interactively simulate adistributionRwhich has thecorrectmarginals and achieves
thecorrelationκuptoδ. Inthiscase,thereisanexplicitboundonthenumberofcopiesofPrequired
and the algorithm also outputs the functions f,g used for the non-interactive simulation. (b) In the
second case, no non-interactive protocol between Alice and Bob can simulate a distribution R such
thatithasthecorrectmarginalsandtargetcorrelationtoerroratmost8δ.
We remark that the requirement for the marginals to match exactly is unimportant. Indeed, any
protocolwherethemarginalsmatchapproximatlycanbe“fixed”tohaveexactmarginals,withasmall
lossinthecorrelation.
The main restriction of Theorem 5 is that the output of the non-interactive protocol is a pair of
bits. Theorem3immediatelyimpliesthefollowingmodificationofTheorem5: wemayreplace 0,1
{ }
by [k] wherever it appears, provided that we also assume that P is a Gaussian measure. In the best
of both worlds, we would be able to replace 0,1 by [k] without adding the assumption that P is a
{ }
Gaussianmeasure. Usingthemethodswedevelophere,thisalsoturnsouttobepossible;weprovide
detailsinafollow-upwork.
4 Preliminaries
Wewillstartbydefiningsometechnicalpreliminarieswhichwillbeusefulfortherestofthepaper.
Definition2. Fork Nand1 i k, lete bethe unitvector alongcoordinate iandlet ∆ bethe convex
i k
∈ ≤ ≤
hullformedby e .
i 1 i k
{ } ≤≤
In this paper, we will be working on the space of functions f : Rn R where the domain is
→
equippedwiththestandardndimensionalnormalmeasure(denotedbyγ ()). Unlessexplicitlymen-
n
·
tionedotherwise,all thefunctionsconsideredin thispaperwillbe inL2(γ ). Akeypropertyofsuch
n
functions is that they admit the so-called Hermite expansion. Let us define a family of polynomials
H :R R(forq 0)as
q
→ ≥
( 1)q dq
H0(x) = 1; H1(x) = x; Hq(x) = −√q! ·ex2/2· dxqe−x2/2.
LetZ denotethesubsetofnon-negativeintegersandS Z n. DefineH : Rn Ras
∗ ∗ S
∈ →
n
H (z) = H (z ).
S Si i
i=1
Y
7
Itiswellknownthattheset HS S Z∗n formsanorthonormalbasisforL2(γn). Inotherwords,every
f L2(γ )maybewrittenas{ } ∈
n
∈
f = f(S) H ,
S
·
S Z∗n
X∈
b
wheref(S)are typically referredtoastheHermite coefficients and expansionisreferredtoas theHer-
mite expansion. The notion of Hermite expansion can be easily extended to f : Rn Rk as follows:
→
Letf =b(f ,...,f )andlet
1 k
f = f (S) H .
i i S
·
S Z∗n
X∈
b
Then, the Hermite expansion of f is given by f(S) H where f(S) = (f (S),...,f (S)). In
S Z∗n · S 1 k
thissetting,wealsohaveParseval’sidentity: ∈
P
b b b b
f(x) 2 γ (x)dx = f(S) 2 (1)
k k2 n k k2
Z S Z∗n
X∈
b
Forf : Rn Rk andd N,definef :Rn Rk by
d
→ ∈ ≤ →
f (x) = f(S) H (x).
d S
≤ ·
S:S d
X| |≤
b
WewilldefineW d[f] = f 2 andW>d[f]= f(S) 2.
≤ k ≤dk2 |S|>dk k2
P
Ornstein-Uhlenbeckoperator b
Definition3. TheOrnstein-Uhlenbeckoperator P isdefinedfort [0, )suchthatforanyf : Rn Rk,
t
∈ ∞ →
(P f)(x) = f(e t x+ 1 e 2t y)dγ (y).
t − − n
· − ·
Zy Rn
∈ p
Notethatiff :Rn ∆ ,thensoisP f foreveryt > 0. AbasicfactabouttheOrnstein-Uhlenbeck
k t
→
operatoristhatthefunctions H areeigenfunctionsofthisoperator. Weleavetheproofofthenext
S
{ }
propositiontothereader.
Proposition6. ForS Z n,P H = e t S H .
∗ t S − ·| | S
∈ ·
Wedefinethenoisestabilityoff :Rn ∆ withnoiseratet [0, )by
k
→ ∈ ∞
Stab (f)= f,P f .
t t
h i
Multivariatepolynomialthresholdfunctions Werecallthedefinitionofpolynomialthresholdfunc-
tionsfromtheintroduction(primarilytodefineanassociatedquantitywhichshallbeusefullater).
Definition. Afunction f : Rn [k]issaid tobeamultivariate PTFisthere exists polynomials p ,...,p :
1 k
→
Rn Rsuchthat
→
j ifp (x) >0 andfori = j, p (x) 0
j i
f(x) = 6 ≤
(1 otherwise
Wethendenotef asf = PTF(p ,...,p ). Further,f issaidtobeadegree-dmultivariatePTFifp ,...,p are
1 k 1 k
degree dpolynomials. Forthe rest ofthis paper, wewillusePTFsto meanmultivariate PTFs. Theunderlying
k willbeclearfromthecontext. Also,letusdefineCollision(f) Rn definedas
⊆
Collision(f) = x Rn : j :p (x) > 0 = 1 .
j
∈ |{ }| 6
(cid:8) (cid:9)
8
Convention We also adopt theconvention that for all subsequentstatements,unlessexplicitly men-
tioned otherwise, the domain is equipped with the standard Gaussian measure over the relevant
dimensions. Also, at several places in the paper, we will establish bounds on one quantity in terms
ofotherswithouttheexplicitmentionofthedependence. Unlessmentionedotherwise,thesebounds
can be made explicit by workingthroughthe proofbut we donot dosoin theinterestofclarity. For
example, if we state that a quantity d = d(k,ǫ), we mean that that d can be bounded as an explicit
functionofk andǫbutwehidetheexplicitdependence.
5 Reduction from arbitrary functions to PTFs
In this section, we will prove that given any k-ary function with a given set of measures for each of
thek-partitions,thereisamultivariatePTFwithnearlythesamemeasuresfortheinducedpartitions
which is a multivariate PTF and (up to an error ǫ), no less noise stable at a fixed noise rate t. This is
thefirststep(“fromgeneralpartitionstoPTF”)oftheproofsketchinSection2.
Wewillneedthefollowingconcentrationboundforlow-degreepolynomials,whichmaybeproved
usingtheGaussianhypercontractiveinequality. See[14]forareference.
Theorem7. Letp : Rn Rbeadegree-d polynomial. Then,foranyt > 0,
→
Pr p(x) E[p(x)] t Var[p] d e t2/d.
−
x | − | ≥ · ≤ ·
(cid:2) p (cid:3)
For technical reasons, we will also require the PTFs to satisfy a property which we refer to as
(d,δ)-balanced definedbelow.
Definition 4. A degree-d multivariate PTF f = PTF(p(1),...,p(k)) is said to be (d,δ)-balanced if each p(i)
hasvariance1and E[p(i)] logd/2(k d/δ).
| |≤ ·
Weaskthereadertoobservethatthefirstcondition(namely,Var(p(i)) = 1)canbeachievedwith-
outloss ofgeneralityby simply scaling all thepolynomials tohave variance 1. This scaling doesnot
changethevalueofthePTFatanypointx. Whiletheconditiononexpectationisnon-trivial,thenext
propositionsaysthatanymultivariatePTFcanbeassumedtobe(d,δ)-balanced whileonlychanging
thevalueofthePTFatδ-fraction ofplaces.
Proposition 8. Let f : Rn [k] defined as f = PTF(p(1),...,p(k)). Then, there is a (d,δ)-balanced
→
multivariate PTFg : Rn [k]defined asg = PTF(q(1),...,q(k))suchthat Pr [g(x) = f(x)] δ. Further,
x
→ 6 ≤
Pr [x Collision(g)] Pr [x Collision(f)]+δ. Infact,thepolynomials q(i) arelineartranslationsofthe
x x
∈ ≤ ∈ { }
polynomials p(i) .
{ }
Proof. Aswehaveobserved,wecanassumewithoutlossofgeneralitythatthepolynomialsp(i) have
variance1. Wedefinethepolynomialsq(i)asfollows. If E[p(i)] logd/2(d k/δ),thenwesetq(i) =p(i).
| |≤ ·
Else,letb = sign(E[p(i)])andwedefine
i
q(i) = p(i) E[p(i)]+b logd/2(d k/δ).
i
− · ·
First, note that as claimed, q(i) are indeed affine translations of p(i). Next, note that by Theorem 7,
whenever q(i) = p(i), the sign of p(i)(x) (for x γ ) is δ/k-close to being fixed. Applying Theorem7
n
6 ∼
again,
Pr sign(q(i)(x)) = sign(p(i)(x)) δ/k.
x γn 6 ≤
∼
Fromthis,itimmediatelyfollowsth(cid:2)at (cid:3)
Pr[f(x) = g(x)] δ andPr[x Collision(f)] Pr[x Collision(g)]+δ.
x 6 ≤ x ∈ ≤ x ∈
9
Thenexttheoremisthemainresultofthissectionnamely,thatgivenanyfunctionf :Rn [k],it
→
is possibleto obtain a multivariate (d,ǫ) balanced PTF g (for some explicit d = O (1)) such that the
ǫ,k
resulting PTF g has nearly the same partition sizes and noise stability. Further, Collision(g) has small
probability.
Theorem9. Letf : Rn [k]satisfyE[f] = µwhereµ = (µ ,...,µ ). Then,foreveryǫ > 0,thereexists a
1 k
→
multivariatePTFg :Rn [k]ofdegreed = d(k,ǫ)suchthat
→
E[g] µ ǫ,
1
• k − k ≤
g,P g f,P f ǫ,
t t
• h i ≥ h i−
Pr[x Collision(g)] ǫ,and
• ∈ ≤
g is(d,ǫ)-balanced.
•
WewillproveTheorem9intwoparts. Thefirststepistoshowthatwecanreplacef byafunction
withexplicitHermitedecay:
Lemma 10. For every f : Rn [k] and every ǫ > 0, there exists a function h : Rn [k] satisfying the
→ →
following:
E[h] = E[f],
•
h,P h f,P f ǫ,
t t
• h i ≥ h i−
W>d[h] ǫforsomeexplicitd= d(k,ǫ).
• ≤
ThesecondstepintheproofofTheorem9goesfromexplicitHermitedecaytoanactualPTF:
Lemma11. Leth : Rn [k]suchthatW>d[h] ǫ. Then,there isaPTFg : Rn [k]ofdegree dsuchthat
→ ≤ →
E [g(x) = h(x)] k2 ǫ. Further,Pr [x Collision(h)] k2 ǫ.
x x
6 ≤ · ∈ ≤ ·
It is easy to see that Theorem 9 follows in a straightforward way by combining Lemma 10 and
Lemma11. Notethattheconditionofg being(d,ǫ)-balanced isobtainedbysimplyapplyingProposi-
tion8.
WebeginwiththeproofofLemma11,whichiseasier.
Proof of Lemma 11: Let us view h : Rn Rk and define h(0) : Rn Rk as h(0)(x) = h(x) 1 1.
→ → − k ·
Here the function 1 is definedas 1 : x (1,1,...,1) for all x Rn. Note that range of the function
7→ ∈
h(0) atallpointsisak-dimensionalvectorwiththeentry(k 1)/k inoneofthecoordinatesand 1/k
− −
ineveryothercoordinate. Callsuchapointak-latticepoint.
Let h(0) : Rn Rk (as defined earlier) correspond to the function obtained by truncating the
d →
Hermitee≤xpansionofh(0) atdegreed. Considerthedegree-dpolynomialsp ,...,p : Rn Rwhere
1 k
→
p istheith coordinateofh(0). Wenowdefineg = PTF(p ,...,p ). Consideranypointx Rn.
i d 1 k ∈
≤
If x Collision(g), then observe that h(0)(x) h(0)(x) 2 k 2. To see this, note that if x
• ∈ k − d k2 ≥ − ∈
Collision(g), thereare twopossibilities. (a)p (x) ≤0 forall 1 i k: Ittheneasily followsthat
i
≤ ≤ ≤
(0)
ℓ distance ofh (x)from any k-lattice pointis at least 1/k. (b) p (x),p (x) > 0 for distinct i,j.
2 d i j
≤ (0)
Againtheℓ distanceofh (x)fromanyk-latticepointisatleast1/k. Thus,inboththesecases,
2 d
h(0)(x) h(0)(x) 2 1 .≤
k − d k2 ≥ k2
≤
10