New Rounding Techniques for the Design and Analysis of Approximation Algorithms

321 Pages·2014·1.76 MB·English

NEW ROUNDING TECHNIQUES FOR
THE DESIGN AND ANALYSIS OF APPROXIMATION ALGORITHMS
A DISSERTATION
SUBMITTED TO THE DEPARTMENT OF MANAGEMENT
SCIENCE AND ENGINEERING
AND THE COMMITTEE ON GRADUATE STUDIES
OF STANFORD UNIVERSITY
IN PARTIAL FULFILLMENT OF THE REQUIREMENTS
FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY
Shayan Oveis Gharan
May 2014
Abstract
We study two of the most central classical optimization problems, namely the Traveling Salesman
problems and Graph Partitioning problems and develop new approximation algorithms for them.
We introduce several new techniques for rounding a fractional solution of a continuous relaxation of
theseproblemsintonearoptimalintegralsolutions. Thetwomostnotableofthosearethemaximum
entropy rounding by sampling method and a novel use of higher eigenvectors of graphs.
iv
To my beloved wife, Farnaz, my mum, my dad, Sheida, Shahram, Shadi, Shahab and all members of my
family.
v
Acknowledgements
Foremost, I would like to express my deepest appreciation to my advisor, Amin Saberi, who intro-
duced me to the field of computing. Amin thought me how to conduct academic research, how to
write a mathematical paper and how to present my work effectively to others; the three skills that
shaped my career. I would like to express my gratitude to Luca Trevisan who introduced me to the
fieldofSpectralgraphtheory. Istartedworkingwith Lucaaftertakingacourseonthissubject and
we have managed to advance this area in several directions since then.
I would like to thank all my teachers at Stanford University for offering advanced courses that
have helped me throughout my career. In particular, I would like to thank Persi Diaconis, Ashish
Goel, Serge Plotkin, Tim Roughgarden, Ryan Williams and Yinyu Ye. In addition, a thank you
to all my coauthors: Arash Asadpour, Michel Goemans, Tsz Chiu Kwok, Bundit Laekhanukit, Lap
Chi Lau, James Lee, Yin Tat Lee, Russell Lyons, Aleksander Madry, Vahideh Manshadi, Vahab
Mirrokni, Mohit Singh, Jan Vondrak and Morteza Zadimoghaddam; working with each one of them
has been a tremendous learning experience.
IamdeeplygratefultotheManagementScienceandEngineeringdepartment,StanfordSchoolof
Engineering, Caroline Pease and Anne Schnoebele for supporting my graduate research during the
past five years. Also, I gratefully acknowledge the National Science Foundation for their financial
support.
I have greatly benefited from the summer internships at IBM Almaden Research Center, Mi-
crosoft Research, New England and Microsoft Research, Redmond. I want to thank my supervisors
in those institutes for giving me the opportunity to work and network with researchers from all
around the world.
Last but not least, my special thanks goes to my wife, Farnaz, and everyone else in my family:
my mum, my dad, Sheida, Shahram, Shadi and Shahab. They have supported me during this PhD
by means that words cannot explain.
vi
Contents
Abstract iv
Acknowledgements vi
1 Introduction 2
1.1 New Approximation Algorithms to the Traveling Salesman Problem . . . . . . . . . 3
1.1.1 Our Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.1.2 Rounding by Sampling and Online Stochastic Matching Problem . . . . . . . 7
1.2 New Analysis of Spectral Graph Algorithms through Higher Eigenvalues . . . . . . . 11
1.2.1 Spectral Clustering Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.2.2 Spectral Graph Algorithms in Theory . . . . . . . . . . . . . . . . . . . . . . 13
1.2.3 Our Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
1.2.4 An Upper Bound on Graph Diameter based on Laplacian Eigenvalues . . . . 19
I New Approximation Algorithms to the Traveling Salesman Problem 23
2 Background 24
2.1 The Traveling Salesman Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.2 Linear and Convex Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.3 Matroids and Spanning trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.4 Linear Programming Relaxation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.4.1 Integrality Gap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.4.2 Integral Polytopes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
2.5 The Christofides’ Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.6 Structure of Minimum Cuts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.6.1 Properties of Minimum Cuts . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
2.6.2 The Cactus Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
2.6.3 Properties of Tree Hierarchy and Cactus Representation . . . . . . . . . . . . 41
vii
2.7 Structure of Near Minimum Cuts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
2.7.1 Structure of (1+η) near min cuts for small η . . . . . . . . . . . . . . . . . 44
2.7.2 Structure of α near minimum cuts for large α . . . . . . . . . . . . . . . . . . 47
2.8 Random Spanning Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
2.8.1 Sampling a λ-Random Tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
2.8.2 Electrical Networks and λ-Random Trees . . . . . . . . . . . . . . . . . . . . 50
2.8.3 Negative Correlation and Concentration Inequalities. . . . . . . . . . . . . . . 52
2.8.4 λ-Random Trees and Determinantal Measures. . . . . . . . . . . . . . . . . . 53
2.9 Strongly Rayleigh Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
2.9.1 Operations defined on Strongly Rayleigh Measures . . . . . . . . . . . . . . . 55
2.9.2 Properties of Strongly Rayleigh Measures . . . . . . . . . . . . . . . . . . . . 56
2.9.3 Properties of Rank Function of Strongly Rayleigh Measures . . . . . . . . . . 58
3 New Machineries 63
3.1 Rounding by Sampling Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
3.1.1 Maximum Entropy Rounding by Sampling Method . . . . . . . . . . . . . . . 66
3.1.2 Computation of Maximum Entropy Distribution . . . . . . . . . . . . . . . . 70
3.1.3 Other Methods for Rounding with Negative Correlation Property . . . . . . . 75
3.2 The Cactus-like Structure of Near Minimum Cuts. . . . . . . . . . . . . . . . . . . . 76
3.2.1 Cut Classes with Inside Atoms . . . . . . . . . . . . . . . . . . . . . . . . . . 86
3.2.2 Applications to TSP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
3.3 Locally Hamiltonian Property of Random Spanning Trees . . . . . . . . . . . . . . . 93
4 Asymmetric TSP 103
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
4.1.1 Notations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
4.2 Thin Trees and Asymmetric TSP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
4.3 Construction of a Thin Tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
4.4 Tight Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
5 Planar Asymmetric TSP 112
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
5.2 Constructing a thin-tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
5.3 Thin trees, Goddyn’s conjecture and ATSP . . . . . . . . . . . . . . . . . . . . . . . 116
5.3.1 Nowhere-zero flows and Jaeger’s conjecture . . . . . . . . . . . . . . . . . . . 118
6 Symmetric TSP 119
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
6.2 The Structure Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
viii
6.3 In Search of Good Edges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
6.4 Inside Good Edges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
6.5 Thread Good Edges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
6.5.1 Threads with non-proper Cut Classes . . . . . . . . . . . . . . . . . . . . . . 144
6.5.2 Threads with Proper Cut Classes . . . . . . . . . . . . . . . . . . . . . . . . . 147
II New Analysis of Spectral Graph Algorithms through Higher Eigen-
values 151
7 Background 152
7.1 Spectral Graph Theory. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
7.2 Laplacian Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
7.3 Cayley Graphs and Their Spectrum . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
7.3.1 The Cycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
7.3.2 The Grid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
7.3.3 The Ladder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
7.3.4 The Hypercube . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
7.4 Random Walks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
7.5 Gaussian Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
7.6 Eigenfunction Computation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
7.7 Expansion, Conductance and Sparsest Cut. . . . . . . . . . . . . . . . . . . . . . . . 172
7.7.1 Continuous Relaxations of Conductance . . . . . . . . . . . . . . . . . . . . . 175
7.7.2 Computability of Conductance . . . . . . . . . . . . . . . . . . . . . . . . . . 176
7.7.3 The Small Set Expansion Problem . . . . . . . . . . . . . . . . . . . . . . . . 177
7.8 Cheeger’s Inequality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
7.8.1 Proof of the Cheeger’s Inequality . . . . . . . . . . . . . . . . . . . . . . . . . 180
7.8.2 Tightness of Cheeger’s inequality . . . . . . . . . . . . . . . . . . . . . . . . . 182
7.9 Random Partitioning of Metric Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . 183
8 New Machineries 187
8.1 Spectral Embedding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
8.2 Beyond log(n) dimensions, Johnson-Lindenstrauss to log(k) dimensional space. . . . 193
8.3 Improved Lower Bounds on Escape Probability . . . . . . . . . . . . . . . . . . . . . 198
8.3.1 Lower Bounds on Uniform Mixing Time of Random Walks . . . . . . . . . . 202
9 Universal Bounds on Laplacian Eigenvalues 205
9.1 introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
9.1.1 Related Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
ix
9.1.2 Notations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
9.2 Regular Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
9.3 General Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
9.4 Vertex-Transitive Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
9.5 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
10 Higher Order Cheeger’s Inequality 221
10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
10.1.1 Finding many sets and small-set expansion . . . . . . . . . . . . . . . . . . . 223
10.1.2 Proof Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
10.1.3 A general algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
10.2 Localizing eigenfunctions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
10.2.1 Smooth localization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
10.2.2 Merging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
10.2.3 Random Partitioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
10.2.4 Higher-order Cheeger inequalities . . . . . . . . . . . . . . . . . . . . . . . . . 234
10.3 Gaps in the Spectrum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
10.4 A New Multiway Cheeger Inequality . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
10.5 Noisy hypercubes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240
10.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242
11 Improved Cheeger’s Inequality 244
11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244
11.1.1 Improvements on Applications of Cheeger’s Inequality . . . . . . . . . . . . . 245
11.1.2 More Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247
11.1.3 Proof Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248
11.2 Proof of the Main Result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
11.2.1 Normalized Euclidean Distance Function. . . . . . . . . . . . . . . . . . . . . 250
11.2.2 Construction of Dense Well Separated Regions . . . . . . . . . . . . . . . . . 251
11.2.3 Lower Bounding the Energy . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252
11.2.4 Improved Cheeger’s Inequality . . . . . . . . . . . . . . . . . . . . . . . . . . 255
11.3 Extensions and Connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255
11.3.1 Spectral Multiway Partitioning . . . . . . . . . . . . . . . . . . . . . . . . . . 256
11.3.2 Balanced Separator. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
12 Almost Optimal Local Graph Clustering 261
12.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
12.1.1 Almost Optimal Local Graph Clustering . . . . . . . . . . . . . . . . . . . . . 262
x
12.1.2 Approximating the Conductance Profile . . . . . . . . . . . . . . . . . . . . . 263
12.1.3 Approximating Balanced Separator . . . . . . . . . . . . . . . . . . . . . . . . 264
12.1.4 Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264
12.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265
12.2.1 Evolving Set Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265
12.2.2 The Volume-Biased Evolving Set Process . . . . . . . . . . . . . . . . . . . . 266
12.2.3 The Diaconis-Fill Coupling . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
12.3 Main Proof . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268
13 Partitioning into Expanders 274
13.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274
13.1.1 Related Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276
13.1.2 Our Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276
13.1.3 Tightness of Existential Theorem . . . . . . . . . . . . . . . . . . . . . . . . 278
13.1.4 Notations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
13.1.5 Overview of the Proof . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
13.1.6 Background on Higher Order Cheeger’s Inequality . . . . . . . . . . . . . . . 280
13.2 Proof of Existential Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
13.3 Proof of Algorithmic Theorem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
13.4 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290
14 Open Problems 291
Bibliography 293
xi
List of Figures
1.2.1Spectral Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.2.2An Application of Spectral Clustering in Image Segmentation . . . . . . . . . . . . . 16
2.4.1An Integrality Gap Example of Held-Karp Relaxation . . . . . . . . . . . . . . . . . 31
2.4.2An Example of an O-join . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.5.3An Illustration of the output of Christofides’ 3/2 Approximation Algorithm for TSP 33
2.6.4An Example of Two Crossing Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.6.5A Connected Family of Crossing Cuts . . . . . . . . . . . . . . . . . . . . . . . . . . 37
2.6.6The Cactus Representation of a Star Graph . . . . . . . . . . . . . . . . . . . . . . . 39
2.6.7An Example of a Tree Hierarchy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
2.7.8A Family of near Minimum Cuts with an Inside Atom . . . . . . . . . . . . . . . . . 45
3.1.1A Fractional solution of the LP Relaxation of TSP as a Convex Combination of
Spanning Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
3.1.2An Example of the Maximum Entropy Distribution of Spanning Trees . . . . . . . . 69
3.2.3Setting in the proof of Lemma 3.2.7. . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
3.2.4An Example of a Chain of near Minimum Cuts . . . . . . . . . . . . . . . . . . . . . 82
3.2.5Setting in the Proof of Claim 3.2.14 . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
3.2.6An Example of G(ψ(C)) for a Feasible solution of Held-Karp Relaxation . . . . . . . 90
3.2.7Each Edge of a Cycle of Lenght n is Contained in n−1 Minimum Cuts . . . . . . . 91
3.3.8An Example of a λ-random Spanning Tree distribution that is not Locally Hamiltonian 95
3.3.9An Example of a λ-random Spanning Tree Distribution µ where two vertices of Ex-
pected 2 do not have even degree in T ∼µ with high Probability . . . . . . . . . . . 99
3.3.10Setting of Lemma 3.3.7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
4.4.1A Tight Example for Theorem 4.3.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
6.2.1AsetU hasanoddnumberofodddegreeverticesofaSpanningTreeT,iff|T∩δ(U)|
is odd. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
6.2.2An Example of a Graph with no Good Edge . . . . . . . . . . . . . . . . . . . . . . . 125
xii