Table Of ContentPhase transition for cutting-plane approach to vertex-cover problem
Timo Dewenter1 and Alexander K. Hartmann1,∗
1Institut fu¨r Physik, Universita¨t Oldenburg, D-26111 Oldenburg, Germany
(Dated: January 10, 2012)
Westudythevertex-coverproblemwhichisanNP-hardoptimizationproblemandaprototypical
modelexhibitingphasetransitionsonrandomgraphs,e.g.,Erd˝os-Renyi(ER)randomgraphs. These
phasetransitions coincide with changesof thesolution spacestructure,e.g, fortheERensembleat
connectivity c = e ≈ 2.7183 from replica symmetric to replica-symmetry broken. For the vertex-
coverproblem,alsothetypicalcomplexityofexactbranch-and-boundalgorithms,whichproceedby
2
exploringthelandscapeoffeasible configurations,changeclose tothisphasetransition from“easy”
1
to “hard”. In this work, we consider an algorithm which has a completely different strategy: The
0
problem is mapped onto a linear programming problem augmented by a cutting-plane approach,
2
hence the algorithm operates in a space outside the space of feasible configurations until the final
n step, where a solution is found. Here we show that this type of algorithm also exhibits an “easy–
a hard” transition around c = e, which strongly indicates that the typical hardness of a problem is
J
fundamentalto the problem and not dueto a specific representation of theproblem.
9
] NP-hard combinatorial optimization problems [1, 2] ers non-feasible (non-combinatorial) assignments to the
n
are fundamental to computational complexity, because variables which are always more optimal than the true
n
- despite much effort no algorithm has been found so far, feasible solution. Cutting planes are constraints which
s
whichisabletosolvetheseproblemsintheworst case in areaddedadditionallyanditerativelytotheproblemun-
i
d polynomial time, leading to the famous P-NP problem. til a feasible solution is found, which is then the optimal
.
t One way to try to understand the root of the apparent solution. Inparticularwestudythevertexcoverproblem
a
m computational hardness is to analyze hard instances of via LP and CP for Erdo˝s-Renyirandomgraphs[19]. We
problems. This has attracted much interest in statisti- show that VC with our LP/CP implementation changes
-
d cal physics [3–5]. Phase transitions on suitably chosen from“easy”to“hard”rightatthesametransitionpoint,
n ensembles of random instances were found, e.g., for the where this change occurs for a branch-and-bound algo-
o Satisfiability Problem [6], the Traveling Salesman Prob- rithm, and where the solution landscape changes from
c
lem [7] or the vertex-cover problem (VC) [8]. For ex- simple(replicasymmetricinthespin-glasslanguage[17])
[
act branch-and-bound algorithms [9, 10] the hardest in- to complex (replica-symmetry broken). Hence, our re-
1
stances are found right at these phase transitions, of- sults indicate that the typical hardness of a problem
v
ten related to a change from a typically polynomially seemstobequiteuniversalsincethechangesfrom“easy”
4
(“easy”) to a typically exponentially (“hard”) region. to “hard” are visible for algorithms which are based on
1
8 Branch-and-boundalgorithmssystematicallyexplorethe fundamentally different notions of configuration space.
1 space of feasible solutions (branching) while trying to Model Let G = (V,E) be an undirected graph with
1. avoid uninteresting configurations via updating efficient N vertices i ∈ V and M edges {i,j} ∈ E. A vertex
0 bounds. The behavior of these exact algorithms can be coverV ⊂V isasubsetofverticessothatforalledges
VC
2 partially understood in terms of an effective dynamics {i,j} ∈ E at least one end i or j is contained in V .
1 VC
inside the phase diagrams [11, 12]. In practice very effi- The vertices i ∈ V are called covered, uncovered else.
: VC
v cient but not exact are stochastic algorithms,e.g. Walk- We are interested in vertex coversof G of minimum car-
Xi SAT [13] or ASAT [14] and message-passing algorithms dinality |VVC|, the minimum vertex covers. The decision
[15, 16], inspired by statistical mechanics methods like problem if a VC with fixed cardinality exists or not be-
r
a the cavity approach [17]. Also these types of algorithms longs to the class of NP-complete problems [1].
rely on either moving in configuration space or on cal-
The analytical solution of VC on Erdo˝s-Renyi graphs
culating iteratively probabilities (weights) for different
exhibits a phase transition at the average connectivity
subspaces of configurations.
c=e≈2.7183: forc<e,thesolutionisreplicasymmet-
Here, we consider a completely different and comple- ric,while for c>e replica symmetry breakingwasfound
mentary type of algorithm, linear programming (LP), [8]. Thiscanbeseenalsonumericallywhenclusteringthe
which is a standard approach for practical optimization minimum vertex covers [20]. Furthermore, in connection
problems [2]. In connection with cutting planes (CP) withtheleaf-removal heuristic [21],thetypical-casecom-
[18], it is a very efficient (but apparently still worst- plexity of a branch-and-bound algorithm changes form
case exponential) approach to combinatorial optimiza- “easy” to “hard” at c=e.
tion problems. This approach is fundamentally different Linear-Programming Approach First, we translate
from the algorithms mentioned above since it does not the VC problem to an integer linear programming(ILP)
move inside the configuration space but instead consid- problem [2], each of the N nodes of the graph is rep-
2
resented by a variable x ∈ {0,1}, i = 1,...,N. The Ingeneral,SXguaranteestoobtainasolutioninacor-
i
value x = 1 denotes a covered, x = 0 indicates an un- ner of the polytope with minimum x. This means that
i i
covered node. The fact that for each edge {i,j} i or j stillnon-integersolutionscanbeobtained,howfrequently
must be covered can be written as x +x ≥ 1. Min- depends also on the heuristics used in the actual SX im-
i j
imizing the cardinality of the cover means we want to plementation. Anyway, an incomplete solution obtained
minimize x . Whenwerelaxtheintegerconstraintto byCP+SXprovidesanotherlowerboundx, usuallybet-
i i
xi ∈ [0,1]P, the set of constraints xi +xj ≥ 1 describes ter (but never worse) than that obtained by SX alone.
a polytope. Now we obtain the following linear program- Node heuristics To complete an incomplete solution,
ming problem (LP): we also applied the following “node” heuristics (NH): It
Minimize x= N x randomly selects a vertex i with an undecided variable
i=1 i x ∈]0,1[and adds x =0 to the LP andsolves it again.
i i
P
Subject to 0≤x ≤1∀i∈V Thisforcesthe SX algorithmto setnodes j adjacentto i
i
x +x ≥1 ∀{i,j}∈E tox =1. AftereachrunoftheSXalgorithmwechecked
i j j
whether still undecided variables are found and if neces-
sary the procedure is repeated. This ensures that finally
This can be solved efficiently, i.e., typically in polyno-
a complete solution, i.e., a vertex cover is found, but it
mial time, by the simplex algorithm (SX) [2, 22]. We
doesn’t have to be a minimum one. Hence the values of
usedthepublicavailablelp solve[23]withBland’sfirst
xobtainedinthiswayareupperboundstothetruemin-
index pivoting [24]. Note that now the solutions are not
imum vertex covers. Note that we also tried a heuristics
guaranteedtobeinteger-valuedanymore,variableswith
where x = 1 is added to the LP, but it provided typi-
x ∈ ]0,1[ we call undecided. Such solutions we call in- j
i callyhighervaluesofxassolution,inparticularforlarge
complete. Thevalueofx= x isalwaysalowerbound
i i graph connectivity c>5.
for the cardinality of a comPplete solution. On the other
Results Next, simulation results for the different
hand, if a solution computed by SX is complete, i.e., all
types of algorithms are presented for ER random graphs
variables are integer-valued, then it is immediately clear
ofN nodes,uptoN =280. Weusedtheensemblewhere
that it is a correct minimum for VC.
for each graph M edges are created randomly with uni-
Cutting-Plane Approach In order to obtain more
form weight, i.e., the connectivity is c=2M/N.
complete solutions, the CP approach [18] can be used.
Fig. 1 shows the fraction p of graphs which exhibit a
The basic idea is to limit the solution space by adding f
complete solution, for the SX (inset) and SX+CP algo-
extraconstraints,whichexcludeincompletesolutions. In
rithms,obtainedfromaveragingover1000realizationsof
principle, many types of extra constraints are possible.
graphs and for different system sizes, respectively. Ap-
Here, we apply the following heuristics, inspired by the
parently, SX is able to find solutions up to about c = 1,
nature of the problem: For any graph which is a cycle,
where a sharp drop of p is visible, resembling a phase
setting all variables of non-isolated nodes to 0.5 (0 else) f
transition. NotethatthepercolationtransitionoftheER
is a solutionofthe LP,but incomplete. Nevertheless,for
ensemble is at c=1. For c<1, ER random graphs con-
acycleofoddlengthl=2k+1,thesize ofthe minimum
sist mainly of trees, which are apparently easy to solve,
cover is always k+1 > k+0.5. Hence, for any cycle of
even for SX. When including CP, more samples can be
length l =2k+1 in a graph, at least k+1 nodes of the
solved,the transition shifts to a point close to the c=e,
cyclemustbe covered. Thus,afteranexecutionofSX,if
where replica symmetry breaking occurs and where the
the solutionis incomplete, we try to detect cycles ofodd
exact configuration-space-basedbranch-and-bound algo-
length l where the condition
rithm(withleafremoval[21])startstoexhibitatypically
l exponential running time.
x ≥ (1)
i (cid:24)2(cid:25) Anindicatorforthe running time ofthe SX+CPalgo-
xiX∈loop rithm is the average number of extra constraints M
extra
is violated (⌈l/2⌉ is the largest integer larger or equal that were added to the LP resulting from CPs (1) to
to l/2) and add this constraint to the LP. Technically, obtain complete solutions. Fig. 2 shows Mextra/N as a
the loops are obtained by searching a random spanning function of connectivity c. Clearly, an increase close to
tree (ST) in the graphvia a random breadth-firstsearch c=e isvisible. NotethatMextra/N increasesalsoforall
and adding a randomly chosen edge, which is part of realizations (see inset), but in this case this is less infor-
the graph but not of the ST. Our algorithm stops, if mative, since the algorithm is stopped if for some time
s = 20M times for a randomly chosen spanning trees no new constraint could be added.
andforallloopsemergingfromthesetreeswedidnotadd Finally, Fig. 3 shows the phase diagram for the VC
a constraint (because the loop was of even length or the problem. In addition to the exact minimum VC and the
constraintwasalreadyfulfilledbythecurrentincomplete analytical solution [8], simulation results for the SX and
solution). Otherwise, SX is executed for the next time theSX+CPalgorithm(lowerbounds)andacombination
and the solution checked for completeness again. of SX/SX+CP with NH (upper bounds) are included.
3
proaches for c ≥ 5, where apparently stronger finite-size
1 1 correctionsoccur. We found, e.g., for SX+CP at c=4 a
value b=0.88(9)whichis compatible with the scalingof
0.9
0.8 the exact value obtained by branch-and-bound [25].
0.8
0.6
0.7 SX
0.4 00..77
0.6
0.2
pf 0.5 00..66 ccoovveerreedd
0
0.4 SX+CP 0 1 2 3 4 5
00..55
uunnccoovveerreedd
0.3 N = 280
N = 200 00..44
0.2 )) 0.59 c = 4.0
N = 150 cc
((
N = 100 xx 0.57
0.1 N = 70 00..33 N,c) 0.55
0 N = 50 x(
0 2 4 6 8 10 12 14 00..22 0.53
SX+NH
c SX+CP+NH 0.51
min. VC 0 100 200 300
00..11
FIG. 1: Fraction pf of complete solutions for CP approach analytic sol. N
as a function of connectivity c for ER random graphs. The SX+CP
insetshowspf fortheSXalgorithm. Theverticallinedenotes 00 SX
c=e,the otherlines are guides to theeyes only. 00 11 22 33 44 55 66 77 88 99 1100 1111
cc
6 FIG. 3: Phase diagram for the fraction of covered vertices
5 N = 280 x. Minimum VC found with exact Branch-and-Bound algo-
all solutions N = 200
rithm/analytics [8] as well as upper and lower bounds ob-
4 N = 150
5 tainedfromthedifferentalgorithms. Inset: Finite-sizescaling
N = 100
3 N = 70 for SX+CP and c=4.0 The vertical line denotes c=e, dot-
N = 50 ted line in the inset a result from the fit (0.52+0.40N−0.88)
4 2 N = 20 and theother lines are guides to theeyes only.
N = 10
N
/ a 3 1 TheSXalgorithmaloneonlyyieldsresultsclosetothe
xtr 0 minimum VC up to c = e. Above this value the critical
Me 0 1 2 3 4 5 6 7 8 fraction of coveredvertices converges towards the trivial
2 solutionxc =0.5. TheSX+CPapproachresultsinabet-
ter lower bound and deviates only for an average degree
complete
c ≥ 5 visibly from the true minimum cover sizes. Com-
solutions
1 paring with the previous results, this means for c >∼ e,
this approach typically does not yield the correct solu-
tion but comes very close to it. The upper bounds also
0
start to deviate significantly for c > e, but stay rather
0 1 2 3 4 5 6 7 8
close. In general, one sees again the importance of the
c
criticallinec=e: Forsmallerconnectivitiescallbounds
FIG.2: AveragenumberofextraconstraintsMextra pernode seem to agree but beyond it they start to diverge,which
forCPalgorithmasafunctionofconlyforcompletesolutions
is incontrastto previousanalyticalbounds [3], whichdo
and for all solutions (inset). The vertical line denotes c=e,
not match the correct result for all connectivities c>0.
theother lines are guides to theeyesonly.
Conclusion/Outlook We studied the vertex-cover
problem for Erdo˝s-Renyi random graphs with a linear
These bounds were obtained, respectively by averaging programming/cutting plane algorithm. The algorithm
the “cover size” x for different system sizes N and dif- shows a clear “easy–hard” signature close to the con-
ferentconnectivitiesc,yieldingx(N,c). Weextrapolated nectivity c = e. This measn that this point denotes a
toinfinitesystemsizesx(c)=limN→∞x(N,c)viafitting phase transition not only for configuration-space-based
thedatatofunctionsx(N,c)=x(c)+aN−b (seeinsetof quantities and algorithms[8, 21] but also for the LP/CP
figure 3) or x(N,c) = [x (c)+a N−b]·[1+f N−g]. The approachwhichoperatesoutside the spaceoffeasibleso-
c
latter was only used for the SX+CP/SX+CP+NH ap- lutions. Thus, the typical hardness of VC is really an
4
intrinsic property of the problem and not bounded to Culture (MWK) of the Lower Saxony State.
specific algorithms. This finding may be related to the
fact thatalsoin the worst-case,allalgorithmsknownfor
NP-hard problems exhibit an exponential running time,
∗
[email protected];
i.e. itcouldhelpinordertounderstandbetter the P-NP
http://www.compphys.uni-oldenburg.de/en/
problem. [1] M. R. Garey and D. S. Johnson, Computers and in-
In principle, the number of loops grows exponentially tractability (W.H. Freemann, San Francisco, 1979).
with the number of nodes. Hence, one could imagine [2] C. Papadimitriou and K. Steiglitz, Combinatorial Opti-
that even within our SX+CP approach, by exhausting mization – Algorithms and Complexity (Dover Publica-
the set of these loop constraints, one arrives at an (ex- tions Inc.,Mineola, NY,1998).
[3] A. K. Hartmann and M. Weigt, Phase Transitions
ponentially slow) but complete algorithm. Nevertheless,
in Combinatorial Optimization Problems (Wiley-VCH,
there are graphs, where our constraints are clearly in-
Weinheim, 2005).
sufficient: A simple example is a complete graph of size [4] M. M´ezard andA.Montanari, Information, Physics and
N = 4, i.e., where each node is connected to any other Computation (Oxford University Press, Oxford, 2009).
node. The result of SX will be to set all x = 0.5, i.e., [5] C. Moore and S. Mertens, The Nature of Computation
i
x= 2. By adding constraints for the four possible loops (Oxford UniversityPress, Oxford, 2011).
of odd length, the result will be x = 2/3 for all nodes, [6] S.KirkpatrickandB.Selman,Science264,1297(1994).
i
[7] I.P.GentandT.Walsh,inProceedings of12thEuropean
i.e., x = 8/3, while the correct minimum-cover size is
Conference on Artificial Intelligence. ECAI ’96 (Wiley,
x=N −1=3.
Chichester, 1996), p. 170.
Onecouldimprovethealgorithminprinciplebyadding [8] M.WeigtandA.K.Hartmann,Phys.Rev.Lett.84,6118
other types of constraints, e.g., general Gomory-Chv´atal (2000).
cuts [18]. Alternatively, one could consider small sub- [9] M. Davis, G. Logemann, and D. Loveland, Com-
graphs G′ = (V′,E′), V′ ⊂ V and E′ ⊂ E, solve them mun. ACM 5, 394 (1962), ISSN 0001-0782, URL
by an exact algorithm yielding the cardinality X′ of the http://doi.acm.org/10.1145/368273.368557.
[10] R. Sedgewick, Algorithms in C (Addison-Wesley, Read-
minimum cover. Then one could add the constraint
ing (MA), 1990).
i∈V′xi ≥ X′. We have performed some preliminary [11] M.WeigtandA.K.Hartmann,Phys.Rev.Lett.86,1658
ePxperiments with these types of cuts, but observed only (2001).
marginal improvements so far, i.e., the overall behavior [12] S. Cocco and R. Monasson, Phys. Rev. Lett. 86, 1654
with the transition close to c=e was preserved. (2001).
In practice often a combination of the branching, i.e. [13] C. H. Papadimitriou, in Foundations of Computer Sci-
configuration-spacebased,andcutting,i.e.,LP-basedap- ence, Annual IEEESymposium on (IEEEComputerSo-
ciety,LosAlamitos,CA,USA,1991),vol.0,pp.163–169,
proaches, are used. Here, branching sets in when all
ISBN 0-8186-2445-0.
available cutting planes are exhausted. Thus, it would [14] J. Ardelius and E. Aurell, Phys. Rev. E 74, 037702
be very interesting to see how this combination of ap-
(2006).
proaches performs on VC for ER graphs. [15] M.M´ezard,G.Parisi,andR.Zecchina,Science297,812
Furthermore, it would be of high interest to analyti- (2002).
cally analyze the cutting plane approach,to see whether [16] M. M´ezard and R. Zecchina, Phys. Rev. E 66, 056126
one can understand why it performs so well for c < e. (2002).
[17] M.M´ezard,G.Parisi,andM.Virasoro,Spinglasstheory
This would lead to a better understanding of the roots
and beyond (World Scientific,Singapore, 1987).
of computationalhardness andcould alsolead to refined
[18] W.J.Cook,W.H.Cunningham,W.R.Pulleyblank,and
boundsontheminimumcoversizes,providingtechniques A. Schriever, Combinatorial Optimization (Wiley, New
applicable to a vast range of problems. York, 1998).
Finally, it could be worthwhile to extend the present [19] P.Erd˝osandA.R´enyi,Publ.Math.Inst.Hungar.Acad.
study to other graphensembles. One could test whether Sci. 5, 17 (1960).
at the same point replica symmetry breaking occurs, hi- [20] W. Barthel and A. K. Hartmann, Phys. Rev. E 70,
066120 (2004).
erarchical clustering of solution space can be found, and
[21] M. Bauer and O. Golinelli, Eur. Phys. J. B 24, 339
the problem becomes hard for configuration-space-based
(2001).
branch-and-boundapproachesaswellasfortheLP-based [22] G. Dantzig, Bull. Amer.Math. Soc. 54, 1074 (1948).
cutting-plane algorithms. [23] M.Berkelaar,K.Eikland,andP.Notebaert,Open-source
We thank O. Melchert for critically reading the program to solve linear programming problems: lp solve,
manuscript and M.-T. Hu¨tt, W. Krauth, M. M´ezard, U. http://lpsolve.sourceforge.net/5.5/ (2010), online;
Nowak,D.SherringtonandM.Weigtforstimulatingdis- Licence: GNU LGPL (Lesser General Public Licence).
[24] R.Bland,MathematicsofOperationsResearch2(1977).
cussions. The simulations were performed at the HERO
[25] A. K. Hartmann, W. Barthel, and M. Weigt, Comp.
clusteroftheUniversityofOldenburgfundedbytheDFG Phys. Comm. 169, 234 (2005).
(INST184/108-1FUGG)andtheminstryofScienceand
