Algorithms and Data Structures: The Basic Toolbox

305 Pages·2008·2.03 MB·English

by  Kurt Mehlhorn

Algorithms and Data Structures
Kurt Mehlhorn • Peter Sanders
Algorithms and
Data Structures
The Basic Toolbox
Prof.Dr.KurtMehlhorn Prof.Dr.PeterSanders
Max-Planck-InstitutfürInformatik UniversitätKarlsruhe
Saarbrücken Germany
Germany [email protected]
[email protected]
ISBN978-3-540-77977-3 e-ISBN978-3-540-77978-0
DOI10.1007/978-3-540-77978-0
LibraryofCongressControlNumber:2008926816
ACMComputingClassification(1998):F.2,E.1,E.2,G.2,B.2,D.1,I.2.8
(cid:2)c 2008Springer-VerlagBerlinHeidelberg
Thisworkissubjecttocopyright.Allrightsarereserved,whetherthewholeorpartofthematerialis
concerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,broadcasting,
reproductiononmicrofilmorinanyotherway,andstorageindatabanks.Duplicationofthispublication
orpartsthereofispermittedonlyundertheprovisionsoftheGermanCopyrightLawofSeptember9,
1965,initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer.Violationsare
liabletoprosecutionundertheGermanCopyrightLaw.
The use of general descriptive names, registered names, trademarks, etc. in this publication does not
imply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevantprotective
lawsandregulationsandthereforefreeforgeneraluse.
Coverdesign:KünkelLopkaGmbH,Heidelberg
Printedonacid-freepaper
9 8 7 6 5 4 3 2 1
springer.com
Toallalgorithmicists
Preface
Algorithmsareattheheartofeverynontrivialcomputerapplication.Thereforeevery
computerscientistandeveryprofessionalprogrammershouldknowaboutthebasic
algorithmictoolbox:structuresthatallowefficientorganizationandretrievalofdata,
frequentlyusedalgorithms,andgenerictechniquesformodeling,understanding,and
solvingalgorithmicproblems.
This book is a concise introduction to this basic toolbox, intended for students
andprofessionalsfamiliarwithprogrammingandbasicmathematicallanguage.We
haveusedthebookinundergraduatecoursesonalgorithmics.Inourgraduate-level
courses, we make most of the book a prerequisite, and concentrate on the starred
sectionsandthemoreadvancedmaterial.Webelievethat,evenforundergraduates,
aconciseyetclearandsimplepresentationmakesmaterialmoreaccessible,aslong
asitincludesexamples,pictures,informalexplanations,exercises,andsomelinkage
totherealworld.
Mostchaptershavethesamebasicstructure.Webeginbydiscussingaproblem
asitoccursinareal-lifesituation.Weillustratethemostimportantapplicationsand
thenintroducesimplesolutionsasinformallyaspossibleandasformallyasneces-
sarytoreallyunderstandtheissuesathand.Whenwemovetomoreadvancedand
optionalissues,thisapproachgraduallyleadstoamoremathematicaltreatment,in-
cludingtheoremsandproofs.Thisway,thebookshouldworkforreaderswithawide
rangeofmathematicalexpertise.Therearealsoadvancedsections(markedwitha*)
wherewerecommendthatreadersshouldskipthemonfirstreading.Exercisespro-
videadditionalexamples,alternativeapproachesandopportunitiestothinkaboutthe
problems. It is highly recommended to take a look at the exercises even if there is
notimetosolvethemduringthefirstreading.Inordertobeabletoconcentrateon
ideasratherthanprogrammingdetails,weusepictures,words,andhigh-levelpseu-
docodetoexplainouralgorithms.Asection“implementationnotes”linkstheseab-
stractideastoclean,efficientimplementationsinrealprogramminglanguagessuch
asC++andJava.Eachchapterendswithasectiononfurtherfindingsthatprovides
aglimpseatthestateoftheart,generalizations,andadvancedsolutions.
Algorithmicsisamodernandactiveareaofcomputerscience,evenatthelevel
of the basic toolbox. We have made sure that we present algorithms in a modern
VIII Preface
way,includingexplicitlyformulatedinvariants.Wealsodiscussrecenttrends,such
as algorithm engineering, memory hierarchies, algorithm libraries, and certifying
algorithms.
Wehavechosentoorganizemostofthematerialbyproblemdomainandnotby
solutiontechnique.Thefinalchapteronoptimizationtechniquesisanexception.We
findthatpresentationbyproblemdomainallowsamoreconcisepresentation.How-
ever,itisalsoimportantthatreadersandstudentsobtainagoodgraspoftheavailable
techniques.Therefore,wehavestructuredthefinalchapterbytechniques,andanex-
tensive index provides cross-references between different applications of the same
technique. Bold page numbers in the Index indicate the pages where concepts are
defined.
Karlsruhe,Saarbrücken, KurtMehlhorn
February,2008 PeterSanders
Contents
1 Appetizer:IntegerArithmetics .................................. 1
1.1 Addition................................................. 2
1.2 Multiplication:TheSchoolMethod .......................... 3
1.3 ResultChecking .......................................... 6
1.4 ARecursiveVersionoftheSchoolMethod .................... 7
1.5 KaratsubaMultiplication ................................... 9
1.6 AlgorithmEngineering..................................... 11
1.7 ThePrograms ............................................ 13
1.8 ProofsofLemma1.5andTheorem1.7 ....................... 16
1.9 ImplementationNotes...................................... 17
1.10 HistoricalNotesandFurtherFindings ........................ 18
2 Introduction................................................... 19
2.1 AsymptoticNotation....................................... 20
2.2 TheMachineModel ....................................... 23
2.3 Pseudocode .............................................. 26
2.4 DesigningCorrectAlgorithmsandPrograms .................. 31
2.5 AnExample–BinarySearch................................ 34
2.6 BasicAlgorithmAnalysis .................................. 36
2.7 Average-CaseAnalysis..................................... 41
2.8 RandomizedAlgorithms.................................... 45
2.9 Graphs .................................................. 49
2.10 PandNP ................................................ 53
2.11 ImplementationNotes...................................... 56
2.12 HistoricalNotesandFurtherFindings ........................ 57
3 RepresentingSequencesbyArraysandLinkedLists ............... 59
3.1 LinkedLists.............................................. 60
3.2 UnboundedArrays ........................................ 66
3.3 *AmortizedAnalysis ...................................... 71
3.4 StacksandQueues ........................................ 74
X Contents
3.5 ListsVersusArrays........................................ 77
3.6 ImplementationNotes...................................... 78
3.7 HistoricalNotesandFurtherFindings ........................ 79
4 HashTablesandAssociativeArrays.............................. 81
4.1 HashingwithChaining..................................... 83
4.2 UniversalHashing......................................... 85
4.3 HashingwithLinearProbing................................ 90
4.4 ChainingVersusLinearProbing ............................. 92
4.5 *PerfectHashing.......................................... 92
4.6 ImplementationNotes...................................... 95
4.7 HistoricalNotesandFurtherFindings ........................ 97
5 SortingandSelection ........................................... 99
5.1 SimpleSorters............................................ 101
5.2 Mergesort–anO(nlogn)SortingAlgorithm .................. 103
5.3 ALowerBound........................................... 106
5.4 Quicksort ................................................ 108
5.5 Selection................................................. 114
5.6 BreakingtheLowerBound ................................. 116
5.7 *ExternalSorting ......................................... 118
5.8 ImplementationNotes...................................... 122
5.9 HistoricalNotesandFurtherFindings ........................ 124
6 PriorityQueues................................................ 127
6.1 BinaryHeaps............................................. 129
6.2 AddressablePriorityQueues ................................ 133
6.3 *ExternalMemory ........................................ 139
6.4 ImplementationNotes...................................... 141
6.5 HistoricalNotesandFurtherFindings ........................ 142
7 SortedSequences .............................................. 145
7.1 BinarySearchTrees ....................................... 147
7.2 (a,b)-TreesandRed–BlackTrees............................ 149
7.3 MoreOperations .......................................... 156
7.4 AmortizedAnalysisofUpdateOperations..................... 158
7.5 AugmentedSearchTrees ................................... 160
7.6 ImplementationNotes...................................... 162
7.7 HistoricalNotesandFurtherFindings ........................ 164
8 GraphRepresentation .......................................... 167
8.1 UnorderedEdgeSequences ................................. 168
8.2 AdjacencyArrays–StaticGraphs ........................... 168
8.3 AdjacencyLists–DynamicGraphs .......................... 170
8.4 TheAdjacencyMatrixRepresentation ........................ 171
8.5 ImplicitRepresentations.................................... 172
Contents XI
8.6 ImplementationNotes...................................... 172
8.7 HistoricalNotesandFurtherFindings ........................ 174
9 GraphTraversal ............................................... 175
9.1 Breadth-FirstSearch....................................... 176
9.2 Depth-FirstSearch ........................................ 178
9.3 ImplementationNotes...................................... 188
9.4 HistoricalNotesandFurtherFindings ........................ 189
10 ShortestPaths ................................................. 191
10.1 FromBasicConceptstoaGenericAlgorithm .................. 192
10.2 DirectedAcyclicGraphs ................................... 195
10.3 NonnegativeEdgeCosts(Dijkstra’sAlgorithm) ................ 196
10.4 *Average-CaseAnalysisofDijkstra’sAlgorithm ............... 199
10.5 MonotoneIntegerPriorityQueues ........................... 201
10.6 ArbitraryEdgeCosts(Bellman–FordAlgorithm)............... 206
10.7 All-PairsShortestPathsandNodePotentials................... 207
10.8 Shortest-PathQueries...................................... 209
10.9 ImplementationNotes...................................... 213
10.10 HistoricalNotesandFurtherFindings ........................ 214
11 MinimumSpanningTrees....................................... 217
11.1 CutandCycleProperties ................................... 218
11.2 TheJarník–PrimAlgorithm................................. 219
11.3 Kruskal’sAlgorithm ....................................... 221
11.4 TheUnion–FindDataStructure.............................. 222
11.5 *ExternalMemory ........................................ 225
11.6 Applications.............................................. 228
11.7 ImplementationNotes...................................... 231
11.8 HistoricalNotesandFurtherFindings ........................ 231
12 GenericApproachestoOptimization ............................. 233
12.1 LinearProgramming–aBlack-BoxSolver.................... 234
12.2 GreedyAlgorithms–NeverLookBack....................... 239
12.3 DynamicProgramming–BuildingItPiecebyPiece ............ 243
12.4 SystematicSearch–WheninDoubt,UseBruteForce........... 246
12.5 LocalSearch–ThinkGlobally,ActLocally ................... 249
12.6 EvolutionaryAlgorithms ................................... 259
12.7 ImplementationNotes...................................... 261
12.8 HistoricalNotesandFurtherFindings ........................ 262
A Appendix ..................................................... 263
A.1 MathematicalSymbols..................................... 263
A.2 MathematicalConcepts .................................... 264
A.3 BasicProbabilityTheory ................................... 266
A.4 UsefulFormulae .......................................... 269