Compared to What?: An Introduction to the Anaylsis of Algorithms (Principles of Computer Science Series)

553 Pages·1991·23.07 MB·English

by  Gregory J.E. Rawlins

compared
to what?
an
_introduction
to the
analysis
of algorithms
_
gregory j. e. rawlins
Compared to What?
An
Introduction
to the
Analysis
of Algorithms
PRINCIPLES OF COMPUTER SCIENCE SERIES
Series Editors
Alfred V. Aho, Bellcore, Morristown, New Jersey
Jeffrey D. Ullman, Stanford University, Stanford, California
Alfred V. Aho and Jeffrey D. Ullman
Foundationso f Computer Science
Egon B6rger, Editor
Trends in Theoretical Computer Science
Ruth E. Davis
Truth, Deduction, and Computation:L ogic and Semanticsf or Computer Science
A. K. Dewdney
The Turing Omnibus: 61 Excursions in Computer Science
Vladimir Drobot
FormalL anguagesa nd Automata Theory
Narian Gehani
Advanced C: Foodf or the Educated Palate
C: An Advanced Introduction,A NSI C Edition
Cfor PersonalC omputers: IBM PC, AT&T PC 6300, and Compatibles
Eitan M. Gurari
An Introduction to the Theory of Computation
Martti Mdintyli
An Introduction to Solid Modeling
Shamim Naqvi and Shalom Tsur
A LogicalL anguagef or Data and Knowledge Bases
Christos Papadimitriou
The Theory of DatabaseC oncurrency Control
GregoryJ. E. Rawlins
Comparedt o What?: An Introduction to the Analysis ofAlgorithms
Stuart C. Shapiro
Lisp. An InteractiveA pproach
CoMMoNLisP:A n InteractiveA pproach
Richard Snodgrass
The InterfaceD escription Language:D efinition and Use
James A. Storer
Data Compression:M ethods and Theory
Steven Tanimoto
Elements ofArtificial Intelligence
Elements ofArtificial Intelligence Using COMMON LISP
Jeffrey D. Ullman
ComputationalA spects of VLSI
Principleso f Databasea nd Knowledge-Base Systems, Volume I: ClassicalD atabaseS ystems
Principleso f Databasea nd Knowledge-Base Systems, Volume II: The New Technologies
Theory of RelationalD atabases
Compared to What?
An
Introduction
to the
Analysis
of Algorithms
Gregory J. E. Rawlins
Computer Science Press
An Imprint of
W. H. Freeman and Company
New York
Library of Congress Cataloging-in-Publication Data
Rawlins, Gregory J. E.
Compared to what?: an introduction to the analysis of algorithms
/ by GregoryJ. E. Rawlins
p. cm.
Includes bibliographical references and index.
ISBN 0-7167-8243-X
1. Computer algorithms. I. Title.
QA76.9.A43R39 1991
005.1--dc20 91-30850
CIP
Copyright © 1992 by Gregory J. E. Rawlins
No part of this book may be reproduced by any mechanical,
photographic, or electronic process, or in the form of a phonographic
recording, nor may it be stored in a retrieval system, transmitted, or
otherwise copied for public or private use, without written permission
from the publisher.
Printed in the United States of America
Computer Science Press
An imprint of W H. Freeman and Company
The book publishing arm of Scientific American
41 Madison Avenue, New York, NY 10010
20 Beaumont Street, Oxford OXI 2NQ, England
2 3 4 5 6 7 8 9 0 RRD 9 9 8 7 6 5 4 3 2
This candle is for you, the beginner.
Turn it into a flame.
What comes from the heart, goes to the heart.
Samuel Taylor Coleridge, Table Talk
CONTENTS
List of Algorithms ................................. xi
List of Tables . . . . . . . . . . . . . . . . . . . . .. .. . . . . . . xiii
Forew ord . . . . .. . . . . . . . . . . . . . . . .. . .. . . . . . . xv
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii
Conventions ............................... xxv
1 Overview 1
1.1 Problem s . . . . . . . . . . . . . . .. . . . . . . . . . . . 7
1.2 M odels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.3 Algorithm s .......................... 13
1.4 A nalysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.5 Now W hat? .......................... 26
1.6 Napkin Mathematics ........................ 27
1.7 Growth Rates ......................... 38
1.8 Back to Reality ........................ 49
1.9 Hard Problems ............................ 52
1.10 Coda-The Continent of Analysis .............. 58
Endnotes . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
Q uestions ......... ............... .. . 67
Part I Fresh Horses 79
2 Searching 81
2.1 Linear Search ......................... 83
2.2 Jum p Search ......................... 98
2.3 Binary Search ........................ 105
2.4 Changing the Model ....................... 123
2.5 Coda-Apples and Oranges ................... 136
Endnotes ............. .. ............ 138
Q uestions ............... ............ 144
3 Selecting 159
3.1 Rankings ............................ 160
3.2 Finding the Best ....................... 164
3.3 Finding the Second Best ..................... 171
vii" CONTENTS
3.4 Finding the Best and Worst ................... 182
3.5 Finding the ith Best .................... 192
3.6 The Partition Problem ...................... 205
3.7 Changing the Model ....................... 207
3.8 Coda-Artists and Artisans .................... 210
Endnotes ........ ... ....... ... ...... 211
Q uestions . .......... .......... ...... 218
4 Sorting 231
4.1 Strategies . ........... .......... ..... 233
4.2 Swap Sorts .... .......................... 235
4.3 Insert Sorts .......................... 238
4.4 Select Sorts .......................... 240
4.5 M erge Sorts .......................... 249
4.6 Split Sorts . .......... .. ........ .. .... 253
4.7 Lower Bounds on Sorting .................... 259
4.8 Optimal Sorting ....................... 268
4.9 Changing the Model ........................ 279
4.10 Coda-Sorting Out Sorting ................... 281
Endnotes ................................ 282
Questions ................................ 287
Part II Sea Change 303
5 Graphs 305
5.1 Structures ........................... 309
5.2 Queues and Partitions ...................... 316
5.3 Connecting Telephones .................... 324
5.4 Partially Sorting ....................... 330
5.5 Exploring Graphs ......................... 335
5.6 Broadcasting Information ................... 340
5.7 Distributing Flow ...................... 343
5.8 Coda-Graph Therapy ..................... 346
Endnotes ......... .. ....... ... ...... 346
Q uestions .......... ........... ...... 351
6 Numbers 357
6.1 Exponential Numbers ...................... 359
6.2 Common Numbers ........................ 362
6.3 Prime Numbers ....................... 366
6.4 Secret Numbers ....................... 373
6.5 Authentic Numbers ........................ 385
6.6 Random Numbers ...................... 393
CONTENTS ix
6.7 Transforming Numbers ...................... 396
6.8 Coda-Straight On Till Morning ............... 403
Endnotes ........................... 404
Q uestions ..... .... ................. . 411
7 Infeasibility 415
7.1 Remembrance of Times Past ................. 416
7.2 Models of Computation .................... 419
7.3 Turing Machines ....................... 424
7.4 Birth of KP-Completeness .................. 428
7.5 Working out the Hierarchy .................. 434
7.6 Solving Hard Problems ...................... 442
7.7 What Is an Algorithm? ................... 443
7.8 W hat Is a Proof? ....................... 447
7.9 Coda-The End of the Beginning .............. 452
Endnotes ........................... 454
Q uestions ........................... 459
Appendices 467
A Mathematical Background 467
A .1 Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467
A .2 Algebra ............................ 471
A. 3 Real Analysis ......................... 473
A .4 Sum s . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . 476
A .5 Probability .......................... 480
B Manipulating Order Notation 483
B. 1 The Sum Rule ........................ 483
B. 2 The Product Rule ......................... 484
B. 3 Sample Algorithm Analyses .................. 485
B. 4 Algorithms That Call Subroutines .............. 487
C Recurrences 491
C. 1 Simple Recurrences ..................... 491
C.2 Recursive Algorithms .................... 493
C. 3 A Tricky Example ......................... 496
C.4 A Long Example ....................... 497
Bibliography .............................. 501
Sym bol Index .............................. 513
Nam e Index ............................... 515
Subject Index .............................. 521
LIST OF ALGORITHMS
1 .1 H ano i . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.2 Fibonacci . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
1.3 Fibonacci II .......... ............... ... 32
1.4 Strange . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
2.1 Linear Search ........................... 84
2.2 Linear Search II ......................... 85
2.3 Jump-Search ................................ 100
2.4 Binary Search ............................... 105
2.5 Binary-Search II ............................. 106
2.6 Coin-FlipSearch ............................. 124
2.7 Linked-List Search ............................ 130
2.8 Linear-Search III ............................. 145
2.9 Binary-Search III ............................. 153
3.1 Find-Max .................................. 165
3.2 Find-SecondMax ........................ 173
3.3 Build-BinomialTree ....................... 176
3.4 Find-Second-Max 11 ....................... 177
3.5 Find-M axM in ........................... 183
3.6 Find-Max-Min II ......................... 188
3 .7 Sp lit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
3 .8 Find . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
3 .9 Select . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
3.10 Find-M ax-M in III ......................... 224
4.1 Bubble-Sort ............................ 236
4.2 Linear-InsertSort ......................... 238
4.3 Linear-Select Sort ........................ 240
4.4 Fix-H eap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248
4.5 H eap -Sort ......... ............ ........ 249
4.6 Linear-M erge Sort ........................ 251
4.7 Linear-M erge ........................... 252
4.8 Q uick-Sort ......... .............. ...... 254
4.9 M erge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275