Foundations of computer science

796 Pages·1995·5.947 MB·English

by  Alfred V. Aho, Jeffrey D. Ullman

TABLEOFCONTENTS v
Table of Contents
F
F F
F
Preface ix
Chapter 1. Computer Science: The Mechanization of Abstraction 1
1.1. What This Book Is About 3
1.2. What This Chapter Is About 6
1.3. Data Models 6
1.4. The C Data Model 13
1.5. Algorithms and the Design of Programs 20
1.6. Some C Conventions Used Throughout the Book 22
1.7. Summary of Chapter 1 23
1.8. Bibliographic Notes for Chapter 1 24
Chapter 2. Iteration, Induction, and Recursion 25
2.1. What This Chapter Is About 27
2.2. Iteration 27
2.3. Inductive Proofs 34
2.4. Complete Induction 44
2.5. Proving Properties of Programs 52
2.6. Recursive Definitions 59
2.7. Recursive Functions 69
2.8. Merge Sort: A Recursive Sorting Algorithm 75
2.9. Proving Properties of Recursive Programs 84
2.10. Summary of Chapter 2 87
2.11. Bibliographic Notes for Chapter 2 88
Chapter 3. The Running Time of Programs 89
3.1. What This Chapter Is About 89
3.2. Choosing an Algorithm 90
3.3. Measuring Running Time 91
3.4. Big-Oh and Approximate Running Time 96
3.5. Simplifying Big-Oh Expressions 101
3.6. Analyzing the Running Time of a Program 109
3.7. A Recursive Rule for Bounding Running Time 116
3.8. Analyzing Programs with Function Calls 127
3.9. Analyzing Recursive Functions 132
3.10. Analysis of Merge Sort 136
3.11. Solving Recurrence Relations 144
3.12. Summary of Chapter 3 154
3.13. Bibliographic Notes for Chapter 3 155
Chapter 4. Combinatorics and Probability 156
4.1. What This Chapter Is About 156
4.2. Counting Assignments 157
4.3. Counting Permutations 160
4.4. Ordered Selections 167
vi TABLEOFCONTENTS
4.5. Unordered Selections 170
4.6. Orderings With Identical Items 178
4.7. Distribution of Objects to Bins 181
4.8. Combining Counting Rules 184
4.9. Introduction to Probability Theory 187
4.10. Conditional Probability 193
4.11. Probabilistic Reasoning 203
4.12. Expected Value Calculations 212
4.13. Some Programming Applications of Probability 215
4.14. Summary of Chapter 4 220
4.15. Bibliographic Notes for Chapter 4 221
Chapter 5. The Tree Data Model 223
5.1. What This Chapter Is About 223
5.2. Basic Terminology 224
5.3. Data Structures for Trees 231
5.4. Recursions on Trees 239
5.5. Structural Induction 248
5.6. Binary Trees 253
5.7. Binary Search Trees 258
5.8. Efficiency of Binary Search Tree Operations 268
5.9. Priority Queues and Partially Ordered Trees 271
5.10. Heapsort: Sorting with Balanced POTs 280
5.11. Summary of Chapter 5 284
5.12. Bibliographic Notes for Chapter 5 285
Chapter 6. The List Data Model 286
6.1. What This Chapter Is About 286
6.2. Basic Terminology 287
6.3. Operations on Lists 291
6.4. The Linked-List Data Structure 293
6.5. Array-BasedImplementation of Lists 301
6.6. Stacks 306
6.7. Implementing Function Calls Using a Stack 312
6.8. Queues 318
6.9. Longest Common Subsequences 321
6.10. Representing Character Strings 327
6.11. Summary of Chapter 6 334
6.12. Bibliographic Notes for Chapter 6 335
Chapter 7. The Set Data Model 337
7.1. What This Chapter Is About 337
7.2. Basic Definitions 338
7.3. Operations on Sets 342
7.4. List Implementation of Sets 351
7.5. Characteristic-Vector Implementation of Sets 357
7.6. Hashing 360
7.7. Relations and Functions 366
7.8. Implementing Functions as Data 373
7.9. Implementing Binary Relations 380
TABLEOFCONTENTS vii
7.10. Some Special Properties of Binary Relations 386
7.11. Infinite Sets 396
7.12. Summary of Chapter 7 401
7.13. Bibliographic Notes for Chapter 7 402
Chapter 8. The Relational Data Model 403
8.1. What This Chapter Is About 403
8.2. Relations 404
8.3. Keys 411
8.4. Primary Storage Structures for Relations 414
8.5. Secondary Index Structures 419
8.6. Navigation among Relations 423
8.7. An Algebra of Relations 428
8.8. Implementing Relational Algebra Operations 436
8.9. Algebraic Laws for Relations 440
8.10. Summary of Chapter 8 449
8.11. Bibliographic Notes for Chapter 8 450
Chapter 9. The Graph Data Model 451
9.1. What This Chapter Is About 451
9.2. Basic Concepts 452
9.3. Implementation of Graphs 459
9.4. Connected Components of an Undirected Graph 466
9.5. Minimal Spanning Trees 478
9.6. Depth-First Search 484
9.7. Some Uses of Depth-First Search 495
9.8. Dijkstra’s Algorithm for Finding Shortest Paths 502
9.9. Floyd’s Algorithm for Shortest Paths 513
9.10. An Introduction to Graph Theory 521
9.11. Summary of Chapter 9 526
9.12. Bibliographic Notes for Chapter 9 527
Chapter 10. Patterns, Automata, and Regular Expressions 529
10.1. What This Chapter Is About 530
10.2. State Machines and Automata 530
10.3. Deterministic and Nondeterministic Automata 536
10.4. From Nondeterminism to Determinism 547
10.5. Regular Expressions 556
10.6. The UNIX Extensions to Regular Expressions 564
10.7. Algebraic Laws for Regular Expressions 568
10.8. From Regular Expressions to Automata 571
10.9. From Automata to Regular Expressions 582
10.10. Summary of Chapter 10 588
10.11. Bibliographic Notes for Chapter 10 589
Chapter 11. Recursive Description of Patterns 591
11.1. What This Chapter Is About 591
11.2. Context-Free Grammars 592
11.3. Languages from Grammars 599
11.4. Parse Trees 602
viii TABLEOFCONTENTS
11.5. Ambiguity and the Design of Grammars 610
11.6. Constructing Parse Trees 617
11.7. A Table-Driven Parsing Algorithm 625
11.8. Grammars Versus Regular Expressions 634
11.9. Summary of Chapter 11 640
11.10. Bibliographic Notes for Chapter 11 641
Chapter 12. Propositional Logic 642
12.1. What This Chapter Is About 642
12.2. What Is PropositionalLogic? 643
12.3. Logical Expressions 645
12.4. Truth Tables 649
12.5. From Boolean Functions to Logical Expressions 655
12.6. Designing Logical Expressions by KarnaughMaps 660
12.7. Tautologies 669
12.8. Some Algebraic Laws for Logical Expressions 674
12.9. Tautologies and Methods of Proof 682
12.10. Deduction 686
12.11. Proofs by Resolution 692
12.12. Summary of Chapter 12 697
12.13. Bibliographic Notes for Chapter 12 698
Chapter 13. Using Logic to Design Computer Components 699
13.1. What This Chapter is About 699
13.2. Gates 700
13.3. Circuits 701
13.4. Logical Expressions and Circuits 705
13.5. Some Physical Constraints on Circuits 711
13.6. A Divide-and-Conquer Addition Circuit 716
13.7. Design of a Multiplexer 723
13.8. Memory Elements 730
13.9. Summary of Chapter 13 731
13.10. Bibliographic Notes for Chapter 13 732
Chapter 14. Predicate Logic 733
14.1. What This Chapter Is About 733
14.2. Predicates 734
14.3. Logical Expressions 736
14.4. Quantifiers 739
14.5. Interpretations 745
14.6. Tautologies 751
14.7. Tautologies Involving Quantifiers 753
14.8. Proofs in Predicate Logic 759
14.9. Proofs from Rules and Facts 762
14.10. Truth and Provability 768
14.11. Summary of Chapter 14 774
14.12. Bibliographic Notes for Chapter 14 775
Index 776
Preface
Thisbookwasmotivatedbythedesireweandothershavehadtofurthertheevolu-
tion of the core course in computer science. Many departments across the country
have revised their curriculum in response to the introductory course in the science
ofcomputing discussedin the “Denning Report,” (Denning, P.J., D. E.Comer, D.
Gries,M.C.Mulder,A. Tucker,J.Turner,andP.R.Young,“Computingas aDis-
cipline,”Comm.ACM32:1,pp.9–23,January1989.). Thatreportdrawsattention
to three working methodologies or processes — theory, abstraction, and design —
as fundamental to all undergraduate programs in the discipline. More recently,
theComputing Curricula 1991 reportofthejointACM/IEEE-CSCurriculumTask
Force echoes the Denning Report in identifying key recurring concepts which are
fundamental to computing, especially: conceptual and formal models, efficiency,
andlevelsofabstraction. Thethemesofthesetworeportssummarizewhatwehave
tried to offer the student in this book.
This book developedfrom notes for a two-quartercourse at Stanford— called
CS109: Introduction to Computer Science — that serves a number of goals. The
first goal is to give beginning computer science majors a solid foundation for fur-
ther study. However, computing is becoming increasingly important in a much
wider range of scientific and engineering disciplines. Therefore, a second goal is
to give those students who will not take advanced courses in computer science the
conceptual tools that the field provides. Finally, a more pervasive goalis to expose
all students not only to programming concepts but also to the intellectually rich
foundations of the field.
OurfirstversionofthisbookwasbasedonprogramminginPascalandappeared
in 1992. Our choice of Pascalas the languagefor example programswas motivated
by that language’s use in the Computer Science Advanced Placement Test as well
as in a plurality of college introductions to programming. We were pleased to see
that since 1992 there has been a significant trend toward C as the introductory
programming language, and we accordingly developed a new version of the book
usingCforprogrammingexamples. Ouremphasisonabstractionandencapsulation
should provide a good foundation for subsequent courses covering object-oriented
technology using C++.
At the same time, we decided to make two significant improvements in the
content of the book. First, although it is useful to have a grounding in machine
architecture to motivate how running time is measured, we found that almost all
curriculaseparatearchitectureintoaseparatecourse,sothechapteronthatsubject
was not useful. Second, many introductory courses in the theory of computing
emphasize combinatorics and probability, so we decided to increase the coverage
and cluster the material into a chapter of its own.
Foundations of Computer Science covers subjects that are often found split
betweenadiscretemathematicscourseandasophomore-levelsequenceincomputer
science in data structures. It has been our intention to select the mathematical
foundations with an eye toward what the computer user really needs, rather than
what a mathematician might choose. We have tried to integrate effectively the
mathematical foundations with the computing. We thus hope to provide a better
feelfor the soulofcomputer sciencethanmightbe foundinaprogrammingcourse,
ix
x PREFACE
a discrete mathematics course, or a course in a computer science subspecialty. We
believe that, as time goes on, all scientists and engineers will take a foundational
coursesimilarto the one offeredatStanfordupon whichthis bookis based. Sucha
courseincomputerscienceshouldbecomeasstandardassimilarcoursesincalculus
and physics.
Prerequisites
Students taking courses based on this book have ranged from first-year undergrad-
uates to graduate students. We assume only that students have had a solid course
in programming. They should be familiar with the programming language ANSI
C to use this edition. In particular, we expect students to be comfortable with C
constructssuchasrecursivefunctions,structures,pointers,andoperatorsinvolving
pointers and structures such as dot, ->, and &.
Suggested Outlines for Foundational Courses in CS
In terms of a traditional computer science curriculum, the book combines a first
course in data structures — that is, a “CS2” course — with a course in discrete
mathematics. We believe that the integration of these subjects is extremely desir-
able for two reasons:
1. Ithelpsmotivatethemathematicsbyrelatingitmorecloselytothecomputing.
2. Computing and mathematics can be mutually reinforcing. Some examples
are the way recursive programmingand mathematical induction are related in
Chapter 2 and the way the free/bound variable distinction for logic is related
tothescopeofvariablesinprogramminglanguagesinChapter14. Suggestions
for instructive programming assignments are presented throughout the book.
There are a number of ways in which this book can be used.
A Two-Quarter or Two-Semester Course
The CS109A-B sequence at Stanford is typical of what might be done in two quar-
ters,althoughthesecoursesareratherintensive,being4-unit,10-weekcourseseach.
These two courses cover the entire book, the first seven chapters in CS109A and
Chapters 8 through 14 in CS109B.
A One-Semester “CS2” Type Course
Itispossibletousethebookforaone-semestercoursecoveringasetoftopicssimilar
to what would appear in a “CS2” course. Naturally, there is too much material in
the book to cover in one semester, and so we recommend the following:
1. Recursive algorithms and programs in Sections 2.7 and 2.8.
2. Big-oh analysis and running time of programs: all of Chapter 3 except for
Section 3.11 on solving recurrence relations.
3. Trees in Sections 5.2 through 5.10.
PREFACE xi
4. Lists: all of Chapter 6. Some may wish to cover lists before trees, which
is a more traditional treatment. We regard trees as the more fundamental
notion, but there is little harm in switching the order. The only significant
dependency is that Chapter 6 talks about the “dictionary” abstract data type
(set with operations insert, delete, and lookup), which is introduced in Section
5.7 as a concept in connection with binary search trees.
5. Sets and relations. Data structures for sets and relations are emphasized in
Sections 7.2 through 7.9 and 8.2 through 8.6.
6. Graph algorithms are covered in Sections 9.2 through 9.9.
A One-Semester Discrete Mathematics Course
For a one-semester course emphasizing mathematical foundations, the instructor
could choose to cover:
1. Mathematical induction and recursive programs in Chapter 2.
2. Big-oh analysis, runningtime, and recurrencerelationsinSections3.4through
3.11.
3. Combinatorics in Sections 4.2 through 4.8.
4. Discrete probability in Sections 4.9 through 4.13.
5. Mathematical aspects of trees in Sections 5.2 through 5.6.
6. Mathematical aspects of sets in Sections 7.2, 7.3, 7.7, 7.10, and 7.11.
7. The algebra of relations in Sections 8.2, 8.7, and 8.9.
8. Graph algorithms and graph theory in Chapter 9.
9. Automata and regular expressions in Chapter 10.
10. Context-free grammars in Sections 11.2 through 11.4.
11. Propositional and predicate logic in Chapters 12 and 14, respectively.
Features of This Book
Tohelpthestudentassimilatethematerial,wehaveaddedthefollowingstudyaids:
1. Each chapter has an outline section at the beginning and a summary section
at the end highlighting the main points.
2. Marginalnotesmarkimportantconceptsanddefinitions. However,itemsmen-
tioned in section or subsection headlines are not repeated in the margin.
3. “Sidebars”areseparatedfromthetextbydoublelines. Theseshortnotesserve
several purposes:
F Someareelaborationsonthetextormakesomefinepointsaboutprogram
or algorithm design.
F Others are for summary or emphasis of points made in the text nearby.
These include outlines of important kinds of proofs, such as the various
forms of proof by induction.
F Afewareusedtogiveexamplesoffallaciousarguments,andwehopethat
the separation from the text in this way will eliminate possible miscon-
struction of the point.
xii PREFACE
F A few give very brief introductions to major topics like undecidability or
the history of computers to which we wish we could devote a full section.
4. Most of the sections end with exercises. There are more than 1000 exercises
or parts spread among the sections. Of these roughly 30% are marked with a
single star, whichindicates that they require more thoughtthan the unstarred
exercises. Approximatelyanother10%ofthe exercisesaredoubly starred,and
these are the most challenging.
5. Chapters end with bibliographic notes. We have not attempted to be exhaus-
tive,butoffersuggestionsformoreadvancedtextsonthesubjectofthechapter
and mention the relevant papers with the most historical significance.
About the Cover
Itisatraditionforcomputersciencetextstohaveacoverwithacartoonordrawing
symbolizingthecontentofthebook. Here,wehavedrawnonthemythoftheworld
as the back of a turtle, but our world is populated with representatives of some of
the other, more advanced texts in computer science that this book is intended to
support. They are:
The teddy bear: R. Sethi, Programming Languages: Concepts and Constructs,
Addison-Wesley, Reading, Mass., 1989.
The baseball player: J. D. Ullman, Principles of Database and Knowledge-Base
Systems, Computer Science Press, New York, 1988.
Thecolumn: J.L.HennessyandD.A.Patterson,ComputerArchitecture: aQuan-
titative Approach, Morgan-Kaufmann,San Mateo, Calif., 1990.
The dragon: A. V. Aho, R. Sethi, and J. D. Ullman, Compiler Design: Principles,
Techniques, and Tools, Addison-Wesley, Reading, Mass., 1986.
The triceratops: J. L. Peterson and A. Silberschatz, Operating Systems Concepts,
second edition, Addison-Wesley, Reading, Mass., 1985.
Acknowledgments
We are deeply indebted to a number of colleaguesand students who have read this
material and given us many valuable suggestions for improving the presentation.
We owe a special debt of gratitude to Brian Kernighan, Don Knuth, Apostolos
Lerios, and Bob Martin who read the original Pascal manuscript in detail and
gaveus manyperceptive comments. We havereceived,andgratefullyacknowledge,
reports of course testing of the notes for the Pascaledition of this book by Michael
Anderson, MargaretJohnson, Udi Manber, JosephNaor, Prabhakar Ragde, Rocky
Ross, and Shuky Sagiv.
There are a number of other people who found errors in earlier editions, both
the original notes and the various printings of the Pascal edition. In this regard,
we would like to thank: Susan Aho, Michael Anderson, Aaron Edsinger, Lonnie
Eldridge,Todd Feldman, Steve Friedland, Christopher Fuselier,Mike Genstil, Paul
GrubbIII,BarryHayes,JohnHwang,HakanJakobsson,ArthurKeller,DeanKelley,
James Kuffner Jr., Steve Lindell, Richard Long, Mark MacDonald, Simone Mar-
tini, Hugh McGuire, Alan Morgan, Monnia Oropeza, Rodrigo Philander, Andrew
Quan, Stuart Reges, John Stone, Keith Swanson, Steve Swenson, Sanjai Tiwari,
Eric Traut, and Lynzi Ziegenhagen.
PREFACE xiii
We acknowledge helpful advice from Geoff Clem, Jon Kettenring, and Brian
Kernighan during the preparation of the C edition of Foundations of Computer
Science.
PeterUllmanproducedanumberofthefiguresusedinthisbook. Wearegrate-
ful to Dan Clayton, Anthony Dayao, Mat Howard, and Ron Underwood for help
withTEXfonts,andtoHesterGlynnandAnne Smithforhelpwiththe manuscript
preparation.
On-Line Access to Code, Errata, and Notes
Youcanobtaincopiesofthemajorprogramsinthisbookbyanonymousftptohost
ftp-cs.stanford.edu. Login with user name anonymousand give your name and
host as a password. You may then execute
cd fcsc
whereyouwillfindprogramsfromthisbook. We alsoplantokeepinthisdirectory
information about errata and what course notes we can provide.
A. V. A.
Chatham, NJ
J. D. U.
Stanford, CA
July, 1994