Introduction to the Theory of Computation

481 Pages·2012·7.43 MB·English

by  Michael Sipser

Intro duction to the Theory of
C N
o m p u t a t i o
t E
hird ditioN

m i C h a E l s i p s E r
Australia • Brazil • Japan • Korea • Mexico • Singapore • Spain • United Kingdom • United States
Copyright 2013 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from
the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to
remove additional content at any time if subsequent rights restrictions require it.
Introduction to the Theory of © 2013 Cengage Learning
Computation, Third International
CENGAGE and CENGAGE LEARNING are registered trademarks of
Edition
Cengage Learning, Inc., within the United States and certain other
Michael Sipser
jurisdictions.
Editor-in-Chief: Marie Lee ALL RIGHTS RESERVED. No part of this work covered by the copyright
Senior Product Manager: herein may be reproduced, transmitted, stored or used in any form or
Alyssa Pratt by any means graphic, electronic, or mechanical, including but not lim-
Associate Product Manager: ited to photocopying, recording, scanning, digitizing, taping, Web distri-
Stephanie Lorenz bution, information networks, or information storage and retrieval sys-
Content Project Manager: tems, except as permitted under Section 107 or 108 of the 1976 United
Jennifer Feltri-George States Copyright Act, without the prior written permission of the
Art Director: GEX Publishing Services publisher.States Copyright Act, or applicable copyright law of another
jurisdiction, without the prior written permission of the publisher.
Associate Marketing Manager:
Shanna Shelton
Cover Designer: GEX Publishing For permission to use material from this text or product,
Services submit all requests online at cengage.com/permissions
Cover Image Credit: © SuperStock / Further permissions questions can be emailed to
SuperStock [email protected]
Library of Congress Control Number: 2012938665
International Edition:
ISBN-13: 978-1-133-18781-3
ISBN-10: 1-133-18781-1
Cengage Learning International Offices
Asia Australia/New Zealand
www.cengageasia.com www.cengage.com.au
tel: [phone] tel: [phone]
Brazil India
www.cengage.com.br www.cengage.co.in
tel: [phone] tel: [phone]
Latin America UK/Europe/Middle East/Africa
www.cengage.com.mx www.cengage.co.uk
tel: [phone] tel: [phone]
Represented in Canada by Nelson
Education, Ltd.
tel: [phone]
www.nelson.com
Cengage Learning is a leading provider of customized learning solu-
tions with office locations around the globe, including Singapore, the
United Kingdom, Australia, Mexico, Brazil, and Japan. Locate your
local office at: international.cengage.com/region
For product information: www.cengage.com/international
Visit your local office: www.cengage.com/global
Visit our corporate website: www.cengage.com
Cengage Learning reserves the right to revise this publication and
make changes from time to time in its content without notice.
The programs in this book are for instructional purposes only.They
have been tested with care, but are not guaranteed for any particular
Printed in Canada intent beyond educational purposes. The author and the publisher do
not offer any warranties or representations, nor do they accept any
1 2 3 4 5 6 7 8 16 15 14 13 12
liabilities with respect to the programs.
Copyright 2013 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from
the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to
remove additional content at any time if subsequent rights restrictions require it.
To Ina, Rachel, andAaron
Copyright 2013 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from
the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to
remove additional content at any time if subsequent rights restrictions require it.
Copyright 2013 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from
the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to
remove additional content at any time if subsequent rights restrictions require it.
C O N T E N T S
PrefacetotheFirstEdition xi
Tothestudent . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
Totheeducator . . . . . . . . . . . . . . . . . . . . . . . . . . xii
Thefirstedition . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
Feedbacktotheauthor . . . . . . . . . . . . . . . . . . . . . . xiii
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . xiv
PrefacetotheSecondEdition xvii
PrefacetotheThirdEdition xxi
0 Introduction 1
0.1 Automata,Computability,andComplexity . . . . . . . . . . . . . 1
Complexitytheory . . . . . . . . . . . . . . . . . . . . . . . . . 2
Computabilitytheory . . . . . . . . . . . . . . . . . . . . . . . 3
Automatatheory . . . . . . . . . . . . . . . . . . . . . . . . . . 3
0.2 MathematicalNotionsandTerminology . . . . . . . . . . . . . . 3
Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Sequencesandtuples . . . . . . . . . . . . . . . . . . . . . . . 6
Functionsandrelations . . . . . . . . . . . . . . . . . . . . . . 7
Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Stringsandlanguages . . . . . . . . . . . . . . . . . . . . . . . 13
Booleanlogic. . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Summaryofmathematicalterms . . . . . . . . . . . . . . . . . 16
0.3 Definitions,Theorems,andProofs . . . . . . . . . . . . . . . . . 17
Findingproofs . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
0.4 TypesofProof . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Proofbyconstruction . . . . . . . . . . . . . . . . . . . . . . . 21
Proofbycontradiction. . . . . . . . . . . . . . . . . . . . . . . 21
Proofbyinduction . . . . . . . . . . . . . . . . . . . . . . . . . 22
Exercises,Problems,andSolutions . . . . . . . . . . . . . . . . . . . 25
v
Copyright 2013 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from
the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to
remove additional content at any time if subsequent rights restrictions require it.
vi CONTENTS
Part One: Automata and Languages 29
1 RegularLanguages 31
1.1 FiniteAutomata . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
Formaldefinitionofafiniteautomaton . . . . . . . . . . . . . 35
Examplesoffiniteautomata . . . . . . . . . . . . . . . . . . . . 37
Formaldefinitionofcomputation . . . . . . . . . . . . . . . . 40
Designingfiniteautomata . . . . . . . . . . . . . . . . . . . . . 41
Theregularoperations . . . . . . . . . . . . . . . . . . . . . . 44
1.2 Nondeterminism . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
Formaldefinitionofanondeterministicfiniteautomaton. . . . 53
EquivalenceofNFAsandDFAs . . . . . . . . . . . . . . . . . 54
Closureundertheregularoperations. . . . . . . . . . . . . . . 58
1.3 RegularExpressions . . . . . . . . . . . . . . . . . . . . . . . . . 63
Formaldefinitionofaregularexpression . . . . . . . . . . . . 64
Equivalencewithfiniteautomata . . . . . . . . . . . . . . . . . 66
1.4 NonregularLanguages . . . . . . . . . . . . . . . . . . . . . . . . 77
Thepumpinglemmaforregularlanguages . . . . . . . . . . . 77
Exercises,Problems,andSolutions . . . . . . . . . . . . . . . . . . . 82
2 Context-FreeLanguages 101
2.1 Context-FreeGrammars . . . . . . . . . . . . . . . . . . . . . . . 102
Formaldefinitionofacontext-freegrammar . . . . . . . . . . 104
Examplesofcontext-freegrammars . . . . . . . . . . . . . . . 105
Designingcontext-freegrammars . . . . . . . . . . . . . . . . 106
Ambiguity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
Chomskynormalform . . . . . . . . . . . . . . . . . . . . . . 108
2.2 PushdownAutomata . . . . . . . . . . . . . . . . . . . . . . . . . 111
Formaldefinitionofapushdownautomaton. . . . . . . . . . . 113
Examplesofpushdownautomata . . . . . . . . . . . . . . . . . 114
Equivalencewithcontext-freegrammars . . . . . . . . . . . . . 117
2.3 Non-Context-FreeLanguages. . . . . . . . . . . . . . . . . . . . 125
Thepumpinglemmaforcontext-freelanguages. . . . . . . . . 125
2.4 DeterministicContext-FreeLanguages. . . . . . . . . . . . . . . 130
PropertiesofDCFLs . . . . . . . . . . . . . . . . . . . . . . . 133
Deterministiccontext-freegrammars . . . . . . . . . . . . . . 135
RelationshipofDPDAsandDCFGs . . . . . . . . . . . . . . . 146
ParsingandLR(k)Grammars . . . . . . . . . . . . . . . . . . . 151
Exercises,Problems,andSolutions . . . . . . . . . . . . . . . . . . . 154
Part Two: Computability Theory 163
3 TheChurch–TuringThesis 165
3.1 TuringMachines . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
FormaldefinitionofaTuringmachine . . . . . . . . . . . . . . 167
Copyright 2013 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from
the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to
remove additional content at any time if subsequent rights restrictions require it.
CONTENTS vii
ExamplesofTuringmachines . . . . . . . . . . . . . . . . . . . 170
3.2 VariantsofTuringMachines . . . . . . . . . . . . . . . . . . . . . 176
MultitapeTuringmachines . . . . . . . . . . . . . . . . . . . . 176
NondeterministicTuringmachines . . . . . . . . . . . . . . . . 178
Enumerators . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
Equivalencewithothermodels . . . . . . . . . . . . . . . . . . 181
3.3 TheDefinitionofAlgorithm . . . . . . . . . . . . . . . . . . . . 182
Hilbert’sproblems . . . . . . . . . . . . . . . . . . . . . . . . . 182
TerminologyfordescribingTuringmachines . . . . . . . . . . 184
Exercises,Problems,andSolutions . . . . . . . . . . . . . . . . . . . 187
4 Decidability 193
4.1 DecidableLanguages. . . . . . . . . . . . . . . . . . . . . . . . . 194
Decidableproblemsconcerningregularlanguages . . . . . . . 194
Decidableproblemsconcerningcontext-freelanguages. . . . . 198
4.2 Undecidability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
Thediagonalizationmethod . . . . . . . . . . . . . . . . . . . 202
Anundecidablelanguage . . . . . . . . . . . . . . . . . . . . . 207
ATuring-unrecognizablelanguage . . . . . . . . . . . . . . . . 209
Exercises,Problems,andSolutions . . . . . . . . . . . . . . . . . . . 210
5 Reducibility 215
5.1 UndecidableProblemsfromLanguageTheory . . . . . . . . . . 216
Reductionsviacomputationhistories. . . . . . . . . . . . . . . 220
5.2 ASimpleUndecidableProblem . . . . . . . . . . . . . . . . . . . 227
5.3 MappingReducibility . . . . . . . . . . . . . . . . . . . . . . . . 234
Computablefunctions . . . . . . . . . . . . . . . . . . . . . . . 234
Formaldefinitionofmappingreducibility . . . . . . . . . . . . 235
Exercises,Problems,andSolutions . . . . . . . . . . . . . . . . . . . 239
6 AdvancedTopicsinComputabilityTheory 245
6.1 TheRecursionTheorem. . . . . . . . . . . . . . . . . . . . . . . 245
Self-reference . . . . . . . . . . . . . . . . . . . . . . . . . . . 246
Terminologyfortherecursiontheorem . . . . . . . . . . . . . 249
Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250
6.2 Decidabilityoflogicaltheories . . . . . . . . . . . . . . . . . . . 252
Adecidabletheory . . . . . . . . . . . . . . . . . . . . . . . . . 255
Anundecidabletheory. . . . . . . . . . . . . . . . . . . . . . . 257
6.3 TuringReducibility. . . . . . . . . . . . . . . . . . . . . . . . . . 260
6.4 ADefinitionofInformation . . . . . . . . . . . . . . . . . . . . . 261
Minimallengthdescriptions . . . . . . . . . . . . . . . . . . . 262
Optimalityofthedefinition . . . . . . . . . . . . . . . . . . . . 266
Incompressiblestringsandrandomness . . . . . . . . . . . . . 267
Exercises,Problems,andSolutions . . . . . . . . . . . . . . . . . . . 270
Copyright 2013 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from
the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to
remove additional content at any time if subsequent rights restrictions require it.
viii CONTENTS
Part Three: Complexity Theory 273
7 TimeComplexity 275
7.1 MeasuringComplexity . . . . . . . . . . . . . . . . . . . . . . . . 275
Big-Oandsmall-onotation . . . . . . . . . . . . . . . . . . . . 276
Analyzingalgorithms . . . . . . . . . . . . . . . . . . . . . . . 279
Complexityrelationshipsamongmodels . . . . . . . . . . . . . 282
7.2 TheClassP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284
Polynomialtime . . . . . . . . . . . . . . . . . . . . . . . . . . 284
ExamplesofproblemsinP . . . . . . . . . . . . . . . . . . . . 286
7.3 TheClassNP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292
ExamplesofproblemsinNP . . . . . . . . . . . . . . . . . . . 295
ThePversusNPquestion . . . . . . . . . . . . . . . . . . . . 297
7.4 NP-completeness . . . . . . . . . . . . . . . . . . . . . . . . . . . 299
Polynomialtimereducibility . . . . . . . . . . . . . . . . . . . 300
DefinitionofNP-completeness . . . . . . . . . . . . . . . . . . 304
TheCook–LevinTheorem . . . . . . . . . . . . . . . . . . . . 304
7.5 AdditionalNP-completeProblems . . . . . . . . . . . . . . . . . 311
Thevertexcoverproblem . . . . . . . . . . . . . . . . . . . . . 312
TheHamiltonianpathproblem . . . . . . . . . . . . . . . . . 314
Thesubsetsumproblem . . . . . . . . . . . . . . . . . . . . . 319
Exercises,Problems,andSolutions . . . . . . . . . . . . . . . . . . . 322
8 SpaceComplexity 331
8.1 Savitch’sTheorem . . . . . . . . . . . . . . . . . . . . . . . . . . 333
8.2 TheClassPSPACE . . . . . . . . . . . . . . . . . . . . . . . . . 336
8.3 PSPACE-completeness . . . . . . . . . . . . . . . . . . . . . . . 337
TheTQBFproblem . . . . . . . . . . . . . . . . . . . . . . . . 338
Winningstrategiesforgames . . . . . . . . . . . . . . . . . . . 341
Generalizedgeography . . . . . . . . . . . . . . . . . . . . . . 343
8.4 TheClassesLandNL . . . . . . . . . . . . . . . . . . . . . . . . 348
8.5 NL-completeness . . . . . . . . . . . . . . . . . . . . . . . . . . 351
Searchingingraphs . . . . . . . . . . . . . . . . . . . . . . . . 353
8.6 NLequalscoNL . . . . . . . . . . . . . . . . . . . . . . . . . . . 354
Exercises,Problems,andSolutions . . . . . . . . . . . . . . . . . . . 356
9 Intractability 363
9.1 HierarchyTheorems . . . . . . . . . . . . . . . . . . . . . . . . . 364
Exponentialspacecompleteness . . . . . . . . . . . . . . . . . 371
9.2 Relativization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376
Limitsofthediagonalizationmethod . . . . . . . . . . . . . . 377
9.3 CircuitComplexity . . . . . . . . . . . . . . . . . . . . . . . . . . 379
Exercises,Problems,andSolutions . . . . . . . . . . . . . . . . . . . 388
10 AdvancedTopicsinComplexityTheory 393
10.1 ApproximationAlgorithms . . . . . . . . . . . . . . . . . . . . . 393
Copyright 2013 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from
the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to
remove additional content at any time if subsequent rights restrictions require it.
CONTENTS ix
10.2 ProbabilisticAlgorithms . . . . . . . . . . . . . . . . . . . . . . . 396
TheclassBPP . . . . . . . . . . . . . . . . . . . . . . . . . . . 396
Primality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399
Read-oncebranchingprograms . . . . . . . . . . . . . . . . . . 404
10.3 Alternation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408
Alternatingtimeandspace . . . . . . . . . . . . . . . . . . . . 410
ThePolynomialtimehierarchy . . . . . . . . . . . . . . . . . . 414
10.4 InteractiveProofSystems . . . . . . . . . . . . . . . . . . . . . . 415
Graphnonisomorphism . . . . . . . . . . . . . . . . . . . . . . 415
Definitionofthemodel . . . . . . . . . . . . . . . . . . . . . . 416
IP=PSPACE . . . . . . . . . . . . . . . . . . . . . . . . . . . 418
10.5 ParallelComputation . . . . . . . . . . . . . . . . . . . . . . . . 427
UniformBooleancircuits . . . . . . . . . . . . . . . . . . . . . 428
TheclassNC . . . . . . . . . . . . . . . . . . . . . . . . . . . 430
P-completeness . . . . . . . . . . . . . . . . . . . . . . . . . . 432
10.6 Cryptography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433
Secretkeys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433
Public-keycryptosystems . . . . . . . . . . . . . . . . . . . . . 435
One-wayfunctions. . . . . . . . . . . . . . . . . . . . . . . . . 435
Trapdoorfunctions . . . . . . . . . . . . . . . . . . . . . . . . 437
Exercises,Problems,andSolutions . . . . . . . . . . . . . . . . . . . 439
SelectedBibliography 443
Index 448
Copyright 2013 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part. Due to electronic rights, some third party content may be suppressed from
the eBook and/or eChapter(s). Editorial review has deemed that any suppressed content does not materially affect the overall learning experience. Cengage Learning reserves the right to
remove additional content at any time if subsequent rights restrictions require it.