Table Of ContentDecision Engineering
Series Editor
Professor Rajkumar Roy
Department of Enterprise Integration
School of Industrial and Manufacturing Science
Cranfield University
Cranfield
Bedford
MK43 0AL
UK
Other titles published in this series
Cost Engineering in Practice
John McIlwraith
IPA – Concepts and Applications in Engineering
Jerzy Pokojski
Strategic Decision Making
Navneet Bhushan and Kanwal Rai
Product Lifecycle Management
John Stark
From Product Description to Cost: A Practical Approach
Volume 1: The Parametric Approach
Pierre Foussier
From Product Description to Cost: A Practical Approach
Volume 2: Building a Specific Model
Pierre Foussier
Decision-Making in Engineering Design
Yotaro Hatamura
Composite Systems Decisions
Mark Sh. Levin
Intelligent Decision-making Support Systems
Jatinder N.D. Gupta, Guisseppi A. Forgionne and Manuel Mora T.
Knowledge Acquisition in Practice
N.R. Milton
Global Product: Strategy, Product Lifecycle Management and the Billion Customer Question
John Stark
Enabling a Simulation Capability in the Organisation
Andrew Greasley
Network Models and Optimization
Mitsuo Gen, Runewei Cheng and Lin Lin
Management of Uncertainty
Gudela Grote
Xinjie Yu · Mitsuo Gen
Introduction
to Evolutionary Algorithms
123
Xinjie Yu, PhD Mitsuo Gen, PhD
Department of Electrical Engineering Fuzzy Logic Systems Institute (FLSI)
Tsinghua University 680-41 Kawazu
100084 Beijing 820-0067 Iizuka
China Japan
[email protected] [email protected]
ISSN 1619-5736
ISBN 978-1-84996-128-8 e-ISBN 978-1-84996-129-5
DOI 10.1007/978-1-84996-129-5
Springer London Dordrecht Heidelberg New York
British Library Cataloguing in Publication Data
A catalogue record for this book is available from the British Library
Library of Congress Control Number: 2010929767
© Springer-Verlag London Limited 2010
Discipulus is a trademark of RML Technologies, 7606 S. Newland St., Littleton, CO 80128, USA,
www.rmltech.com
Mathematica® is a registered trademark of Wolfram Research, Inc., 100 Trade Center Drive,
Champaign, IL 61820-7237, USA, www.wolfram.com
MATLAB® is a registered trademark of The MathWorks, Inc., 3 Apple Hill Drive, Natick, MA,
01760-2098 USA, www.mathworks.com
Apart from any fair dealing for the purposes of research or private study, or criticism or review, as
permitted under the Copyright, Designs and Patents Act 1988, this publication may only be
reproduced, stored or transmitted, in any form or by any means, with the prior permission in writing of
the publishers, or in the case of reprographic reproduction in accordance with the terms of licences
issued by the Copyright Licensing Agency. Enquiries concerning reproduction outside those terms
should be sent to the publishers.
The use of registered names, trademarks, etc. in this publication does not imply, even in the absence of
a specific statement, that such names are exempt from the relevant laws and regulations and therefore
free for general use.
The publisher makes no representation, express or implied, with regard to the accuracy of the
information contained in this book and cannot accept any legal responsibility or liability for any errors
or omissions that may be made.
Cover design: eStudioCalamar, Figueres/Berlin
Printed on acid-free paper
Springer is part of Springer Science+Business Media (www.springer.com)
Toourfamilies,friends,andstudents
Preface
Thisisatextbookonevolutionaryalgorithms(EAs).Inpreparingtheproposaland
themanuscript,thefollowingquestionswerealwayskeptinourminds.
∙ Isthisbookconvenientforteaching,studying,andself-study,i.e.,havethecon-
tentsbeenarrangedinapedagogicallysoundway?
∙ DoesthisbookintroducethestateoftheartofEAs?
∙ Doesthisbookcoverthecontentsascomprehensivelyaspossible?
∙ Does this book contain specific programming codes so that the reader can use
themdirectly?
Forthefirsttwoquestions,wewouldliketosayyes.Wewantthistextbooktobe
intuitivebecauseintuitiveideas,andnotstrictproofs,leadtothekindofinnovation
weseek.Wealsowanttobuildabridgeconnectingthebasicsandthecuttingedge
sothatourstudentscanreachthepeakquicklyyetwithasolidgraspofthemate-
rial. Our answer to the remaining two questions is no. This textbook is neither an
encyclopedianoracookbook.Weselectedonlyinterestingandimportanttopicsand
leftthereadertoimplementthecodesforthealgorithmsafterwehavedeliberately
removedallunderstandingobstacles.Thereasonforthelatterconsiderationliesin
thebeliefcontainedinthefollowingsaying:“TellmeandI’llforget;showmeandI
mayremember;letmetry,andIcanunderstand.”
EAshaveattractedconsiderableinterestinrecentyears.Tounderstandhow“hot”
the topic is, consider the number of papers indexed by the Science Citation Index
(SCI)everyyear.IfthereaderhastheaccesstotheSCI,he/sheisencouragedtodo
asearchtoverifythehotnessofthistopic.
Thereasonthetopicissohotismainlybecauseofitseffectsonvariousproblems,
i.e., it really works. We will introduce various application examples to show how
simpleideascanbeexpandedtosolvecomplexproblems.
Theprocedureof designingoranalyzing analgorithm forsolving optimization
orlearningproblemsisfullofchallenges,i.e.,problemsaredifficult.Thisisirrele-
vant.Wewillaccompanyandassistthereaderwhennecessary.Wewilldemonstrate
thebasicideas behindthescaryequationsand tryourbesttomakethingseasyto
understand.Incidentally,thiscumbersomeprocedureisalsointeresting.Beforeand
vii
viii Preface
duringtheprocedureofclimbingMountFuji,wethoughtitwouldbetorture.While
atthesummit,wefeltthatallthesufferingswereinteresting.
Theprerequisitesofthisbooklieintwoaspects.Inthemathematicspart,abasic
understandingoflinearalgebra,multivariablecalculus,probability,andstatisticsis
necessary.1 The other demand is derived from a proficiency in programming. We
assumeafirmgraspofatleastoneprogramminglanguage,i.e.,youcanimplement
an algorithm. Since these requirements are in the common curriculum of junior
undergraduateprograms,seniorundergraduateorgraduatestudentsshouldbeable
toreadthistextbookwithoutknowledgeobstacles.
Thepedagogicalapproachesusedinthistextcanbesummarizedasfollows:
∙ Buildingthemansionfromthebricks.Wewillalwaysfocusonthekeyelements
ofanalgorithmbecauselatertheymightbecomeyourtools.
∙ Fromspecifictogeneral.Wewillalwaysdiscussspecificsimpleexamplesbefore
formalexpressions.
∙ Fromideatoimplementation.Wewillalwaysintroducetheinitialnotionsbefore
discussingtheconcretecontents.
∙ Explaining the critical part but leaving questions unanswered deliberately. We
willleavesomeobstaclesdeliberatelyinthecontexttoactivateyourthoughts.
R.W. Hamming2 gave his motto as the following statement: “The purpose of
computingisinsight,notnumbers”inhisfamousbookNumericalMethodsforSci-
entistsandEngineers.Herewewouldliketouseitagaintorepresentourownatti-
tudetowardEAs.Asalgorithmdesigners,wecaremoreaboutthesolutionlandscape
oftheproblemandthecorrespondingsearchabilityofthealgorithms,3althoughwe
do seek the optimal solution to the problem. From this perspective, there will be
less “how-to” in this textbook for specific instructions. On the contrary, there will
beplentyof“why-to”stoexplaintheinsightsandtherationaleofagivenalgorithm
so that one can generate one’s own cookbook according to these “why-to”s. We
believethat’swhatthereaderwants!
Otherteachingandself-studyconsiderationsincludeplentyoffootnotesformen-
tioningeasilyignoredyetimportantpoints,“SuggestionsforFurtherReading”sec-
tionsinmostchaptersforfurtherstudy,exerciseswithvariousdegreesofdifficulty
todeepenone’sunderstandingandevenpromoteone’sown“small-scale”research.
The pedagogical relationship among the ten chapters of this textbook is illus-
tratedbyFig.0.1,wherethedottedcirclesseparatethetenchaptersintothreecat-
egories:thebasics,optimizationproblems,andotherbranches.Thesolidlinesare
the suggested pedagogical path for teaching or self-study. Chapters 1 to 3 are all
necessaryforsubsequentchapters.Afterthat,readerscanskiparounddependingon
personalinterests.Dashedlinesrepresenttheimprovements,e.g.,anunderstanding
1Knowledgeofoperationsresearch,e.g.,linearprogramming,nonlinearprogramming,andcom-
binatorialoptimization,mightcontributetoone’sunderstandingbutnotbenecessary.
2FatheroftheHammingdistance,whichwillbeusedofteninthistextbook.
3Thesetermswillbeexplainedexplicitlyandusedthroughoutthebook.Heretheycanbetaken
astheproceduretofindtheoptimalsolution.
Preface ix
ofChap.4mightcontributeto,butnotbeaprerequisiteof,understandingChap.6
andviceversa.WehopeFig.0.1helpsthereaderformamentalmapfordifferent
branchesandapplicationsofEAs.
(cid:2)
(cid:2)
(cid:2) .(cid:27)(cid:10)(cid:9)(cid:2)(cid:7)(cid:6)(cid:2)(cid:22)(cid:23)(cid:24)(cid:2)
(cid:2)
(cid:2) (cid:3)(cid:4)(cid:5)(cid:6)(cid:2)(cid:7)(cid:8)(cid:9)(cid:10)(cid:11)(cid:12)(cid:13)(cid:14)(cid:9)(cid:15)(cid:11)(cid:8) *(cid:21)(cid:8)(cid:21)(cid:9)(cid:15)(cid:14)(cid:2)(cid:27)(cid:20)%(cid:11)(cid:10)(cid:15)(cid:9)(cid:4)(cid:18)(cid:2)
(cid:2) (cid:22)(cid:26)(cid:11)(cid:20)(cid:13)(cid:9)(cid:15)(cid:11)(cid:8)(cid:2)(cid:24)(cid:9)(cid:10)(cid:27)(cid:9)(cid:21)%((cid:2)
(cid:2) (cid:3)(cid:4)(cid:16)(cid:6)(cid:2)(cid:17)(cid:15)(cid:18)(cid:19)(cid:20)(cid:21)(cid:2)(cid:22)(cid:23)(cid:24) (cid:22)(cid:26)(cid:11)(cid:20)(cid:13)(cid:9)(cid:15)(cid:11)(cid:8)(cid:27)(cid:10)((cid:2)(cid:19)(cid:10)(cid:11)%(cid:10)(cid:27)(cid:18)(cid:18)(cid:15)(cid:8)%
(cid:2) (cid:17)(cid:15)(cid:18)(cid:19)(cid:20)(cid:21)+(cid:2)(cid:24)(cid:21)(cid:27)(cid:10)(cid:14)(cid:4)(cid:2)
(cid:2) (cid:17)(cid:14)(cid:27)(cid:9)(cid:9)(cid:21)(cid:10)(cid:2)(cid:24)(cid:21)(cid:27)(cid:10)(cid:14)(cid:4)(cid:2)
(cid:3)(cid:4)(cid:25)(cid:6)(cid:2)(cid:23)(cid:12)(cid:26)(cid:27)(cid:8)(cid:14)(cid:21)(cid:12)(cid:2)(cid:22)(cid:23)(cid:24)
(cid:2) ,(cid:15)’’(cid:21)(cid:10)(cid:21)(cid:8)(cid:9)(cid:15)(cid:27)(cid:20)(cid:2)(cid:21)(cid:26)(cid:11)(cid:20)(cid:13)(cid:9)(cid:15)(cid:11)(cid:8)(cid:2)
(cid:2)
(cid:2) (cid:3)(cid:4)(cid:28)(cid:6)(cid:2)(cid:3)(cid:11)(cid:8)(cid:24)(cid:9)(cid:10)(cid:27)(cid:15)(cid:8)(cid:21)(cid:12) (cid:3)(cid:4)#(cid:6)(cid:2)(cid:17)$(cid:27)(cid:10)(cid:18)(cid:2)
(cid:2) (cid:15)(cid:8)(cid:9)(cid:21)(cid:20)(cid:20)(cid:15)%(cid:21)(cid:8)(cid:14)(cid:21)(cid:2)
(cid:2)
(cid:23)(cid:8)(cid:9)(cid:2)(cid:14)(cid:11)(cid:20)(cid:11)(cid:8)((cid:2)(cid:11)(cid:19)(cid:9)(cid:15)(cid:18)(cid:15)-(cid:27)(cid:9)(cid:15)(cid:11)(cid:8)(cid:2)
(cid:2) (cid:3)(cid:4)(cid:29)(cid:6)(cid:2)(cid:30)(cid:13)(cid:20)(cid:9)(cid:15)(cid:18)(cid:11)(cid:12)(cid:27)(cid:20)(cid:2) .(cid:27)(cid:10)(cid:9)(cid:15)(cid:14)(cid:20)(cid:21)(cid:2)(cid:24)$(cid:27)(cid:10)(cid:18)(cid:2)(cid:11)(cid:19)(cid:9)(cid:15)(cid:18)(cid:15)-(cid:27)(cid:9)(cid:15)(cid:11)(cid:8)
(cid:2)
(cid:2)
(cid:3)(cid:4)&(cid:6)(cid:2)(cid:23)(cid:10)(cid:9)(cid:15)’(cid:15)(cid:14)(cid:15)(cid:27)(cid:20)(cid:2)
(cid:2) (cid:3)(cid:4)(cid:31)(cid:6)(cid:2)(cid:30)(cid:13)(cid:20)(cid:9)(cid:15)(cid:11) !(cid:21)(cid:14)(cid:9)(cid:15)(cid:26)(cid:21)(cid:2) (cid:15)(cid:18)(cid:18)(cid:13)(cid:8)(cid:21)(cid:2)(cid:24)((cid:24)(cid:9)(cid:21)(cid:18)(cid:2)
(cid:2)
(cid:2)
(cid:3)(cid:4)(cid:5))(cid:6)(cid:2)*(cid:21)(cid:8)(cid:21)(cid:9)(cid:15)(cid:14)(cid:2)
(cid:2) (cid:3)(cid:4)"(cid:6)(cid:2)(cid:3)(cid:11)(cid:18) (cid:15)(cid:8)(cid:27)(cid:9)(cid:11)(cid:10)(cid:15)(cid:27)(cid:20) (cid:19)(cid:10)(cid:11)%(cid:10)(cid:27)(cid:18)(cid:18)(cid:15)(cid:8)%(cid:2)
(cid:2)
(cid:2) .(cid:27)(cid:10)(cid:9)(cid:2)(cid:16)(cid:6)(cid:2).(cid:10)(cid:11) (cid:20)(cid:21)(cid:18)(cid:24)(cid:2) .(cid:27)(cid:10)(cid:9)(cid:2)(cid:25)(cid:6)(cid:2)/(cid:9)(cid:4)(cid:21)(cid:10)(cid:24)(cid:2)
(cid:2)
Fig.0.1Thepedagogicalrelationshipofthetenchaptersinthistextbook
ThisbookistheresultofthedevelopmentofthecourseEvolutionaryComputa-
tionandItsApplicationsgivenbythefirstauthoratTsinghuaUniversitysince2002
andthenumerousintensivecoursestaughtbythesecondauthoraroundtheworld.
Toomanynamesrequirementionintheacknowledgments.Butwewishtoexpress
ourappreciationforourstudentsfirst,especiallythoseintheseminarEvolutionary
Algorithms 2010. Without your critical comments during the seminar, we would
havenoappreciationofthereader’sposition,whichisperhapsthemostdistinctive
featureofthistextbook.Manyofoldfriendsgaveencourageandcommentsonthe
book,includingRunweiCheng,BaodingLiu,KapHwanKim,etc.Thefastresponse
andtheprofessionalskillofClaireProtheroughatSpringerandNadjaKrokeatle-
teximpressedusdeeply.Thefinal,butmostimportant,acknowledgementbelongs
toLinWangandEikoGenbecauseyouarethebackbonesofourlives.
BeijingChina, XinjieYu
KitakyushuJapan, MitsuoGen
May2010
Contents
PartI EvolutionaryAlgorithms
1 Introduction................................................... 3
1.1 WhatAreEvolutionaryAlgorithmsUsedFor? .................. 3
1.2 WhatAreEvolutionaryAlgorithms?........................... 6
SuggestionsforFurtherReading................................... 8
References..................................................... 9
2 SimpleEvolutionaryAlgorithms................................. 11
2.1 IntroductoryRemarks....................................... 11
2.2 SimpleGeneticAlgorithm ................................... 14
2.2.1 AnOptimizationProblem ............................. 14
2.2.2 RepresentationandEvaluation ......................... 15
2.2.3 Initialization ........................................ 17
2.2.4 Selection ........................................... 18
2.2.5 VariationOperators .................................. 20
2.2.6 SimpleGeneticAlgorithmInfrastructure................. 22
2.3 EvolutionStrategyandEvolutionaryProgramming .............. 25
2.3.1 EvolutionStrategy ................................... 25
2.3.2 EvolutionaryProgramming............................ 27
2.4 Direction-basedSearch...................................... 28
2.4.1 DeterministicDirection-basedSearch ................... 28
2.4.2 RandomDirection-basedSearch........................ 32
2.5 Summary ................................................. 35
SuggestionsforFurtherReading................................... 36
ExercisesandPotentialResearchProjects........................... 37
References..................................................... 37
3 AdvancedEvolutionaryAlgorithms .............................. 39
3.1 ProblemsWeFace.......................................... 39
3.2 EncodingandOperators..................................... 40
xi
