Parallelizing simulated annealing algorithms based on high-performance computer

29 Pages·2007·0.65 MB·English

JGlobOptim(2007)39:261–289
DOI10.1007/s10898-007-9138-0
ORIGINAL PAPER
Parallelizing simulated annealing algorithms based
on high-performance computer
Ding-JunChen · Chung-YeolLee ·
Cheol-HoonPark · PedroMendes
Received:28July2005/Accepted:12January2007/Publishedonline:20February2007
©SpringerScience+BusinessMediaB.V.2007
Abstract Weimplementedfiveconversionsofsimulatedannealing(SA)algorithm
fromsequential-to-parallelformsonhigh-performancecomputersandappliedthem
to a set of standard function optimization problems in order to test their perfor-
mances. According to the experimental results, we eventually found that the tradi-
tionalapproachtoparallelizingsimulatedannealing,namely,parallelizingmovesin
sequential SA, difficultly handled very difficult problem instances. Divide-and-con-
querdecompositionstrategyusedinasearchspacesometimesmightfindtheglobal
optimum function value, but it frequently resulted in great time cost if the random
searchspacewasconsiderablyexpanded.Themosteffectivewaywefoundinidenti-
fyingtheglobaloptimumsolutionistointroducegeneticalgorithm(GA)andbuilda
highlyhybridGA+SAalgorithm.Inthisapproach,GAhasbeenappliedtoeachcool-
ingtemperaturestage.Additionally,theperformanceanalysesofthebestalgorithm
among the five implemented algorithms have been done on the IBM Beowulf PCs
Clusterandsomecomparisonshavebeenmadewithsomerecentglobaloptimization
algorithmsintermsofthenumberoffunctionalevaluationsneededtoobtainaglobal
minimum,successrateandsolutionquality.
Keywords Simulated annealing · Genetic algorithms · Parallel and distributed
processing·Message-passinginterface(MPI)·High-performancecomputing
B
D.-J.Chen( )·C.-Y.Lee·C.-H.Park
Department of Electrical Engineering and Computer Science, Korea Advanced Institute of
ScienceandTechnology,Daejeon305-701,RepublicofKorea
e-mail:[email protected]
P.Mendes
VirginiaBioinformaticsInstitute,VirginiaTech,Blacksburg,VA24060,USA
262 JGlobOptim(2007)39:261–289
1 Introduction
Manyapplicationproblemsinengineeringareformulatedasoptimizationproblems;
forexample,indiversedisciplinessuchasphysics,chemistryandmolecularbiology,the
optimizationproblemminf(→x),subjecttoonlylower-andupper-boundconstraintson
thevariables,isfrequentlymet.Inordertosolvesuchatypeofoptimizationproblems,
alargenumberofoptimizationmethodshavebeendeveloped.Someofdeterministic
methodsapplydeterministicheuristics,suchasmodifyingthesearchtrajectoryintra-
jectorymethodsandaddingpenaltiesinpenalty-basedmethods,tobringasearchout
ofalocalminimum[1].Ingeneral,thesemethodsdonotworkwellifthesearchspace
istoolargeortheoptimizationfunctionhasmanydistinctlocalminima.Inthelast
20years, numerous stochastic optimization algorithms have emerged as a powerful
andbroadmethodforsolvingoptimizationproblems.Theseprobabilisticglobalopti-
mization methodsrelyonprobability tomakedecisions. Ingeneral,theyhavehigh
computational demand. Simulated annealing is one of the most popular stochastic
globaloptimizationmethodsandhasbeensuccessfullyappliedtosolvemanynonlin-
earoptimizationproblems[2–6].Uptotoday,manynovelextensionsofthesimulated
annealingmethodhavebeenproposedforoptimizingdifficultfunctions,suchasFast
Simulated Annealing and Very Fast Simulated Re-annealing (VFSR) [2–4]. Those
methods mainly focus on finding a proper temperature annealing schedule so as to
makeSAconvergetothelowestfunctionalminimum(namely,theglobalminimum)
asquicklyaspossible.Someresearchershaveevertriedtointroducetheparallelismin
SA.Actually,duetothefactthatSAisanaturallysequentialalgorithm,itisdifficult
to parallelize SA without changing its serial nature. In the last decade, methods to
efficientlyparallelizeSAhavealwaysbeenacontroversialissueandwerediscussed
byEllenetal.[7],Hammaetal.[8],Ingber[5],Yongetal.[9]andChenetal.[10].The
latestfiveparallelSAalgorithmsdevelopedbyEsinandLinet[11]areimplemented
fortheDistributedMemoryMIMDsystemsandbasedupon8PentiumII350MHZ
PCsconnectedby10Mbit/secondEthernetusingtheParallelVirtualMachine(PVM)
Linux programming environment. The study based on such a computing platform
impossiblyoffersuscomprehensiveperformanceanalyses,wethink.Themainreason
is that we actually do not know some performances how to vary if the parallel SA
runswithover8PCs,suchasthescalabilityoftheparallelSA.
Withrapidadvancesinpersonalcomputersandnetworksinthelastdecade,itis
possibletouseSAtofindquicklyaglobaloptimumwiththelowesttime-consuming
costbyconvertingexistingsimulatedannealingsoftwarefromsequential-to-parallel
formsonahigh-performancecomputer.Thisapproachisbecomingmoreappealingto
usbecausePCsclustersaremoreeconomicalthansupercomputersandthedeveloped
parallel software is highly portable. Certainly, it is unrealistic to expect to find one
generalparallelSAapproachtoworkwellforeverykindofnonlinearoptimization
problem.Wethink,however,wemayattempttofindsuchaparallelstochasticopti-
mizationapproachthatfitsmanymoreproblemsofthetypewearesolving.Thus,in
ordertothoroughlyinvestigatetheperformanceofparallelapproachesforSAona
high-performance computer and to try to find a as robust and good parallel SA as
possible,wehavecarriedoutfivedifferentconversionsofsimulatedannealingsoft-
ware from sequential-to-parallel forms on IBM Beowulf PCs xSeries clusters using
Message-PassingInterface(MPI)andappliedthemtoasetofstandardmultimodal
functionsforevaluatingtheirperformances.
JGlobOptim(2007)39:261–289 263
The organization of this paper is as follows: In Sect. 2, five strategies are used
and detailed implementations on how to convert the existing sequential simulated
annealing software from sequential-to-parallel forms are presented. The computa-
tionalresults,detailedperformanceanalysesofalgorithmsanddiscussionsaregiven
inSect.3.SomeconclusionsaregiveninSect.4.Testfunctionsusedfortestingthe
proposedparalleloptimizationalgorithmsaregiveninAppendixA.
2 Parallelizingsimulatedannealing
2.1 StrategiesusedforparallelizingSA
Inthispaper,weconsiderapplyingSAonlytofunctionminimizationproblems.The
duality principle suggests that we can convert a maximization problem into a mini-
mizationonebymultiplyingtheobjectivefunctionby−1.Therefore,sequentialSA
generally may work as follows: At the beginning, SA randomly generates a single
point, and then begins to perform the following iterative operations. A neighbor is
createdfromacurrentsolutionbychangingapartofthesolutionandiftheneigh-
bor’scostfunction(or“energy”)islessthanthatoftheexistingsolution,theneighbor
replaces the current solution. Otherwise, the neighbor is accepted probabilistically.
The probability of acceptance depends on the current annealing temperature con-
trolled by the cooling scheme. At last, SA stops when it does not make significant
progressoveranumberofiterations[2,3,12,13].
Asdescribedabove,itiseasytoseethatsimulatedannealingisinherentlysequen-
tial, thereby leading to long computation time, especially in the case of applying
algorithm to problems with large search spaces. Naturally, a more efficient way to
speedupthesimulatedannealingalgorithmandmakeSAamoreattractivesolution
for optimization problems is to add parallelism independent of the problem. How-
ever, different people may take different strategies in implementing a parallelizing
SA,suchasMobparallelannealing[14],Time-homogeneousparallelannealing[15],
Parallelrecombinativesimulatedannealing(PRSA)[15]andParallelsystolicSA[16].
OurideaonparallelizingSAbasedonhigh-performancecomputingistofocuson
twoconsiderations.Onegoodconsiderationistoabandonsomesearchspaceandthus
makeSAalwaysruninpromisingsearchspacewhereaglobaloptimizationsolution
mayexist.AnotherconsiderationisthatSA’ssearchspacealwaysremainsunchanged.
Inthiscase,ifdifferentrandomnumberstreamsareusedoneachprocessor,theneach
processorwillprobablyfindanindependentlocalminimum,andeventuallywechoose
thesmallestoneamongthemasthebestsolutionatthetemperatureofeachcooling
stage.Therefore,animportantmattertothinkaboutishowtoobtaindifferentand
goodinitialsolutionsoneachparticipatingprocessorsandmakeeachsolutioncan-
didate independently and randomly move in its own search space with periodically
communicatingwitheachother.
Five strategies of parallelizing SA algorithms are illustrated in Fig. 1 and first
implementedhereonanIBMBeowulfPCsxSeriesclusterusingMPICH(apopular
implementationversionofMPIspecification[17]).TheprincipalfeatureoftheMPI
programisthataprogramconsistsofa(typicallyfixed)setofheavyweightprocesses
(ingeneral,eachCPUcanjustonlyallowoneprocesstorunonit),eachwithaunique
identifier(processIDfromintegers0toP−1,heresupposedPisthetotalnumber
ofprocesses).Ifnecessary,processesmayinteractbyexchangingtypedmessages,by
264 JGlobOptim(2007)39:261–289
Strategy taken:
Approach-1
Search space
(V1)
decompostion
Strategy taken:
Using single initial
Approach-2
point and doing
(V2)
multiple parallel moves
Strategy taken:
Sequential Using multiple initial Approach-3
SA points and doing (v3)
multiple parallel moves
Strategy taken:
Only applying GA to
multiple initial points Approach-4
and doing multiple (v4)
parallel moves
Strategy taken:
Applying GA to Approach-5
multiple points at (v5)
each temperature
stage and doing
multiple parallel moves
Fig.1 StrategiesusedforparallelizingSA
engagingincollectivecommunicationoperations,orbyprobingforpendingmessages.
Incaseofourparallelcomputingplatform,allprocessesareonlyexecutedoncom-
putingnodesoftheIBMBeowulfPCscluster.Thenumberofcomputingnodesthat
maybeusedtorunanMPIprogramdependsupontheapplicationsubmittedtothe
PortableBatchSystem(PBS)runningonaheadnode.
2.2 Detaileddescriptionsoffiveapproaches
Approach-1: Sequential SA algorithm works on the whole search space directly. If
the search space is huge, it is not easy for SA to find the global optimum solution.
EventhoughSAcanoccasionallysucceedingettingone,itwilltakeaverylongtime.
It is a good strategy to decompose a large search space into smaller ones and then
let SA work on them separately. This kind of Divide-and-conquer decomposition
strategyhasbeenappliedinmanysearchalgorithmsforbothcontinuousanddiscrete
problems [18–21]. Unpromising sub-spaces are excluded from further search, and
promisingonesarerecursivelydecomposedandevaluated.
If→x isone-dimensionparameterxinobjectivefunctionf(→x),thenitisveryeasy
toevenlypartitionthesearchspace;If→x istwo-dimensionparameters(x ,x ),then
1 2
either of x and x can be used to equally partition search space. Suppose p is the
1 2
numberofdesiredsearchsub-spaceandx isusedtopartitionsearchspace,therefore
2
thesearchsub-spacecanberepresentedasfollows:
RangeofSub-space1:x ∈(mn ,mx )andx ∈(mn ,mn +(mx −mn )/p)
1 1 1 2 2 2 2 2
RangeofSub-space2:
x ∈(mn ,mx )andx ∈(mn +(mx −mn )/p,mn +2×(mx −mn )/p)
1 1 1 2 2 2 2 2 2 2
RangeofSub-space3:
x ∈(mn ,mx )andx ∈(mn +2×(mx −mn )/p,mn +3×(mx −mn )/p)
1 1 1 2 2 2 2 2 2 2
JGlobOptim(2007)39:261–289 265
Therestmaybededucedbyanalogy.Heremn ,mx representthelowerboundary
1 1
andupperboundaryofx ,respectively;mn ,mx representthelowerboundaryand
1 2 2
upper boundary of x , respectively. If →x is n-dimension parameters (x ,x ,...,x ),
2 1 2 n
then any one of parameters x ,x ,...,x may be chosen to evenly partition search
1 2 n
space.Thenextsearchspaceisdeterminedbyparallelprocess0.Parallelprocess0is
responsibleforcollectingtheminimumobtainedbyeachparallelprocessandpicks
up the best of them as the promising solution, and then the range of the sub-space
generatedthebestminimumsolutionischosenasthenextsearchspace.Anyoneof
parametersx ,x ,...,x inthenextsearchspacemaybechosentoevenlypartition
1 2 n
thenextsearchspace.
Basedontheideamentionedabove,Approach-1maybedescribedasfollows:
1. AnMPIprograminitiallygeneratesPprocessesthathaveranks0,1,...,P−1.
2. Processeswithrank0equallydividethesearchspaceintoPpartsanddistribute
thesub-spacetoPprocessestoleteachprocesshaveasub-space.
3. Each process runs a sequential SA algorithm and gets the optimum in its sub-
space.1
4. The process with rank 0 is responsible for collecting the optimum obtained by
eachprocessandfordecidingifthestopcriterionismetornot.
4.1 Ifthestopcriterion2ismet,thealgorithmterminates.
4.2 Ifnot,process0hastodeterminethenextsearchspaceandthengoesback
tostep2.
EachSAalgorithmrunsinitssub-spacelikeasequentialSA.Differentpeoplemay
usedifferentstrategiestoimplementsequentialSA.Supposetherealreadyisaninitial
solution→x(x ,x ,...,x )toobjectivefunctionf(→x).Ourprincipleinimplementation
1 2 n
canbebrieflydescribedasfollows:
current_solution←initia_solution
current_cost←evaluate(current_solution)
T ←T
initial
while(stop_criterion_not_meet)
for i=1toiterations(T)
new_solution←move(current_solution)
(cid:1)(cid:1)cost←new_cost−current_cost (cid:2)
if (cid:1)cost≤0ORe−(cid:1)cost/T >Random(0,1)
/∗acceptnewsolution∗/
current_solution←new_solution
current_cost←new_cost
endif
endfor
T ←newx_temp(T)
endwhile
TheprimaryparametersusedbyeachSAalgorithmineachsub-spaceareasfollows:
Initialtemperature:T =1.0;
initial
1 Intermsoftheusedstopcriterion,eachparallelprocessmayterminateindifferentways,thereby
makingruntimecostdifferent.Inordertoguaranteethatprocess0cancollecttheoptimumminimum
obtainedbyeachparallelprocesses,synchronouspointissethereafterthe3rdstepofV1ends.
2 Stoppingcriterionhereisthatthealgorithmterminateswhenthedifferencebetweentheprevious
andthelatestoptimalsolutionislessthanorequalto10−10.
266 JGlobOptim(2007)39:261–289
Temperaturecoolingrulenewx_temp(T):Tt+1 =Tt×0.85;
Ateachtemperature,thenumberofiterationsisequalto1000×n(nisthenumber
ofparameters,→x(x ,x ,...,x ));
1 2 n
Stop_criterion: if ((|f(→x)Gmin−f(→x)Lmin| < 1e−10) || (Tt+1 < 1e−04)) is true,
then SA terminates (Here f(→x) Represents the global minimum found so far;
Gmin
f(→x) Representstheglobalminimumnewlyfoundatcurrenttemperature);
Lmin
Inaddition,functionmove(current_solution)isimplementedasfollows:
for(i=0,i<n;i++)
{
stepsize=(mx −mn)×Random(0,1);
i i
if(Random(0,1)>=0.5)
new_x ←current_x+stepsize;
i i
else
new_x ←current_x−stepsize;
i i
if((new_x <mn)||(new_x >mx))
i i i i
Enforce_bound(new_x,i);
i
}
AbovefunctionEnforce_bound(Parametervalue,i)isimplementedasfollows:
{
if(Parametervalue>=mx)Parametervalue= mx −DBL_EPSILON;
i i
else
if(Parametervalue>mx)Parametervalue=mx;
i i
if(Parametervalue<=mn)Parametervalue=mn +DBL_EPSILON;
i i
else
if(Parametervalue<mn)Parametervalue=mn;
i i
}
(Random(0,1)isarandomnumbergeneratorbetween0and1;mn,mx representthe
i i
lowerboundaryandupperboundaryofx,respectively.)
i
Approach-2:Acommonapproachtoparallelizingsimulatedannealingistogenerate
severalperturbationsinthecurrentsolutionsimultaneously.Someofthem,thosewith
smallvariance,locallyexploretheregionaroundthecurrentpoint,whilethosewith
largervariancesgloballyexplorethefeasibleregion.Ifeachprocesshasgotdifferent
perturbationormovegeneration,eachprocesswillprobablygetadifferentsolution
attheendofiterations.Approach-2maybedescribedasfollows:
1. TheMPIprograminitiallygeneratesPprocessesthathaveranks0,1,...,P−1.
2. TheMPIprograminitiallygeneratesastartingpointatrandomavailableforall
processesandallprocessessetT =T .
0
3. Processwithrank0randomlygeneratesPdifferentstepsizesanddistributesthem
toeachparticipatingprocess.
4. AtthecurrenttemperatureT,eachprocessbeginstoexecuteiterativeoperations.
5. Attheendofiterations,processwithrank0isresponsibleforcollectingthesolu-
tion obtained by each process at current temperature and broadcasts the best
solutionofthemamongallparticipatingprocesses.
6. Iftheterminationconditionisnotmet,eachprocesscoolsthetemperatureand
goesbacktostep4,elsealgorithmterminates.
ThecoreofthesimulatedannealingalgorithmistheMetropolisprocedurewhich
simulatestheannealingprocessatagiventemperatureT.Theperturbationmethods
JGlobOptim(2007)39:261–289 267
to generate a new solution significantly affect the algorithm convergence. Suitable
perturbationwaysareexpectedtogeneratediversesolutionsateachgiventemper-
ature. In Approach-2, at the beginning the process 0 randomly generates different
step sizes for each parameter with the following way and distributes them to each
participatingprocess:
for(j=0;j<P;j++)
for(i=0;i<n;i++)
{
do
Stepsize[j][i]=(mx −mn)×Random(0,1)+(mx −mn)×(j/(2×P));
i i i i
while(Stepsize[j][i]>=(mx −mn));
i i
}
(mn,mx arethelowerboundaryandupperboundaryofparameterx,respectively;
i i i
n is the number of parameters of candidate solution →x(x ,x ,...,x ) to objective
1 2 n
functionf(→x))
Therefore,ateachparallelprocesseachparameterhascompletelydifferentstep
sizefromothersparallelprocesses.Thenthefollowingperturbationwaysareusedin
eachparallelprocess:
for(i=0,i<n;i++)
{
if(Random(0,1)>=0.5)
new_x ←current_x +stepsize[i];
i i
else
new_x ←current_x −stepsize[i];
i i
if((new_x <mn)||(new_x >mx))
i i i i
Enforce_bound(new_x i);
i,
}
However,sometimesthenew(cid:1)solutionsgeneratedbyperturbationfunctions(cid:2)didnot
maketheconditionexpression (cid:1)cost≤0 OR e−(cid:1)cost/T >Random(0,1) betrue
andthusnewneighborhoodsolutionisnotalwaysacceptedasnextcandidatesolution.
Sothenumberoftheacceptedneighborhoodsolutionsneedstobecounted.Inthis
case,thestepsizesinperturbationwaysshouldbechangedaccordingtothenumber
oftheacceptedneighborhoodsolutions.Supposethatthenumberofiterationsisset
to100×10×n(thisimportantcontrolparameterisdeterminedmanually.),thismeans
thestepsizecanbechangedatmost100timesandeachparametersx hasopportunity
i
togenerate10neighborhoodperturbations.Therefore,theframeworkofMetropolis
procedurecanbedescribedasfollows:
for(i=0;i<100;i++) //Adjusting stepsize
for(j=0;j<10;j++) //adjusting all parameters
for(k=0;k<n;k++) //adjusting oneparameter
{
···
codes to implement Metropolis procedure
...
}
Supposethatforparameterx therearecc timestoaccepttheneighborhoodpertur-
i i
bationasnextcandidatesolution.Thestepsizesfortheparameterx willbechanged
i
inthefollowingway:
268 JGlobOptim(2007)39:261–289
for(i=0,i<n;i++)
{
rate=cc/10;
i
if(rate>0.6)stepsize[i]=stepsize[i]×(1+5×(rate−0.6));
if(rate<0.4)stepsize[i]=stepsize[i]/(1+5×(0.4−rate));
}
Of course, in general, the above step size stepsize[i] for each parameter x should
i
alwaysbewithintherangeof(mx −mn).Thisworkcanbedonebyanadditional
i i
function. Therefore, some parallel processes with small variances in step sizes, will
locally explore the region around the current point, while those parallel processes
withlargervariancesinstepsizeswillgloballyexplorethefeasibleregion.
Approach-3: The quality of an SA’s performance is usually affected by the configu-
ration of the starting solution. We may further improve Approach 2 by changing a
single start-solution into multiple start-solutions. Thus, we get the Approach 3 and
maythinkofitasamultistartsearchtechnique.TogetApproach3,wejustneedto
modifythestep2inApproach2asfollows:
2.Processwithrank0initiallygeneratesPdifferentstartingpointsatrandomand
distributesthemtoeachparticipatingprocessandallprocessessetT =T .
0
Inprocess0,multipledifferentstartingpointsarerandomlygeneratedasfollows:
for(i=0;i<P;i++)
for(j=0;j<n;j++)
x[i][j]=mn +(mx −mn)×Random(0,1);
j j j
(mn,mx arethelowerboundaryandupperboundaryofparameterx,respectively;
j j j
n is the number of parameters of candidate solution →x(x ,x ,...,x ) to objective
1 2 n
functionf(→x))
Approach-4andApproach-5:Geneticalgorithmsarepopulation-basedoptimization
algorithmsbasedonDarwinianmodelsofnaturalselectionandevolution.Thebasic
ideabehindGAisthataninitialpopulationofcandidatesolutionsofsizeN ischo-
senatrandomandeachsolutionisevaluatedaccordingtothespecifiedoptimization
function [21]. If genetic operators (Crossover and Mutation) are chosen properly
and applied to a population by a specific number of iterations (generations), then
eventually the population will have better solutions. Thus, GA improves the whole
population.InApproach3,theinitialmultiplestarting-solutionsarerandomlygener-
atedtoreducetheeffectcausedbytheconfigurationofthestartingsolution.Infact,
thosestartingsolutionsmaynotbeverygoodandcanbefurtheroptimized.Inorder
toguaranteethequalityofinitialstartingsolutions,GAmaybeappliedtothoseinitial
solutions.Therefore,Approachs4and5maybeelaboratedasfollows,respectively:
Apprach-4:
1. TheMPIprograminitiallygeneratesPprocessesthathaveranks0,1,...,P−1.
2. Processwithrank0initiallygeneratesPrandomstart-solutionsandrunsGAby
afixednumberofiterations(supposethatthemaximumgenerationissetto1,000
ormore).AftertheGAends,processwithrank0distributesPindividualsinthe
finalpopulationtoeachparticipatingprocessasaninitialstartingsolution.
3. AllprocessessetcurrenttemperatureT =T ;
0
4. At the current temperature T, each process begins to execute those iterative
operations.
JGlobOptim(2007)39:261–289 269
5. Attheendoftheiterations,processwithrank0isresponsibleforcollectingthe
solutionobtainedbyeachprocessatcurrenttemperatureandbroadcaststhebest
solutionofthemamongallparticipatingprocesses.
6. Iftheterminationconditionisnotmet,eachprocesscoolsthetemperatureand
goesbacktostep4,otherwisethealgorithmterminates.
Approach-5:
1. TheMPIprograminitiallygeneratesPprocessesthathaveranks0,1,...,P−1.
2. Processwithrank0initiallygeneratesPrandomstart-solutionsandrunsGAby
afixednumberofiterations(supposethatthemaximumgenerationissetto1,000
ormore).AftertheGAends,processwithrank0distributesPindividualsinthe
finalpopulationtoeachparticipatingprocessasaninitialstartingsolution.
3. AllprocessessetcurrenttemperatureT =T ;
0
4. At the current temperature T, each process begins to execute those iterative
operations.
5. Attheendoftheiterations,processwithrank0isresponsibleforcollectingthe
solutionsobtainedbyeachprocessatcurrenttemperatureandthenemployinga
GAtoevolvethembyafixednumberofgenerationsandfinallybroadcastingthe
bestsolutionofthepopulationachievedbyGAamongallparticipatingprocesses.
6. Iftheterminationconditionisnotmet,eachprocesscoolsthetemperatureand
goesbacktostep4,otherwisethealgorithmterminates.
Inside above Approachs 4 and 5, in the case of minimizing f(→x) is subject only
tolower-andupper-boundconstraintsonvariables,thedesignedGAhasusedreal
parametersaschromosomecodesandmayworkasfollows:
1. GeneratePindividualsatrandomasinitialsolutions.
2. Evaluatetheinitialpopulation.
3. Apply crossover operators to the current population to get 2P individuals
(PparentindividualsplusPchildrenindividuals).
4. Apply mutation operators to each child individual and evaluate the P mutated
individuals.
5. Accordingtothefitnessvalue,bytournamentcompetitionstrategy,selectPindi-
vidualsfrom2Pindividualsasthenextevolvedpopulation.
6. Determineiftheiterationreachesthemaximumgenerationornot:
6.1.Ifyes,thealgorithmterminates.
6.2.Ifno,thealgorithmneedstodofollowingoperation:
• If the best stored individual is not updated consecutively and the number of
counterscumulativelyreaches10times(or30timesor50times),10%(or30%
or50%)individualslocatedatthebottomofthenextevolvedpopulationwill
be replaced by randomly generated individuals, and then the algorithm goes
backtostep3andcontinues.3
3 Inouralgorithm,thebestindividualfoundsofarisstoredadditionally.Attheeachgeneration,if
thebestindividualcurrentlygeneratedinpopulationisbetterthanthepreviouslystoredindividual
inevaluationfitness,thenitwillreplacethestoredbestindividual.Acounterissettocounttimes.
Supposethebeststoredindividualhasnotbeenconsecutivelyupdatedby10times,then10%indi-
vidualslocatedatthebottomofthepopulationwillbereplacedbyrandomlygeneratedindividuals.
Therest(30%or50%)maybededucedbyanalogy.Obviouslythisoperationcanbedoneafterthe
individualsinpopulationaresortedintermsofevaluationfitness.
270 JGlobOptim(2007)39:261–289
X_0 Y_0 X_0 Y_0
X_1 Y_1 Y_1 X_1
X_2 Y_2 Y_2 X_2
Crossover
X_3 Y_3 Y_3 X_3
. . . .
. . . .
X_n-3 Y_n-3 X_n-3 Y_n-3
X_n-2 Y_n-2 X_n-2 Y_n-2
X_n-1 Y_n-1 X_n-1 Y_n-1
Mother with N Father with N Child 1 with N Child 2 with N
parameters parameters parameters parameters
Fig.2 CrossoveroperationwithNum_crossover=3andFirst_position=2
If GA is applied to a problem, in general, there are at least four issues, namely,
chromosomecode,fitnessevaluation,selectionstrategyandgeneticoperations,which
needtobeaddressed.Asmentionedabove,SAisanindividual-basedoptimization
algorithmandalgorithmonlyoptimizeonecandidatesolutiontooptimizationfunc-
tion f(→x), so the candidate solution →x(x ,x ,...,x ) can be coded as a vector(one
1 2 n
dimensional array). In GA, suppose that an initial population comprise p individ-
uals, so the individuals will have to be coded with two-dimensional array, such as
individual[i][j],(0<=i<p;0<=j<n)(wherepispopulationsizeandnisthenum-
berofparameters).Therefore,thefitnessevaluationfunctionisnaturallyequivalent
to the optimization function f(→x) itself. Genetic operations generally include cross-
overandmutation.Beforecrossoveroperationbegins,itisnecessarytoknowwhere
thecrossoverpositionwillbeandhowmanyparameterswillbeusedtocrossover.
In the implementation of GA, the algorithm always randomly generates a number
Num_crossover that is no more than half of a chromosome length. In addition, the
algorithmalsorandomlychoosesthefirstpositionintermsofthefollowingrule:
(cid:1) (cid:3) (cid:2)
Num_crossover=floor Num_parameter 2∗Random_num[0,1]
First_position=floor(1+Random_num[0,1]∗(Num_parameter−Num_crossover))
(*floor(doublex)isoneofC/C++mathematicallibraryfunctionsandcomputesthe
largestintegralvaluenotgreaterthanx.)
Forexample,Fig.2showstheprocessofcrossoveroperationwithNum_crossover
= 3 and First_position = 2 (supposing here the number of parameters is N ≥ 7 at
lease.).Mutationoperationmaybedoneasfollows:mutationoperatoratfirstneedsto
randomlychooseaparameterx,(1<i<n)fromachildindividual→x(x ,x ,...,x )
i 1 2 n
andreplaceitwitharandomlygeneratednumber.Ofcourse,theboundarycondition
mn <= x <= mx should be always satisfied. After the population is operated by
i i i
crossoverandmutation,thenumberofparentpopulationplusoffspringisdefinitely
more than the current parent population size. Tournament competition strategy is
usedtokeepthesizeofnextparentpopulationunchanged.