Table Of ContentRapid Adaptation of Video Game AI
Rapid Adaptation of Video Game AI
PROEFSCHRIFT
terverkrijgingvandegraadvandoctor
aandeUniversiteitvanTilburg,
opgezagvanderectormagnificus,
prof.dr.Ph.Eijlander,
inhetopenbaarteverdedigentenoverstaanvaneen
doorhetcollegevoorpromotiesaangewezencommissie
indeaulavandeUniversiteit
opwoensdagmaartom:uur
door
SanderCornelusJohannesBakkes
geborenopjanuariteSwalmen
Promotor: Prof.dr.H.J.vandenHerik
Copromotor: Dr.ir.P.H.M.Spronck
Ledenvandebeoordelingscommissie:
Prof.dr.A.P.J.vandenBosch
Prof.dr.J-J.Ch.Meyer
Prof.dr.ir.G.vanOortmerssen
Prof.dr.E.O.Postma
Dr.D.W.Aha
e research reported in this thesis has been funded by NWO, the Netherlands Or-
ganization for Scientific Research, in the framework of the Rapid Online Learning for
EntertainmentComputing(ROLEC)project,withgrantnumber...
e research reported in this thesis is part of the Interactive Collaborative Information
Systems(ICIS)project,supportedbytheDutchMinistryofEconomicAffairs,withgrant
numberBSIK.
SIKSDissertationSeriesNo.-
e research reported in this thesis has been carried out under the auspices of SIKS, the
DutchResearchSchoolforInformationandKnowledgeSystems.
TiCCPh.D.SeriesNo.
ISBN----
©SanderBakkes
CoverdesignbyJobPereboom,StudioKonstruktief
All rights reserved. No part of this publication may be reproduced, stored in a retrie-
val system, or transmitted, in any form or by any means, electronically, mechanically,
photocopying,recordingorotherwise,withoutpriorpermissionoftheauthor.
Preface
Intheearlys,thevideo-gameindustryhasgrowntosurpasstheHollywoodmoviein-
dustryinrevenues.Inrecentyears,videogaminghasbecomeamass-marketentertainment
industryonaparwithTV,movies,andmusic.Modernvideogameshavebecomeincrea-
singlyrealisticintheirvisualandauditorypresentation.However,theartificialintelligence
(AI)ofvideogameshasnotreachedahighdegreeofrealismyet.Amajordisadvantageof
currentgameAIisitsinabilitytoadaptadequatelytogamecircumstances.edisadvan-
tagecanberesolvedbyendowinggameAIwithadaptivebehaviour,i.e.,theabilitytoadapt
togamecircumstances,generallybymeansoflearningadequategamebehaviourwhilethe
gameisinprogress.
CurrentapproachestoadaptivegameAItypicallyrequirenumeroustrialstolearnef-
fectivebehaviourinonlinegameplay(i.e.,gameadaptationisnotrapid).Inaddition,game
developersareconcernedthatapplyingadaptivegameAImayresultinuncontrollableand
unpredictablebehaviour(i.e.,gameadaptationisnotreliable).esecharacteristicsham-
pertheincorporationofadaptivegameAIinactual,commerciallyreleasedvideogames.e
generalgoalofourresearchistoinvestigatetowhatextentitispossibletoestablishgame
AIcapableofadaptingrapidlyandreliablytogamecircumstances.Ourfindings,thatare
discussedinthisthesis,maybeusedbygamedevelopersforincorporationinactualvideo
games.
isthesiswouldnothavebeenpossiblewithouttheassistanceandencouragementofa
largegroupofpeople.Iwasmostfortunatethatmysupervisor,JaapvandenHerik,allowed
me to pursue my goals and I owe many thanks to his support and commitment. I highly
appreciatehiscommentsandadviseonmyscientificwritings,aswellashiseffortsinarran-
gingadditionalfundingforfinalisingtheresearch.Aspecialgratitudegoesouttomydaily
advisor,PieterSpronck,forsupportingmesoverydedicatedly,forhisresearchinspiration,
andforthecharmhebringstoacademia.
Moreover,IwouldliketothankmycolleaguesatTilburgUniversityandMaastrichtUni-
versity.esupportivestaffofbothuniversitiesdeservemyappreciationfortheirhelpin
manyrespects.IwouldespeciallyliketomentionPeterGeurtzfortechnicalassistancein
thepast,andJokeHellemons,LeenJacobs,andDennisTantradjajafortheirongoingefforts
in different areas of support. e research atmosphere that I experienced within the uni-
versitywallsisonethatIhappilylookbackupon.Iconsidermyselfluckytohaveworkedin
aninternationalenvironment,andsecretlyenjoyedthedailydosageofslightlyunorthodox
behaviourandthoughtsthatitbroughtabout.Forthat,Iacknowledgegratefullyalldirect
colleagues,androommatesovertheyears.IwouldespeciallyliketothankStevendeJong,
GuillaumeChaslot,AlessandroRossi,SanderSpek,AndreasRüdel,SophiaKatrenko,and
AnneBoldersfortheirongoingassistance,hospitality,andfriendship.
During my research, I had the pleasure to supervise and cooperate with several stu-
dents. I explicitly thank Laurens van der Blom, Philip Kerbusch, Bart Mehlkop, and Fre-
derik Schadd, whose research efforts and their results have been integrated in the thesis.
Inaddition,IthankTomNowellandAlexanderSeizingerfortheirgenerousassistancein
developingAIfortheopen-sourceSgameenvironment.
ii Preface
Outwiththeacademia,Ifoundthesupportofindividualsandorganisationsforexpres-
singmyselfintheformofphotographyandDJ’ing.HereIemphasisethecollectiveofphoto-
graphers‘Nxtlvl’,andtheDJ’sof‘Loki’.Iacknowledgeinparticularthepersonsbehindsocial
gatheringsinvarious(improvised)settings,andinthe‘Ipanema’,‘Zondag’,and‘Landbouw-
belang’,forbringingtogetherthemanypeoplethatIwillmisswhenobligationswillchange
mylifeafterthethesisdefence.
Inconclusiontotheseacknowledgements,Iwouldliketothankfriendsfrommyhome-
townSwalmenandmybrotherMilan,withwhomitwasapleasuretogrowup.Ichbedank
mienleefomaveur’tmichmitgaevevanhäörwieslaeveslessedieouchopvandaagnaggeljig
zin;enmieneljers,veurhäörónveurwaardelikkesjteunenvertroewe.Finalmente,gostaria
deexpressaraminhafelicidadeporreceberoamordaCândida.
SanderBakkes
March,
Contents
Preface i
Contents iii
Introduction 1
. Games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
. GameAI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
.. Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
.. Gamedevelopers’approachtogameAI . . . . . . . . . . . . . . . . 4
.. AcademicapproachtogameAI . . . . . . . . . . . . . . . . . . . . . 5
. Problemstatementandresearchquestions . . . . . . . . . . . . . . . . . . . 6
. Researchmethodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
. esisstructure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Background 11
. Videogames . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
.. History. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
.. Gamegenres . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
.. Gamestudies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
.. Stateoftheindustry . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
. ResearchintogameAI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
.. EntertainmentandgameAI . . . . . . . . . . . . . . . . . . . . . . . 16
.. Non-adaptivegameAI . . . . . . . . . . . . . . . . . . . . . . . . . . 17
.. AdaptivegameAI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
. Chaptersummary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
IncrementaladaptivegameAI 21
. WhatisincrementaladaptivegameAI? . . . . . . . . . . . . . . . . . . . . . 22
. IncrementaladaptivegameAIwithTEAM . . . . . . . . . . . . . . . . . . . 23
.. TeamAI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
.. eTEAMadaptationmechanism . . . . . . . . . . . . . . . . . . . 24
.. TEAMvs.QuakeIIICTFteamAI . . . . . . . . . . . . . . . . . . . 27
.. DiscussionofTEAM . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
. IncrementaladaptivegameAIwithTEAM . . . . . . . . . . . . . . . . . . 32
.. eTEAMadaptationmechanism . . . . . . . . . . . . . . . . . . 32
.. TEAMvs.QUAKEIIICTFteamAI . . . . . . . . . . . . . . . . . . 33
.. DiscussionofTEAM . . . . . . . . . . . . . . . . . . . . . . . . . . 36
. Practicalapplicability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
. Chapterconclusions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
iv Contents
Case-basedadaptivegameAI 41
. Whatiscase-basedadaptivegameAI? . . . . . . . . . . . . . . . . . . . . . 41
.. Case-basedreasoning . . . . . . . . . . . . . . . . . . . . . . . . . . 42
.. Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
.. Contributionsandlimitationsoftheapproach . . . . . . . . . . . . . 44
. Case-basedadaptivegameAIformapadaptation . . . . . . . . . . . . . . . 45
.. Approachtomapadaptation . . . . . . . . . . . . . . . . . . . . . . 45
.. Experimentwithmapadaptation . . . . . . . . . . . . . . . . . . . . 47
.. Discussionoftheresults . . . . . . . . . . . . . . . . . . . . . . . . . 55
. Case-basedadaptivegameAIinasimulatedgame . . . . . . . . . . . . . . . 55
.. Simulatedvideogame . . . . . . . . . . . . . . . . . . . . . . . . . . 56
.. Gatheringdomainknowledge . . . . . . . . . . . . . . . . . . . . . . 57
.. Exploitingdomainknowledge . . . . . . . . . . . . . . . . . . . . . . 59
.. Experimentswithexploitingdomainknowledge . . . . . . . . . . . 62
.. Discussionoftheresults . . . . . . . . . . . . . . . . . . . . . . . . . 63
. Towardscase-basedadaptationinvideogames. . . . . . . . . . . . . . . . . 64
. Chaptersummary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
eevaluationfunction 67
. Evaluationfunctions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
.. Evaluationfunctionsinclassicgames . . . . . . . . . . . . . . . . . . 68
.. Evaluationfunctionsinvideogames . . . . . . . . . . . . . . . . . . 69
.. Establishinganevaluationfunction . . . . . . . . . . . . . . . . . . . 71
.. Exploitinggameknowledge . . . . . . . . . . . . . . . . . . . . . . . 73
. AnevaluationfunctionforSPRING . . . . . . . . . . . . . . . . . . . . . . . 73
.. egameenvironment . . . . . . . . . . . . . . . . . . . . . . . . . . 73
.. Towardsacasebaseofgameknowledge . . . . . . . . . . . . . . . . 76
.. Ourevaluationfunction . . . . . . . . . . . . . . . . . . . . . . . . . 77
.. Phaseofthegame. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
.. Materialstrength . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
.. Commandersafety . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
. Validatingtheevaluationfunction . . . . . . . . . . . . . . . . . . . . . . . . 79
.. Experimentalsetup . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
.. Twomeasuresforperformanceassessment . . . . . . . . . . . . . . 82
.. Measuredperformance . . . . . . . . . . . . . . . . . . . . . . . . . 82
.. Discussionoftheresults . . . . . . . . . . . . . . . . . . . . . . . . . 86
. Chapterconclusions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
eadaptationmechanism 89
. Adaptationmechanisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
.. Rapidadaptation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
.. Reliableadaptation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
. AnadaptationmechanismforSPRING . . . . . . . . . . . . . . . . . . . . . 93
.. Generaladaptationprocedure . . . . . . . . . . . . . . . . . . . . . . 93
Contents v
.. Gameindexing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
.. Clusteringofobservations . . . . . . . . . . . . . . . . . . . . . . . . 94
.. InitialisationofgameAI . . . . . . . . . . . . . . . . . . . . . . . . . 95
.. Similaritymatching . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
.. Onlinestrategyselection . . . . . . . . . . . . . . . . . . . . . . . . . 96
. Validatingtheadaptationmechanism . . . . . . . . . . . . . . . . . . . . . . 97
.. Experimentalsetup . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
.. Performanceassessment . . . . . . . . . . . . . . . . . . . . . . . . . 101
.. Exp:Resultsofgameadaptation . . . . . . . . . . . . . . . . . . . . 101
.. Exp:Resultsofdifficultyscaling . . . . . . . . . . . . . . . . . . . . 104
.. Discussionoftheresults . . . . . . . . . . . . . . . . . . . . . . . . . 105
. Chapterconclusions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
Opponentmodelling 109
. Opponentmodelling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
.. Opponentmodellinginclassicgames. . . . . . . . . . . . . . . . . . 110
.. Opponentmodellinginvideogames . . . . . . . . . . . . . . . . . . 111
. EstablishingopponentmodelsinSPRING . . . . . . . . . . . . . . . . . . . 113
.. Approachtoestablishingopponentmodels . . . . . . . . . . . . . . 113
.. Implementationoftheapproach . . . . . . . . . . . . . . . . . . . . 114
.. Experimentswiththeimplementation . . . . . . . . . . . . . . . . . 116
.. Discussionoftheresults . . . . . . . . . . . . . . . . . . . . . . . . . 122
. ExploitingopponentmodelsinSPRING . . . . . . . . . . . . . . . . . . . . 123
.. Approachtoexploitingopponentmodels . . . . . . . . . . . . . . . 123
.. Anexperimentwithexploitingopponentmodels . . . . . . . . . . . 125
.. Discussionoftheresults . . . . . . . . . . . . . . . . . . . . . . . . . 127
. Chapterconclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
Integratingthethreecomponents 129
. Howtointegratethethreecomponents . . . . . . . . . . . . . . . . . . . . . 129
. Incorporatingopponentmodelling . . . . . . . . . . . . . . . . . . . . . . . 130
.. Establishingopponentmodels . . . . . . . . . . . . . . . . . . . . . . 131
.. Exploitingopponentmodels . . . . . . . . . . . . . . . . . . . . . . . 132
. Experimentswithcase-basedadaptivegameAI . . . . . . . . . . . . . . . . 134
.. Experimentalsetup . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
.. Performanceassessment . . . . . . . . . . . . . . . . . . . . . . . . . 134
.. Generatedopponentmodels. . . . . . . . . . . . . . . . . . . . . . . 135
.. Resultsofgameadaptation . . . . . . . . . . . . . . . . . . . . . . . 135
.. Discussionoftheresults . . . . . . . . . . . . . . . . . . . . . . . . . 138
. Practicalapplicability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
.. Scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
.. Dealingwithimperfectinformation . . . . . . . . . . . . . . . . . . 140
.. Generalisationtodifferentgames . . . . . . . . . . . . . . . . . . . . 141
.. Acceptancebygamedevelopers . . . . . . . . . . . . . . . . . . . . . 142
vi Contents
. Chapterconclusions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
Conclusions 145
. Answerstotheresearchquestions . . . . . . . . . . . . . . . . . . . . . . . . 145
. Answertotheproblemstatement . . . . . . . . . . . . . . . . . . . . . . . . 147
. Futureresearch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
References 151
Appendices 171
A Gameenvironments 171
A. QuakeIIICTF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
A. SPRING . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
B FeaturesofgameobservationsinSPRING 177
C EvaluationfunctionforSPRING 179
D ParametersofstrategicbehaviourinSPRING 183
Summary 185
Samenvatting 189
Curriculumvitae 193
Publications 195
SIKSdissertationseries 197
TiCCPh.D.series 205
Description:Case-based adaptive game AI in a simulated game . Video ga- mes, with the notable exception of turn-based strategy games, usually progress at a fast pace
