Human Adversaries in Security Games: Integrating Models of Bounded Rationality and Fast ...

204 Pages·2014·5.25 MB·English

HumanAdversariesinSecurityGames:
IntegratingModelsofBoundedRationalityandFastAlgorithms
by
RongYang
ADissertationPresentedtothe
FACULTYOFTHEUSCGRADUATESCHOOL
UNIVERSITYOFSOUTHERNCALIFORNIA
InPartialFulfillmentofthe
RequirementsfortheDegree
DOCTOROFPHILOSOPHY
(ComputerScience)
April2014
Copyright 2014 RongYang
Acknowledgments
Fiveyearsago,IdecidedtojoinUSCpursingaPhDdegree. Thiswasoneofthemostimportant
decisions I have made in my life. Finishing the PhD wasn’t easy, but I have really enjoyed the
past five years with the privilege to have worked with a number of extraordinary people, who
havegivenmeinvaluableadvicesonbothresearchandlife.
Firstofall,IwouldliketogivemyspecialthankstomyPhDadvisorMilindTambe,without
whom I would not been anywhere near where I am now. When I first joined the TEAMCORE
research group, little did I know about what it means to do research. Milind, with his endless
patience, guided me through each step to do meaning research. He taught me the importance of
conducting research that has real-world impact. His dedication for his students has encouraged
ustonotonlydobetterresearchbutalsobeabetterperson. Hisunboundedpassionforresearch
andunbeatableworkingattitudehasbeenandwillalwaysbetheinspirationtome. Iwillalways
remember the days when he worked with me until the last minute on the paper deadlines! I
am also grateful for his support as I started my family during the course of pursing my PhD.
His understanding and consideration has made the toughest time of my life so much smoother.
Milind, thanksforalwaysbeingtherewhenIneededadiscussionregardingresearch, abreakto
takecareofmybaby,andaheartfeltadviceforfuturecareerpath.
ii
Next, I would like to thank other members of my thesis committee: Fernando Ordo´n˜ez,
JonathanGratch,RajivMaheswaran,RichardJohn,andVincentConitzer,forprovidingvaluable
feedbacktomyresearchandpushingmetothinkdeeper. Asmyresearchbeingattheintersection
ofmanydisciplines,IwouldneverbeabletopushmyresearchtotheheightIhaveachievedwith-
outhavinganinterdisciplinarycommitteetohelpshapingmyidea. AspecialthankstoFernando,
for his tremendous guidance to me and ceaseless confidence in me, without what my research
would have suffered greatly. You introduced me to the world of large-scale optimization tech-
niques. IwillalwaysrememberthedayswhenIcouldjustwalkintoyourofficetoaskquestions
wheneverIencounteredproblemsinmyresearch. ThanksforflyingoverallthewayfromChilito
supportmyresearch. Richard,thanksforopeningupthedoortohumansubjectresearchforme.
Without your guidance, I would never be able to reach the achievement in my research. I really
appreciateyouralwaysbringingadifferentperspectivetomyresearch. Asacomputerscientist,
IenjoyinteractingwithhumansubjectsinmyexperimentasmuchasIdowithmycodes,thanks
toyourhelp.
IwouldalsoliketothankthemanyexcellentresearchersthatIhavehadtheprivilegetowork
with over the years. This list includes Christopher Kiekintveld, Matthew Taylor, Bo An, Albert
Xin Jiang. James Pita, Jun-young Kwak, Sarit Kraus, Thanh Nguyen, Fei Fang, Benjamin Ford
and Debrun Kar. I thank all the students who helped develop the games for my experiment:
MayureshJanorkar, MohitGoenka, KarthikRajagopalandNoahOlsman. Iamalsogratefulfor
the support from the Army Research Office, the US Coast Guard and IBM on my research over
the years. They have provided me not only the opportunity to work on real-world problems of
research interest, but also the possibility to develop my own research interest. I would like to
iii
particularly thank Janusz Marecki for his help in applying for the IBM fellowship and for being
thebestacademicbrotherIcouldeveraskfor.
DuringmytimeatUSC,IenjoyedmyselfverymuchasamemberofthelargeTEAMCORE
family. IappreciatethetimesharedwiththoseduringmyPhDcareer: MatthewBrown,Leandro
Marcolino,ChaoZhang,YundiQian,KumarAmulyaYadav,HaifengXu,FrancescoDelleFave
and William Haskell. Special thanks to Manish Jain for all the advices you have given me over
the years; Jason Tsai for being a great officemate with the patience to listen to my stories from
shopping for shoes to taking care of my baby boy; Eric Shieh for the delicious food you have
broughttotheoffice; ZhengyuYinforthemanyafternoonsthatyouhavespenddiscussingwith
me about research; Jun-young Kwak for always having the good recommendations for Korean
restaurants.
Finally,Iwouldliketothankmyfamilyfortheirsupportovertheyears. Inparticular,thanks
to my parents for always believing in me, for supporting my decision of flying aboard to pursue
my career, and for coming over to the US to help take care of Dylan. I would also like to thank
my in-laws for the support during the toughest time of my PhD career. Lastly, I would like to
thank my dear husband Qing for always being there for me through happiness and toughness,
forbeingthebestfriendofmylife,forsupportingmetopursuemyowncareerandforbringing
Dylan,themostpreciousgiftever,tomylife,.
iv
Table of Contents
Acknowledgments ii
ListofFigures ix
Abstract xi
Chapter1: Introduction 1
1.1 ProblemAddressed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2.1 StochasticModelsofAdversaryDecisionMaking . . . . . . . . . . . . 4
1.2.2 AlgorithmsforOptimizingDefenderStrategy . . . . . . . . . . . . . . . 5
1.2.3 AdaptiveResourceAllocationandApplicationforProtectingWildlife . . 7
1.3 OverviewofThesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Chapter2: Background 10
2.1 StackelbergGames . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.1.1 BayeisanStackelbergGames . . . . . . . . . . . . . . . . . . . . . . . . 12
2.1.2 StrongStackelbergEquilibrium . . . . . . . . . . . . . . . . . . . . . . 13
2.1.3 StackelbergSecurityGames . . . . . . . . . . . . . . . . . . . . . . . . 14
2.2 LosAngelesInternationalAirport . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.3 BaselineSolvers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.3.1 DefenderOptimalStrategyagainstaperfectlyrationaladversary . . . . . 19
2.3.2 DefenderOptimalStrategyagainstthe(cid:15) optimaladversaryresponse . . 20
−
2.4 HumanSubjectExperiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Chapter3: RelatedWork 24
3.1 BehavioralGameTheory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.2 EfficientComputationofDefenderOptimalStrategy . . . . . . . . . . . . . . . 26
3.3 RobustDefenderStrategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.4 LearningAdversaryBehavioralinRepeatedGames . . . . . . . . . . . . . . . . 29
Chapter4: ModelingAdversaryDecisionMaking 30
4.1 ModelsforPredictingAttackerBehaviors . . . . . . . . . . . . . . . . . . . . . 31
4.1.1 ProspectTheory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
4.1.2 QuantalResponse. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
v
4.1.3 QuantalResponsewithRank-relatedExpectedUtility . . . . . . . . . . 35
4.2 ComputingOptimalDefenderStrategy . . . . . . . . . . . . . . . . . . . . . . . 36
4.2.1 ComputingagainstaPT-adversary . . . . . . . . . . . . . . . . . . . . . 36
4.2.1.1 BRPT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
4.2.1.2 RPT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
4.2.2 ComputingagainstaQR-adversary . . . . . . . . . . . . . . . . . . . . 42
4.2.3 ComputingagainstaQRRU-adversary . . . . . . . . . . . . . . . . . . . 43
4.3 ParameterEstimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4.3.1 SelectingPayoffStructures . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.3.2 ParameterEstimationforProspectTheory . . . . . . . . . . . . . . . . . 49
4.3.3 ParameterEstimationfortheQRModel . . . . . . . . . . . . . . . . . . 51
4.3.4 ParameterEstimationfortheQRRUModel . . . . . . . . . . . . . . . . 52
4.4 ExperimentalResultsandDiscussion . . . . . . . . . . . . . . . . . . . . . . . . 54
4.4.1 ASimulatedOnlineSSG . . . . . . . . . . . . . . . . . . . . . . . . . . 54
4.4.2 ExperimentalSettings . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
4.4.3 AlgorithmParameters . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
4.4.4 QualityComparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
4.4.4.1 AveragePerformance . . . . . . . . . . . . . . . . . . . . . . 60
4.4.4.2 PerformanceDistribution . . . . . . . . . . . . . . . . . . . . 66
4.4.5 ModelPredictionAccuracy . . . . . . . . . . . . . . . . . . . . . . . . 69
Chapter5: QuantalResponseModelwithSubjectiveUtility 74
5.1 TheSUQRModel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
5.1.1 LearningSUQRParameters . . . . . . . . . . . . . . . . . . . . . . . . 76
5.1.2 PredictionAccuracyofSUQRmodel . . . . . . . . . . . . . . . . . . . 77
5.2 ImprovingMATCH . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
5.2.1 Selectingβ forMATCH: . . . . . . . . . . . . . . . . . . . . . . . . . . 80
5.3 ExperimentalResults . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
5.3.1 ResultswithAMTWorkers,8-targetGames . . . . . . . . . . . . . . . . 81
5.3.2 SU-BRQRvs MATCH . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
5.3.3 SU-BRQRvsImproved MATCH . . . . . . . . . . . . . . . . . . . . . . 83
5.3.4 ResultswithNewExperimentalScenarios . . . . . . . . . . . . . . . . . 84
5.3.4.1 SecurityIntelligenceExperts,8-targetgames . . . . . . . . . . 85
5.3.4.2 SU-BRQRvs DOBSS . . . . . . . . . . . . . . . . . . . . . . 85
5.3.4.3 SU-BRQRvs MATCH . . . . . . . . . . . . . . . . . . . . . . 85
5.3.5 BoundedRationalityofHumanAdversaries . . . . . . . . . . . . . . . . 86
5.3.6 AMTWorkers,24-targetGames . . . . . . . . . . . . . . . . . . . . . . 87
5.3.6.1 SU-BRQR vs MATCH with Parameters Learned from the 8-
targetGames . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
5.3.6.2 SU-BRQRvsDOBSSwithRe-estimatedParameters . . . . . 88
5.3.6.3 SU-BRQRvsMATCHwithRe-estimatedParameters . . . . . 88
vi
Chapter6: ModelingHumanAdversariesinNetworkSecurityGames 90
6.1 ProblemDefinition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
6.2 AdversaryModel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
6.2.1 BasicQuantalResponseModel . . . . . . . . . . . . . . . . . . . . . . 95
6.2.2 QuantalResponsewithHeuristics . . . . . . . . . . . . . . . . . . . . . 95
6.3 ModelParameterEstimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
6.3.1 DataCollection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
6.3.2 TrainingtheQRModel . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
6.3.3 TrainingtheQRHModel . . . . . . . . . . . . . . . . . . . . . . . . . . 100
6.4 ComputingDefenderResourceAllocationStrategy . . . . . . . . . . . . . . . . 102
6.4.1 BestResponsetoQRmodel . . . . . . . . . . . . . . . . . . . . . . . . 103
6.4.2 BestResponsetoQRHmodel . . . . . . . . . . . . . . . . . . . . . . . 104
6.5 ExperimentResults . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
6.5.1 ExperimentSettings . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
6.5.2 ExperimentResults . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
Chapter7: ComputingDefenderOptimalStrategy 111
7.1 ProblemDefinition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
7.1.1 ResourceAssignmentConstraint . . . . . . . . . . . . . . . . . . . . . . 113
7.2 BinarySearchMethod . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
7.3 GOSAQ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
7.3.1 GOSAQwithNoAssignmentConstraint . . . . . . . . . . . . . . . . . . 117
7.3.2 GOSAQwithAssignmentConstraints . . . . . . . . . . . . . . . . . . . 119
7.4 PASAQ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
7.4.1 PASAQwithNoAssignmentConstraint . . . . . . . . . . . . . . . . . . 122
7.4.2 PASAQWithAssignmentConstraints . . . . . . . . . . . . . . . . . . . 125
7.5 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
7.5.1 NoAssignmentConstraints . . . . . . . . . . . . . . . . . . . . . . . . 127
7.5.2 WithAssignmentConstraints . . . . . . . . . . . . . . . . . . . . . . . 129
Chapter8: Scaling-up 133
8.1 Generalized PASAQ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
8.2 COCOMO–ABranch-and-PriceAlgorithm . . . . . . . . . . . . . . . . . . . . 137
8.3 BLADE–ACutting-PlaneAlgorithm . . . . . . . . . . . . . . . . . . . . . . . . 140
8.3.1 Master . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
8.3.2 SeparationOracle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
8.3.3 WBLADE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
8.3.4 QualityandRuntimeTrade-off . . . . . . . . . . . . . . . . . . . . . . . 148
8.4 Experimentalresults . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
Chapter9: AdaptiveResourceAllocationanditsApplicationtoWildlifeProtection 153
9.1 Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
9.2 Modelin PAWS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
9.2.1 StackelbergGameFormulation . . . . . . . . . . . . . . . . . . . . . . 156
9.2.2 BehavioralHeterogeneity . . . . . . . . . . . . . . . . . . . . . . . . . 159
9.2.3 AdaptingPatrollingStrategyusingHistoricalCrimeData . . . . . . . . . 161
vii
9.3 ResearchAdvancesin PAWS . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
9.3.1 LearntheBehavioralModel . . . . . . . . . . . . . . . . . . . . . . . . 162
9.3.1.1 LearningwiththeIdentifiedData . . . . . . . . . . . . . . . . 163
9.3.1.2 LearningwiththeAnonymousData . . . . . . . . . . . . . . . 164
9.3.1.3 CombiningtheTwoKindsofData . . . . . . . . . . . . . . . 165
9.3.2 AdaptingPatrollingStrategy . . . . . . . . . . . . . . . . . . . . . . . . 167
9.4 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
9.4.1 GeneralGameSettings . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
9.4.2 ResultsfortheDeploymentArea . . . . . . . . . . . . . . . . . . . . . . 170
Chapter10:Conclusion 174
10.1 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
10.2 FutureWork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
Bibliography 181
AppendixA: ErrorBoundof PASAQ 188
viii
List of Figures
1.1 USCoastGuardattheportofBoston . . . . . . . . . . . . . . . . . . . . . . . 6
1.2 QENP:Theintendedsiteofdeployment. RangerphototakenbyAndrewLemieux. 8
2.1 LAXSecurity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
4.1 ProspectTheoryempiricalfunctionforms . . . . . . . . . . . . . . . . . . . . . 32
4.2 Piecewiseapproximationoftheweightingfunction . . . . . . . . . . . . . . . . 40
4.3 PayoffStructureClusters(color) . . . . . . . . . . . . . . . . . . . . . . . . . . 47
4.4 GameinterfaceforoursimulatedonlineSSG . . . . . . . . . . . . . . . . . . . 54
4.5 Defenderaverageexpectedutilityachievedbydifferentstrategies . . . . . . . . . 59
4.6 Defenderaverageexpectedutility(normalizedbetween0and1)achievedbydif-
ferentstrategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
4.7 DefenderaverageexpectedutilityachievedbyQRmodelbasedstrategies . . . . 64
4.8 Distributionofdefender’sexpectedutilityagainsteachindividualsubject . . . . 67
4.9 Distributionofdefender’sexpectedutilityagainsteachindividualsubject . . . . 68
6.1 GameInterface(colored) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
6.2 GraphsTestedinDataCollection . . . . . . . . . . . . . . . . . . . . . . . . . . 98
6.3 GraphsTestedinEvaluationExperiments . . . . . . . . . . . . . . . . . . . . . 106
6.4 AverageDefenderExpectedUtility . . . . . . . . . . . . . . . . . . . . . . . . . 109
6.5 AverageDefenderExpectedUtility . . . . . . . . . . . . . . . . . . . . . . . . . 110
ix
7.1 PiecewiseLinearApproximation . . . . . . . . . . . . . . . . . . . . . . . . . . 120
7.2 SolutionQualityandRuntimeComparison,withoutassignmentconstraints(bet-
terincolor) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
7.3 SolutionQualityandRuntimeComparison,withassignmentconstraint(betterin
color) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
8.1 BranchingTree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
8.2 Minimizingweighted1-normdistance . . . . . . . . . . . . . . . . . . . . . . . 147
8.3 RuntimeComparisonofthe BLADEfamily . . . . . . . . . . . . . . . . . . . . 150
8.4 Comparing COCOMOand BLADE,QRModel . . . . . . . . . . . . . . . . . . 151
8.5 RuntimeComparison,QR-Sigmoidmodel . . . . . . . . . . . . . . . . . . . . . 152
9.1 Lioness photo courtesy of John Coppinger, Remote Africa Safaris Ltd. Poacher
snarephototakenbyAndrewLemieux. . . . . . . . . . . . . . . . . . . . . . . 155
9.2 EmpiricalMarginalPDFoftheSUQRparameteramongallthe760subjects . . . 160
9.3 Simulationresultsoverround . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
9.4 SlowCapturev.s. FastCapture . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
9.5 ComparingcumulativeEUatround20 . . . . . . . . . . . . . . . . . . . . . . . 170
9.6 TheQENPareaofinterestforoursimulation . . . . . . . . . . . . . . . . . . . 172
9.7 Simulationresultsoverroundforthe64sq. kmgridareainQENP . . . . . . . . 173
9.8 Patrollingcoveragedensityinthepark . . . . . . . . . . . . . . . . . . . . . . . 173
x