An approach to multi-agent planning with incomplete information

0.27 MB·English

by  Alejandro Torreño

An approach to multi-agent planning with incomplete
information
AlejandroTorren˜o and EvaOnaindia and O´scarSapena1
Abstract. Multi-agentplanning(MAP)approacheshavebeentyp- Inadistributedversionofthislatterwork,authorsuseadistributed
ically conceived for independent or loosely-coupled problems to CSPsolvertohandlecoordination[15].
enhance the benefits of distributed planning between autonomous Theaforementionedapproachesareconceivedforloosely-coupled
agents as solving this type of problems require less coordination problems (LCP), where agents have little interaction between each
betweentheagents’sub-plans.However,whenitcomestotightly- other,astheseprocessesarelikelytobeinefficientintightly-coupled
coupledagents’tasks,MAPhasbeenrelegatedinfavourofcentral- problems(TCP)[15].Thisway,thecouplinglevelofacooperative
ized approaches and little work has been done in this direction. In multi-agentsystemisformallydefinedasasetofparameterstolimit
thispaper,wepresentageneral-purposeMAPcapabletoefficiently thecombinatorialblow-upofplanningcomplexity[2].Ontheother
handleplanningproblemswithanylevelofcouplingbetweenagents. hand,theseMAPmodelsdonotconsidersystemscomposedofmul-
5
Weproposeacooperativerefinementplanningapproach,builtupon tiple entities distributed functionally or spatially but rather agents
1
thepartial-orderplanningparadigm,thatallowsagentstoworkwith endowedwiththesamecapabilitiesandactingundercompleteinfor-
0
incompleteinformationandtohaveincompleteviewsoftheworld, mation.Whencapabilitiesaredistributedacrosstheagents’domains,
2
i.e.beingignorantofotheragents’information,aswellasmaintain- agentshavenecessarilytointeracttosolvetheMAPproblemwhile

n ingtheirownprivateinformation.Weshowvariousexperimentsto beingunawareoftheotheragents’abilitiesorinformationaboutthe
a
comparetheperformanceofoursystemwithadistributedCSP-based world,i.e.workingunderincompleteinformation.
J
MAPapproachoverasuiteofproblems. In this paper, we present a general-purpose MAP model able to
8
workwithinherentlydistributedentitiesandsuitableforbothLCP
2 1 INTRODUCTION
and TCP domains. Similarly to [11], we use an iterative planning

Multi-agent planning (MAP) refers to any planning or plan execu- refinement procedure that uses single-agent planning technology.
]
I tionactivitythatinvolvesseveralagents.Ingeneralterms,MAPis Particularly, our model builds upon a partial-order planning (POP)
A
about the collective effort of multiple planning agents to combine paradigm, which also allow us to represent a collection of acting
. their knowledge, information, and capabilities so as to develop so- entitiesasasingleagent.POPisaverysuitableapproachforcen-
s
c lutions to problems that each could not have solved as well (if at tralized MAP with a small number of coordination points between
[ all) alone [9]. There exists a great variety of tools and techniques agents [14], and the application of a multi-agent POP refinement

forMAP.Agent-orientedMAPapproachesputtheemphasisondis- frameworkalsorevealsasaveryappropriatemechanismtoaddress
1
tributedexecution,plansynchronizationandcollaborativeactivityat tightly-coupledproblems.
v
6 run-timeplanningtoensurethattheagent’slocalobjectiveswillbe Thispaperisorganizedasfollows.Thenextsectionpresentsthe
5 met[8,16].AnotherresearchlineinMAPfocusesoncoordination specificationofaMAPtask.Following,weexplainthePOPrefine-
2 ofalreadycompletedplansthatagentshaveconstructedtoachieve mentapproachandtheextensionswehaveintroducedtodealwith
7 theirindividualgoals,asforexampleplanmerging[17,6,5].Incon- a multi-agent representation and incomplete information. The next
0 trast,thecooperativedistributedplanning(CDP)approachputsthe sectionsdescribeourMAPtasktheoreticalmodelandtherefinement
1. emphasisonplanningandhowitcanbeextendedintoadistributed planningalgorithmcarriedoutbytheagents.Following,weshowthe
0 environment, on building a competent plan carried out by multiple resultsofthetestswehaveperformed,andfinally,weconcludeand
5 agents[8].InCDP,agentstypicallyexchangeinformationabouttheir outlinethefuturelinesofresearch.
1 plans,whichtheyiterativelyrefineandreviseuntiltheyfittogether
2 MULTI-AGENTPLANNINGTASK
: well.
v
Following the cooperative approach, differences among MAP In our approach, the planning formalism of an agent is based on a
i
X modelslieintheintegrationoftheplanningandcoordinationstages STRIPS-likemodelofclassicalplanningunderpartialobservability.
r [9,7].SomerecentworksonfullycooperativeMAPhaveemerged Themodelallowsagentstorepresenttheirpartialviewoftheworld
a lately.Theworkin[14]considersagentsashavingsequentialthreads throughtheadoptionoftheopenworldassumption.Statesarerep-
ofexecutionandinteractiononlyoccurswhendistributingsub-plans resentedintermsofstatevariables.O isafinitesetofobjectsthat
to individual agents for plan execution. This approach follows a modeltheelementsoftheplanningdomain;V isafinitesetofstate
single-agentplanninganddistributedcoordination.Acentralizedal- variableseachwithanassociatedfinitedomain,Dv,ofmutuallyex-
gorithmforMAPcanbefoundin[2],wheremultipleagentsdoplan- clusivevalues.ValuesinDv denoteobjectsoftheplanningdomain,
ningoveracentralizedplaninterleavingplanningandcoordination. i.e., ∀v ∈ V, Dv ⊆ O. A state is a set of positive fluents of the
form(cid:104)v,d(cid:105),andnegativefluentsoftheform(cid:104)v,¬d(cid:105),meaningthat
1 UniversitatPolite`cnicadeVale`ncia,CaminodeVeras/n,46022Valencia, thevariabletakesonthevaluedor¬d,respectively.Aformula(v,d)
Spain evaluatestotrueifthefluent(cid:104)v,d(cid:105)ispresentinthestateanditeval-
uatestofalseotherwise.Morespecifically,(v,d)evaluatestofalseif Plansthatagentsbuildareconcurrentmulti-agent(MA)plansas
thefluent(cid:104)v,¬d(cid:105)isinthestate,orifnofluentrelatingthevariable, twodifferentactionsinΠcannowbeexecutedconcurrentlybytwo
v,andthevalue,d,ispresentinthestate,inwhichcasewesaythe different agents. Some MAP models adopt a simple form of con-
currentvalueofv isunknown.Wewillgenerallyrefertoasfluents currency: two actions can happen simultaneously if none of them
bothpositiveandnegativefluents. changesthevalueofastatevariablethattheotherreliesonoraffects,
Actions are given as tuples a = (cid:104)pre(a),eff(a)(cid:105), where pre(a) too[3].Weimposetheadditionalconcurrencyconstraintthatthepre-
denotestheformulasthatmustholdinastateSforatobeapplicable, conditions of two actions have to be mutually consistent [1]. This
andeff(a)representsthenewfluentsintheresultingstateS(cid:48).Effects definitionofconcurrencyisstraightforwardlyextendedtoajointac-
oftheform(v=d)addafluent(cid:104)v,d(cid:105)intheresultingstateaswellas tiona = (cid:104)a1,...,an(cid:105).Agentsaddressconcurrencyinconsistencies
asetoffluents{(cid:104)v,¬d (cid:105)},∀d (cid:54)=d,d ∈D ,reflectingthat(v,d ) throughthedetectionofthreatsoverthecausallinksoftheiractions.
j j j v j
evaluatestofalseintheresultingstate.Effectsoftheform(v (cid:54)= d) Thisway,concurrencyissuesbetweentwodifferentactionsmaynot
addafluent(cid:104)v,¬d(cid:105)totheresultingstate,whichimpliesthecurrent ariseuntiltheirpreconditionsaresupportedthroughcausallinks.
valueofvisunknownunlessthereisafluent(cid:104)v,d(cid:48)(cid:105)inS(cid:48),d(cid:54)=d(cid:48). ArefinementplanΠdevisedbyanagentioverabaseplanΠg is
aconcurrentMAplanthatresultsfromthecompositionofΠg anda
WedefineaMAPtaskasatupleT =(cid:104)AG,V,A,I,G(cid:105)where:
refinementstepΠ proposedbyagenti.Thisrefinementplan,which
i
• AG ={1,...,n}isafinitenon-emptysetofplanningagents. could eventually become a base plan, is defined as Π = Πg ◦Π ,
• V = {V }n ,whereV isthesetofstatevariablesmanagedby i
i i=1 i where◦representsthecompositionoperation.AcompositeplanΠ
agenti.Variablescanbesharedbytwoormoredifferentagents.
isaconcurrentMAplanifforeverypairofunequalactionsa and
• A = {A }n , where A is the set of actions that agent i can i
i i=1 i a ,i(cid:54)=j,∀p ∈pre(a ),p (cid:54)∈openGoals(Π),∀p ∈pre(a ),p (cid:54)∈
j i i i j j j
perform. Given two different agents i, j, A and A can share
i j openGoals(Π),a anda areconcurrentlyconsistent.
i j
somecommonactionsorbetwodisjointsets.
Inourmodel,eachagentimplementsaPOPplannertocompute
• I = {I }n , where I is the set of fluents known by agent i
i i=1 i refinement plans over a base plan Π. If an agent is not capable to
thatrepresentstheinitialstateoftheagent.Iftwoagentssharea
come up with a concurrent MA plan, then the agent refrains from
variablevthentheyalsoshareallofthefluentsregardingv.
suggestingsucharefinement.Ifnoagentelicitsaconsistentrefine-
• G ={G }n ,whereG isasetofformulasknowntoagentithat
i i=1 i mentplanforabaseplan,theplannodeispruned.
mustholdinthefinalstateanddenotethetop-levelgoalsofT.
As defined above, state variables may not be known to all agents. Algorithm1Dis-RPGconstructionforanagenti
Givenastatevariablev ∈ V andv (cid:54)∈ V ,∀j (cid:54)= i,v issaidtobe BuildinitialRPG
i j i
privatetoagenti.Additionally,agentscanhavedifferentvisionsof repeat
thedomainofastatevariable;thatis,noteveryvalueinavariable ∀j (cid:54)=i,isendsjshareablefluentsSF ∈RPG oftheform
i→j i
domainhastobevisibletoallagents.Givenanagenti,wedenoteits (cid:104)v,d(cid:105)or(cid:104)v,¬d(cid:105),wherev∈V ∩V andd∈D ∩D
i j vi vj
viewofthedomainofavariablevasD ⊆ D .Thus,thedomain ∀j (cid:54)=i,ireceivesfromjshareablefluentsSF ∈RPG of
vi v j→i j
of a state variable v can be defined as D = {D }n . Agents’ theform(cid:104)v,d(cid:105)or(cid:104)v,¬d(cid:105),wherev∈V ∩V andd∈D ∩D
v vi i=1 i j vi vj
incomplete views on the state variables and their domains directly
affectthevisibilityofthefluents. RF ←∅
• Anagentihasfullvisibilityofafluent(cid:104)v,d(cid:105)or(cid:104)v,¬d(cid:105)ifv∈Vi ∀j (cid:54)=i,RFi ←RFi∪SFj→i
andd∈Dvi. forallreceivedfluentsf ∈RFido
• An agent i has partial visibility of a fluent (cid:104)v,d(cid:105) or (cid:104)v,¬d(cid:105) if iff (cid:54)∈RPGithen
v ∈ Vi butd (cid:54)∈ Dvi.GivenastateS,where(cid:104)v,d(cid:105) ∈ S,agenti InsertfinRPGi
willseeinsteadafluent(cid:104)v,⊥(cid:105),where⊥istheundefinedvalue. costRPGi(f)←cost(f)
• Anagentihasnovisibilityofafluent(cid:104)v,d(cid:105)or(cid:104)v,¬d(cid:105)ifv(cid:54)∈V . endif
i
if(f ∈RPG )∧(cost (f)>cost(f))then
OurMAPmodelcanbeviewedasaPOP-based,multi-agentre- i RPGi
cost (f)←cost(f)
finementplanningframework,ageneralmethodbasedontherefine- RPGi
endif
mentofthesetofallpossiblepartial-orderplans[12].Anagentpro-
endfor
poses a plan Π that typically enforces some top-level goals of the
ExpandRPG
planningtask;then,therestofagentscollaborateontherefinement i
untilRF =∅
of this base plan Π by proposing refinement steps that solve some i
opengoalsinopenGoals(Π).Thisway,agentscooperativelysolve
theMAPtaskbyconsecutivelyrefininganinitiallyemptyplanΠ. 3 REFINEMENTPLANNING
ArefinementstepΠi devisedbyanagentioverabaseplanΠg, Thecooperativerefinementplanningalgorithmstartswithaprelim-
whereg ∈ openGoals(Πg),isatripleΠi = (cid:104)∆,OR,CL(cid:105),where inary information exchange by which agents communicate share-
∆∈AiisasetofactionsandORandCLaresetsoforderingsand able information. After this initial stage, agents execute the multi-
causal links over ∆, respectively. Πi is a plan free of threats [18] agent refinement planning algorithm, which comprises two inter-
that solves g as well as all the new open goals that arise from this leavedstages.First,agentsindividuallyelicitrefinementplansover
resolution and can only be achieved by agent i, (cid:104)v,d(cid:105) or (cid:104)v,¬d(cid:105), a centralized base plan through their embedded POP. Later, agents
where (v ∈ Vi) ∧ (v (cid:54)∈ Vj,∀j (cid:54)= i). That is, when solving an jointlyselectthemostpromisingrefinementasthenextbaseplan.
opengoalofabaseplan,anagentiwillalsoachievethenewarising
3.1 Informationexchange
open goals concerning fluents that are only visible to i, so are not
visibletotherestofagents,leavingtherestofgoalsunsolved.Let AgentsreceivetheinformationontheMAPtaskthroughasetofdef-
g ∈ openGoals(Πg)beaformulaoftheform(v,d)or(v,¬d);an initionfiles.ThesefilesareencodedinaMAPlanguagethatextends
agenticomputesarefinementstepoverΠgiffv∈V . PDDL3.1 [13], including a :shared-data section to configure
i
the agent’s vision of the planning task and which fluents it shares theagents’internalPOPforthispurpose)thatallowsagentstoes-
andwithwhom. timate the quality of the plans. Since agents do not have complete
Prior to executing the refinement procedure, agents share infor- informationontheMAPtaskortherefinementplans,theyevaluate
mationbybuildingadistributedRelaxedPlanningGraph(dis-RPG), plansaccordingtoitsownviewofeachrefinementplanΠ,i.e.,agent
basedontheapproachof[19].Agentsexchangethefluentsdefined ievaluatesarefinementplanΠaccordingtoF(view (Π))(seeAl-
i
asshareableinthe:shared-datasectionoftheMAPdefinition gorithm2).
files.Fluentsarelabeledwiththelistofagentsthatcanachievethem,
Algorithm2Refinementplanningprocessforanagenti
givingeachagentaviewofthepossibleinteractionsthatcanariseat
Π←Π
planningtimewithotheragents.Additionally,thedis-RPGprovides 0
R=∅
anestimateofthebestcosttoachieveeachfluent,ahelpfulinforma-
repeat
tiontodesignheuristicstoguidetheproblem-solvingprocess.
Selectopengoalg∈openGoals(Π)
Algorithm1summarizestheconstructionofthedis-RPG.Agents
RefinebaseplanΠgindividually
compute an initial RPG and expand it by following the procedure
∀j (cid:54)=i,sendRefinements (Πg)toagentj
in[10].TheRPGcontainsasetoffluentandactionlevelsthatare i
∀j (cid:54)=i,receiveRefinements (Πg)
interleaved. The first fluent level contains the fluents that are part j
Refinements(Πg)←Refinements (Πg)
oftheinitialstate,andthefirstactionlevelincludesalltheactions i
∀j (cid:54)=i,Refinements(Πg)←Refinements(Πg)∪
whose preconditions appear in the first fluent level. The effects of
Refinements (Πg)
theseactionsareplacedinthesecondfluentlevel,andthiswaythe j
forallplansΠ∈Refinements(Πg)do
graphisexpandeduntilnonewfluentsarefound.
EvaluateΠaccordingtoF(view (Π))
Once all the agents have computed their initial RPGs, the itera- i
endfor
tivedis-RPGcompositionbegins.AsdepictedinAlgorithm1,agents
R←R∪Refinements(Πg)
starteachiterationbyexchangingthethefluentsshareablewithother
Selectbest-valuedplanΠ ∈R
agents.AnagentiwillsendagentjthesetoffluentsSF thatare best
i→j
Π←Π
visibletoagentj,i.e.,thenewfluentsoftheform(cid:104)v,d(cid:105)or(cid:104)v,¬d(cid:105), best
ifopenGoals(Π)=∅then
wherev∈V ∩V andd∈D ∩D .Likewise,agentiwillreceive
i j vi vj return Π
fromallagentsj (cid:54)=itheshareablefluentstheyhavegenerated.
endif
AgentiupdatesthenitsRPG withthesetofnewfluentsithas
i
untilR=∅
received, RF . If a fluent f is not yet in RPG , it is stored ac-
i i
cording to cost(f). If f is already in RPG , its cost is updated if
i Onceevaluated,thenewrefinementplansarestoredinthesetof
cost (f) > cost(f). Hence, agents only store the best esti-
RPGi refinementsR.Next,eachagentvotesforthebest-valuedcandidate
matedcosttoreacheachfluent.AfterupdatingRPG ,agentiex-
i Π ∈ R.Incaseofadraw,thebatonagentwillchoosethenext
pands it by checking whether the new inserted fluents trigger new best
baseplanamongthemostvotedalternatives.
actionsinRPG ornot.Thefluentsproducedaseffectsofthesenew
i Oncearefinementplanisselected,agentsadoptitasthenewbase
actionswillbesharedinthenextiteration.
planΠ.IfopenGoals(Π) = ∅,asolutionplanisreturned.Assome
Theprocessfinisheswhentherearenonewfluentsinthesystem.
opengoalsmightnotbevisibletosomeagents,everyagentimust
Following, agents start the refinement planning process to build a
confirmthatΠisasolutionplanaccordingtoview (Π),i.e.,Πis
solutionplanjointly. i
asolutioniff∀i ∈ AG,openGoals(view (Π)) = ∅.Iftheplanhas
i
3.2 Multi-agentrefinementplanning still pending goals, the baton agent selects the next open goal g ∈
openGoals(Π) to be solved, and a new iteration of the refinement
Therefinementplanningprocessisbasedonademocraticleadership planningprocessstarts.
bywhichabatonisscheduledamongtheagentsfollowingaround- Theplanningalgorithmcarriedoutbytheagentscanberegarded
robinstrategy.Agentscarryouttwointerleavedstages:theindividual as a joint exploration of the refinement space. Nodes in the search
constructionofrefinementplansthroughaPOP,andacoordination tree represent refinement plans and each iteration of the algorithm
processbywhichagentsjointlysearchtherefinementspace. expandsadifferentnode.
Algorithm 2 describes the refinement planning process. Each
3.3 Soundnessandcompleteness
agent i computes a finite set of refinement plans for Πg,
Refinements (Πg), through its embedded POP planner. The in- Thealgorithmwehavepresentedcanberegardedasamulti-agent
i
ternalPOPsystemfollowsanA∗searchalgorithmguidedbyastate- extension of the POP algorithm. A partial-order plan is sound if it
of-the-artPOPheuristicfunction[18].Theresultingrefinementplans is a threat-free plan. In our algorithm, we address inconsistencies
areexchangedbytheagentsinthesystemfortheirevaluation(send among the concurrent MA plans by detecting and solving threats.
andreceiveoperationsinAlgorithm2). Thus,inordertoprovethatouralgorithmissound,weshouldensure
Agent i has a local, partial vision of each refinement plan, thatallthethreatsamongthecausallinksofaconcurrentMAplan
view (Π),accordingtoitsvisibilityovertheplanningtaskT.Thus, arecorrectlydetectedandsolved.
i
whenreceivingarefinementplanΠ,agentiwillonlyviewtheopen Undercompleteinformation,threatsonaMAconcurrentplanwill
goals(v,d) ∈ openGoals(Π) | v ∈ V .Withrespecttothefluents, becorrectlydetectedbyanyagent,asallthefluentsintheplanare
i
agentiwillonlyviewthosefluentsforwhichithasfullvisibility.If fullyvisible.Inourincompleteinformationmodel,weshouldstudy
ihaspartialvisibilityofafluent(cid:104)v,d(cid:105)or(cid:104)v,¬d(cid:105),itwillseeinstead howvisibilityoverfluentsaffectsthedetectionofthreats.
afluent(cid:104)v,⊥(cid:105),where⊥standsfortheundefinedvalue.Thisnotion Let Π be a MA concurrent plan and let (cid:104)v,d1(cid:105) be a fluent in a
ofpartialviewdirectlyaffectstheevaluationoftherefinements. causallinkcl∈CL(Π).Supposethatanagentibuildsarefinement
Theevaluationofrefinementplansiscarriedoutthroughautility Π(cid:48)overΠthataddsanewactionattotheplanwhichisnotordered
functionF (currently,weusethesameheuristicfunctionthatguides withrespecttocl andhasaneffect(v = d2).Thiseffectcausesa
threatoverclasitconflictswith(cid:104)v,d (cid:105).ForΠ(cid:48)tobesound,agenti actionsofanotheragent.Basedonthisconcept,itdefinesthenotion
1
shouldbeabletodetectsuchathreatwhatevervisibilityithasover of coupling level as the average rate of public actions of an agent.
thefluent(cid:104)v,d (cid:105): A high value of coupling level results in many agent coordination
1
• If i has full visibility over (cid:104)v,d (cid:105), the inconsistency between cl points, thus giving rise to tightly-coupled problems. The approach
1
anda willbecorrectlydetected. followedbyPlanningFirstisespeciallyeffectivewhendealingwith
t
• Ifihasnovisibilityover(cid:104)v,d (cid:105),thenv(cid:54)∈V .Inthiscase,agenti loosely-coupledproblems(LCP)[15],butitsperformancedecreases
1 i
doesnothaveanactiona withaneffectinvolvingvariablev,i.e., whentacklingtightly-coupledproblems(TCP).
t
suchathreatcanneveroccur.
Time comparison - Rovers domain
• If i has partial visibility over (cid:104)v,d (cid:105), agent i will see instead a
1 MAP-POP
fluent(cid:104)v,⊥(cid:105).Since⊥=(cid:54) d ,thethreatwillbedetectedandsolved. 2500 Planning First
2
Therefore,allthethreatsoverMAconcurrentplansarealwaysde-
tectedandresolved,whichprovesthatourMAPalgorithmissound. 2000
Asforcompleteness,wecannotensurethatourMAPalgorithmis
cidnoumcthepisloevwteeo.rrAak,cbctahoserednipnulgmantboemrthaoeyfnrneoofittinobenemofiefnnrtietpfie.lnaeHnmsenethncaett,pawlanenaawgreeenhtimacvapenliucpisrteolyd- Time (seconds) 11050000
pruningtherefinementsearchspace.Nevertheless,agentsrelyonan
A∗POPsearchprocesstobuildtherefinementplans,whichinmost
cases returns good refinement plans that guide the MAP algorithm 500
towardsasolutionplan.Theempiricalresultsshowninthenextsec-
tionconfirmourclaim. 0
2 4 6 8 10 12 14
Time comparison - Satellite domain Agents
35 Figure2. ScalabilityresultsfortheRoversdomain
MAP-POP
Planning First Thetestspresentedhereinvolvethreeofthebenchmarkdomains
30 used on the IPCs: satellite, rovers and logistics, which are the do-
mainsusedintheresultspresentedin[15]aswell.Thesedomains
25
giverisetoproblemsofdifferentcouplinglevels.Thesatelliteprob-
me (seconds) 1250 ltaeonmdinsttaeakrreaeciLtmCwaPigtehassebatyhchethdoeitmfhfeserer;elvntehtsea.ygRemonvotesvre(s,thpceraolsbiabltereamltleistettshe)neidarrteionnsptorrtuesmleikenentltyas
Ti
mediumcouplinglevel:roveragentsareindependentbuttheyhave
10
accesstocertainsharedresourcesintheirenvironment,namelythe
rockandsoilsamplestheycollectandanalyze.Thelogisticsprob-
5
lemsfallintotheTCPcategorysinceagents(trucksandplanes)have
0 to cooperate to transport the different packages to the target loca-
2 4 6 8 10 12 14
Agents tionsandproblemspresentseveralcoordinationpoints(locations)at
Figure1. ScalabilityresultsfortheSatellitedomain
whichagentscaninteract.
4 EXPERIMENTALRESULTS WeadaptedtheSTRIPSproblemfilesusedintheIPCstobothour
MAPlanguageandNissim’sMA-STRIPSlanguage.Problemsfrom
Wedesignedandexecutedasetofteststocomparetheperformance
the IPCs turned outto be complex instances for PlanningFirst be-
andscalabilityofourMAP-POPapproachwithanotherstate-of-the-
causeagentshavenecessarilytointeracttoeachotherandcooperate
art MAP system. Comparing the performance of multi-agent plan-
tofindasolutionplanfortheseproblemsandPlanningFirstworks
ningsystemsisnotaneasytaskduetotwomainreasons.First,most
better when plans for each agent can be computed (mostly) inde-
MAPapproachesarenotgeneral-purposebutdomain-dependentsys-
pendently.Forthisreason,weencodedanadditionalsetofproblems
temsspecificallydesignedtoaddressaparticularproblem,mosttyp-
limitingcooperationandinteractionsamongagentsasmuchaspossi-
icallytrafficcontrolorreal-timeplanningapplications.Second,un-
ble.Particularly,intheseadditionalproblems,agentscansolvegoals
like single-agent planners that have been promoted and populated
throughthecelebrationoftheInternationalPlanningCompetitions2 independently,i.e.,anagentisabletosolveagoalorsetofgoalsby
itselfwithoutneedofinteractingwiththerestofagents(wewillrefer
(IPC)and,therefore,havebeenmadepubliclyavailable,itisdifficult
totheseproblemsasindependentproblemsintheremainder).
tofindamulti-agentplannerabletorunthebenchmarkdomainsand
planningproblemsuitescreatedfortheIPCs.
Despite these drawbacks, we could assess the performance of
Table1showstheresultswhencomparingthequalityofthesolu-
MAP-POPandcomparetheresultswiththoseobtainedinthePlan-
ningFirstapproachpresentedin[15]3.PlanningFirstisaMAPsys- tionplansobtainedwithMAP-POPandPlanningFirstandtheexe-
cutiontimes4.Thequalityofthesolutionplansisassessedthrough
temthatalsomakesuseofsingle-agentplanningtechnology.More
threeparameters:a)thenumberofactionsoftheplan;b)thedura-
precisely,itbuildsuponasingle-agentstate-basedplanner[4],and
tionoftheplan,i.e.thenumberoftimeunitsortimestepsrequired
handlesagentcoordinationbysolvingadistributedCSP.
toexecutetheplan;andc)thenumberofagentsthattakepartinthe
Planning First defines public actions as the actions of an agent
solutionplan.Thislatterparametergivesanideaofhowtheefforton
whose descriptions contain atoms affected by and/or affecting the
solvingtheproblemhasbeendistributedamongtheagents.
2http://ipc.icaps-conference.org/
3WewanttoespeciallythankRazNissimforprovidinguswiththesource 4 Allthetestswereperformedonasinglemachinewitha2.83GHzIntel
codeofhisPlanningFirstsystemfortestingandcomparisonpurposes. Core2QuadCPUand8GBRAM.
%Coupling #Domain MAP-POP PlanningFirst
Problem #Agents
level actions #Acts #TS #Partics Time #Acts #TS #Partics Time
IPCSat1 1 1,2 54 9 8 1 0,23 10 9 1 0,14
IPCSat4 2 29,3 2082 21 11 2 18,80 †
IPCSat10 5 23,7 1786 29 20 3 90,3 †
IPCSat16 10 18,3 7196 51 24 5 73,7 †
IPCSat17 12 14,3 8324 46 16 4 53,9 †
IndSat1 2 5,2 40 9 4 2 0,83 9 4 2 0,16
IndSat2 4 1,4 274 14 3 4 2,20 14 4 4 0,31
IndSat3 7 0,3 1820 32 4 7 6,5 32 4 7 4,1
IndSat4 8 0,3 2082 28 3 8 8,7 28 4 8 11,1
IndSat5 14 0,1 11020 63 4 14 32,5 †
IPCRov1 1 1,2 81 10 7 1 0,344 11 7 1 0,359
IPCRov2 1 2,3 45 8 4 1 0,390 9 5 1 0,312
IPCRov7 3 77,4 157 18 6 3 8,578 †
IPCRov14 4 58,7 797 35 21 2 81,874 †
IPCRov15 4 85 536 42 16 4 42,014 †
IndRov1 2 45,5 160 24 11 2 3,609 22 7 2 2,75
IndRov2 3 45,5 239 36 11 3 5,500 33 7 3 12,141
IndRov3 4 45,5 318 48 11 4 9,188 44 7 4 120,719
IndRov4 5 45,5 397 70 11 5 14,141 55 7 5 674
IndRov5 6 45,5 476 72 11 6 20,688 66 7 6 2594,515
IPCLog2 3 20 52 27 9 3 18,187 †
IPCLog4 4 12,3 116 37 13 4 33,765 †
IPCLog5 4 14 116 31 11 4 40,188 †
IPCLog7 5 9,8 206 46 15 5 96,484 †
IPCLog9 5 11,7 206 45 17 5 239,578 †
IndLog1 3 44,4 20 6 6 2 1,579 9 8 3 0,578
IndLog2 3 55,5 20 10 9 3 2,250 10 9 3 0,609
IndLog3 4 65 42 13 10 4 3,225 9 8 4 66,187
IndLog4 4 70 42 14 6 4 3,766 14 6 4 284,094
IndLog5 6 54,1 98 21 6 6 13,578 †
Table1. PerformancecomparisonbetweenMAP-POPandPlanningFirst
ProblemslabeledwithIPCaredirectlytakenfromtheIPCbench- dealingwithsmallproblems,butitsperformancedecreaseswhenthe
marks,whileproblemslabeledwithIndaretheextrasetofindepen- sizeoftheproblemincreases.Forinstance,whilethefirstthreeprob-
dentproblemswecreatedtoassessPlanningFirstperformance(for lemsaresolvedfasterbyPlanningFirst,itisslowerthanMAP-POP
eachdomain,weshowtheresultsof5outofthe20IPCproblemswe when solving IndSat4, and it does not find a solution to the most
testedaswellas5independentproblems).Thenextthreecolumnsin complexinstance,IndSat5.MAP-POPprovesalsotobemoreeffec-
thetableshowthedifficultyoftheplanningproblems:#Agentsin- tiveatparallelizingactionsinthisdomainasitobtainsplansofequal
dicates the number of agents involved in the problem; %Coupling orshorterdurationthanPlanningFirst.
levelestimatesthecouplingleveloftheproblemastheaveragerate With respect to the rovers domain, our results confirmed that
ofinstantiatedpublicactionsofagents(takingintoconsiderationthe MAP-POPsolves15outofthe20IPCproblemsforthisdomain.For
notionofpublicandprivateactiondefinedin[15]),and#Domain thefiveIPCroversproblemsshowninTable1,wecanseethework-
actions refers to the total number of instanced actions. The results loadinthisdomainisbetterdistributedthaninthesatellitedomain
foreachplannerincludethenumberofactions(#Acts)andtimesteps asmostoftheagentsparticipateinthesolutionplan,whichconsider-
(#TS)ofthesolutionplan,respectively.#Particsindicatesthenum- ablyreducesthedurationoftheplan.Forinstance,thesolutionplan
berofagentsthattakepartinthesolutionplan,andTimeshowsthe forproblemIPCRov7contains18actionsandissolvedinjust6time
totalexecutiontime.Adagger(†)indicatesthatPlanningFirstwas steps.PlanningFirstsolvesonlythetwosmallestIPCproblems.For
notabletosolvetheproblem. theindependentproblemswemodeled,PlanningFirstobtainsbetter-
For the most loosely-coupled problems, the satellite domain, qualitybutmorecostlysolutionsthanMAP-POP.Thedifferencesin
MAP-POP exhibited an excellent performance as our results con- executiontimearefarmorenoticeablethaninthesatellitedomain.
firmedthatitwasabletosolve18outof20IPCproblems.Forthe Thisisduetotothemoretightly-couplednatureoftheproblemsof
fiveIPCproblemsforthesatellitedomainshowninTable1,wecan this domain (45.5% coupling level for the independent problems),
seethatourapproachdealsveryefficientlywithcomplexproblems whichaffectsnegativelytheperformanceofPlanningFirst.
upto12agents.Itisalsonoticeablethatatleastonethirdofthepar- Finally, the logistics domain has proven to be the most com-
ticipatingagentstakepartinthesolutionplans,whichhasapositive plexoneforbothmulti-agentapproaches.Agentsinthisdomainare
impactontheplanduration,asactionsarecarriedoutinparallelby trucksandairplanesthatmustcooperateinmostofthecasestotrans-
differentagents.AlthoughtheIPCsatelliteproblemsdonotpresent portpackages.Hence,solutionsfortheseproblemsaremorecostly
ahighcouplinglevel(lessthan30%ofpublicactionsintheworst thanintheroversandsatellitedomains,astheyrequireagentcoor-
case),PlanningFirstonlysolvesthefirstIPCproblem,astheseprob- dination,animportantfeaturetodeterminetheefficiencyofaMAP
lemsrequirecooperationamongagentsanditismorenecessaryfor approach.OurresultsconfirmedthatMAP-POPlosesperformance
larger instances. As for the additional problems we encoded (Ind- inthisdomain,beingabletosolveonly9outof20IPCproblems.
Sat1,...,IndSat5),wecanseethatPlanningFirstisnotabletosolve However,itdistributestheworkloadeffectivelysincealloftheagents
thelargestone,IndSat5.PlanningFirstisfasterthanMAP-POPwhen participate in all the solution plans obtained. Planning First shows
also a poorer performance in this domain as it is not able to solve system that handles agent coordination through a distributed CSP.
anyoftheIPCproblems.Theseresultsareinlinewiththeconclu- ResultsshowthatMAP-POPefficientlysolvesloosely-coupledprob-
sions exposed in [15], which reveals the difficulty of a CSP-based lemsbutitalsoshowscompetitivewhensolvingproblemsthathave
approachtodealefficientlywithproblemsthatexhibitahighlevel a higher coupling level and when computing plans that require the
ofinter-agentinteraction.Asfortheindependentproblems,someof cooperationamongagents.Hence,wecanconcludethatMAP-POP
thesolutionsobtainedbyMAP-POPhavebetterqualityintermsof isanefficient,domain-independentandgeneral-purposeframework
actions and duration than the solutions of Planning First. In addi- tosolveMAPproblems.
tion, Planning First is still remarkably slower than MAP-POP, be- ACKNOWLEDGEMENTS
ing unable to solve the IndLog5 problem, even though we defined
ThisworkhasbeenpartlysupportedbytheSpanishMICINNunder
rather small instances (notice the differences in execution time for
projects Consolider Ingenio 2010 CSD2007-00022 and TIN2011-
the instance IndLog4). Again, Planning First only performs better
27652-C03-01,andtheValencianPrometeoproject2008/051.
thanMAP-POPinthesmallerproblems.
The second test compares the scalability of both MAP frame-
REFERENCES
works, i.e. to which extent their efficiency is affected by the num-
berofagents.Inordertodoso,wehaverunfourteendifferenttests [1] C. Boutilier and R.I. Brafman, ‘Partial-order planning with concur-
rentinteractingactions’,JournalofArtificialIntelligenceResearch,14,
forboththesatelliteandtheroversdomains.Eachtestincreasesthe
105–136,(2001).
numberofagentsinthetaskbyone,fromoneagenttofourteen.The [2] R.I. Brafman and C. Domshlak, ‘From one to many: Planning for
problems are modeled so that each of the participant agents has to looselycoupledmulti-agentsystems’,in18thInternationalConference
achieveoneoftheproblem’sgoalsbyitself. onAutomatedPlanningandScheduling(ICAPS),pp.28–35,(2008).
Figure1showsthescalabilityresultsforthesatellitedomain.As [3] M.BrennerandB.Nebel,‘Continualplanningandactingindynamic
multiagentenvironments’,JournalofAutonomousAgentsandMultia-
itcanbeobserved,PlanningFirstshowabetterperformancewhen
gentSystems,19(3),297–331,(2009).
solving small problems (up to seven agents). However, its perfor- [4] A.I.Coles,M.Fox,D.Long,andA.Smith,‘Teachingforward-chaining
mancedecreasesquicklyasweexecutelargerproblems.MAP-POP planning with JavaFF’, in Colloquium on AI Education, 23th AAAI
isfasteratsolvingthe8-agentsatelliteproblem,andPlanningFirst ConferenceonArtificialIntelligence,(2008).
[5] J.S.CoxandE.H.Durfee,‘Anefficientalgorithmformultiagentplan
isunabletofindasolutionforthe9-agentproblemupwards.MAP-
coordination’,in4thInternationalConferenceonAutonomousAgents
POP,however,findsasolutionforthe14probleminstances,andex- andMultiagentSystems(AAMAS),pp.828–835,(2005).
ecutiontimessufferonlyaslightincreasebetweenproblems. [6] J.S. Cox, E.H. Durfee, and T. Bartold, ‘A distributed framework for
Thedifferencesinperformanceofbothsystemsaremorenotice- solvingthemultiagentplancoordinationproblem’,in4thInternational
ableinthemoretightly-coupledroversdomain.Theresultsofthis ConferenceonAutonomousAgentsandMultiagentSystems(AAMAS),
pp.821–827,(2005).
test are depicted in figure 2. In this case, Planning First requires
[7] M.deWeerdtandB.Clement,‘Introductiontoplanninginmultiagent
more than 40 minutes to solve the 6-agent rovers problem, while systems’,MultiagentandGridSystems,5(4),345–355,(2009).
MAP-POP takes only 20 seconds. Again, MAP-POP solves all the [8] M.E.desJardins,E.H.Durfee,C.L.Ortiz,andM.J.Wolverton,‘Asur-
problemswithoutlosingperformanceinthelargerinstances. veyofresearchindistributedcontinualplanning’,AIMagazine,20(4),
13–22,(1999).
In conclusion, MAP-POP proves to be a more robust approach
[9] E.H. Durfee, ‘Distributed problem solving and planning’, in Multi-
than Planning First as it can tackle larger and more complex plan- agentSystemsandApplications(EASSS),volumeLNAI2086,pp.118–
ningproblems.Moreover,whilePlanningFirstisdesignedforsolv- 149.Springer-Verlag,(2001).
ingLCP,MAP-POPisageneral-purposemethodthattacklesprob- [10] J. Hoffmann and B. Nebel, ‘The FF planning system: Fast planning
generationthroughheuristicsearch’,JournalofArtificialIntelligence
lemsofdifferentcouplinglevels.AlthoughMAP-POPbehavesbetter
Research,14,253–302,(2001).
inLCPproblems,itcanalsosolvecomplexTCPproblems.Scalabil-
[11] A.JonssonandM.Rovatsos,‘Scalingupmultiagentplanning:Abest-
ityresultsshowthatPlanningFirstperformsbetterwhendealingwith response approach’, in 21st International Conference on Automated
simpleproblemsthatinvolvefewagents.However,MAP-POPscales PlanningandScheduling(ICAPS),pp.114–121.AAAI,(2011).
upfarbetter,beingabletosolvemuchlargerplanningproblems. [12] S.Kambhampati,‘Refinementplanningasaunifyingframeworkfor
plansynthesis’,AIMagazine,18(2),67–97,(1997).
5 CONCLUSIONSANDFUTUREWORK [13] D.L.Kovacs,‘CompleteBNFdescriptionofPDDL3.1’,Technicalre-
port,(2011).
Thispaperpresentsageneral-purposeMAPmodelsuitabletocope [14] J.Kvarnstro¨m,‘Planningforlooselycoupledagentsusingpartialor-
derforward-chaining’,in21stInternationalConferenceonAutomated
withawidevarietyofMAplanningdomainsunderincompletein-
PlanningandScheduling(ICAPS),pp.138–145.AAAI,(2011).
formation. The ability to define incomplete views of the world for [15] R.Nissim,R.I.Brafman,andC.Domshlak,‘Ageneral,fullydistributed
the agents allows us to deal with more real problems, from inher- multi-agent planning algorithm’, in 9th International Conference on
entlydistributeddomains-functionallyorspatially-toproblemsthat AutonomousAgentsandMultiagentSystems(AAMAS),pp.1323–1330,
(2010).
handleglobalandcentralizedsourcesofinformation.Currently,we
[16] M.Tambe,‘Towardsflexibleteamwork’,JournalofArtificialIntelli-
aretestingourplanneronlarge-sizelogisticsapplicationsinwhich
genceResearch,7,83–124,(1997).
agents are geographically distributed and are completely unaware [17] H.Tonino,A.Bos,M.deWeerdt,andC.Witteveen,‘Plancoordination
of the other agent’s information except for the coordination points byrevisionincollectiveagentbasedsystems’,ArtificialIntelligence,
withintheirworkingareas. 142(2),121–145,(2002).
[18] H.L.S.YounesandR.G.Simmons,‘Vhpop:Versatileheuristicpartial
TheMAPresolutionprocessisaPOP-basedrefinementplanning
orderplanner’,JournalofArtificialIntelligenceResearch,20,405–430,
approachthatiterativelycombinesplanningandcoordinationwhile (2003).
maintainingforeachagentonlytheinformationthatisvisibletothe [19] J.F.Zhang,X.T.Nguyen,andR.Kowalczyk,‘Graph-basedmulti-agent
planningentity.ThisPOPapproachcenteredaroundthegradualcon- replanningalgorithm’,in6thInternationalConferenceonAutonomous
AgentsandMultiagentSystems(AAMAS),(2007).
structionofajointsolutionplanfortheMAPtaskhighlybenefitsthe
resolutionofcooperativedistributedplanningproblems.
We have compared our MAP approach against Planning First, a