FMAP: Distributed Cooperative Multi-Agent Planning

0.88 MB·English

by  Alejandro Torreño

Noname manuscript No.
(will be inserted by the editor)
FMAP: Distributed Cooperative Multi-Agent Planning
Alejandro Torren˜o · Eva Onaindia · O´scar Sapena
5
Received:date/Accepted:date
1
0
2
Abstract ThispaperproposesFMAP(ForwardMulti- Keywords Distributed Algorithms · Multi-Agent

n
Agent Planning), a fully-distributed multi-agent plan- Planning · Heuristic Planning · Privacy
a
ning method that integrates planning and coordina-
J
tion. Although FMAP is specifically aimed at solving
8
2 problems that require cooperation among agents, the 1 Introduction
flexibility of the domain-independent planning model

]
I allows FMAP to tackle multi-agent planning tasks of Multi-agent planning (MAP) introduces a social ap-
A any type. In FMAP, agents jointly explore the plan proachtoplanningbywhichmultipleintelligententities
s. space by building up refinement plans through a com- worktogethertosolveplanningtasksthattheyarenot
c plete and flexible forward-chaining partial-order plan- able to solve by themselves, or to at least accomplish
[
ner. The search is guided by hDTG, a novel heuris- them better by cooperating [40]. MAP places the focus

1 tic function that is based on the concepts of Domain onthecollectiveeffortofmultipleagentstoaccomplish
v Transition Graph and frontier state and is optimized tasks by combining their knowledge and capabilities.
0
5 to evaluate plans in distributed environments. Agents The complexity of solving a MAP task directly de-
2 inFMAPapplyanadvancedprivacymodelthatallows pendsonitstypology.Inordertoillustratethefeatures
7 themtoadequatelykeepprivateinformationwhilecom- of a MAP task, let us introduce a brief application ex-
0 municating only the data of the refinement plans that ample.
.
1 is relevant to each of the participating agents. Exper-
0 imental results show that FMAP is a general-purpose Example 1 Consider the transportation task in Fig. 1,
5
approachthatefficientlysolvestightly-coupleddomains which involves three different agents. There are two
1
: that have specialized agents and cooperative goals as transport agencies (ta1 and ta2), each of which has a
v
well as loosely-coupled problems. Specifically, the em- truck (t1 and t2, respectively). The two agencies work
i
X pirical evaluation shows that FMAP outperforms cur- intwodifferentgeographicalareas,ga1andga2,respec-
r rent MAP systems at solving complex planning tasks tively. The third agent is a factory, f, which is placed
a
thatareadaptedfromtheInternationalPlanningCom- intheareaga2.Tomanufactureproducts,factoryf re-
petition benchmarks. quires raw materials (rm) that are gathered from area
ga1.Inthistask,ta1andta2havethesamecapabilities,
but they act in different areas; i.e., they are spatially
AlejandroTorren˜o
distributed agents. Additionally, the factory agent f is
Departamento deSistemasInform´aticosyComputaci´on
functionallydifferentfromta1andta2.Thegoalofthis
UniverstitatPolit`ecnicadeVal`encia
CaminodeVera,s/n,46022,Valencia,Spain task is for f to manufacture a set of final products. In
E-mail:[email protected] ordertocarryoutthetask,ta1willsenditstruckt1to
EvaOnaind´ıa loadtherawmaterialsrmlocatedinl2andthentrans-
E-mail:[email protected] portthemtoastoragefacility(sf)thatisplacedinthe
O´scarSapena intersection of both geographical areas. Then, ta2 will
E-mail:[email protected] completethedeliverybyusingitstruckt2totransport
2 AlejandroTorren˜oetal.
ga1 ga2 ever,doesnottakeintoconsiderationonekeyaspectof
t1 l3
l1 MAPtasks:thepresenceofcooperative goals;i.e.,goals
that cannot be solved individually by any agent since
f
they require the cooperation of specialized agents. Ex-
sf
ample1illustratesatightly-coupledtaskwithonesuch
l2 t2
rm l4 goal since none of the agents can achieve the manu-
facturing of the final products by itself. Instead, they
must make use of their specialized capabilities and in-
Fig. 1 Exampleofatransportationtask
teract with each other to deliver the raw materials and
manufacture the final products.
the materials from sf to f, which will in turn manu-
In this paper, we present FMAP (Forward MAP),
facture the final products. Therefore, this task involves
whichisadomain-independentMAPsystemthatisde-
threespecializedagentsthatarespatiallyandfunction-
signed to cope with a great variety of planning tasks of
ally distributed which must cooperate to accomplish a
differentcomplexityandcouplinglevel.FMAPisafully
common goal.
distributed method that interleaves planning and coor-
dinationbyfollowingacooperativerefinementplanning
Example 1 emphasizes most of the key elements of
approach. This search scheme allows us to efficiently
aMAPtask.First,thespatialand/orfunctionaldistri-
coordinate agents’ actions in any type of planning task
butionofplanningagentsgivesrisetospecializedagents
(either loosely-coupled or tightly-coupled) as well as to
thathave differentknowledgeandcapabilities.Inturn,
handle cooperative goals.
this information distribution stresses the issue of pri-
FMAP relies on a theoretical model which defines
vacy, which is one of the basic aspects that should be
a more sophisticated notion of privacy than most of
considered in multi-agent applications [33].
the existing MAP systems. Instead of using a single
Since the three parties involved in Example 1 are
set of private data, FMAP allows agents to declare
specialized in different functional or geographical ar-
the information they will share with each other. For
eas of the task, most of the information managed by
instance, the transport agency ta2 in Example 1 will
factory f is not relevant for the transport agencies and
share with factory f information that is likely to be
vice-versa.Thesameoccurswiththetransportagencies
different from the information shared with agent ta1.
ta1 and ta2. Additionally, agents may not be willing to
Our system enhances privacy by minimizing the infor-
share the sensitive information of their internal proce-
mation that agents need to disclose. FMAP is a com-
dures with the others. For instance, ta1 and ta2 are
pleteandreliableplanningsystemthathasproventobe
cooperating in this particular delivery task, but they
very competitive when compared to other state-of-the-
might be potential competitors since they work in the
art MAP systems. The experimental results will show
same business sector. Therefore, agents in a MAP con-
that FMAP is particularly effective for solving tightly-
text want to minimize the information they share with
coupled MAP problems with cooperative goals.
each other, either for strategic reasons or simply be-
Thisarticleisorganizedasfollows:section2presents
cause it is not relevant for the rest of the agents in
some related work on multi-agent planning, with an
order to address the planning task.
emphasis on issues like the coupling level of planning
Besides the need for computational or information
tasks, privacy, or cooperative goals. Section 3 formal-
distribution, privacy is also one of the reasons to adopt
izes the notion of a MAP task; section 4 describes the
amulti-agentapproach.Thisaspect,however,hasbeen
main components of FMAP, the search procedure, and
traditionallyrelegatedinMAP,particularlybytheplan-
theDTG-basedheuristicfunction;finally,section5pro-
ning community [21]. While some approaches define a
videsathoroughexperimentalevaluationofFMAPand
basic notion of privacy [2,25], others allow agents to
section 6 concludes the paper.
share detailed parts of their plans or do not take pri-
vate information into account at all [22].
The complexity of a MAP task is often described
by means of its coupling level [4], which is measured 2 Related work
as the number of interactions that arise among agents
during the resolution of a MAP task. According to this In the literature, there are two main approaches for
parameter, MAP tasks can be classified into loosely- solving MAP tasks like the one described in Exam-
coupled tasks (which present few interactions among ple 1. Centralized MAP involves using an intermediary
agents) and tightly-coupled tasks (which involve many agentthathascompleteknowledgeofthetask.Thedis-
interactions among agents). The coupling level, how- tributed ordecentralizedapproachspreadstheplanning
FMAP:Distributed CooperativeMulti-AgentPlanning 3
responsability among agents, which are in charge of in- assumes that each agent is able to solve a subset of the
teracting with each other to coordinate their local so- task’s goals by itself. However, some approaches use
lutions, if necessary [28,18]. The adoption of a central- plan merging to coordinate local plans of specialized
izedapproachisaimedatimprovingtheplannerperfor- agents. In this case, the effort is placed on discovering
mancebytakingadvantageoftheinherentstructureof the interaction points among agents through the pub-
the MAP tasks [22,8]. Centralized approaches assume lic information that they share. For instance, Planning
a single planning entity that has complete knowledge First [25] introduces a cooperative MAP approach for
of the task, which is rather unrealistic if the parties loosely-coupledtasks,inwhichspecializedagentscarry
involved in the task have sensitive private information out planning individually through a state-based plan-
that they are not willing to disclose [32]. In Example ner. The resulting local plans are then coordinated by
1, the three agents involved in the task want to protect solving a distributed Constraint Satisfaction Problem
the information regarding their internal processes and (CSP) [16]. This combination of CSP and planning to
business strategies, so a centralized setting is not an solve MAP tasks was originally introduced by the MA-
acceptable solution. STRIPS framework [4].
WethenfocusonfullydistributedMAP,thatis,the Another major research trend in MAP interleaves
problemofcoordinatingagentsinasharedenvironment planningandcoordination,providingamoreunifiedvi-
whereinformationisdistributed.ThedistributedMAP sion of cooperative MAP. One of the first approaches
setting involves two main tasks: the planning of local to domain-independent MAP is the Generalized Par-
solutions and the coordination of the agents’ plans into tial Global Planning (GPGP) framework [23]. Agents
a global solution. Coordination can be performed at in GPGP have a partial view of the world and commu-
one or various stages of the distributed resolution of a nicate their local plans to the rest of the agents, which
MAP task. Some techniques are used for problems in in turn merge this information into their own partial
which agents build local plans for the individual goals global plan in order to improve it. Approaches to con-
thattheyhavebeenassigned.MAPisaboutcoordinat- tinualplanning(interleavingplanningandexecutionin
ing the local plans of agents so as to mutually benefit a world undergoing continual change), assume there is
by avoiding the duplication of effort. In this case, the uncertainty in the world state and therefore agents do
goalisnottobuildajointplanamongentitiesthatare nothaveacompleteviewoftheworld[5].Specificallyin
functionallyorspatiallydistributedbutrathertoapply [5],agentshavealimitedknowledgeoftheenvironment
plan merging to coordinate the local plans of multiple and limited capabilities, but the authors do not explic-
agents that are capable of achieving the problem goals itly deal with a functional distribution among agents
by themselves [7]. or cooperative goals. TFPOP is a fully centralized ap-
Thereisalargebodyofworkonplan-mergingtech- proach that combines temporal and forward-chaining
niques. The work in [7] introduces a distributed co- partial-order planning to solve loosely-coupled MAP
ordination framework based on partial-order planning tasks [22]. The Best-Response Planning algorithm de-
thataddressestheinteractionsthatemergebetweenthe parts from an initial joint plan that is built through
agents’ local plans. This framework, however, does not thePlanningFirstMAPsystem[25]anditerativelyim-
considerprivacy.Theproposalin[36]isbasedontheit- proves the quality of this initial plan by applying cost
erativerevisionoftheagents’localplans.Agentsinthis optimal planning [17]. Agents can only access the pub-
modelcooperatebymutuallyadaptingtheirlocalplans, lic information of the other agents’ plans thereby pre-
with a focus on improving their common or individual serving privacy, and they optimize their plans with the
benefit. This approach also ignores privacy and agents aim to converge to a Nash equilibrium regarding their
are assumed to be fully cooperative. The approach in preferences. MAP-POP is a fully distributed method
[39] uses multi-agent plan repair to solve inconsisten- that effectively maintains the agents’ privacy [38,37].
cies among the agents’ local plans while maintaining Agents in MAP-POP perform an incomplete partial-
privacy. µ-SATPLAN [9] extends a satisfiability-based order planning search to progressively develop and co-
planner to coordinate the agents’ local plans by study- ordinate a joint plan until its completion.
ing positive and negative interactions among them. Finally, MAPR is a recent planner that performs
Plan-mergingtechniquesarenotverywellsuitedfor goalallocationtoeachagent[2].Agentsiterativelysolve
copingwithtightly-coupledtasksastheymayintroduce the assigned goals by extending the plan of the previ-
exponentially many ordering constraints in problems ous agent. In this approach, agents work under lim-
that require great coordination effort [7]. In general, ited knowledge of the environment by obfuscating the
plan merging is not an effective method for attaining private information in their plans. MAPR is particu-
cooperativegoalssincethisresolutionschemegenerally larly effective for loosely-coupled problems, but it can-
4 AlejandroTorren˜oetal.
not deal with tasks that feature specialized agents and of fluents that defines the initial state of T . Since
MAP
cooperative goals since it assumes that each goal is specialized agents are allowed, they may only know a
achieved by a single agent. Section 5 will show a com- subset of I. Given two agents i and j, Ii ∩ Ij may
parative performance evaluation between MAPR and or may not be ∅; in any case, the initial states of the
FMAP, our proposed approach. agentsnevercontradicteachother.G isthesetofgoals
of T , i.e., the values of the state variables that
MAP
agents have to achieve in order to accomplish T .
MAP
3 MAP task formalization Finally, A=(cid:83) Ai is the set of planning actions of
i∈AG
the agents. Ai and Aj of two specialized agents i and j
Agents in FMAP work with limited knowledge of the
will typically be two disjoint sets since the agents have
planningtaskbyassumingthatinformationthatisnot
their own different capabilities; otherwise, Ai and Aj
representedinanagent’smodelisunknowntotheagent.
may overlap. A includes two fictitious actions α and
i
The states of the world are modeled through a finite
α that do not belong to the action set of any par-
f
set of state variables, V, each of which is associated to
ticular agent: α represents the initial state of T ,
i MAP
a finite domain, D , of mutually exclusive values that
v i.e., PRE(α ) = ∅ and EFF(α ) = I, while α repre-
i i f
refertotheobjectsintheworld.Assigningavaluedto
sentstheglobalgoalsofT ,i.e.,PRE(α )=G,and
MAP f
a variable v ∈V generates a fluent. A positive fluent is
EFF(α )=∅.
f
a tuple (cid:104)v,d(cid:105), which indicates that the variable v takes
the value d. A negative fluent is of the form (cid:104)v,¬d(cid:105),
AsdiscussedinExample1,ourmodelconsidersspe-
indicating that v does not take the value d. A state S
cializedagentsthatcanbefunctionallyand/orspatially
is a set of positive and negative fluents.
distributed. This specialization defines the local view
Anactionisatupleα=(cid:104)PRE(α),EFF(α)(cid:105),where
thateachagenthasoftheMAPtask.Localviewsarea
PRE(α)isafinitesetoffluentsthatrepresentsthepre-
typical characteristic of multi-agent systems and other
conditions of α, and EFF(α) is a finite set of positive
distributedsystems.Forinstance,distributedCSPsuse
and negative variable assignments that model the ef-
local views, such that agents only receive information
fects of α. Executing an action α in a world state S
about the constraints in which they are involved [16,
leads to a new world state S(cid:48) as a result of applying
41]. Next, we define the information of an agent i on a
EFF(α) over S. An effect of the form (v = d) assigns
planning task T .
the value d to the variable v, i.e., it adds the fluent MAP
(cid:104)v,d(cid:105) to S(cid:48) as well as adding a set of fluents (cid:104)v,¬d(cid:48)(cid:105) The view of an agent i on a MAP task TMAP is de-
for each other value d(cid:48) in the variable domain in order fined as TMiAP = (cid:104)Vi,Ai,Ii,G(cid:105). Vi is the set of state
variables known to agent i; Ai ⊆ A is the set of its
to have a consistent state representation. Additionally,
any fluent in S of the form (cid:104)v,¬d(cid:105) or (cid:104)v,d(cid:48)(cid:48)(cid:105), d(cid:48)(cid:48) (cid:54)=d, is capabilities (planning actions); Ii is the subset of flu-
removed in state S(cid:48). This latter modification removes ents of the initial state I that are visible to agent i;
and G is the set of global goals of T . Since agents
any fluent that contradicts (cid:104)v,d(cid:105). On the other hand, MAP
anassignment(v (cid:54)=d)addsthefluent(cid:104)v,¬d(cid:105)toS(cid:48) and in FMAP are fully cooperative, they are all aware of
removes (cid:104)v,d(cid:105) from S(cid:48), if such a fluent exists in S. the global goals of the task. Obviously, because of spe-
cialization, a particular agent may not understand the
Forinstance,letussupposethatthetransportation
goals as specified in G; defining G as global goals im-
task in Example 1 includes a variable pos-rm that de-
pliesthatallagentscontributetotheachievementofG,
scribesthepositionoftherawmaterialsrm,whichcan
either directly (achieving a g ∈ G) or indirectly (intro-
beanyofthelocationsinthetask.LetS beastatethat
ducing actions whose effects help other agents achieve
includes a fluent (cid:104)pos-rm,l2(cid:105), which indicates that rm
g).
is placed in its initial location (see Fig. 1). Agent ta1
performs an action to load rm into its truck t1, which The state variables of an agent i are determined
includes an effect of the form (pos-rm = t1). The ap- by the view the agent has on the initial state, Ii, the
plication of this action results in a new world state S(cid:48) planningactionsitcanperform,Ai,andthesetofgoals
thatwillincludeafluent(cid:104)pos-rm,t1(cid:105)andfluentsofthe of TMAP. This also affects the domain Dv of a variable
form (cid:104)pos-rm,¬l(cid:105) for each other location l (cid:54)= t1; the v.WedefineDvi ⊆Dv asthesetofvaluesofthevariable
fluent (cid:104)pos-rm,l2(cid:105) will no longer be in S(cid:48). v that are known to agent i.
Consider again the pos-rm variable in Example 1.
Definition 1 AMAPtaskisdefinedasatupleT = The domain of pos-rm contains all the locations in
MAP
(cid:104)AG,V,I,G,A(cid:105). AG = {1,...,n} is a finite non-empty the transportation task, including the factory f, the
setofagents.V =(cid:83) Vi,whereVi isthesetofstate storage facility sf, and the trucks; that is, D =
i∈AG pos-rm
variables known to an agent i. I = (cid:83) Ii is a set {l1,l2,l3,l4,f,sf,t1,t2}. However, agents ta1 and ta2
i∈AG
FMAP:Distributed CooperativeMulti-AgentPlanning 5
have local knowledge about the domain of pos-rm be- The introduction of new actions in a partial plan
cause some of the values of such variable refer to ob- may trigger the appearance of flaws. There are two
jects of T that are unknown to them. Hence, ta1 types of flaws in a partial plan: preconditions that are
MAP
will manage Dta1 = {l1,l2,sf,t1}, while ta2 will notyetsolved(orsupported)throughacausallink,and
pos-rm
manage Dta2 ={l3,l4,sf,f,t2}. (cid:104)v,d(cid:105)
pos-rm threats. A threat over a causal link α → β is caused
AgentsinFMAPinteractwitheachotherbysharing
by an action γ that is not ordered w.r.t. α or β and
information about their state variables. For each pair
might potentially modify the value of v [12] ((v (cid:54)=d)∈
of agents i and j, the public information they share EFF(γ) or (v = d(cid:48)) ∈ EFF(γ), d(cid:48) (cid:54)= d), making the
is defined as Vij = Vji = Vi ∩Vj. Additionally, some
causal link unsafe. Threats are addressed by introduc-
of the values in the domain of a variable can also be
ing either an ordering constraint γ ≺ α (this is called
public to both agents. The set of values of a variable v
demotion because the causal link is posted after the
that are public to a pair of agents i and j is defined as
threatening action) or an ordering β ≺γ (this is called
Dij =Di ∩Dj.
v v v promotion because the causal link is placed before the
AsExample1indicates,thepos-rmvariableispub- threatening action) [12].
lic to agents ta1 and ta2. The values that are public to Aflaw-free planisathreat-freepartialplaninwhich
bothagentsaredefinedastheintersectionofthevalues thepreconditionsofalltheactionsaresupportedthrough
that are known to each of them, Dptao1s-tram2 ={sf}. This causal links.
way, the only public location of rm for agents ta1 and
Planning agents in FMAP cooperate to solve MAP
ta2 is the storage facility sf, which is precisely the in-
tasksbyprogressivelyrefininganinitiallyemptyplanΠ
tersection between the two geographical areas. Hence,
until a solution is reached. The definition of refinement
if agent ta1 places rm in sf, it will inform ta2 accord-
plan is closely related to the internal forward-chaining
ingly, and vice versa. This allows agents ta1 and ta2 to
partial-order planning search performed by the agents.
work together while minimizing the information they
Refinement planning is a technique that is widely used
share with each other.
bymanyplanners,specificallyinanytimeplanning,where
Our MAP model is a multi-agent refinement plan-
a first initial solution is progressively refined until the
ning framework, which is a general method based on
deliberation time expires [31]. We define a refinement
the refinement of the set of all possible plans. The in-
plan as follows:
ternal reasoning of agents in FMAP is configured as a
Partial-OrderPlanning(POP)searchprocedure.Other
Definition 3 A refinement plan Π = (cid:104)∆ , OR ,
r r r
local search strategies are applicable, as long as agents
CL (cid:105)overapartialplanΠ =(cid:104)∆,OR,CL(cid:105)isaflaw-free
r
build partial-order plans. The following concepts and
partial plan that extends Π, i.e., ∆⊂∆ , OR⊂OR
r r
definitionsarestandardtermsfromthePOPparadigm
and CL ⊂ CL . Π introduces a new action α ∈ ∆
r r r
[12], which have been adapted to state variables. Ad-
in Π, resulting in ∆ = ∆∪α. All the preconditions
r
ditionally, definitions also account for the multi-agent
in PRE(α) are linked to existing actions in Π through
nature of the planning task and the local views of the
causal links; i.e., all preconditions are supported: ∀p ∈
task by the agents. p
PRE(α), ∃ β →α∈CL , where β ∈∆.
r
Definition 2 Apartial-order planorpartialplanis
RefinementplansinFMAPincludeactionsthatcan
a tuple Π =(cid:104)∆,OR,CL(cid:105). ∆={α|α∈A} is the set of
be executed in parallel by different agents. Some MAP
actions in Π. OR is a finite set of ordering constraints
models consider that two parallel or non-sequential ac-
(≺) on ∆. CL is a finite set of causal links of the form
tions are mutually consistent if neither of them modi-
(cid:104)v,d(cid:105) (cid:104)v,¬d(cid:105)
α → β or α → β, where α and β are actions in fiesthevalueofastatevariablethattheotherrelieson
(cid:104)v,d(cid:105) or affects [5]. We also consider that the preconditions
∆.Acausallinkα → β enforcesprecondition(cid:104)v,d(cid:105)∈
of two mutually consistent actions have to be consis-
PRE(β) through an effect (v = d) ∈ EFF(α) [12].
tent [3]. Hence, two non-sequential actions α∈Ai and
(cid:104)v,¬d(cid:105)
Similarly, a causal link α → β enforces (cid:104)v,¬d(cid:105) ∈
β ∈Aj are mutually consistent if none of the following
PRE(β) through an effect (v (cid:54)= d) ∈ EFF(α) or (v =
conditions hold:
d(cid:48))∈EFF(α), d(cid:48) (cid:54)=d.
– ∃(v = d) ∈ EFF(α) and ∃((cid:104)v,d(cid:48)(cid:105) ∈ PRE(β) ∨
Anempty partialplanisdefinedasΠ =(cid:104)∆ ,OR , (cid:104)v,¬d(cid:105)∈PRE(β)),wherev ∈Vij,d∈Dij,d(cid:48) ∈Dj
0 0 0 v v
CL (cid:105),whereOR andCL areemptysets,and∆ con- and d (cid:54)= d(cid:48), or vice versa; that is, the effects of α
0 0 0 0
tains only the fictitious initial action α . A partial plan and the preconditions of β (or vice versa) do not
i
Π for a task T will always contain α . contradicteachotherunderthespecifiedconditions.
MAP i
6 AlejandroTorren˜oetal.
– ∃(v = d) ∈ EFF(α) and ∃((v = d(cid:48)) ∈ EFF(β)∨ i w.r.t. agent j. Instead of (cid:104)v,d(cid:105), agent i will send
(v (cid:54)= d) ∈ EFF(β)), where v ∈ Vij, d ∈ Dij, d(cid:48) ∈ agent j a fluent (cid:104)v,⊥(cid:105), where ⊥ is the undefined
v
Dj andd(cid:54)=d(cid:48),orviceversa;thatis,theeffectsofα value. Hence, preconditions of the form (cid:104)v,d(cid:105) will
v
andtheeffectsofβ (orviceversa)donotcontradict be sent as (cid:104)v,⊥(cid:105), effects of the form (v =d) will be
each other under the specified conditions. (cid:104)v,d(cid:105)
replaced by (v =⊥), and causal links α → β will
– ∃(cid:104)v,d(cid:105)∈PRE(α)and∃((cid:104)v,d(cid:48)(cid:105)∈PRE(β)∨(cid:104)v,¬d(cid:105)∈
(cid:104)v,⊥(cid:105)
adopt the form α → β.
PRE(β)), where v ∈ Vij, d ∈ Dij, d(cid:48) ∈ Dj and
v v
d (cid:54)= d(cid:48), or vice versa; that is, the preconditions of
Ifanagentj receivesafluent(cid:104)v,⊥(cid:105),⊥isinterpreted
α and the preconditions of β (or vice versa) do not
as follows: ∀d ∈ Dj, (cid:104)v,¬d(cid:105). That is, ⊥ indicates that
contradicteachotherunderthespecifiedconditions. v
v is not assigned any of the values known to agent j
Agentsaddressparallelismbytheresolutionofthreats (Dj). This mechanism is used to inform an agent that
v
over the causal links of the plan. Thus, consistency be- aresourceisnolongeravailableinitsinfluencearea.For
tweenanytwonon-sequentialactionsintroducedbydif- instance,supposethatagentta2inExample1acquires
ferent agents is always guaranteed as refinement plans therawmaterialrmfromsf byloadingitintoitstruck
are flaw-free plans. t2.Agentta2communicatestota1thatrmisnolonger
Finally, a solution plan for T is a refinement in sf, but agent ta1 does not know about the truck t2.
MAP
plan Π = (cid:104)∆, OR, CL(cid:105) that addresses all the global Tosolvethisissue,ta2sendsta1thefluent(cid:104)pos-rm,⊥(cid:105),
goals G of T . A solution plan includes the ficti- meaning that the resource rm is no longer available in
MAP
tious final action α and ensures that all its precondi- thegeographicalareaofagentta1.Consequently,ta1is
f
tions (note that PRE(α ) = G) are satisfied; that is, nowawarethatrmisnotlocatedinanyofitsaccessible
f
g positions Dta1 ={l1,l2,sf,t1}.
∀g ∈ PRE(αf), ∃ β → αf ∈ CL, β ∈ ∆, which is the pos-rm
necessary condition to guarantee that Π solves T . Fig. 2 shows the view that the transport agents ta1
MAP
and ta2 in Example 1 have of a simple refinement plan
Π . In this plan, agent ta1 drives the truck t1 from
r
3.1 Privacy in partial plans l1 to l2 and loads rm into t1. As shown in Fig. 2a,
viewta1(Π ) contains all the information of both ac-
r
Everytimeanagentirefinesapartialplanbyintroduc-
tions in the plan since agent ta1 has introduced them.
ing anew action α∈Ai, itcommunicatesthe resulting
Agent ta2, however, does not know about the truck
refinement plan to the rest of the agents in T . As
MAP t1, and hence the variable pos-t1, which models the
statedabove,theinformationthatispublictoapairof
position of t1, is private to ta1 w.r.t. ta2. This way,
agents is defined according to the common state vari-
all the preconditions and effects related to the fluents
ables and domain values. In order to preserve privacy, (cid:104)pos-t1,l1(cid:105)and(cid:104)pos-t1,l2(cid:105)areoccludedinviewta2(Π )
r
agent i will only communicate to agent j the fluents in
(see Fig. 2b). Additionally, the causal links regarding
action α whose variables are common to both agents.
these two fluents are replaced by ordering constraints
The information of a refinement plan Π that agent j inviewta2(Π ).Ontheotherhand,thevariablepos-rm
r
receives from agent i configures its view of that plan,
ispublictobothagents,buttheloadactionreferstothe
viewj(Π). More specifically, given two agents i and j locationst1andl2,whicharenotinDta2 .Therefore,
and a fluent (cid:104)v,d(cid:105), where v ∈ Vi and d ∈ Di (equiva- pos-rm
v fluents (cid:104)pos-rm,l2(cid:105) and (cid:104)pos-rm,t1(cid:105) are partially pri-
lently for a negative fluent (cid:104)v,¬d(cid:105)), we distinguish the vate to agent ta1 w.r.t. ta2. This way, in viewta2(Π ),
r
three following cases:
the precondition (cid:104)pos-rm,l2(cid:105) and the effect (pos-rm=
t1) of the load action are replaced by (cid:104)pos-rm,⊥(cid:105) and
– Public fluent: if v ∈ Vij and d ∈ Dij, the fluent
v
(pos-rm =⊥), respectively. The fluent (cid:104)pos-rm,l2(cid:105) is
(cid:104)v,d(cid:105)ispublictobothiandj,andthusagentiwill
(cid:104)pos-rm,l2(cid:105)
send agent j all the causal links, preconditions, and alsoreplacedby(cid:104)pos-rm,⊥(cid:105)inthecausallinkα →
i
effects regarding (cid:104)v,d(cid:105). loadt1rml2.
– Private fluent to agent i: if v (cid:54)∈ Vij, the fluent
(cid:104)v,d(cid:105) is private to agent i w.r.t. agent j, and thus
agentiwilloccludethepreconditionsandeffectsre-
3.2 MAP definition language
garding (cid:104)v,d(cid:105) to agent j. Causal links of the form
(cid:104)v,d(cid:105)
α → β will be sent to agent j as ordering con- There is a large body of work on planning task specifi-
straints α≺β. cationlanguages.Sinceplanninghasbeentraditionally
– Partiallyprivatefluenttoagenti:ifv ∈Vij but regarded as a centralized problem, the most popular
d(cid:54)∈Dij,thefluent(cid:104)v,d(cid:105)ispartiallyprivatetoagent definition languages, such as the different versions of
v
FMAP:Distributed CooperativeMulti-AgentPlanning 7
!"#$%12()*+, )0*
)0, "0 !i !f
./ !"#$%&'()*', !!""##$$%%1&2'(())**++,,
- !"#$%&'()*', 01345)&')*')*+ !"#$%&'()*+, *#.0)&')12)*+ -
i f
7"#$%&'8*+/ 7"#$%128&'/ " ! #$%&'( ! " ! #$%&'( !
00 i )*(+*(+, f 03 i ),(+/(. f
!"#$%12()⊥,
6/ - - !"#$%-12()⊥, - "01 !i )*#($+%*&('-(. !f "02 !i ),#($+%/&('-(. !f
i 0 1 f
7"#$%128)⊥/
Fig. 3 FMAPmulti-agentsearchtreeexample
Fig. 2 A refinement plan Π as viewed by: a) agent ta1 b)
r
agent ta2
in which the partial plans of the nodes are built with
the contributions of one or more agents.
PDDL (the Planning Domain Definition Language1), Fig.3showsthefirstlevelofthemulti-agentsearch
aredesignedtomodelsingle-agentplanningtasks.MAP treethatwouldbegeneratedforthetransportationtask
introduces a set of requirements that are not present of Example 1. At this level, agents ta1 and ta2 each
in single-agent planning, such as privacy or specialized propose two refinement plans, specifically the plans to
agents,whichmotivatethedevelopmentofspecification move their trucks within their geographical areas. In
languages for multi-agent planning. each of these refinement plans, the agent adds one ac-
TherearemanydifferentapproachestoMAPasde- tion and the corresponding orderings and causal links.
scribed in section 2. MA-STRIPS [4], which was de- Agent f does not contribute here with any refinement
signed as a minimalistic extension to STRIPS [10], is plan because the initial empty plan Π does not have
0
one of the most common MAP languages. It allows thenecessarysupportinginformationforf toinsertany
defining a set of agents and associating the planning of its actions. In a subsequent iteration (expansion of
actions they can execute. FMAP presents several ad- the next tree node), agents can in turn create new re-
vancedfeaturesthatmotivatedthedefinitionofourown finementplans.Forinstance,ifnodeΠ inFig.3isse-
00
PDDL-based specification language (the language syn- lectednextforexpansion,the threeagentsintheprob-
tax is detailed in [38]) rather than using MA-STRIPS. lem (ta1, ta2, or f) will try to create refinement plans
SincetheworldstatesinFMAParemodeledthrough overΠ byaddingoneoftheiractionsandsupporting
00
statevariablesinsteadofpredicates,ourMAPlanguage it through the necessary causal links and orderings.
is based on PDDL3.1 [20], the latest version of PDDL. Agents keep a copy of the multi-agent search tree,
Unlike its predecessors, which model planning tasks storing the local view they have of each of the plans
through predicates, PDDL3.1 incorporates state vari- in the tree nodes. Given a node Π in the multi-agent
ablesthatmaptoafinitedomainofobjectsofthetask. search tree, an agent i maintains viewi(Π) in its copy
Inasingle-agentlanguage,theuserspecifiesthedo- of the tree.
main of the task (planning operators, types of objects,
FMAP applies a multi-agent A* search that iter-
andfunctions)andtheproblem tobesolved(objectsof
atively explores the multi-agent tree. One iteration of
the task, initial state, and goals). In FMAP, we write a
FMAP involves the following: 1) agents select one of
domain and a problem file for each agent, which define
the unexplored leaf nodes of the tree for expansion; 2)
the typology of the agent, and the agent’s local view
agents expand the selected plan by generating all the
of the MAP task, respectively. The domain files keep
refinementplansoverthisnode;and3)agentsevaluate
the structure of a regular PDDL3.1 domain file. The
the resulting successor nodes and communicate the re-
problemfiles,however,areextendedwithanadditional
sultstotherestoftheagents.Insteadofusingabroad-
:shared-data section, which specifies the information castcontrolframework,FMAPusesdemocraticleader-
that an agent can share with each of the other partici-
ship,inwhichacoordinatorroleisscheduledamongthe
pating agents in the task.
agents. One of the agents adopts the role of coordina-
tor at each iteration, thus leading the procedure in one
iteration (initially, the coordinator role is randomly as-
4 FMAP refinement planning procedure
signed to one of the participating agents). More specif-
ically, a FMAP iteration is as follows:
FMAP is based on a cooperative refinement planning
procedureinwhichagentsjointlyexploreamulti-agent,
– Base plan selection: Among all the open nodes
plan-spacesearchtree.Amulti-agentsearchtreeisone
(unexplored leaf nodes) of the multi-agent search
1 http://en.wikipedia.org/wiki/Planning_Domain_ tree,thecoordinatoragentselectsthemostpromis-
Definition_Language ingplan,Πb,asthebaseplantorefineinthecurrent
8 AlejandroTorren˜oetal.
iteration. Πb is selected according to the evaluation 3 ! "#$%&' "#$%&' !
of the open nodes (details on the node evaluation 001 i ()'*)'*+ ()'*+',- f
and selection are presented in section 4.3). In the
initialiteration,thebaseplanistheemptyplanΠ0. /1 !i ()"'#*$)%'&*+' ()"'#*$+%'&,'- FS ()"'#,$-%'&*+' *.#0/"'*'(+)' !f
– Refinement plan generation:AgentsexpandΠ
b
and generate its successor nodes. A successor node
! "#$%&' "#$%&' !
isarefinementplanoverΠb thatanagentgenerates 21 i ()'*)'*+ *./"'()' ()'*+',- f
individuallythroughitsembeddedforward-chaining #0'*+
partial-order planner (see subsection 4.1).
Fig. 4 Loading rm in plan Π : a) inserting actions from
001
– Refinement plan evaluation:Eachagentievalu- afrontierstateb)insertingactionsusing FLEX
ates its refinement plans Π by applying a classical
r
A* evaluation function (f(Π ) = g(viewi(Π )) +
r r
ents according to the level of privacy defined between
h(viewi(Π ))).Theexpressiong(viewi(Π ))stands
r r
them, as described in subsection 3.1. This way, ta1 will
for the number of actions of Π . Since agents view
r send viewta2(Π ) and viewta2(Π ) to agent ta2, and
all the actions of the plans (but not necessarily all 00 01
viewf(Π ) and viewf(Π ) to agent f.
theirpreconditionsandeffects),g(viewi(Π ))isequiv- 00 01
r
The following subsections analyze the key elements
alenttog(Π ).h(viewi(Π ))appliesourDTG-based
r r
of FMAP, that is, the search algorithm that agents
heuristic(seesubsection4.3)toestimatethecostof
use for the generation of the refinement plans and the
reaching a solution plan from Π .
r
heuristicfunctiontheyuseforplanevaluation.Wealso
– Refinement plan communication: Each agent
include a subsection that addresses the completeness
communicatesitsrefinementplanstotherestofthe
andcorrectnessofthealgorithmaswellasasubsection
agents. The information that an agent i communi-
that describes the limitations of FMAP.
cates about its plan Π to the rest of the agents
r
depends on the level of privacy specified with each
of them. Along with the refinement plan Π , agent
r 4.1 Forward-Chaining Partial-Order Planning
i communicates the result of the evaluation of Π ,
r
f(Π ).
r Agents in FMAP use an embedded flexible forward-
Once the iteration is completed, the leadership is chainingPOPsystemtogeneratetherefinementplans;
handedtoanotheragent,whichadoptsthecoordinator thiswillbereferredtoasFLEXintheremainderofthe
role, and a new iteration starts. The next coordinator paper. Similarly to other approaches, FLEX explores
agent selects the open node Π that minimizes f(Π) thepotentialofforwardsearchtosupportpartial-order
as the new base plan Π , and then, agents proceed to planning. OPTIC [1], for instance, combines partial-
b
expand it. This iterative process carries on until Π order structures with information on the frontier state
b
becomes a solution plan that supports the final action of the plan. Informally speaking, the frontier state of
α , or when all the open nodes have been visited, in the partial plan of a tree node is the resulting state
f
which case, the agents will have explored the complete after executing the actions in such a plan. Given a re-
search space without finding a solution for the MAP finement plan Π = (cid:104)∆,OR,CL(cid:105), we define its frontier
task T . state FS(Π) as the set of fluents (cid:104)v,d(cid:105) achieved by
MAP
A refinement plan Π is evaluated only by the agent actions α ∈ ∆ | (cid:104)v,d(cid:105) ∈ EFF(α), such that any ac-
that generates it. The agent communicates Π along tion α(cid:48) ∈ ∆ that modifies the value of the variable v
with f(Π) to the rest of the agents. Therefore, the de- ((cid:104)v,d(cid:48)(cid:105)∈EFF(α(cid:48))|d(cid:54)=d(cid:48)) is not reachable from α by
cisiononthenextbaseplanisnotaffectedbytheagent following the orderings and causal links in Π.
that plays the coordinator role since all of the agents The only actions that OPTIC adds to a plan are
manage the same f(Π) value for every open node Π. those whose preconditions hold in the frontier state.
In the example depicted in Fig. 3, agent ta1 evalu- This behaviour forces OPTIC to some early commit-
ates its refinement plans, Π and Π , and communi- ments; however, this does not sacrifice completeness,
00 01
catesthemalongwithf(Π )andf(Π )toagentsta2 because search can backtrack. Also, TFPOP [22] ap-
00 01
and f; likewise, ta2 with ta1 and f. In this first level plies a centralized forward-chaining POP for multiple
of the tree, agents ta1 and ta2 have a complete view agents,keepingasequentialexecutionthreadperagent.
of the refinement plans, that they have generated since The aforementioned approaches only permit intro-
theseplansonlycontainanactionthattheythemselves ducing actions that are applicable in the frontier state
introduced. However, when ta1 and ta2 communicate of the plan. In contrast, FLEX allows inserting actions
their plans to each other, they will only send the flu- at any position of the plan without assuming that any
FMAP:Distributed CooperativeMulti-AgentPlanning 9
action in the plan has already been executed. This is a Base plan
moreflexibleapproachthatisalsomorecompliantwith & % drive %
00 i t1 l1 l2 f
the least-commitment principle that typically guides
Candidate Actions ⊆ !"#$
backward-chaining POP. Fig. 4 shows the advantages Potentially
!"#$ Supportable load t1 drive drive drive drive
of our flexible search strategy. Consider the refinement Actions rm l2 t1 l2 sf t1 l2 l1 t1 l1 sf t1 l1 l2
Estimate
plan Π , which is the result of a refinement of agent
001
ta1 on plan Π00 (see Fig. 3) after including the action Independent POP
(drive t1 l1 sf). This is not the best course of action trees for each action
for taking the raw material rm to the factory f as ta1
& & & Leaf nodes
should load rm into t1 before moving to sf. The fron- 000 001 002 (Refinement plans)
tier state FS(Π001) reflects the state of the world after Fig. 5 FLEX algorithm as applied by agent ta1 over plan
executingtheplanΠ001,inwhichthetruckt1wouldbe Π00
at sf. Planners like OPTIC would only introduce ac-
tionsthatareapplicableinthefrontierstateFS(Π ).
001
coversallthepossiblewaystosupportthepreconditions
In this example, OPTIC would first insert the action
of α through causal links.
(drive t1 sf l2) to move the truck t1 back to l2 in or- f
der to be able to apply the action (load t1 rm l2) (see If a solution plan is not found, agent i analyzes
Fig. 4a). FLEX, however, is able to introduce actions all its planning actions Ai and estimates if they are
at any position in the plan, so the load action can be supportable in Πb. Given an action α ∈ Ai, the func-
directly placed between both drive actions, thus mini- tion potentiallySupportable(α,Πb) checks if ∀(cid:104)v,d(cid:105) ∈
mizing the length of the plan (see Fig. 4b). PRE(α), ∃β ∈ ∆(Πb) | (v = d) ∈ EFF(β), i.e., the
agent estimates that α is supportable if for every pre-
condition of α there is a matching effect among the
actions of Π .
Algorithm 1: FLEX search algorithm for an b
agent i Fig. 5 shows an example of potentially supportable
actions. Agent ta1 evaluates all the actions in Ata and
RPi←∅
if potentiallySupportable(αf,viewi(Πb))then finds five candidate actions. In αi, the initial state of
returnsolutionPlans Π , the truck t1 is at location l1. Consequently, ta1
00
CandidateActions←∅ considers(drivet1l1sf)and(drivet1l1l2)aspoten-
forall the α∈Ai do
tial candidate actions for its refinements. Note that ac-
if potentiallySupportable(α,viewi(Π ))then
b
CandidateActions←CandidateActions∪α tion(drivet1l1l2)isalreadyincludedinplanΠ00.Ac-
tions(drivet1l2sf),(drivet1l2l1),and(loadt1rml2)
forall the α∈CandidateActionsdo
are also classified as candidates since they are applica-
Plans←{viewi(Π )}
b
repeat ble after the action (drivet1l1l2), which is already in
Selectandextract Πs∈Plans plan Π00.
Flaws(Π )←
s It is possible to introduce an action multiple times
unsupportedPrecs(α,Π )∪Threats(Π )
s s
if Flaws(Π )=∅then in a plan; for instance, a truck may need to travel back
s
RPi←RPi∪Πs andforthbetweentwodifferentlocationsseveraltimes.
else Forthisreason,ta1againconsiders(drivet1l1l2)asa
Selectandextract Φ∈Flaws(Πs) candidate action when refining Π00, even if this action
Plans←Plans∪solveFlaw(Π ,Φ)
s is already included in Π . By estimating potentially
00
untilPlans=∅ supportable actions in any position of the plan, FLEX
returnRPi follows the least commitment principle and does not
leave out any potential refinement plan.
The potentiallySupportable procedure is an esti-
Algorithm 1 summarizes the FLEX procedure in- mate because it does not actually check the possible
voked by an agent i to generate refinement plans, and flaws that arise when supporting an action. Hence, an
Fig.5showshowagentta1inExample1usestheFLEX agentanalyzesthealternativesthatsupporteachcandi-
algorithm to refine plan Π in Fig. 3. The first oper- date action α by generating a POP search tree for that
00
ation of an agent i that executes FLEX is to check particular action (repeat loop in Algorithm 1). All the
whether the fictitious final action α is supportable in leaf nodes of the tree (stored in the Plans list in Algo-
f
Π , that is, if a solution plan can be obtained from Π . rithm 1) are explored, thereby covering all the possible
b b
Ifso,theagentwillgenerateasetofsolutionplansthat ways to introduce α in Π .
b
10 AlejandroTorren˜oetal.
Asinbackward-chainingPOP,FLEXintroducesthe someofthecausallinksoftheplan.Supposethatagent
action α in Π by supporting itspreconditions through iintroducesanactionα withaneffect(v =d(cid:48))inplan
b t
causal links and solving the threats that arise during Π. Additionally, there is a causal link in Π of the form
thesearch.Thesetofflaw-freeplansobtainedfromthis (cid:104)v,d(cid:105)
cl = α → α introduced by an agent j; as cl is not
0 1
search are stored in RPi as valid refinement plans of
ordered with respect to α , this situation generates a
t
agent i over Πb. This procedure is carried out for each threat. According to viewi(Π), agent i may find one of
candidate action. Completeness is guaranteed since all
the following situations:
thepossiblerefinementplansoveragivenbaseplanare
generated by the agents involved in T . – If (cid:104)v,d(cid:105) is public to i and j, then cl is in viewi(Π),
MAP
Fig. 5 shows that, for every candidate action, ta1 and thus the threat between cl and αt will be cor-
performsanindependentPOPsearchaimedatsupport- rectly detected and solved by promoting or demot-
ingtheaction.Actions(loadt1rml2),(drivet1l2sf), ing αt.
and (drive t1 l2 l1) lead to three different refinement – If (cid:104)v,d(cid:105) is private to j w.r.t. i, then αt cannot con-
plans over Π : {Π , Π ,Π }. These plans will tain an effect (v = d(cid:48)) because v (cid:54)∈ Vi. Therefore,
00 000 001 002
thenbeinsertedintota1’scopyofthemulti-agentsearch the threat described above can never occur in Π.
tree. Agent ta1 will also send the information of these – If (cid:104)v,d(cid:105) is partially private to j w.r.t. i, then cl =
(cid:104)v,d(cid:105) (cid:104)v,⊥(cid:105)
plans to agents ta2 and f according to the level of pri- α → α will be seen as cl = α → α in
0 1 0 1
vacy defined with each one. ta2 and f also store the viewi(Π).Since⊥=(cid:54) d,agentiwillbeabletodetect
received plans in their copies of the tree. and address the threat between α and cl.
t
Candidate action (drivet1l1sf) does not produce
Consequently,anagentcanalwaysdetectthearising
valid refinement plans because it causes an unsolvable
threatswhenitaddsanewaction,α ,intheplan.Now,
threat. This is because truck t1 cannot simultaneously t
we should study whether the potential threats caused
move to two different locations from l1, which causes a
by actions in Π on the causal links that support the
conflict between the existing action (drive t1 l1 l2) ∈
action α are correctly detected by agent i. Suppose
∆(Π )and(drivet1l1sf).Similarly,action(drivet1l1l2) t
does0n0otyieldanyvalidrefinements.Theresultingplan that there is a causal link cl(cid:48) = β (cid:104)v→(cid:48),e(cid:105) αt, and an
wouldhavetwoactions(drivet1l1l2)inparallel,both action γ with an effect (v(cid:48) = e(cid:48)) which is not ordered
of which are linked to αi, which causes an unsolvable withrespecttoαt.Again,agentimayfinditselfinthree
threat because t1 cannot perform two identical drive different scenarios according to its view of (v(cid:48) =e(cid:48)):
actions in parallel.
– If (v(cid:48) = e(cid:48)) is public to i and j, the threat between
cl(cid:48) and γ will be correctly detected by i.
4.2 Completeness and Soundness – If (v(cid:48) = e(cid:48)) is private to j w.r.t. i, then none of
the variables in PRE(α ) are related to v(cid:48) because
t
As explained in the previous section, agents refine the v(cid:48) (cid:54)∈Vi. Thus, this threat will never arise in Π.
base plan concurrently by analyzing all of the possible – If (v(cid:48) =e(cid:48)) is partially private to j w.r.t. i, (v(cid:48) =e(cid:48))
ways to support their actions in the base plan. Since willbeseenas(v(cid:48) =⊥)inviewi(Π).Since⊥(cid:54)=e,the
this operation is done by every agent and for all their threat between γ and cl(cid:48) will be correctly detected
actions, we can conclude FMAP is a complete proce- by agent i.
dure that explores the whole search space.
Note that privacy does not prevent agents from de-
Asforsoundness,apartial-orderplanissoundifitis
tecting and solving threats nor does it affect the com-
a flaw-free plan. The FLEX algorithm addresses incon-
plexityoftheprocess.Ifthefluentispublicorpartially
sistencies among actions in a partial plan by detecting
private, the agent that is refining the plan will be able
and solving threats.
to detect the threat because it either sees the value of
When an agent i introduces an action α in a base
thevariableorsees⊥,andbothcontradictthevalueof
plan Π, FLEX studies the threats that α causes in the
the variable in the causal link. If the fluent is private,
causal links of Π and the threats that the actions of Π
thenthereisnosuchthreat.ThisprovesthatFMAPis
may cause in the causal links that support the precon-
sound.
ditionsofα.Inbothcases,iisabletodetectallthreats
whatever its view of the plan is, viewi(Π). That is,
FMAP soundness is guaranteed regardless of the level 4.3 DTG-based Heuristic Function
of privacy defined between agents.
Withregardtothethreatscausedbytheeffectsofa ThelastaspectofFMAPtoanalyzeishowagentseval-
newaction,privacymaypreventtheagentfromviewing uate the refinement plans. FMAP guides the search