Ambiguity and Incomplete Information in Categorical Models of Language

0.11 MB·

by  Dan Marsden

Ambiguity and Incomplete Information in Categorical
Models of Language
Dan Marsden
UniversityofOxford
[email protected]
We investigatenotionsof ambiguityand partialinformationin categoricaldistributionalmodelsof
naturallanguage.Probabilisticambiguityhaspreviouslybeenstudiedin[27,26,16]usingSelinger’s
CPM construction. Thisconstructionworkswell for modelsbuilt uponvectorspaces, as has been
shown in quantum computational applications. Unfortunately, it doesn’t seem to provide a satis-
factory method for introducing mixing in other compact closed categories such as the category of
sets and binary relations. We therefore lack a uniform strategy for extending a category to model
impreciselinguisticinformation.
In this work we adopt a differentapproach. We analyze differentforms of ambiguousand in-
completeinformation,bothwithandwithoutquantitativeprobabilisticdata. Eachschemethencor-
respondsto a suitableenrichmentofthe categoryin which we modellanguage. We view different
monadsasencapsulatingtheinformationalbehaviourofinterest,byanalogywiththeiruseinmod-
ellingsideeffectsincomputation.PreviousresultsofJacobsthenallowustosystematicallyconstruct
suitablebasesforenrichment.
Weshowthatwecanfreelyenricharbitrarydaggercompactclosedcategoriesinordertocapture
allthephenomenaofinterest,whilstretainingtheimportantdaggercompactclosedstructure. This
allowsustoconstructamodelwithrealconvexcombinationofbinaryrelationsthatmakesnon-trivial
useofthescalars. Finallywerelateourvariousdifferentenrichments,showingthatfinitesubconvex
algebraenrichmentcoversalltheeffectsunderconsideration.
1 Introduction
Thecategoricaldistributional approachtonaturallanguageprocessing[6]aimstoconstructthemeaning
of a sentence from its grammatical structure and the meanings of its parts. The grammatical structure
oflanguage canbedescribed usingpregroup grammars[20]. Thedistributional approach tonaturallan-
guagemodelsthemeaningsofwordasvectorsofstatisticsinfinitedimensionalrealvectorspaces. Both
the category of real vector spaces and linear maps, and pregroups are examples of monoidal categories
inwhichobjectshaveduals. Thiscommonstructure isthekeyobservation thatallowsustofunctorially
transfer grammaticalstructure tolinearmapsmodelling meaninginacompositional manner.
Recent work in categorical models of language has investigated ambiguity, for example words with
multiple meanings, [27,26,16]. Thesepapers exploitanalogies withquantum mechanics, usingdensity
matricestomodelpartialandambiguousinformation. Inordertodothis,thecategoryinwhichmeanings
areinterpreted mustchange. Selinger’s CPMconstruction [29]isexploited toconstruct compactclosed
categories in which ambiguity can be described. This construction takes a compact closed category C
and produces a new compact closed category CPM(C). When applied to the category of finite dimen-
sionalHilbertspacesandlinearmaps,FdHilb,theresultingcategoryCPM(FdHilb)isequivalenttothe
categoryoffinitedimensional Hilbertspacesandcompletelypositivemaps. Inthiswaytheconstruction
takesthesettingforpurestatequantummechanics andproduces exactlytherightsettingformixedstate
R.DuncanandC.Heunen(Eds.):QuantumPhysicsandLogic(QPL)2016
EPTCS236,2017,pp.95–107,doi:10.4204/EPTCS.236.7
96 MixingandAmbiguityinCategoricalModelsofLanguage
quantum mechanics. It is therefore tempting to consider the CPM construction as a mixing device for
compactclosedcategories. Thisperspectivewasadoptedin[27]bystartingwiththecategoryRelofsets
andbinaryrelationsandinterpreting CPM(Rel)asatoymodelofambiguity. Asarguedin[24,9],there
areaspectsofCPM(Rel)thatconflictwithitsinterpretation asasettingformixing. Specifically:
• Therearepurestatesthatcanbeformedasaconvexmixtureoftwomixedstates
• Thereareconvexcombinations ofdistinctpurestatesthatgivepurestates
ItcouldbearguedthatthisanomalousbehaviourisduetotherestrictednatureofthescalarsinRel. This
thenpointstoanotherweaknessofCPM(Rel)asasettingforambiguityandmixing. Wemaywishtosay
that the word bank is 90% likely tomean a financial institution and 10% likely to refer tothe boundary
of a river. We can express this in CPM(FdHilb) as the scalars are sufficiently rich. In CPM(Rel) the
bestwecandoissaythatit’seitheroneortheother, withallthequantitative information beinglost.
In this paper we investigate some alternative models of ambiguity in the compact closed setting.
We consider a variety of different interpretations of what it might mean to have ambiguous or limited
information, andhowthesecanbedescribed mathematically. Specifically:
• Insection4wedescribeconstructionsthatmodelambiguousandincompleteinformationinanon-
quantitativemanner. Weshowthatcompactclosedcategoriescanbefreelyextendedsoastoallow
themodelling ofincomplete information, ambiguityandamixtureofbothphenomena.
• Insection5weextendourconstructions todescribequantitativeambiguousandincompleteinfor-
mation. Again we show that compact closed categories can be freely extended in order to model
suchfeatures.
• Proposition 5.10showsthatthesevariousinformational notions embedintoeachother.
Ourgeneralperspectiveistoenrichthehomsetsofourcategoriesinordertomodelthelanguagefeatures
we are interested in. In order to do this systematically, we exploit some basic monad theory. Monads
arecommonlyusedtodescribecomputational effectssuchasnon-determinism, exceptions andcontinu-
ations [25,31]. Inourcaseweinstead viewthemasmodelsofinformational effectsinnatural language
applications. Monads have previously been used in models of natural language, see for example [30].
WorkofJacobs [11]provides theconnection betweenacertain classofmonadsandcategories thatpro-
vide good bases for enriched category theory. We aim to give explicit constructions of the categories
of concrete interest throughout, rather than pursuing a policy of maximum abstraction. Our systematic
approach meansweshouldbeabletoincorporate additional informational featuresinasimilarmanner.
Weassumefamiliarity withelementary category theory, andthenotionsofcompactclosed anddag-
gercompactclosedcategories [1].
2 Monads
We will now outline the necessary background on monads required in later sections, and introduce the
monadsthatwillbeofparticularinterest. Thematerialinthissectionisstandard,goodsourcesforfurther
background are [22,2,4, 2,14]. In this paper wewillonly beinterested in monads on thecategory Set
ofsets and total functions, although wewillstate somedefinitions moregenerally whereitiscleaner to
doso.
Definition2.1(Monad). AmonadonacategoryC istripleconsisting of:
• Anendofunctor T :C →C
D.Marsden 97
• Aunitnaturaltransformation h :1⇒T
• Amultiplicationnaturaltransformation m :TT ⇒T
suchthatthefollowingthreeaxiomshold:
m ◦(h ∗T)=T m ◦(T∗h )=T m ◦(m ∗T)=m ◦(T∗m )
Wenow introduce the monads of interest in this paper, and relate them to computational behaviour.
Similarly to [28], we emphasize that monads are induced by algebraic operations modelling computa-
tional, orinourcaseinformational, behaviour.
Definition2.2(LiftMonad). Theliftmonad((−) ,h ,m )isdefinedasfollows:
⊥
• Thefunctorcomponent isgivenbythecoproduct offunctors 1+{⊥}:Set→Set.
• Theunitandmultiplication aregivencomponentwise by:
xifx∈X
h (x)=x m (x)=
X X
(⊥otherwise
Theliftmonadiscommonlyusedtodescribe computations thatcandiverge.
Definition2.3(PowersetMonads). ThefinitepowersetmonadPw hasfunctorcomponentthecovariant
finitepowersetfunctor. Theunitsendsanelementtothecorresponding singleton, andthemultiplication
is given by taking unions. The non-empty finite powerset monad P+ arises in an analogous way by
w
restricting the sets under consideration. The finite powerset monad is used to model finitely bounded
non-determinism, andthenon-emptyfinitepowersetmonadeliminatesthepossibility ofdivergence.
Definition2.4(FiniteDistribution Monads). Thefinitedistributionmonadhasfunctorcomponent:
D:Set→Set
(cid:229)
X 7→{d :X →[0,1]|d hasfinitesupportand d(x)=1} (1)
x
f :X →Y 7→l dy. (cid:229) d(x)
x∈f−1{y}
Theunitandmultiplication aregivencomponentwise by:
h (x)=d m (d)(x)= (cid:229) d(e)e(x)
X x X
e∈supp(e)
whered istheDiracdeltafunction andsupp(e)isthesupport ofe.
x
The finitesubdistribution monad(S,h ,m ) has identical structure, except that weweaken the con-
ditioninequation(1)to:
(cid:229)
d(x)≤1
x
Soourfinitedistributionsarenowsub-normalizedratherthannormalizedto1. Boththefinitedistribution
andsubdistribution monadsareusedtomodelprobabilistic computations. Intuitivelythesubdistribution
monadprovides scopefordiverging behaviour inthe“missing”probability mass.
Remark2.5. Weadoptaconvenient notationalconvention from[13]andwritefinitedistributions asfor-
mal sums(cid:229) p|xi, where weabuse the physicists ket notation toindicate the sum is aformal construc-
i i i
tion. Usingthisnotation, theunitofthe(sub)distribution monadisthemapx7→|xiandmultiplication is
givenbyexpanding outsumsofsumsintheusualmanner.
98 MixingandAmbiguityinCategoricalModelsofLanguage
Eachmonadcanbecanonically relatedtoacertaincategoryofalgebras.
Definition 2.6 (Eilenberg-Moore Algebras). Let (T,h ,m ) be a monad on C. An Eilenberg-Moore
algebra[7]forT consists ofanobjectAandamorphisma:TA→Asatisfying thefollowingaxioms:
a◦h =1 a◦m =a◦Ta
A A
AmorphismofEilenberg-Moore algebrasoftype(A,a)→(B,b)isamorphism inC suchthat:
h◦a=b◦Th
Thecategory ofEilenberg-Moore algebras andtheirhomomorphisms willbedenotedEM(T).
Example2.7. Forthemonadsunderconsideration wenotethat:
• The Eilenberg-Moore category of the lift monad is equivalent to the category of pointed sets and
functions thatpreservethedistinguished element, denotedSet .
•
• The Eilenberg-Moore category of the finite powerset monad is equivalent to the category of join
semilattices andhomomorphisms, denoted JSLat.
• The Eilenberg-Moore category of the non-empty finite powerset monad is equivalent to the cate-
goryofaffinejoinsemilattices andhomomorphisms, denotedAJSLat.
• TheEilenberg-Moore category of thefinitedistribution monad isthe category ofconvex algebras
andfunctions commutingwithformingconvexcombinations, denotedConvex. Thiscategoryhas
receivedagreatdealofattention inforexample[12,15,8].
• The Eilenberg-Moore category of the finite subdistribution monad is the category of subconvex
algebras,thatisalgebrasthatcanformsubconvexcombinations ofelementsinacoherentmanner.
Themorphismsarefunctionsthatcommutewithformingsubconvexcombinations. Wedenotethis
categorySubconvex.
Weconsider commutativemonadsonthecategorySet,andspecialize theirdefinitionappropriately.
Definition2.8(CommutativeMonad). Let(T,h ,m )beaSetmonad. Thereareacanonicalstrengthand
costrength natural transformations:
st :X×TY →T(X×Y) st’ :TX×Y →T(X×Y)
X,Y X,Y
(x,t)7→T(l y.(x,y))(t) (t,y)7→T(l x.(x,y))(t)
Themonadissaidtobeacommutativemonad[19]ifthefollowingequationholdsforallX,Y:
m ◦T(st’ )◦st =m ◦T(st )◦st’
X×Y X,Y TX,Y X×Y X,Y X,TY
Thiscompositeisthencalledthedoublestrength,denoteddst.
Remark 2.9. Monads are intimately related to the topic of universal algebra. The Eilenberg-Moore
algebrasforaSetmonadcanbepresentedbyoperationsandequations,ifwepermitinfinitaryoperations.
All the monads in this paper are in fact finitary monads, meaning they can presented by operations of
finite arity. Let f and y be operations of arities m and n respectively. These operations are said to
commutewitheachotherifthefollowingequationholds:
y (f (x ,...,x ),...,f (x ,...,x ))=f (y (x ,...,x ),...,y (x ,...,x ))
1,1 1,m n,1 n,m 1,1 n,1 1,m n,m
Ifweunravel thedefinition ofcommutativemonad, itsaysthatalltheoperations inapresentation com-
mutewitheachother. Wecanalsophrasethisaseveryoperation beingahomomorphism. Moredetailed
discussion ofconnections touniversalalgebraandpresentations canbefoundin[23].
D.Marsden 99
Lemma 2.10. Each of the lift, powerset, finite powerset, finite non-empty power, distribution and sub-
distribution monadsarecommutative.
Remark2.11. Itisinteresting thatallthenotionsofpartialinformation andambiguityconsidered inthis
paper give rise to commutative monads. Possibly we could regard this as showing these informational
effectsareindependent oftheorderinwhichtheyarebuiltup?
Clearlymanymonadsarenotcommutative:
Example 2.12. The list monad has a functor component that sends a set to finite lists of its elements.
The unit maps an element to the corresponding single element list and the multiplication concatenates
lists of lists. The Eilenberg-Moore algebras of this monad are arbitrary, not necessarily commutative,
monoids. Unsurprisingly, thismonadisnotcommutative.
The following proposition captures the essential properties of Eilenberg-Moore categories of com-
mutativemonadsthatwewillneed,allinoneplace. Thekeysymmetricmonoidalclosedstructureisdue
toworkofJacobs[11],theotherproperties arewellknown.
Proposition 2.13. Let(T,h ,m )beacommutativemonadonSet. Thecategory EM(T):
• Isasymmetricmonoidal closedcategory.
• Hasuniversal bimorphismsforthemonoidaltensor.
• Hasmonoidalunitgivenbythefreealgebra({∗},!).
• Thetensorproduct m ⊗m isisomorphic to m .
X Y X×Y
• Iscomplete.
• Iscocomplete.
Proof. Completeness andcocompleteness ofcategories thataremonadicoverSetisstandard. Thecate-
gory Set is a SMCC via its products and exponentials. Set is complete so we can use [11, lemma 5.3].
Theadditional properties ofthemonoidalstructure comefrom[11,lemmas5.1,5.2].
Remark 2.14. Weavoid technical discussion ofuniversal bimorphisms, detailscanbefoundin[18,11].
The essential idea is to generalize the universal property of the tensor product of vector spaces, and
their relationship to bilinear functions. So in the set theoretic case, homomorphisms out of our tensors
will bijectively correspond to functions out of the cartesian product that are homomorphisms in each
component separately.
3 Enriched Categories
Anenrichedcategoryisacategoryinwhichthehomsetshaveadditionalstructurethatinteractswellwith
composition.
Example3.1. Thefollowingarenaturalexamplesofenriched categories:
• Asatrivialexample,ordinary locallysmallcategories areSet-enriched.
• A category is poset enriched if it’s homsets have a poset structure and composition in monotone
withrespecttothatstructure. Forexamplethecategory Relisposetenriched.
• Inthecategorical quantum mechanicscommunity, acategoryissaidtohaveasuperpositionrule
[10], or be a process theory with sums [5], if its homsets carry commutative monoid structure
thatissuitably compatiblewithcomposition (andpossibly additional structure).
100 MixingandAmbiguityinCategoricalModelsofLanguage
Wehaveinsufficient spaceforadetailed outline ofthepartsofenriched category theorywerequire,
werefer thereader to[17,4]for background. Theinformal discussion above should hopefully besuffi-
cienttounderstand thediscussions inlatersections.
The idea of this paper is that complete and cocomplete categories with symmetric monoidal closed
structureprovideaverygoodbaseofenrichmentforenrichedcategorytheory. Ifweselectacommutative
monadthatcaptures thelinguistic feature weareinterested in,wecanthenconsider categories enriched
inthecorresponding algebraic structure.
TheuniversalbimorphismpropertyofthemonoidalstructureofEilenberg-Moorecategoriesofcom-
mutativemonadsallowsusprovideconcrete conditions forenrichment inourcategories ofinterest:
Proposition 3.2. Acategory C:
• IsSet -enriched ifitshomsetshavepointedsetstructures suchthat:
•
⊥◦ f =⊥ and f ◦⊥=⊥ (2)
• IsAJSLat-enriched ifitshomsetshaveaffine joinsemilatticestructures suchthat:
(f ∨g)◦h=(f ◦h)∨(g◦h) and f ◦(g∨h)=(f ◦g)∨(f◦h) (3)
• IsJSLat-enrichedifitshomsetshavejoinsemilatticelatticestructuressuchthatboththeequations
of (2)and (3)hold.
• IsConvex-enriched ifitshomsetshaveconvexalgebra structures suchthat:
(cid:229) (cid:229) (cid:229) (cid:229)
( p f)◦g= p(f ◦g) and f ◦( pg)= p(f ◦g) (4)
i i i i i i i i
i i i i
• IsSubconvex-enrichedifitshomsetshavesubconvexalgebrastructuressuchthattheequations(4)
holdforallsubconvex combinations.
For a given cocomplete symmetric monoidal closed category V, we can form the free V-enriched
category over an ordinary category. This construction willbe exploited in the later sections, details can
befoundin[17,4].
4 Unquantified Mixing
We begin by considering probably the simplest case, in which we have incomplete information. For
example, I simply don’t know the meaning of the word “logolepsy”. In order to model this, we enrich
ourcategory inpointedsets,withthedistinguished elementdenotingmissinginformation.
Definition4.1. Forcategory C wedefinethecategoryC ashaving:
⊥
• Objects: ThesameobjectsasC
• Morphisms: WedefineC (A,B)=C(A,B)
⊥ ⊥
Identities areasinC. CompositionisgivenasinC,extendedwiththerules:
⊥◦ f =⊥ and g◦⊥=⊥
Proposition 4.2. Foracategory C,C isthefreepointedsetenrichedcategory overC.
⊥
D.Marsden 101
Theorem 4.3. If C is a compact closed category then C is a compact closed category. The monoidal
⊥
structure onmorphismsextends thatinC asin(5).
⊥if f =⊥or f′=⊥ ⊥if f =⊥
f ⊗ f′= (5) f†⊥ = (6)
⊥ (f ⊗ f′ otherwise (f† otherwise
Thereisanidentityandsurjective onobjectsstrictmonoidalembedding C →C .
⊥
If C is a dagger compact closed category then C is a dagger compact closed category with the
⊥
dagger extending thatofC asin(6).
Proof. Wesketch thebasic ideas. Wemustcheck that theextended definitions ofthe monoidal product
and if necessary the dagger are functorial. We then wish to inherit the associator, left and right unitor
and symmetry from the base. In order to do so we must check they remain natural with respect to the
extended functor actions onmorphisms. Cupsandcapsandotherstructure andaxiomsarethenbroadly
speaking inherited fromC.
Although C gives a new compact closed category, it is not particularly exciting. As soon as we
⊥
compose or tensor with a ⊥ element, the whole term becomes ⊥. This is consistent with the intuition
thatifwehavenoideaaboutpartoftheinformation werequire, wecannotknowthewholeeither.
Wetooktheopportunity tosketchtheproofthatC iscompactclosedasitiseasiesttofollowinthis
⊥
simplecase. Laterproofsofsimilarclaimsareanalogous.
Remark 4.4. Although we have inherited some good properties from the base category in the construc-
tion 4.1, clearly not everything can be preserved. We are expanding the morphisms between each pair
of objects, for example in Rel we now have three scalars, which we can interpret as true, false and
⊥
unknown. Thisexpansion willinterfere with(co)limits from the base category. Forexample ifwehave
azeroobjectinC,thatobjectwillnolongerbeazeroobjectinC astherewillbe2morphismsoftype
⊥
0→0.
Notingthesimilaritywiththebehaviourofthe⊥elementswiththatofzeromorphismsincategories
withzeroobjects, wenotethat:
Lemma4.5. Everycategory withazeroobjectisSet -enriched.
•
Although theliftmonadandSet -enrichment areextremelystraightforward, weshallreturntothem
•
later, ininteraction withdifferenttypesofambiguity.
Iftheprevious modelcaptured incomplete information, wenowmovetoconsider ambiguity. Inthis
caseweintendsituationswhereseveralthingsarepossible,forexampleabatiseitherawingedmammal
orsportingequipment. Inparticular, unlikethepreviousmodel,wehavecompleteinformationaboutthe
available possibilities, we are simply unaware of which one applies. If we don’t have any sense of the
relative likelihoods of the possibilities, we are just left with a non empty finite set of alternatives, and
thispointsusinthedirection ofthemonadP+.
w
Definition4.6. Forcategory C wedefinethecategoryC ashaving:
Pw+
• Objects: ThesameobjectsasC
• Morphisms: WedefineCPw+(A,B)=Pw+(C(A,B))
Wedefinecomposition asfollows:
V◦U ={v◦u|v∈V,u∈U}
Identities arethengivenbythesingletons containing theidentities fromC.
102 MixingandAmbiguityinCategoricalModelsofLanguage
Proposition 4.7. Foracategory C,C isthefreeaffinejoinsemilattice enrichedcategory overC.
Pw+
Theorem 4.8. IfC is a compact closed category then C is a compact closed category. The action of
Pw+
thetensoronmorphismsextends thatofC asin(7).
U⊗Pw+U′={u⊗u′|u∈U,u′∈U′} (7) U†Pw+ ={u† |u∈U} (8)
Thereisanidentityandsurjective onobjectsstrictmonoidalembedding C →C .
Pw+
If C is a dagger compact closed category then C is a dagger compact closed category with the
Pw+
dagger extending thatofC asin(8).
As a final possibility, what if we wish to consider both ambiguous and incomplete information? It
is then natural to consider the finite powerset monad as the source of our enrichment. The definition is
almostidentical tothatofC :
Pw+
Definition4.9. Forcategory C wedefinethecategoryC ashaving:
Pw
• Objects: ThesameobjectsasC
• Morphisms: WedefineCPw (A,B)=Pw (C(A,B))
Wedefinecomposition asfollows:
V◦U ={v◦u|v∈V,u∈U}
Identities arethengivenbythesingletons containing theidentities fromC.
Proposition 4.10. Foracategory C,C isthefreejoinsemilatticeenriched category overC.
Pw
Theorem4.11. IfC isacompactclosed category thenC isacompactclosed category. Theaction of
Pw
thetensoronmorphismsextends thatofC asin(9).
U⊗ U′={u⊗u′|u∈U,u′ ∈U′} (9) U†Pw ={u†|u∈U} (10)
Pw
Thereisanidentityandsurjective onobjectsstrictmonoidalembedding C →C .
Pw
If C is a dagger compact closed category then C is a dagger compact closed category with the
Pw
dagger extending thatofC asin(10).
Definition4.12. Weconsider twosub-classes ofmonad:
• A monad is said to be affine [21, 11] if the component of its unit at the terminal object is an
isomorphism.
• Amonadissaidtoberelevant[11]ifdst◦d =Td
Techniques forextracting theaffineandrelevantpartsofacommutativemonadcanbefoundin[11].
Remark 4.13. Againwecanthink about thenotion ofaffinemonad intermsofpresentations, aswedid
withcommutativity. Anoperation issaidtobeidempotent if:
y (x,x,...,x)=x
Analgebraic theory is affineifallits operations are idempotent. Inparticular this means the theory can
havenoconstants ornon-trivial unaryoperations. Itmakesintuitivesensethatdescriptions ofambiguity
should lead to affine algebraic theories. We cannot just conjure up elements out of thin air, and being
confused betweenxandxshould providethesameinformation asknowingxdirectly.
Wenownoteafundamental relationship betweenthethreemonadsconsidered inthissection.
Remark 4.14. As observed in [11], P+ and the lift monad are the affine and relevant parts of the finite
w
powersetmonad. Infactthefinitepowersetmonadcanbeconstructedfromthenon-emptyfinitepowerset
monad and the lift monad using adistributive law [3], and soin amathematical sense itis precisely the
description ofincompleteinformation combined withnon-quantitative ambiguity. Asimilarpatternwill
berepeatedinthenextsection.
D.Marsden 103
5 Quantified Mixing
We now move to the setting that has typically been considered in categorical models of mixing and
ambiguityuntilnow,probabilisticmixtures. Herewereturntothesituationwhereourstateofknowledge
isforexamplethattheword“bank”suggests with90%confidenceafinancialbankand10%confidence
ariverbank. Wenowhavequantitativeinformation, anditshouldbepossibletoencodethisinformation
inourhomsets.
Definition5.1. Forcategory C wedefinethecategoryC ashaving:
D
• Objects: ThesameobjectsasC
• Morphisms: WedefineC (A,B)=D(C(A,B))
D
Composition isgivenasfollows:
(cid:229) (cid:229) (cid:229)
q |g i◦ p|fi= pq |g ◦ fi
j j i i i j j i
j i i,j
Example 5.2. Describing mixing in CPM(Rel), as discussed in the introduction, was unsatisfactory
as we could not encode quantitative data about our state of knowledge. The category Rel encodes a
D
convex set of weights on its morphisms. For example the scalars in Rel correspond to the closed real
D
interval[0,1].
Proposition 5.3. Forcategory C,C isthefreeconvex algebraenrichedcategory overC.
D
Theorem 5.4. If C is a compact closed category then C is a compact closed category. The action of
D
themonoidalstructure onmorphismsextends thatofC asin(11).
(cid:229) (cid:229) (cid:229) ((cid:229) p|fi)†D =(cid:229) p|f†i (12)
p|fi⊗ q |g i= pq |f ⊗g i (11) i i i
i i D j j i j i j
i i
i j i,j
Thereisanidentityandsurjective onobjectsstrictmonoidalembedding C →C .
D
If C is a dagger compact closed category then C is a dagger compact closed category with the
D
dagger extending thatofC asin(12).
Proposition 5.5. Forthefinitesubdistribution monadwehave1:
• Thefinitedistribution monadistheaffinepartofthesubdistribution monad.
• Theliftmonadistherelevantpartofthesubdistribution monad.
• The finite subdistribution monad can be constructed using a distributive law combining the finite
distribution monadandtheliftmonad.
Remark 5.6. As we saw for unquantified ambiguity, quantified ambiguity is affine, so forming combi-
nations is idempotent. Intuitively, quantified confusion between x and itself is the same as knowing x.
Similarlytotheunquantified case,weseethatthesubdistribution monadisexactlytheresultofcombin-
ingquantified ambiguityandincomplete information.
Definition5.7. Forcategory C wedefinethecategoryC ashaving:
S
• Objects: ThesameobjectsasC
• Morphisms: WedefineC (A,B)=S(C(A,B))
S
1possiblytheseobservationsarewellknownorfolklore,butIamunawareofasuitablepriorreference
104 MixingandAmbiguityinCategoricalModelsofLanguage
Composition isgivenasfollows:
(cid:229) (cid:229) (cid:229)
q |g i◦ p|fi= pq |g ◦ fi
j j i i i j j i
j i i,j
Proposition 5.8. Forcategory C,C isthefreesubconvex algebraenrichedcategory overC.
S
Theorem5.9. IfC isacompactclosedcategorythenC isacompactclosedcategory. Theactionofthe
S
monoidalstructure onmorphismsextends thatofC asin(13).
(cid:229) (cid:229) (cid:229) ((cid:229) p|fi)†S =(cid:229) p|f†i (14)
p|fi⊗ q |g i= pq |f ⊗g i (13) i i i
i i S j j i j i j
i i
i j i,j
Thereisanidentityandsurjective onobjectsstrictmonoidalembedding C →C .
D
If C is a dagger compact closed category then C is a dagger compact closed category with the
S
dagger extending thatofC asin(14).
The various free models of ambiguity and incomplete information that we have constructed can be
embedded intoeachotherasfollows:
Proposition 5.10. Forcategory C thereareidentity andsurjective onobjectsembeddings:
E
C Pw+,Pw C E
Pw+ Pw ⊥,Pw
C
⊥
C C
D E S E⊥,S
D,S
where:
(cid:229) if f =⊥
0/ if f =⊥ E (f)= 0/
E (f)= ⊥,S
⊥,Pw ({f}otherwise (|fiotherwise
(cid:229) (cid:229)
E ( p f)= p f
E (U)=U D,S i i i i
Pw+,Pw i i
where(cid:229) denotestheemptyformalsum.
0/
Proposition 5.10 shows that C allows us to model non-quantitative ambiguity and partial infor-
Pw
mation. Analogously, the category C allows the description of quantitative incomplete and ambiguous
S
information. It seems natural to try and describe non-quantitative ambiguity using convex mixtures by
restrictingattentiontouniformdistributions. Assuch,onemightexpectverticalarrowsindiagram(5.10)
withembeddingsC →C andC →C . Thisturnsoutnottobefunctorial, essentiallyasaresultof:
Pw+ D Pw S
Counterexample 5.11. Uniform distributions are not closed under composition in the free Convex or
Subconvex enrichments. For uniform mixture e=0.5|fi+0.5|f ◦ fi, e◦e is not in general a uniform
mixtureoncewecollectterms.
ItseemsthebestwecandoistoobservethatunquantifiedphenomenagivegeneralConvex-enriched
andSubconvex-enriched categories, beyondthefreeones:
Theorem 5.12. For every category C, C is Convex-enriched, and C is Subconvex-enriched. In
Pw+ Pw
bothcasessumsaregivenbysupports:
(cid:229)
p|fi={f |i∈I}
i i i
i∈I