Extracting Drug-Drug Interaction from the Biomedical Literature Using a Stacked Generalization ...

12 Pages·2013·0.72 MB·English

Extracting Drug-Drug Interaction from the Biomedical
Literature Using a Stacked Generalization-Based
Approach
Linna He, Zhihao Yang*, Zhehuan Zhao, Hongfei Lin, Yanpeng Li
SchoolofComputerScienceandTechnology,DalianUniversityofTechnology,Dalian,China
Abstract
Drug-drug interaction (DDI) detection is particularly important for patient safety. However, the amount of biomedical
literatureregardingdruginteractionsisincreasingrapidly.Therefore,thereisaneedtodevelopaneffectiveapproachfor
the automatic extraction of DDI information from the biomedical literature. In this paper, we present a Stacked
Generalization-based approach for automatic DDI extraction. The approach combines the feature-based, graph and tree
kernels and, therefore, reduces the risk of missing important features. In addition, it introduces some domain knowledge
basedfeatures(thekeyword,semantictype,andDrugBankfeatures)intothefeature-basedkernel,whichcontributetothe
performance improvement. More specifically, the approach applies Stacked generalization to automatically learn the
weightsfromthetrainingdataandassignthemtothreeindividualkernelstoachieveamuchbetterperformancethaneach
individualkernel.Theexperimentalresultsshowthatourapproachcanachieveabetterperformanceof69.24%inF-score
compared with othersystems inthe DDI Extraction2011 challengetask.
Citation: He L, Yang Z, Zhao Z, Lin H, Li Y (2013) Extracting Drug-Drug Interaction from the Biomedical Literature Using a Stacked Generalization-Based
Approach.PLoSONE8(6):e65814.doi:10.1371/journal.pone.0065814
Editor:LuisM.Rocha,IndianaUniversity,UnitedStatesofAmerica
ReceivedJanuary1,2013;AcceptedApril30,2013;PublishedJune13,2013
Copyright:(cid:2)2013Heetal.Thisisanopen-accessarticledistributedunderthetermsoftheCreativeCommonsAttributionLicense,whichpermitsunrestricted
use,distribution,andreproductioninanymedium,providedtheoriginalauthorandsourcearecredited.
Funding: This work is supported by grants from the Natural Science Foundation of China (No.61070098 and 61272373, http://www.nsfc.gov.cn/Portal0/
default152.htm)andtheFundamentalResearchFundsfortheCentralUniversities(No.DUT13JB09,http://www.moe.edu.cn).Thefundershadnoroleinstudy
design,datacollectionandanalysis,decisiontopublish,orpreparationofthemanuscript.
CompetingInterests:Theauthorshavedeclaredthatnocompetinginterestsexist.
*E-mail:[email protected]
Introduction biomedical texts. Segura-Bedma etal. applied a linguistic rule-
basedapproachtoextractDDIs[7].Then,theyproposedanother
Adrug-druginteraction(DDI)occurswhenonedruginfluences approach called the shallow linguistic (SL) kernel [8] to extract
theleveloractivityofanother[1].Apatientmaytakeavarietyof
DDIs [1].
drugs at one time. However, one drug may influence the others,
In the DDI Extraction 2011 challenge task [9], more
and sometimes these influences result in side effects that are
approaches were proposed to extract DDIs from the biomedical
dangerous to patients. Therefore, DDI detection is important for
literature. Thomas etal. [10] used an approach called majority
patientsafety.Doctorsshouldnotprescribecombinationsofdrugs
voting ensembles (WBI-5) that consists of three methods, namely
that have side effects when taken together. Furthermore, DDI
the all-paths graph (APG) kernel [6], the shallow linguistic (SL)
detection is also important for pharmacists; if pharmacists are
kernelandMoara,whichisanimprovedsystemthatparticipated
awareoftheinteractionsthatmayoccurbetweendrugs,theycan
in the BioNLP’09 Event Extraction Challenge [11]. Among
list theseinteractionsinthespecificationssothatthepatients will
others, the APG kernel obtains the best performance with an F-
know whichdrugs cannot betaken together.
score of63.53%.WhenfurthercombinedwithSLandMoara,it
Asdrug-druginteractionsarefrequentlyreportedinjournalsof
obtains a performance of 65.74%for theF-score,ranking first in
clinical pharmacology and technical reports, the biomedical
the DDI Extraction 2011 challenge task. Moreover, Chowdhury
literature is the most effective source for the detection of DDIs
etal. [12] applied different machine learning techniques that
[1]. The development of Information extraction (IE) tools for
include a feature-based method and a kernel-based method
automatically extracting DDIs from the biomedical literature is
consisting of a mildly extended dependency tree (MEDT) kernel
important to reduce the time that professionals must spend
[13],aphrasestructuretree(PST)kernel[14],andaSLkernelto
reviewing the relevant literature. Meanwhile, it is essential for
extract DDIs. The union of the feature-based and kernel-based
improving andupdatingthedrug knowledge databases [1].
methodsobtainsaperformancewithanF-scoreof63.98%ranking
In fact, information extraction from the biomedical literature
second in the task. Bjo¨rne etal. proposed the Turku Event
has been a topic of intense investigation during recent years [2].
Extraction system to extract DDIs, and their result ranks fourth
For example, many kernel-based methods, such as subsequence
withanF-scoreof62.99%[15].What’smore,Minardetal.used
kernels [3], tree kernels [4], shortest path kernels [5], and graph
onlyafeature-basedkernelthatincludesmanytypesoffeaturesto
kernels [6], have been proposed and successfully used to extract
extractDDIs[16].Theirapproachintroducesthefeatureselection
protein-proteininteractions(PPIs).However,fewapproacheshave
according to the features’ F-measure improved interaction
been proposed to solve the problem of extracting DDIs from the
PLOSONE | www.plosone.org 1 June2013 | Volume 8 | Issue 6 | e65814
ExtractingDrugInteractionsfromBiomedicalText
Table1.Featuresgeneratedinourfeature-basedkernelfortheinstance‘‘Plasmaconcentrationsofdrug1aredecreasedwhen
administeredwith drug2 containingdrug0 ordrug0.’’
Feature Featurevalue
Abon n=1 left_area=Plasma,left_area=concentrations,left_area=of;Inner_area=are,Inner_area=decreased,…,Inner_area=with;
Right_area=containing,…,Right_area=drug0.
n=2 left_area=Plasmaconcentrations,left_area=concentrationsof;Inner_area=aredecreased,…,Inner_area=administeredwith;
Right_area=containingdrug0,…,Right_area=ordrug0.
n=3 left_area=Plasmaconcentrationsof;inner_area=aredecreasedwhen,…,inner_area=whenadministeredwith;Right_area=containing
drug0or;Right_area=drug0ordrug0.
San n=1 D1_left=Plasma,D1_left=concentrations,D1_left=of;D1_right=are,D1_right=decreased,…;D2_left=decreased,…;
D2_right=containing,…
n=2 D1_left=Plasmaconcentrations,D1_left=concentrationsof;D1_right=aredecreased,D1_right=decreasedwhen,…;
D2_left=decreasedwhen;D2_left=whenadministered,…;D2_right=containingdrug0…
n=3 D1_left=Plasmaconcentrationsof;D1_right=aredecreasedwhen,D1_right=decreasedwhenadministered;D2_left=decreasedwhen
administered,D2_left=whenadministeredwith;D2_right=containingdrug0or,D2_right=drug0ordrug0
Cpn n=1 D1_left‘D2_left‘distance=Plasma‘are‘5,D1_left‘D2_left‘distance=Plasma‘decreased‘5,D1_left‘D2_left‘distance=Plasma‘when‘5…;
D1_right‘D2_left‘distance=are‘are‘5…;D1_left‘D2_right‘distance=Plasma‘containing‘5…;
D1_right‘D2_right‘distance=are‘containing‘5…
n=2 D1_left‘D2_left‘distance=Plasmaconcentrations‘aredecreased‘5,…;
D1_right‘D2_left‘distance=aredecreased‘aredecreased‘5,…;D1_left‘D2_right‘distance=Plasmaconcentrations‘containing
drug0‘5,…;D1_right‘D2_right‘ditance=aredecreased‘containingdrug0‘5,…
n=3 D1_left‘D2_left‘distance=Plasmaconcentrationsof‘aredecreasedwhen‘5,…;D1_right‘D2_left‘distance=aredecreasedwhen‘are
decreasedwhen‘5,…;D1_left‘D2_right‘distance=Plasmaconcentrationsof‘containingdrug0or‘5,…;D1_right‘D2_right‘distance=are
decreasedwhen‘containingdrug0or‘5…
Negativeword no=-1,not=21,…
Keyword decrease=1,activate==21,…
Semantictype Semtype1=‘‘carb,phsu’’,semtype2=‘‘gngm’’
NameIsDrug entity1=21entity2=21
DrugBank indication=0,pharmacology=0,description=0
doi:10.1371/journal.pone.0065814.t001
detection.Asaresult,theirperformanceranksfifthwithanF-score distinctive types of kernels to extract DDIs: namely, the feature-
of59.65%.Onthewhole,theresearchofDDIextractionfromthe based, graph andtree kernels.
biomedicalliteratureisstillatanearlystage,anditsperformance
hasmuchroomtoimprove(intheDDIExtraction2011challenge Feature-based Kernel
task, thebest performance achieved is65.74%inF-score [10]). Before the features used in our feature-based kernel are
In this paper, we propose a Stacked generalization-based introduced, some notions of words, n-grams, areas and positions
approach[17]toextractDDIsfromthebiomedicalliterature.The in a sentence with regard to two interacting drugs (which are
approachintroducesStackedgeneralizationtoautomaticallylearn replaced by ‘‘drug1’’and‘‘drug2’’ inour experiments) aregiven.
the weights from the training data and assigns them to three
individual kernels, feature-based, tree and graph kernels, and 1.Vocabularies ofwords:VW={wordsintraining data}.
achieves much better performance than each individual kernel. 2.Vocabularies of word-level n-grams: VN={1–3 grams in
Theperformanceofourapproachissuperiortothoseof[10,12]. training data}
The key reasons are as follows: 1) In addition to the commonly
3.Assume that words in an DDI instance are indexed by the
used word features, our feature-based kernel introduces the
following sequence:
semantic type feature, the keyword feature and the DrugBank
(http://www.drugbank.ca/) features. The introduction of these Indices=(0,…,d1, …,d2, …,END).
features allows for the utilization of domain knowledge and Where d1 istheindex of thetoken ‘‘drug1’’, d2 istheindex of
improvestheperformanceeffectively(nearly1.2percentagepoint the token ‘‘drug2’’, and END is the index of the last token of the
in the F-score). 2) The features in individual kernels are sentence.
complementary, and their combination with Stacked generaliza-
tion canachieve betterperformance than each individual kernel. 41..General areas: GA={Left_Area, Inner_Area, Right_-
Area}={[0, d1–1], [d1+1, d2–1], [d2+1, END]} – text
Methods snippets split by ‘‘drug1’’ and ‘‘drug2’’ in each sentence
denoted by‘‘Left_Area drug1 Inner_Areadrug2Right_Area’’.
A kernel can be thought of as a similarity function for pairs of
52..Surrounding areas: SA={D1_Left, D1_Right, D2_Left,
objects. Different kernels calculate the similarities with different
D2_Right}={[d1–4, d1–1], [d1+1, d1+4], [d2–4, d2–1],
aspects between two sentences. Combining the similarities can
[d2+1,d2+4]}–textssurrounding‘‘drug1’’or‘‘drug2’’within
reducetheriskofmissingimportantfeaturesandthusproducesa
a 4-wordwindow.
new useful similarity measure. Our method combines several
PLOSONE | www.plosone.org 2 June2013 | Volume 8 | Issue 6 | e65814
ExtractingDrugInteractionsfromBiomedicalText
63..Conjunct positions: CP={ D1_direction ‘ D2_direction ‘ contains only the tokenized sentences, while the MMTx format
distance | direction [{Left, Right}, distance=discretized (d2- contains thetokenized sentencesalongwiththePOStag andthe
d1) [{0,1, 2, 3, 4, 5, (6,7), (8,10), (11,15), (16,20), semantic type for each token. An experienced pharmacist
(21,30), (31,40), (40,) } – conjunctions of partial elements reviewed the UMLS Semantic Network as well as the semantic
(twoareasadjacenteachdrug)andthediscretizedwordcount annotation provided by MMTx and recommended the inclusion
between thetwodrugs. of the following UMLS semantic types as possible types of
interacting drugs [9]: Clinical Drug (clnd), Pharmacological
Lexical features. Our lexical features take three unordered Substance (phsu), Antibiotic (antb), Biologically Active Substance
sets of words as the feature vectors (an example of the features (bacs), Chemical Viewed Structurally (chvs) and Amino Acid,
generated bythe feature-based kernel isgiveninTable 1): Peptide, or Protein (aapp). Because DDIs may exist between two
entitieswiththespecificsemantictypes,weintroducethesemantic
1.Area-bag-of-n-grams (Abon): features derived from VW6GA
type feature, whichisextracted fromtheMMTxformat dataset.
and VN6GA. The features of VW6GA, e.g., ‘‘left_area=-
DrugBank feature. DrugBank database is a rich resource
Plasma’’, ignore word positions in the current area, and the
combining chemicaland pharmaceutical information forapprox-
features of VN6GA, e.g., ‘‘left_area=Plasma concentrations’’, imately 4,900 pharmacological substances [18]. For each drug,
simply enrich the bag-of-words representation by bigrams DrugBank contains more than 100 data fields including drug
andtrigrams. synonyms, brand names, chemical formulas and structures, drug
2.Surrounding area-n-grams (San): features from VN6SA, e.g., categories, ATC and AHFS codes (i.e., codes of standard drug
‘‘D1_Right=increasethe’’.Theyareusedtohighlightn-grams families), mechanism ofaction, indication, dosage forms,toxicity,
in the ‘‘indicating areas’’ (i.e., D1_left, D1_right, D2_left and etc.
D2_right)because,intuitively,featuressurroundingcandidate Based on the idea that the information in the DrugBank fields
drug pairs are more indicative. may help classify the DDI instance, we introduce the DrugBank
3.Conjunction position n-grams (Cpn): features from VN6CP. features.ThreeDrugBankfieldsarefoundthroughexperimentsto
The feature set is the conjunctions of a subset of the be helpful for the performance improvement of DDI extraction,
surrounding position-n-gram features (the features from namely, Indication, Pharmacology and Description. For the two
VN6SP, which gives the information of specific distances drugs in one DDI instance, we construct their corresponding N-
from the n-grams in SA to focusing drugs, e.g., ‘‘-1_from_- dimensional feature vectors a and b of words using the three
D1=of’’)andthedistanceoftwodrugs,e.g.,‘‘D1_left‘D2_left‘ DrugBank fields: if a word wi(i=1,…,N) exists in the DrugBank
field of one drug, the corresponding vector element is set to 1;
distance=of‘with‘5’’.Itcansimultaneouslycapturethelexical
otherwise,0.Then,thesimilaritybetweenaandbiscalculatedby
informationaroundbothdrugs.Theonlyproblemisthatthey
theCosine similarity defined as follows:
maysuffer fromdatasparseness.
In addition to the commonly used lexical features, five extra N
feaNtuergeastaivreeinwtroordducfeedatinutroe.theInfeamtuorest-bacsaesdes,ketrhneel.existence of Similarity~ a:b ~ iP~1aibi ð1Þ
negative words in a DDI instance denotes that there is no DDaDDDDbDD sffiPffiNffiffiffiffiffiaffiffiffiffi2ffiffiffi|ffiffiffiffiffiffiPffiNffiffiffiffiffibffiffiffiffi2ffiffi
interaction between the two drugs. Therefore, we introduce the i i
i~1 i~1
negativewordfeature.Thenegativewordsusedinourexperiment
include ‘‘no’’, ‘‘not’’, ‘‘neither’’, ‘‘fail’’, ‘‘fail to’’, ‘‘fails to’’,
‘‘failed’’, ‘‘failed to’’ and ‘‘failure’’. If one of the negative words If the Cosine similarity of one DrugBank field of two drugs is
existsinaninstance,thefeaturevaluewillbesetto1;otherwise0. more than or equal to 0.4 (which is determined experimentally),
NameIsDrug feature. In both the training and test sets of thecorresponding feature isset to1;otherwise0.
theDDIExtraction2011challengetask,‘‘drug’’,‘‘drugs’’,‘‘Drug’’
and‘‘Drugs’’aresometimeslabeledasdrugnamesandsometimes Graph Kernel
not. There are approximately 520 such labeled instances in the A graph kernel calculates the similarity between two input
trainingset,asshowninthefollowingexample:‘‘Theinteractions graphs by comparing the relations between common vertices
between Betaseron and other drugs have not been fully (nodes).Theweightsoftherelationsarecalculatedusingallwalks
evaluated’’.Tosolvetheproblem,weintroducetheNameIsDrug (possiblepaths)betweeneachpairofvertices.Ourmethodfollows
feature. If only one of the two entities is referred to by the terms theall-pathsgraphkernelproposedbyAirolaetal.[6].Thekernel
‘‘drug’’ or ‘‘drugs’’ or ‘‘Drug’’ or ‘‘Drugs’’, we set the feature as representsthetargetpairusinggraphmatricesbasedontwosub-
‘‘entity1=1 entity2=21’’ or ‘‘entity1=21 entity2=1’’. Other- graphs, one representing the parse structure sub-graph, and the
wise,thefeatureissetas‘‘entity1=1entity2=1’’or‘‘entity1=21 other representing the linear order sub-graph (see Figure 1). The
entity2=21’’. first sub-graph represents the parse structure of a sentence and
Keyword feature. The existence of an interaction keyword includes word or link vertices. A word vertex contains its lemma
(the verb or their variants expressing drug interaction relation, and POS, while a link vertex contains its link. In addition, both
suchas‘‘activate,’’‘‘activation,’’‘‘counteract,’’‘‘inhibit,’’)between types of vertices contain their positions, which differentiate them
twodrugnamesoftenimpliestheexistenceoftheDDI.Thus,itis fromothervertices inFigure 1.Thesecondsub-graphrepresents
chosen as a feature in our feature-based kernel. To identify the the word sequence in the sentence, and each of its word vertices
keywordsintexts,wemanuallycreatedaninteractionkeywordlist contains its lemma, its relative position to the target pair and its
of 506 entries, which includes the interaction verbs and their POS.
variants. For the calculation, two types of matrices, specifically edge
Semantictypefeature. ThedatasetofDDIExtraction2011 matrix A and label matrix L, are used. We assume that V
challenge task is made available in two formats: the so-called representsthesetofverticesinthegraph,andLrepresentstheset
unified format and the MMTx format. The unified format of possible labels that vertices can have. We represent the graph
PLOSONE | www.plosone.org 3 June2013 | Volume 8 | Issue 6 | e65814
ExtractingDrugInteractionsfromBiomedicalText
Figure1.Graphrepresentationgeneratedfromanexamplesentence.Thecandidateinteractionpairismarkedas‘‘drug1’’and‘‘drug2’’,the
otherdrugsaremarkedas‘‘drug0’’.Theshortestpathbetweenthedrugsisshowninbold.Inthedependencybasedsubgraphallnodesinashortest
patharespecializedusingapost-tag(IP).Inthelinearordersubgraphpossibletagsare(B)efore,(M)iddle,and(A)fter.
doi:10.1371/journal.pone.0065814.g001
withanadjacentmatrixA[RDVD|DVD whoserowsandcolumnsare whereNiisthesetofnodesintreeTi,andwenotethatC(n1,n2)
indexedbythevertices,and½A(cid:2) containstheweightoftheedge canbecomputedinpolynomialtimeduetothefollowingrecursive
i,j
connecting v[V and v[V if such an edge exists, and it is 0 definition:
i j
otherwise.Theweightisapredefinedconstantwherebytheedges
(1) Ifthecontext-freeproductions-Context-FreeGrammarrules-
ontheshortestpathsareassignedaweightof0.9andotheredges
at n andn are different, C(n ,n )~0;
receive aweight of 0.3.Additionally, werepresent thelabels as a 1 2 1 2
label allocation matrix L[RDlD|DVD so that L ~1 if the j-th vertex (2) Ifbothn1 andn2 arePOStags,C(n1,n2)~l;otherwisegoto
i,j
has the i-th label, and L ~0 otherwise. Using the Neumann (3).
i,j
(3) Compute C(n ,n ) recursively as:
Series,a graph matrix G iscalculatedas: 1 2
?
G~LTXAnL~LT((I{A){1{I)L ð2Þ
n~1 C(n ,n )~lncP(n1)(1zC(ch(n ,i),ch(n ,i))):
1 2 1 2
i~1
This matrix sums up the weights of all the walks between any wherenc(n )isthenumberofchildrenofn inthetree.ch(n,i)is
1 1
pairofverticesso,asaresult,eachentryrepresentsthestrengthof thei-th childof noden andl isthedecayfactor.
the relation between a pair of vertices. Using two input graph Parse tree kernel. The parse tree encapsulates the relation
matrices G9 and G0, the graph kernel K(G9, G0) is the sum of the instance between two focused entities. Thus, we should know
productsofthecommonrelations’weights,givenbyEquation(3). which portion of a parse tree is important in the tree kernel
calculation.
K(G0,G00)~XXG0G00 ð3Þ A standard tree kernel is often defined on the minimum
ij ij complete tree (MCT) that contains both candidate entities in a
i~1 j~1
parsetree.TheMCTisthecompletesub-treerootedbythenode
of the nearest common ancestor of the two entities under
consideration. However, the MCT may introduce some random
Tree Kernel noisefeatures.Thus,weintroducedtheSPT,whichisthesmallest
commonsub-tree including thetwofocused entities.
Asaspecializedconvolutionkernel,thetreekernelusedinour
In some cases, the information contained in the SPT is not
work aims to capture structured information in terms of
sufficient to determine the relationship between two candidate
substructures. T and T represent two parse trees. The
1 2
entities. By analyzing the experimental data, we found that in
convolution tree kernel K(T ,T ) countsthenumber of common
1 2
thesecasestheSPTsusuallyhavefewerthanfourleafnodesand,
sub-trees as the syntactic structure similarity between two parse
therefore, include little information except the two entity names.
trees T andT [19]:
1 2
Therefore, we employ a simple heuristic rule to expand the SPT
span:weadoptSPTasourtreespan.Ifthenumberofleafnodes
X
K(T1,T2)~ C(n1,n2) ð4Þ in an SPT is smaller than four, the SPT is expanded to a higher
n1[N1,n2[N2 level,i.e.,theparentnodeoftherootnodeoftheoriginalSPTis
usedas thenewroot node.
PLOSONE | www.plosone.org 4 June2013 | Volume 8 | Issue 6 | e65814
ExtractingDrugInteractionsfromBiomedicalText
Dependency path tree kernel. Information of dependency combining multiple classifiers that have been learned for a
path tree is another type of tree structure information that is classification task. The performance of Stacked generalization is
provided by the parser dependency analysis output in our tree very competitive compared with arcing and bagging (which are
kernel.Fordependencybasedparserepresentations,adependency presented byBreiman[23,24])[25].
path is encoded as a flat tree as depicted as follows: (DEPEN- Stacked generalization is a layered architecture. The classifiers
DENCY (NSUBJ (interacts DRUG1)) (PREP (interacts with)) atthelayer-0(level-0)receivetheoriginaldataastheirinputs,and
(POBJ(withDRUG2)))correspondingtothesentence‘‘DRUG1 each classifier outputs a prediction for its own sub-problem.
interacts with DRUG2’’. Because a tree kernel measures the Successive layers receive the predictions of the layer immediately
similarity of the trees by counting the common subtrees, it is preceding it as an input, and finally a single classifier at the top
expected that the system finds effective subsequences of depen- leveloutputsthefinalprediction.Stackedgeneralizationattempts
dency paths. The dependency path tree is similar to SPT, which to minimize the generalization error by using the classifiers at
alsoneedsextensioninsomecases.Byanalyzingtheexperimental higher layers to learn the type of errors made by the classifiers
data, we determined that if the length of the dependency path immediately below. Here we show how stacking with two layers
between two focus drugs is shorter than three, it is extended. (level) works.
Meanwhile,iftwoedgesarepresenttotheleftofthefirstdrugin As shown in Figure 2, given a dataset
the whole dependency parse path, they will be included in the D ~f(x ,y ),n~1,:::,Ng, where x is a vector representing
level0 n n n
dependency path. Otherwise, the two edges to the right of the the attribute values of the nth instance and y is the class value.
n
seconddrugwillbeincludedinthedependencypath.Theoptimal Then, randomly divide D into J almost equal size parts
level0
extension threshold 3 is determined through 10-fold cross- D1,:::DJ and define D{j~Dlevel0{Dj, where j=1, …, J.Dj and
validationonthetrainingset.Theoriginaltrainingsetisrandomly D{j areusedasthetestandtrainingsetsforthejthfoldofaJ-fold
partitionedinto10equalsizesubsamples.Ofthe10subsamples,a cross-validation, respectively. There are T base classifiers, which
single subsample is retained as the validation data for testing the wecallthelevel-0generalizers.Thetthbaseclassifieristrainedusing
model,andtheremaining9subsamplesareusedastrainingdata. instances of the training data set D{j to output the hypothesis
The cross-validation process isthenrepeated 10times,with each H({j,t), for t=1,…, T. Theseare called the level-0 hypotheses. For
ofthe10subsamplesusedexactlyonceasthevalidationdata.The level0
10 results from the folds then are averaged to produce a single eachinstancexn inDj,thetestsetofthejthcross-validationfold,
estimation. let p denote the prediction of the hypothesis H({j,t) on x . By
tn level0 n
processingthewholeJ-foldcross-validation,thelevel-1trainingset
Combination of Kernels using Stacked Generalization D ~f(p ,:::,p ),n~1,:::Ngisassembledfromtheoutputsof
level1 1n Tn
Each kernel has its own pros and cons. The parse tree kernel theThypotheses.Thenaclassifierthatwecallthelevel-1generalizer
does not output certain shallow relations, and conversely, the isusedtoderiveahypothesisH fromthelevel-1trainingdata
level1
dependency path kernel ignores some deep information. The D . Figure 2 illustrates the training process. To complete the
level1
feature-based kernel cannot capture the sentence structure. The
graphkernelcantreattheparser’soutputandwordfeaturesatthe
sametime,butitmaymisssomedistantwordsandsimilaritiesof
paths among more than threeelements [20].
Each kernel calculates the similarity between two sentences
fromdifferentaspects.Thuscombiningthesimilaritiescanreduce
theriskofmissingimportantfeatures.Torealizethecombination
ofdifferenttypesofkernelsbasedondifferentparsestructures,the
normalized output of several kernels K can bedefined as:
m
M
X
K(x,x’)~ s K (x,x’) ð5Þ
m m
m~1
M
X
s ~1,s §0,Vm ð6Þ
m m
m~1
where M represents the number of kernel types, and s is the
m
weight of each K . In both PPI and DDI tasks, most methods of
m
individual kernel combination assign the same weight to each
individual kernel [12,20,21], and sometimes their combined
kernels fail to achieve the best performance [20,21]. The reason
isthatthateachindividualkernelhasadifferentperformance,and
only when the kernels with better performances are assigned
higherweightscanthecombinationofindividualkernelsproduce
thebestresult[22].However,themanualselectionofappropriate
weight foreach kernel isa time-consuming andimperfect art.
Figure2.ThetrainingprocessofStackedgeneralization.TheJ-
In our method, Stacked generalization [17] is introduced to foldcross-validationprocessinlevel-0;andthelevel-1datasetDlevel1
automatically learn the weights assigned to the individual kernels attheendofthisprocessisusedtoproducethelevel-1hypothesisH.
fromthetrainingdata. Stacked generalizationisan approachfor doi:10.1371/journal.pone.0065814.g002
PLOSONE | www.plosone.org 5 June2013 | Volume 8 | Issue 6 | e65814
ExtractingDrugInteractionsfromBiomedicalText
training process, the final level-0 hypotheses Ht ,t~1,...,T, demonstratedin[25]tobemoresuitableforthelevel-1generalizer
level0
are derivedusing allthedata inD . than C4.5 (a decision tree learning algorithm), NB (a reimple-
level0
Now let us consider the classification process, which uses the mentation of a Naive Bayesian classifier) and IB1 (a variant of a
hypotheses Ht ,t~1,...,T, in conjunction with H . When lazy learning algorithm that employs the p-nearest-neighbor
level0 level1
presented with a new instance, it is first classified using Ht . method using a modified value-difference metric for nominal
level0 andbinary attributes).
Thus an input vector (p ,:::,p ) is generated and then classified
1 T
ThefoundationsofSVMhavebeendevelopedbyVapnik[27]
using H , which outputs a prediction for the instance. The
level1
and are gaining popularity due to their many attractive features
algorithm of thestackingprocess isshown inFigure 3.
andpromisingempiricalperformance.SVMsarebinaryclassifiers
In our method, the feature-based, graph and tree kernels are
for a set of training data (x,y), i~1,:::,n,x[RN,y[fz1,{1g
used asthelevel-0 generalizers. As forthelevel-1 generalizer, we i i i i
wherex isafeaturevectorofthejthtrainingsample,andy isthe
compared the effects of three different learning algorithms: j j
class label associated with the jth training sample. The decision
Multiple Linear Regression (MLR) [26], Support Vector Ma-
functionisdefined by
chines(SVM)[27]andRanking-SVM[28].MLRisanadaptation
ofaleast-squareslinearregressionalgorithmthatBreimanusedin
!
rveaglureesdsioanttrsiebtutitnegssc[a2n9].beAntyranclsafossrimficeadtioinntoproablemmultwi-irtehsproenasle- y(x)~sgn Xyjajw(xj):w(x)zb ð7Þ
regression problem. If the original classification problem has I j[SV
classes, it is converted into I separate regression problems, where wherewisanonlinearmappingfunctionfromRNtoRH(N,,H),
the problem for class l has instances with responses equal to one a,b[R,a§0,andSVisasetofsupportexamples.Themapping
j j
when they have class l and zero otherwise. MLR was
functionwshouldbedesignedsuchthatalltrainingexamplesare
Figure3.Thealgorithmofthestackingprocess.
doi:10.1371/journal.pone.0065814.g003
PLOSONE | www.plosone.org 6 June2013 | Volume 8 | Issue 6 | e65814
ExtractingDrugInteractionsfromBiomedicalText
Table2. Performancesofdifferent featuresinthe feature-based kernel.
Feature Precision Recall F-score MCC AUC
Lexicalfeatures Abon 57.40 70.86 63.43 58.92 90.7
San 50.05 66.36 57.06 51.73 89.8
Cpn 54.61 47.81 50.99 45.65 86.0
Abon+San 57.20 73.64 64.39 60.11 91.2
Abon+San+Cpn 58.49 72.98 64.94 60.67 91.8
Lexicalfeatures+Negativeword 60.80 70.46 65.28 60.96 93.0
Lexicalfeatures+Negativeword+NameIsDrug 59.40 72.58 65.44 61.06 92.1
Lexicalfeatures+Negativeword+NameIsDrug+Keyword 59.09 73.64 65.57 61.38 92.8
Lexicalfeatures+Negativeword+NameIsDrug+Keyword+Semantictype 60.88 71.52 65.77 61.63 92.7
Lexicalfeatures+Negativeword+NameIsDrug+Keyword+Semantictype+Indication 59.50 73.51 65.91 61.76 92.0
Lexicalfeatures+Negativeword+NameIsDrug+Keyword+Semantic 59.87 74.70 66.47 62.42 92.2
type+Indication+Pharmacology
Lexicalfeatures+Negativeword+NameIsDrug+Keyword+Semantic 60.30 74.44 66.63 62.58 92.2
type+Indication+Pharmacology+Description
doi:10.1371/journal.pone.0065814.t002
linearly separable in RH space. SVMs take a maximal margin M+NtrainingsamplesforSVM,whilethereareM*Nsamplesfor
strategy in that the parameters are chosen so that the minimum theRankingSVMbecauseitssamplesarebasedonexamplepairs.
distance between examples and the separating hyperplane (i.e., Moresamples mayachieve betterperformance.
margin) ismaximized.
Ranking SVM uses SVM for the task of pairwise classification Results and Discussion
[28]. In essence, Ranking SVM is the classification of the
differences of positive and negative document pairs’ feature Experimental Settings
vectors. Compared with the regular classification SVM, Ranking Our approach was evaluated on the DDI Extraction 2011
SVMtakestherankinginformationintoaccount,whichishelpful challenge task dataset [9] and, therefore, the results can be
to enhance the classification performance. In our Stacked comparedwiththoseoftheparticipatingsystems.Itconsistsof579
Generalization-based approach, Ranking SVM uses the output documents describing DDIs. These documents are randomly
scoresofthelevel-0generalizers(thefeature-based,graphandtree selectedfromtheDrugBankdatabaseandanalyzedbytheUMLS
kernels)asitsfeatures.Thedifferenceofthepositiveandnegative MetaMap Transfer (MMTx) tool. A total of 2402 positive
instances’scoresoutputtedbythelevel-0generalizersisusedasthe instances and 21425 negative instances are identified in the
ranking information for Ranking SVM. In addition, when there training dataset (435 documents). For the test dataset (144
areMpositiveexamplesandNnegativeexamples,thereareonly
Table3. Performancesofeach individual kerneland theircombinations.
Method Precision Recall F-score MCC AUC
Feature-basedkernel 60.30 74.44 66.63 62.58 92.2
Treekernel 48.06 59.21 53.06 49.94 84.6
Graphkernel 62.84 61.59 62.21 57.71 90.3
Feature-basedkernel(0.5)+Graph(0.5) 63.86 73.25 68.23 64.30 92.7
Feature-basedkernel(0.33)+Graph(0.33)+Tree(0.33) 65.05 71.26 68.01 64.05 92.7
Feature-basedkernel(0.45)+Graph(0.4)+Tree(0.15) 66.79 72.19 69.38 65.60 92.9
SVMasthelevel-1classifier 62.21 70.86 68.46 62.06 92.8
MLRasthelevel-1classifier 66.17 71.25 68.62 64.74 92.7
5–5RankingSVMasthelevel-1classifier 67.18 70.99 69.03 65.21 92.7
6–4RankingSVMasthelevel-1classifier 67.43 70.46 68.91 65.10 92.8
7–3RankingSVMasthelevel-1classifier 66.22 72.45 69.20 65.38 92.9
8–2RankingSVMasthelevel-1classifier 70.46 67.55 69.06 65.33 92.9
9–1RankingSVMasthelevel-1classifier 70.39 67.68 69.01 65.37 92.9
RankingSVMasthelevel-1classifier 66.18 72.58 69.24 65.43 92.9
Theweightsofeachindividualkernelincombinedkernelsareintheparenthesesafterthekernelname.
doi:10.1371/journal.pone.0065814.t003
PLOSONE | www.plosone.org 7 June2013 | Volume 8 | Issue 6 | e65814
ExtractingDrugInteractionsfromBiomedicalText
Results
Table4.Performancecomparisonwithothermethodsonthe
DDIExtraction 2011challenge task dataset. In this section, we discuss the effectiveness of different features
used in the feature-based kernel, the effectiveness of different
kernels, and the comparison of our results with those of earlier
Methods Precision Recall F-score MCC AUC works.
Effectiveness of different features in the feature-based
Thomasetal.[10] 60.54 71.92 65.74 61.50 –
kernel. The classification performances of different features in
Chowdhuryetal.[12] 58.59 70.46 63.98 58.25 – thefeature-basedkernelisshowninTable2.Amongotherlexical
Ourcombinedkernel-1 65.05 71.26 68.01 64.05 92.7 features, the Abon feature achieves the best performance (an F-
Ourcombinedkernel-2 66.18 72.58 69.24 65.43 92.9 score of 63.43%). When it is combined with the San and Cpn
features, the F-scores are improved by approximately 1 and 1.5
doi:10.1371/journal.pone.0065814.t004 percentage units, respectively, and an F-score of 64.94% is
achieved. As introduced in Section Methods, our feature-based
documents), there are 755 positive instances and 6271 negative kernelusesfiveothertypesoffeaturesbesidesthelexicalfeatures.
instances. Experimental result shows that, when these five features are
In our implementation,weusedtheSVMlight package(http:// introduced,theoverallperformanceisimprovedby1.7percentage
svmlight.joachims.org/) developed by Joachims for our feature- units in F-score (from 64.94% to 66.63%). Among others, the
basedkernel.Thelinearkernelisusedwithaparametercof0.006 keyword, semantic type, and Drugbank features employ domain
obtainedfromthe10-foldcross-validationonthetrainingdataset, knowledge and contribute to a performance improvement of 1.2
andtheotherparametersaredefault.Asforthegraphkernel,the percentage units inF-score (from 65.44%to66.63%).
all-paths graph kernel proposed by Airola etal. (http://mars.cs. Effectiveness of different kernels. The performance of
utu.fi/PPICorpora/GraphKernel.html) is used. For the tree differentkernelsisshowninTable3.Thefeature-basedkernelhas
kernel, we chose Tree Kernel Toolkit developed by Moschitti superior performance (66.63% in F-score) compared with the
with the default parameters (http://dit.unitn.it/˜moschitt/Tree- other two kernels. The key reason is that, in the feature-based
Kernel.htm). kernel, in addition to the commonly used word features, the
The existing DDI extraction system evaluations use the keyword,semantictype,andDrugBankfeaturesareintroducedto
balanced F-score measure for quantifying the performance of improve the performance. The introduction of these features is a
the systems [9]. This metric is defined as F-score=(2PR)/(P+R) wayofemployingdomainknowledgeandisshowntoimprovethe
wherePdenotestheprecisionandRdenotestherecall.Therefore, performance effectively. The performance (62.21% in F-score) of
theF-scoreisalsousedinourworktomeasuretheperformanceso the graph kernel ranks second because it considers the parser’s
that it can be compared with those of other existing DDI outputandwordfeaturestogether.Finally,theperformanceofthe
extraction systems. In addition, we provide our performance in tree kernel ranks thelowest (53.06%inF-score).
termsofMCC(MatthewsCorrelationCoefficient)andAUC(Area In addition, the experimental results show that when the
Under rocCurve). feature-based kernel is combined with the graph kernel, the
Table5. ExamplesofDDI instances. Thefocused entitiesof eachpair aretypeset inbold.
Corpus’s
Instances Ourresult annotation
P1 AmiodaronemaysuppresscertainCYP450enzymes,includingCYP1A2,CYP2C9,CYP2D6,andCYP3A4. negative positive
P2 Amiodaronemaysuppresscertaincyp450enzymes,includingCYP1A2,CYP2C9,CYP2D6,andCYP3A4. negative negative
P3 Amiodaronemaysuppresscertaincyp450enzymes,includingCYP1A2,CYP2C9,CYP2D6,andCYP3A4. negative positive
P4 Amiodaronemaysuppresscertaincyp450enzymes,includingCYP1A2,CYP2C9,CYP2D6,andCYP3A4. negative negative
P5 Althoughnotstudiedwithalosetron,inhibitionofN-acetyltransferasemayhaveclinicallyrelevant negative positive
consequencesfordrugssuchasisoniazid,procainamide,andhydralazine.
P6 Intestinaladsorbents(e.g.,charcoal)anddigestiveenzymepreparationscontainingcarbohydrate-splitting negative positive
enzymes(e.g.,amylase,pancreatin)mayreducetheeffectofAcarboseandshouldnotbetakenconcomitantly.
P7 Whenadministeredconcurrently,thefollowingdrugsmayinteractwithbeta-adrenergicreceptorblocking negative positive
agents:Anesthetics,general:exaggerationofthehypotensioninducedbygeneralanesthetics.
P8 Aswithsomeothernondepolarizingneuromuscularblockingagents,thetimeofonsetofneuromuscular negative positive
blockinducedbyNUROMAXislengthenedandthedurationofblockisshortenedinpatientsreceiving
phenytoinorcarbamazepine.
P9 Quinolones,includingcinoxacin,mayenhancetheeffectsoforalanticoagulants,suchaswarfarinorits positive negative
derivatives.
P10 Therefore,cautionshouldbeusedwhenadministeringCYP3A4inhibitorswithIRESSA. positive negative
P11 Althoughnodrug-druginteractionstudieshavebeenconductedinvivo,itisexpectedthatnosignificant positive negative
interactionwouldoccurwhennitazoxanideisco-administeredwithdrugsthateitheraremetabolizedby
orinhibitcytochromeP450enzymes.
P12 ItisrecommendedthatifCASODEXisstartedinpatientsalreadyreceivingcoumarinanticoagulants, positive negative
prothrombintimesshouldbecloselymonitoredandadjustmentoftheanticoagulantdosemaybenecessary.
doi:10.1371/journal.pone.0065814.t005
PLOSONE | www.plosone.org 8 June2013 | Volume 8 | Issue 6 | e65814
ExtractingDrugInteractionsfromBiomedicalText
Table6. Analysisofthe false negatives.
Errorcause Errornumber Errorproportion(%) Example
Annotationconsistency 71 35.5 P1,P3
‘‘Drugs’’annotationerror 15 7.5 P5
Negativeworderror 12 6 P6
DDIextractionerror FailuretoextracttheDDI 37 18.5 P7
UnobviousDDI 65 32.5 P8
Totals 200 100
doi:10.1371/journal.pone.0065814.t006
performanceisimprovedby1.6percentageunitsinF-score(from The feature-based, tree and graph kernels are trained on M
66.63%to68.23%).Whenfurthercombinedwiththetreekernel, (M,N) datasets (called the training set), respectively. Then, their
if the kernels are assigned the same weight, the overall test results on other L (L,N and L+M=N) datasets (called the
performancewilldecreasefrom68.23%to68.01%;ifthekernels validationset)areusedasthetrainingdatafortheRankingSVM
areassignedtheoptimalweights(theweightsaretunedonthetest model. In the Stacked generalization presented in this paper, the
set and the performance achieved with them can be regarded as level-1 training set for Ranking SVM is assembled by processing
the best performance that could be achieved on the test set), the the whole N-fold cross-validation, and its size is N. With more
overall performance willimprove from68.23%to69.38%. samples, it is reasonable that Ranking SVM in Stacked
Theexperimentalresultsdemonstratethat,ontheonehand,as generalization achievesa better performance.
discussedinSectionMethods,becausethedifferentkernelscalculate To further verify the idea, the experiments similar to those in
the similarity between two sentences from different aspects, the [30] are conducted on the DDI Extraction 2011 corpus. The
combinationofkernelscoversmoreknowledgeandiseffectivefor trainingsetofDDIExtraction2011isdividedintothetrainingset
DDI extraction; on the other hand, if combined with the same and validation set of different sizes (5–5, 6–4, 7–3, 8–2, 9–1,
weight, the performance of the combined kernel will deteriorate respectively). Take 5–5 for an example. The training set of DDI
owing to the introduction of an individual kernel with poor Extraction 2011 is divided into 10 datasets of equal size. Five
performance. datasetsareusedasthetrainingset,andtheotherfiveareusedas
The performances of MLR, SVM and Ranking-SVM as the the validation set. The feature-based, tree and graph kernels are
level-1 generalizer of Stacked generalization are also shown in trainedonthetrainingset,respectively.Thentheirtestresultson
Table3.BothSVMandRanking-SVMusethelinearkernelwith the validation set are used as the training data for the level-1
theparameterscof0.1and10,respectively.Theperformancesof generalizer Ranking SVM model. The trained Ranking SVM
MLR and SVM are almost the same (68.62% to 68.46% in F- model is then tested on the test set of the DDI Extraction 2011.
score), while that of Ranking-SVM is better (69.24%) and very The experimental results are also shown in Table 3. The
close to the optimal performance (69.38%) achieved with the performances of 5–5, 6–4, 7–3, 8–2, 9–1 Ranking SVM models
optimal weights tuned on the test set. The performance of the range from 68.91% to 69.20% in F-score, which are inferior to
stackedgeneralizationisalsothebestintermsofMCCandAUC. that of Ranking SVM in the Stacked generalization presented in
Thereasonmaybethat,asdiscussedinSectionMethods,Ranking thispaper(69.24%).Whatismore,inthemethodof[30],thesizes
SVMtakestherankinginformationintoaccount,whichishelpful of the training set and validation set need to be tuned to achieve
to enhance the classification performance. In addition, for given the optimal performance while it is not needed in the method of
training samples, Ranking SVM can have more samples because Stacked generalization.
its samples are based on example pairs, and more samples may PerformancecomparisonwithothermethodsintheDDI
achieve betterperformance. Extraction2011challengetask. AsmentionedintheSection
In [30], we have proved the effectiveness of Ranking SVM in Introduction, several kernels have been proposed for information
combining theoutputs of individual kernels inthePPI extraction extraction from the biomedical literature, including the tree
task. However,theexperimentalmethodin[30]isdifferent from kernels, shortest path kernels, and graph kernels. Each kernel
theonepresentedinthispaper.In[30],totrainaRankingSVM utilizesaportionofthestructurestocalculatetheusefulsimilarity,
model,thewholecorpusispartitionedintoNdatasetsofequalsize. whereby any one kernel cannot retrieve the other important
Table7. Analysisofthe false positives.
Errorcause Errornumber Errorproportion(%) Example
Generaldrugnameerror 48 24 P9
Non-drugnameannotationerror 26 13 P10
‘‘Drugs’’annotationerror 12 6 P11
DDIextractionerror 114 57 P12
Totals 200 100
doi:10.1371/journal.pone.0065814.t007
PLOSONE | www.plosone.org 9 June2013 | Volume 8 | Issue 6 | e65814
ExtractingDrugInteractionsfromBiomedicalText
information that may be retrieved by other kernels. The (69.24%inF-score),inwhichStackedgeneralizationisintroduced
combination of multiple kernels can gather more information to automatically learn the weights assigned to the individual
andcoversomeoflosses.Therefore,inrecentyears,studieshave kernelsfromthetrainingdata,anditsperformanceisverycloseto
proposedtheuseofmultiplekernelstoretrievethewidestrangeof the optimal performance (69.38%) achieved with the weights
important informationin agivensentence [31,32]. tuned on thetest set.
Similarly, in the DDI Extraction 2011 challenge task, the best Inaddition,intheDDIExtraction2011challengetask,Bjo¨rne
performing methods are also multiple kernel-based ones [10,12] etal. [15], Minard etal. [16] and Garcia-Blasco etal. [36] only
(theresultsaresummarizedinTable4).Thomasetal.obtainedan used the feature-based method. Bjo¨rne etal. applied their open
F-scoreof65.74%usingthecombinationofAPG,SLandmoara source Turku Event Extraction System, which was the best
[10]. Their APG is the same as ours. However, its performance performing system in the popular BioNLP 2009 Shared Task.
(63.53%F-score)ontheDDIExtraction2011testsetisbetterthan They adapted theTurku System totheDDI taskby extending it
that of our APG (62.21% F-score). The reason may be that the with a new example builder module, which converts the DDI
preprocessing methods are different: Thomas etal. used the corpus into machine learning classification examples, taking into
Charniak-Lease parser with a self-trained re-ranking model account information specific for drug-drug interactions. The
augmented for biomedical texts to parse each sentence, whereas features they used are tokens, dependency n-grams built from
weusedtheStanfordparser.TheSLkernelisdefinedasthesum the shortest path of dependencies, path terminal token attributes
of two kernels, the global context (GC) kernel (which is based on andsentencewordcount.What’smore,theymarkedasafeature
thewordsoccurringinthesentencefore-between,betweenandbetween- for each candidate pair whether it is present in DrugBank and
afterrelativetothepairfocusedentities)andthelocalcontext(LC) whetheritisthereasaknowninteractingpair;theyalsousedthe
kernel (which uses the surface (capitalization, punctuation, information from the MMTx format dataset. In our method, we
numerals) and shallow linguistic (POS-tag, lemma) features alsousedtheinformationfromDrugbankandtheMMTxformat
generated from tokens to the left and right of the entities of the dataset,buttheinformationweusedismuchricherthantheirs,as
entity pair) [31]. Actually, the SL kernel is similar to the feature- described in Section Methods. The experimental results show that
based kernel used in our method. However, the results of our our feature-based kernel outperforms theirs (62.99% in F-score).
feature-based kernel (66.63% in F-score) outperforms that of SL The reason may be that they used so many features that some
(60.05% in F-score). The reason is that our feature-based kernel noisewasintroduced.TheshallowsyntacticfeaturessuchasPOS
introduces some domain knowledge based features, such as the added to a lexical feature set are reported not to increase the
keyword, semantic type, and DrugBank features, which prove to performanceoftheclassifier,whilethedeepplusshallowsyntactic-
beeffectiveinimprovingtheDDIextractionperformance.What’s and lexical-feature based classifier showed a poor performance
more, the shallow syntactic features, such as POS, added to a when the set of lexical features was limited [37]. Both Minard
lexicalfeaturesetarereportednottoincreasetheperformanceof etal. and Garcia-Blasco etal. used the lexical features. Minard
the classifier in [32,33]. Moara uses case-based reasoning (CBR) etal.usedalltheaccessibleinformationofthecorpusasfeatures,
forclassifyingthedrugpairs.CBRisamachinelearningapproach and they used the technology of information gain to select 1010
thatrepresentsdatawithasetoffeatures[34].Ourfeature-based important features. Their method achieves a performance of
kernel(66.63%inF-score)outperformsthatofMoara(44.4%inF- 59.65% in F-score. Garcia-Blasco etal. used both the lexical
score) because it introduces richer features. Chowdhury etal. features and the domain knowledge, such as keywords derived
obtained an F-score of 63.98% by combining the feature-based from the training corpus, drug semantic types, classes and drug
method and the kernel-based method consisting of the MEDT, IDs derived from the MMTx format dataset. Their method
PST, and SL kernels [12]. The feature-based method (FBM) achieves a performance of 58.29% in F-score. Our feature-based
contains two types of features: lexical features (Word features, kerneloutperformstheabovethreemethodsbecauseitintegrates
Morphosyntactic feature, etc.) and advanced features (Trigger both the lexical features and domain knowledge based features,
word andNegation). Thesetwotypes offeatures arealsousedin suchas thekeyword, semantictype, andDrugBank features.
our feature-based kernel, and, in addition, we added other
features, including domain knowledge features, which is the Error Analysis
reason that the performance of FBM is inferior to ours. In their Confined to the complexity of natural language, extracting
kernel-based method, MEDT is a version of the expanded DT DDIs from the biomedical literature remains a challenging task,
kernel. DT is a dependency tree that is a representation that and it is difficult to achieve a satisfactory performance. We
denotes the grammatical relations between words in a sentence manuallyanalyzed200positiveinstancesfromthetestdatasetthat
[35].Thedependencypathtreekernelusedinourpaperalsouses wereclassifiedasnegativeinstancesbyourclassificationmodel(i.e.
theinformationderivedfromDTand,insomecases,itisextended false negatives. Some examples are given in Table 5). A detailed
toincludemoreinformationasdescribedinSectionMethods.Their analysis of all types of false negative errors is shown in Table 6.
PST kernel is similar to the SPT kernel in our method, which is Firstly,thereistheproblemofthecorpusannotationconsistency.
basedonthesmallestcommonsubtreeofaphrasestructureparse Forexample,theinstancesP1,P2,P3andP4inTable5shouldbe
tree including the two entities involved in a relation. The classified as the same class since the interaction patterns in them
difference is that, like for the dependency path tree kernel, we are exactly the same. However, according to the corpus’s
also extendSPT insome cases. annotation, P1 and P3 are labeled as positive instances while P2
In our method, the feature-based kernel alone yields a and P4 are labeled as negative instances. The corpus annotation
comparable performance (66.63% in F-score) with the above error accountsfor35.5%of the200falsenegative errors.
twomethods.Withtheintroductionofthegraphandtreekernels Secondly, some general words (e.g., ‘‘drugs’’) are sometimes
ourmethodachievesevenbetterperformance.Inourexperiments, labeledasdrugnamesinboththetrainingsetandtestset.Totrain
wetried twodifferent multiple kernel combination methods.The a better model, we introduce the NameIsDrug feature as described
firstoneisdenotedas‘‘Ourcombinedkernel-1’’inTable4,which in Section Methods. However, our classification model fails to
assigns the same weights to all three kernels. Its performance address such a problem perfectly, and it causes 15 errors (for
(68.01%inF-score)isinferiortothatof‘‘Ourcombinedkernel-2’’ example,theexampleP5islabeledasapositiveinstancewhileitis
PLOSONE | www.plosone.org 10 June2013 | Volume 8 | Issue 6 | e65814