DTIC ADA460922: Inducing Multilingual Text Analysis Tools via Robust Projection across Aligned Corpora

9 Pages·0.16 MB·English

by  Defense Technical Information Center

Inducing Multilingual Text Analysis Tools
via Robust Projection across Aligned Corpora
David Yarowsky Grace Ngai Richard Wicentowski
Dept. ofComputerScience Dept. ofComputerScience Dept. ofComputerScience
JohnsHopkinsUniversity JohnsHopkinsUniversity JohnsHopkinsUniversity
Baltimore,MD21218USA Baltimore,MD21218USA Baltimore,MD21218USA
[email protected] [email protected] [email protected]
ABSTRACT
Annotations From Existing English Tools
Thispaperdescribesasystemandsetofalgorithmsforautomati-
[PLACE]
callyinducingstand-alonemonolingualpart-of-speechtaggers,base
noun-phrase bracketers, named-entity taggers and morphological [NatiJoJnal laNwNSs] appVlByGing iInN [ HNoNnPg KNoNnPg]
0 1 2 3 4 5
analyzersforan arbitraryforeignlanguage. Casestudiesinclude
French,Chinese,CzechandSpanish.
ExistingtextanalysistoolsforEnglishareappliedtobilingual 0 [1 2] 3 4 5 [6 7 8 9 10]
text corporaand their output projectedonto the second language
IN NNP NNP VBGVBG JJ JJ JJ NNS NNS
viastatisticallyderivedwordalignments.Simpledirectannotation
In Hong implementing of national law(s)
projectionisquitenoisy, however, even withoptimalalignments. Kong
Thusthispaperpresentsnoise-robusttagger,bracketerandlemma- [PLACE]
tizertrainingprocedurescapableofaccuratesystembootstrapping
Induced Annotations for Chinese
fromnoisyandincompleteinitialprojections.
Performance of the induced stand-alone part-of-speech tagger
Annotations From Existing English Tools
appliedtoFrenchachieves96%corepart-of-speech(POS)tagac-
curacy, andthecorrespondinginducednoun-phrasebracketer ex- [aD T signifJiJcant prodNuNcer ] fIoNr [ crJuJde oNNil]
ceeds91%F-measure.Theinducedmorphologicalanalyzerachie- 18 19 20 21 22 23
vesover99%lemmatizationaccuracyonthecompleteFrenchver-
balsystem.
This achievement is particularly noteworthy in that it required [u1n0 produ1c1teur impo12rtant ] d13e [ pe1t4role b1r5ut ]
absolutelynohand-annotatedtrainingdatainthegivenlanguage, DT NN JJ NN JJ
andvirtuallynolanguage-specificknowledgeorresourcesbeyond Induced Annotations for French
rawtext. Performancealsosignificantlyexceedsthatobtainedby
directannotationprojection. Figure1:Projectingpart-of-speechtags,named-entitytagsand
noun-phrasestructurefromEnglishtoChineseandFrench.
Keywords
English Bridge Lemmas
multilingual,textanalysis,part-of-speechtagging,nounphrasebrac-
BELIEVE GROW
keting,namedentity,morphology,lemmatization,parallelcorpora
believed believing believe growing grow
1. TASKOVERVIEW (cid:0) (cid:1)(cid:2) (cid:3)(cid:4)(cid:4)(cid:5)(cid:6)(cid:12) (cid:10)(cid:13) (cid:14) (cid:11)
Afundamentalroadblocktodevelopingstatisticaltaggers,brack- (cid:0) (cid:1)(cid:12) (cid:10)(cid:13) (cid:14) (cid:6)(cid:2) (cid:7)(cid:8)(cid:9)(cid:10)(cid:11)
etersandotheranalyzersformanyoftheworld’s200+majorlan-
guagesistheshortageorabsenceofannotatedtrainingdataforthe croyaient croissant croire croiVtre
largemajorityoftheselanguages.Ideally,onewouldliketolever-
French Inflections (cid:0) (cid:1)(cid:2) (cid:3)(cid:4)(cid:4)(cid:5)(cid:6)(cid:2) (cid:7)(cid:8)(cid:9)(cid:10)(cid:11) French Roots
Figure2:FrenchmorphologicalanalysisviaEnglish
agethelargeexistinginvestmentsinannotateddataandtools for
resource-rich languages (such as English and Japanese) to over-
cometheannotatedresourceshortageinotherlanguages.
Toshowthebroadpotentialofourapproachandmethods,this
paper will investigate four fundamental language analysis tasks:
POS tagging, base noun phrase (baseNP) bracketing, named en-
. titytagging,andinflectionalmorphologicalanalysis,asillustrated
Report Documentation Page Form Approved
OMB No. 0704-0188
Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and
maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information,
including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington
VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it

does not display a currently valid OMB control number.
1. REPORT DATE 3. DATES COVERED
2001 2. REPORT TYPE 00-00-2001 to 00-00-2001
4. TITLE AND SUBTITLE 5a. CONTRACT NUMBER
Inducing Mutlilingual Text Analysis Tools via Robust Projection across
5b. GRANT NUMBER
Aligned Corpora
5c. PROGRAM ELEMENT NUMBER
6. AUTHOR(S) 5d. PROJECT NUMBER
5e. TASK NUMBER
5f. WORK UNIT NUMBER
7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION
John Hopkins University,Center for Language and Speech REPORT NUMBER
Processing,Department of Computer Science,Baltimore,MD,21218
9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S)
11. SPONSOR/MONITOR’S REPORT
NUMBER(S)
12. DISTRIBUTION/AVAILABILITY STATEMENT
Approved for public release; distribution unlimited
13. SUPPLEMENTARY NOTES
14. ABSTRACT
15. SUBJECT TERMS
16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF 18. NUMBER 19a. NAME OF
ABSTRACT OF PAGES RESPONSIBLE PERSON

a. REPORT b. ABSTRACT c. THIS PAGE 8
unclassified unclassified unclassified
Standard Form 298 (Rev. 8-98)
Prescribed by ANSI Std Z39-18
inFigures1and2. Thesebedrocktoolsareimportantcomponents with additional distinctions in verb tense, noun number and pro-
ofth(cid:15)elanguageanalysispipelinesformanyapplications,andtheir noun type as captured in the English tagset inventory. Although
lowcostextensiontonewlanguages,asdescribedhere,canserve impoverishedrelativetosomelanguages,andincapableofresolv-
asabroadlyusefulenablingresource. ing details such as grammaticalgender, this Brown-corpus-based
tagsetgranularityissufficientformanyapplications.Furthermore,
2. BACKGROUND manyfiner-grainedpart-of-speechdistinctionsareresolvedprimar-
ilybymorphology,ashandledinSection7. Finally,ifonedesires
Previousresearchonthewordalignmentofparallelcorporahas
toinduceafiner-grainedtaggingcapabilityforcase,forexample,
tendedtofocusontheiruseintranslationmodeltrainingforMT
oneshouldprojectfromareferencelanguagesuchasCzech,where
rather than on monolingual applications. One exception is bilin-
caseislexicallymarked.
gualparsing. Wu(1995,1997)investigatedtheuseofconcurrent
Figure 3 illustrates six scenarios encountered when projecting
parsingofparallelcorporainatransductioninversionframework,
POS tags from English to a language such as French. The first
helpingtoresolveattachmentambiguitiesinonelanguagebythe
two show straightforward 1-to-1 projections, which are encoun-
coupledparsingstateinthesecondlanguage. JonesandHavrilla
tered in roughly two-thirds of English words. Phrasal (1-to-N)
(1998)utilizedsimilarjointparsingtechniques(twisted-pairgram-
alignments offer greater challenges, as typically only a subset of
mars)forwordreorderingintargetlanguagegeneration.
the aligned words accept the English tag. To distinguish these
However, with these exceptions in the field of parsing, to our
cases,weinitiallyassignposition-sensitivephrasalparts-of-speech
knowledge no one has previously used linguistic annotation pro-
jection via aligned bilingual corpora to induce traditional stand- viasubscripting(e.g.Les/NNS(cid:16) lois/NNS(cid:17) ),andsubsequentlylearn
a probablistic mapping to core, non-phrasal parts of speech (e.g.
alone monolingual text analyzers in other languages. Thus both
ourproposedprojectionandinductionmethods,andtheirapplica- P(cid:18) DT(cid:19)NNS(cid:16)(cid:21)(cid:20) )thatisusedalongwithtagsequenceandlexicalprior
modelstore-tagthesephrasalPOSprojections.
tion to multilingualPOStagging, named-entityclassificationand
morphologicalanalysisinduction,appearstobehighlynovel.
Tagger Output DT NNS VBG NN
English The laws ... ... living room ...
3. DATARESOURCES
The data sets used in these experiments included the English- French Les lois ... O ... salon ...
Induced Tags DT NNS NN
FrenchCanadianHansards,theEnglish-ChineseHongKongHan-
sards, and parallel Czech-English Reader’s Digest collection. In Tagger Output NNS NNS NNS NNS
addition, multiple versions of the Bible were used, including the English Laws ... O Laws ... ... potatoes ... ... veterans ...
FrenchDouay-RheimsBible,SpanishReinaValeraBible,andthree
French Les lois ... Les lois ... ... pommes de terre ... ... anciens combattants ...
English Bible Versions (King James, New International and Re- Induced Tag NNSa NNSb O NNS NNSa NNSb NNSc NNSa NNSb
visedStandard), automaticallyverse-alignedinmultiplepairings. Correct Tag (DT) (NNS) (DT) (NNS) (NNS) (IN) (NN) (JJ) (NNS)
Allcorporawereautomaticallyword-alignedbythenowpublicly
Figure3:FrenchPOStagprojectionscenarios
availableEGYPTsystem(Al-Onaizanetal.,1999),basedonIBM’s
Model3statisticalMTformalism(Brownetal., 1990). Thetag-
gingandbracketingtasksutilizedapproximately2millionwords 4.2 Noise-robust POSTagger Training
in each language, with the sample sizes for morphology induc-
Even at the relatively low tagset granularity of English, direct
tion given in Table 3. All word alignments utilized strictly raw-
projection of core POS tags onto French achieves only 76% ac-
word-basedmodelvariantsforEnglish/French/Spanish/Czechand
curacy using EGYPT’s automatic word alignments (as shown in
character-based model variants for Chinese, with no use of mor-
Table 1). Part of this deficiency is due to word-alignment error;
phologicalanalysisorstemming,POS-tagging,bracketingordic-
whenwordalignmentsweremanuallycorrected,directprojection
tionaryresources.
core-tagaccuracyincreasedto85%.Also,standardbigramtaggers
trained on the automatically projected data achieve only modest
4. PART-OF-SPEECHTAGGERINDUCTION
successatgeneralization(86%whenreappliedtothenoisytrain-
Part-of-speechtaggingisthefirstoffourapplicationscoveredin ingdata).Morehighlylexicalizedlearningalgorithmsexhibiteven
thispaper.ThegoalofthisworkistoprojectPOSanalysiscapabil- greaterpotentialforovermodelingthespecificprojectionerrorsof
itiesfromonelanguagetoanotherviaword-alignedparallelbilin- thisdata.
gualcorpora. Todoso,weuseanexistingPOStagger(e.g. Brill, Thus our research has focused on noise-robust techniques for
1995) to annotate the English side of the parallel corpus. Then, distillingaconservativebuteffectivetaggerfromthischallenging
asillustratedinFigure1forChineseandFrench,therawtagsare rawprojectiondata.Inparticular,wemodifystandardn-grammod-
transferred via the word alignments, yielding an extremely noisy elingtoseparatethetrainingofthetagsequencemodel(cid:22)(cid:23)(cid:18)(cid:25)(cid:24)(cid:26)(cid:20) from
initialtrainingsetforthe2ndlanguage.Thethirdcrucialstepisto thelexicalpriormodels(cid:22)(cid:23)(cid:18)(cid:28)(cid:27)(cid:29)(cid:19)(cid:24)(cid:26)(cid:20) ,andapplydifferentconfidence
generalizefromthesenoisyprojectedannotationsinarobustway, weightingandsignalamplificationtechniquestoboth.
yieldingastand-alonePOStaggerforthenewlanguagethatiscon-
4.2.1 LexicalPriorEstimation
siderablymoreaccuratethantheinitialprojectedtags.
Additionaldetailsofthisalgorithmaregiven inYarowsky and Figure4illustratestheprocessofhierarchicallysmoothingthe
Ngai(2001).Duetolackofspace,thefollowingsectionswillserve lexicalpriormodel(cid:22)(cid:23)(cid:30) (cid:18)(cid:25)(cid:31) (cid:19)!(cid:26)(cid:20) . Onemotivatingempiricalobservation
primarilyasanoverviewofthealgorithmanditssalientissues. isthatwordsinFrench,EnglishandCzechhaveastrongtendency
4.1 Part-of-speechProjectionIssues toexhibitonlyasinglecorePOStag(e.g." or# ),andveryrarely
havemorethan2.InEnglish,withrelativelyhigh(cid:22)(cid:23)(cid:18) POS(cid:19)!(cid:26)(cid:20) am-
First,becauseofconsiderablecross-languagedifferencesinfine- biguity,only0.37%ofthetokensintheBrownCorpusarenotcov-
grainedtagsetinventories,thisworkfocusesonaccuratelyassign- eredbyawordtype’stwomostfrequentcoretags,andinFrench
ingcorePOScategories(e.g. noun,verb,adverb,adjective,etc.), the percentageof tokens is only0.03%. Thuswe employ an ag-
Evaluateon EvaluateonUnseen
E-FAlignedFrench MonolingualFrench
Core EngEqv Core EngEqv
Model Tagset Tagset Tagset Tagset
(a)Directtransfer(onauto-aligneddata) .76 .69 N/A N/A
(b)Directtransfer(onhand-aligneddata) .85 .78 N/A N/A
(c)Standardbigrammodel(onauto-aligneddata) .86 .82 .82 .68
(d)Noise-robustbigraminduction(onauto-aligneddata) .96 .93 .94 .91
(e)Fullysupervisedbigramtraining(ongoldstandard) .97 .96 .98 .97
Table1:Evaluationof5POStaggerinductionmodelson2Frenchdatasetsand2tagsetgranularities
gressive re-estimationinfavor ofthisbias, amplifyingthe model 5. NOUNPHRASEBRACKETER
probabilityof the majority POS tag, and reducing or zeroing the INDUCTION
model probability of 2nd or lower ranked core tags proportional
Ourempiricalstudiesshowthatthereisaverystrongtendency
to their relative frequency with respect to the majority tag. This
fornounphrasestocohereasaunitwhentranslatedbetweenlan-
processisthenappliedrecursively,similarlyamplifyingtheproba-
guages,evenwhenundergoingsignificantinternalre-ordering.This
bilityofthemajoritysubtagswithineachcoretag. Furtherdetails,
strongnoun-phrasecohesioneventendstoholdforrelativelyfree
includingthehandlingof1-to-Nphrasalalignmentprojections,are
word orderlanguagessuchasCzech, wherebothnative speakers
giveninYarowskyandNgai(2001).
andparallelcorpusdataindicatethatnominalmodifierstendtore-
maininthesamecontiguouschunkasthenounstheymodify.This
property allows collective word alignments to serve as a reliable
P(Tag) P(Tag) P(Tag) b5a.s1isfoBrbarascekNetPproPjercotijoencatsiwonell.Methodology
Tag Tag Tag Theprojectionprocessbeginsbyautomaticallytaggingandbrac-
ketingtheEnglishdata,usingBrill(1995)andRamshaw&Marcus
DirectlyProjectedTag Smoothed&(cid:26)% ’)(+*,.- (1994),respectively.
Word J N V R I N V NN NNS VBN VBG As illustrated in Figure 5, each word within an English noun
achat 0 62 48 0 1 $ .76 .24 .73 .03 .03 .21 phraseisthensubscriptedwiththenumberofitsNPinthesentence,
cadre 2 35 7 1 1 .90 .10 .86 .04 .03 .00
andthissubscriptisprojectedontothealignedFrench(orChinese)
cadres 1 5 0 0 0 .94 .00 .04 .90 .00 .00
words.Inthemostcommoncase,thecorrespondingFrench/Chinese
pre´vu 1 11 48 0 0 .09 .91 .08 .01 .86 .00
nounphraseissimplythemaximalspanoftheprojectedsubscript.
Figure4:Hierarchicalsmoothingof(cid:22)/(cid:30) (cid:18)(cid:25)(cid:31)0(cid:19)!(cid:26)(cid:20) tagprobabilities Figure6showssomeoftheprojectionchallengesencountered.
NearlyallsuchcasesofinterwovenprojectedNPsareduetoalign-
menterrors,andastronginductivebiastowardsNPcohesionwas
4.2.2 TagSequenceModelEstimation utilizedtoresolvetheseincompatibleprojections.
Incontrast,thetrainingofthetagsequencemodel(cid:22)(cid:23)(cid:18)(cid:25)(cid:31)21+(cid:19)(cid:31)2143650798:8;8(cid:20)
focusesonconfidenceweightingandfilteringofprojectedtraining [J 1 N 1 ] VBD [ N 2 N 2 ] IN [ N3]
subsequences.Thecontributionofeachcandidatetrainingsentence
O
is weighted proportionally with both its EGYPT/GIZA sentence- [DT N J ] VBD [ N de N ] [ DT [ N ]
(1) (1) (1) (2) (2) (3)
levelalignmentscoreandanagreementmeasurebetweenthepro-
jected tags and the 1st iteration lexical priors, a rough measure Figure5:StandardNPprojectionscenarios.
of alignment reasonableness. Given the observed bursty distri-
bution of alignment errors in the corpus, this downweighting of [DT 1 J 1 N 1 ] VBD [ N 2 N2 ]
low-confidencealignmentregionssubstantiallyimprovessequence
O
modelqualitywithtolerablereductionintrainingvolume.
[DT N ] VBD [ N } { J de N ]
(1) (1) (2) (1) (2)
4.3 EvaluationofPOSTaggerInduction
Figure6:ProblematicNPprojectionscenarios.
Asshown inTable1, performanceisevaluated ontwo evalua-
5.2 BaseNPTrainingAlgorithm
tion data sets, including an independent 200K-word hand-tagged
FrenchdatasetprovidedbyUniversite´ deMontre´al,whichisused Forstand-alonetooldevelopment,theRamshaw&MarcusIOB
togaugestand-alonetaggerperformance.Signalamplificationand bracketingframeworkandafasttransformation-basedlearningsys-
noisereductiontechniquesyielda71%errorreduction,achievinga tem (Ngai and Florian, 2001) were applied to the noisy baseNP-
coretagsetaccuracyof96%,closelyapproachingtheupper-bound projecteddatadescribedabove.
97% performanceofan equivalent bigrammodel traineddirectly AswithPOStaggerinduction,bracketerinductionisimproved
onan80%subsetofthehand-taggedevaluationset(using5-fold byfocusingtrainingonthehighestqualityprojecteddataandex-
cross-validation). Thus robust training on 500K words of very cluding regions with the strongest indications of word-alignment
noisy but automatically-derived tag projections can approach the error. Thussentenceswiththelowest25%ofmodel-3alignment
performanceobtainedbyfullysupervisedlearningon80Kwords scoreswereexcludedfromtraining,asweresentenceswherepro-
ofhand-taggedtrainingdata. jectedbracketingsoverlappedandconflicted(alsoanindicatorof
alignmenterrors). Datawithlower-confidencePOStaggingwere scribedinSection7.Foragivensecond-languagewordFWandall
not<filtered,however,asthisfilteringreducesrobustnesswhenthe Englishwords=>(cid:27)?1 alignedtoit:
stand-alone bracketers are applied to noisy tagger output. Addi-
tionaldetailsareprovidedinYarowskyandNgai(2001).
Currenteffortstofurtherimprovethequalityofthetrainingdata (cid:22)(cid:23)(cid:18)NEclass@A(cid:19)FW(cid:20)CBED (cid:22)(cid:23)(cid:18)NEclass@A(cid:19)EW1 (cid:20)F(cid:22) (cid:16) (cid:18) EW1 (cid:19)FW(cid:20)
includeuseofiterativeEMbootstrappingtechniques.Separatepro- 1
jection of bracketings from aligned parallel data with a 3rd lan- (cid:22)(cid:23)(cid:18) PLACE(cid:19)Core´e(cid:20)GBH(cid:22)(cid:23)(cid:18)PLACE(cid:19)Korea(cid:20)I(cid:22)6(cid:16)J(cid:18)Korea(cid:19)Core´e(cid:20)LKM8:8;8
guagealsoshowspromiseforprovidingindependentsupervision,
whichcanfurtherhelpdistinguishconsensussignalfromnoise. Theco-training-basedalgorithmgiveninCucerzanandYarowsky
(1999) was then used to train a stand-alone named entity tagger
5.3 BaseNPProjectionEvaluation fromtheprojecteddata. Seedwordsforthisalgorithmwerethose
Because no bracketed evaluation data were available to us for FrenchwordsthatwerebothPOS-taggedaspropernounsandhad
French or Chinese, a third party fluent in these languages hand- anabove-thresholdentity-classconfidencefromthelexicalprojec-
bracketedasmall,held-out40-sentenceevaluationsetinbothlan- tionmodels.
guages, using a set of bracketing conventions that they felt were Performancewasmeasuredintermsofper-wordentity-typeclas-
appropriateforthelanguages.Table2showstheperformancerela- sification accuracy on the French Hansard test data, using the 4-
tivetotheseevaluationsets,asmeasuredbyexact-matchbracketing class inventory listed above. Classification accuracy of raw tag
precision(Pr),recall(R)andF-measure(F). projections was only 64% (based on automatic word alignment).
Incontrast,thestand-aloneco-training-basedtaggertrainedonthe
ExactMatch AcceptableMatch projectionsachieved85%classificationaccuracy,illustratingitsef-
Method Pr R F Pr R F fectivessatgeneralizationinthefaceofprojectionnoise. Notably,
Chinese: mostofitsobservederrorscanbetracedtoentityclassificationer-
Direct(auto) .26 .58 .36 .48 .58 .51 rorsfromtheoriginalEnglishtagger.Infact,whenevaluatedonthe
Direct(hand) .47 .61 .53 .86 .86 .86 EnglishtranslationoftheFrenchtestdataset, the Englishtagger
French: onlyachieved86%classificationaccuracyonthisdirectlycompa-
Direct(auto) .43 .48 .45 .60 .58 .59 rabledataset.Itappearsthattheprojection-inducedFrenchtagger
Direct(hand) .56 .51 .53 .74 .70 .72 achievesperformancenearlyashighasitsoriginaltrainingsource.
FTBL(auto) .82 .81 .81 .91 .91 .91 Thusfurtherimprovementsshouldbeexpectedfromhigherquality
Englishtrainingsources.
Table2:PerformanceofBaseNPinductionmodels
7. MORPHOLOGICALANALYSIS
Itisimportanttonote, however, thatmanydecisionsregarding INDUCTION
BaseNPbracketingconventionsareessentiallyarbitrary,andagree-
Bilingualcorporacanalsoserveasaverysuccessfulbridgefor
ment rates between additional human judges on these data were
aligningcomplexinflectedwordformsinanewlanguagewiththeir
measured at 64% and 80% for French and Chinese respectively.
rootforms,evenwhentheirsurfacesimilarityisquitedifferentor
Sincethetranslingualprojectionsareessentiallyunsupervisedand
highlyirregular.
havenodataonwhichtomimicarbitraryconventions,itisalsorea-
sonabletoevaluatethedegreetowhichtheinducedbracketingsare
deemedacceptableandconsistentwiththearbitrarygoldstandard Potential English Bridge Words
(e.g. no crossing brackets). To this end, an additionalpool of 3 believed believing believe growing grow
judgeswereaskedtofurtheradjudicatethedifferencesbetweenthe
goldstandardandtheprojectionoutput,annotatingsuchsituations
aseitheracceptable/compatibleorunacceptable/incompatible.
Overall,thesetranslingualprojectionresultsarequiteencourag-
ing. FortheChinese, they aresimilartoWu’s78%precisionre- croyaient croyant croissant croire croître
sultfortranslingual-grammar-basedNPbracketing,andespecially
promisinggiventhatnowordsegmentation(onlyrawcharacters) French Inflections French Roots
wereused. ForFrench,theincreasefrom59%to91%F-measure
Figure7:Direct-bridgeFrenchinflection/rootalignment
forthestand-aloneinducedbracketershowsthatthetrainingalgo-
rithmisabletogeneralizesuccessfullyfromthenoisyrawprojec-
AsillustratedinFigure7,theassociationbetweenaFrenchver-
tiondata,distillingareasonablyaccurate(andtransferable)model
bal inflection(croyant) andits correctroot (croire), rather thana
ofbaseNPstructurefromthishighdegreeofnoise.
similarcompetitor(croˆıtre),canbeidentifiedbyasingle-steptran-
sitiveassociationviaanEnglishbridgeword(believing).However,
6. NAMEDENTITYTAGGERINDUCTION
inthe caseofmorphologyinduction,suchdirectassociationsare
Multilingualnamedentitytaggerinductionisbasedonthe ex- relativelyraregiventhatinflectionsinasecondlanguagetendtoas-
tended combination of the part-of-speech and noun-phrase brac- sociatewithsimilartensesinEnglishwhilethesingular/infinitive
ketingframeworks. Theentityclasstagsusedforthisstudywere forms tend to associate with analogous singular/infinitive forms,
FNAME,LNAME,PLACEandOTHER(otherentitiesincludingor- andthuscroyaient(believed)anditsrootcroirehavenodirectEn-
ganizations). They were derived from an anonymously donated glishlinkinouralignedcorpus.
MUC-6namedentitytaggerappliedtotheEnglishsideoftheFrench- However,Figure2(firstpage)illustratesthatanexistinginvest-
EnglishCanadianHansardsdata. mentinalemmatizerforEnglishcanhelpbridgethisgapbyjoining
Initialclassificationproceedsonaper-wordbasis,usinganag- amulti-steptransitiveassociationcroyaient$ believed$ believe$
gressivelysmoothedtransitiveprojectionmodelsimilartothosede- croire. Figure8illustrateshowthistransitivelinkageviaEnglish
English Bridge Lemmas Forexample:
BELIEVE THINK (cid:22)(cid:23)(cid:18)commencer(cid:19)commenc¸a(cid:20)GBu(cid:22)(cid:23)(cid:18)c¸a $ cer(cid:19)commenc¸a(cid:20)GB
believed believing believe thought think (cid:22)(cid:23)(cid:18)c¸a$ cer(cid:20)(cid:133)K 5+(cid:22)(cid:23)(cid:18)c¸a$ cer(cid:19)a(cid:20)tK (cid:22)(cid:23)(cid:18)c¸a$ cer(cid:19)c¸a(cid:20)2K
|k(cid:132)K (cid:22)(cid:23)(cid:18)c¸a $ cer(cid:19)|nc¸a(cid:20)tK (cid:22)(cid:23)(cid:18)c¸a$ ce|(cid:135)r(cid:134)(cid:19)enc¸a(cid:20)(cid:133)KM8;8;8
|k(cid:136) |k(cid:137)
NPO(cid:25)\ Z]4^_VQWXUYZ[ NPO‘Q(cid:25)RS4SUT‘V\ Z]U^ [ (cid:22)(cid:23)(cid:18)ployer(cid:19)ploie(cid:20)GBH(cid:22)(cid:23)(cid:18)ie$ yer(cid:19)ploie(cid:20)}B
(cid:22)(cid:23)(cid:18)ie$ yer(cid:20)(cid:133)K 5+(cid:22)(cid:23)(cid:18)ie$ yer(cid:19)e(cid:20)(cid:133)K (cid:22)(cid:23)(cid:18)ie$ yer(cid:19)ie(cid:20)2K
croyaient croissant croire croiVtre |k(cid:132) K (cid:22)(cid:23)(cid:18)ie$ yer|(cid:19)oie(cid:20)LK (cid:22)(cid:23)(cid:18)ie$ y|(cid:135)er(cid:134)(cid:19)loie(cid:20)tKu8;8;8
|k(cid:136) |k(cid:137)
Animportantpropertyofthetrie-basedmodelsistheireffective-
French Inflections NPO(cid:25)Q:RS4SUTVQ;WX4YZ[ French Roots nessatclusteringwordsthatexhibitsimilarmorphologicalbehav-
ior,bothreducingmodelsizeandfacilitatinggeneralizationtopre-
Figure8:Multi-bridgeFrenchinflection/rootalignment viouslyunseenexamples. ThispropertyisillustratedinFigure9,
showingasample(inflection$ root)triebranchforFrenchverbal
inflections, with suffix histories (cid:127) =’oie’, (cid:127) =’noie’, (cid:127) =’roie’, etc.
Ateachhistorynode,thehierarchicallysmoothedprobabilitiesof
lemmatizationcanbepotentiallyutilizedforallotherEnglishlem- severaly $ w (inflection$ root)changesaregiven. Notethatthe
mas (such as THINK)with which croyaient and croire also asso- relativeprobabilitiesofthecompetinganalysesie$ irandie$ yer
ciate,offeringgreaterpotentialcoverageandrobustnessviamulti- differ substantially for diffent suffix histories, and that there are
plebridges. subexceptionsthattendtoclusterbyaffixhistory. Thisallowsfor
Formally, thesemultipletransitive linkages can be modeled as thesuccessfulanalysisof8ofthe9italicizedtestwordsthathad
shownbelow, bysummingoverallEnglishlemmas(=badcfehg )with notbeenseeninthebilingualprojectiondataorwheretheMProj
whicheitheracandidateforeigninflection(i infl)oritsroot(i root) modelyieldednorootcandidateabovethreshold.
exhibitanalignmentintheparallelcorpus: root a (cid:222) b P(a (cid:222) b |h)
ie(cid:222) ir 0.003
h=e e(cid:222) re 0.002
e ie(cid:222) yer 0.0008
(cid:22)6eCjk(cid:18)Ui6l+m+mon+(cid:19)i 1)p(cid:21)q a (cid:20)CBHD (cid:22) (cid:16) (cid:18)Uirl2m+mono(cid:19)= adcfe go(cid:20)F(cid:22) (cid:16) (cid:18)U= adcfe g0(cid:19)i 1:psq a (cid:20)
oie(cid:222) eoir 0.0001
1
ie(cid:222) ir 0.021
Forexample: h=’e’ e(cid:222) re 0.016
i ie(cid:222) yer 0.005
(cid:22)reCjJ(cid:18)croire(cid:19)croyaient(cid:20)GB oie(cid:222) eoir 0.0005
(cid:22)6(cid:16)J(cid:18)croire(cid:19)BELIEVE(cid:20)I(cid:22)6(cid:16)F(cid:18) BELIEVE(cid:19)croyaient(cid:20)2K ie(cid:222) ir 0.313
(cid:22) (cid:16) (cid:18)croire(cid:19)THINK(cid:20)F(cid:22) (cid:16) (cid:18) THINK(cid:19)croyaient(cid:20)tKM8;8;8 h=’ie’ ie(cid:222) yer 0.083
o e(cid:222) re 0.030
Thisprojection/bridge-basedsimilaritymeasure(cid:22) mp(cid:18)Ui root(cid:19)i infl(cid:20) oie(cid:222) eoir 0.007
canbequiteeffectiveonitsown,asshownintheMProjonlyentries atermoie(cid:222) atermoyer3
apitoie(cid:222) apitoyer3 ie(cid:222) yer 0.424
inTable3(formultipleparallelcorporain3differentlanguages), nettoie(cid:222) nettoyer3 ie(cid:222) ir 0.309
especiallywhenrestrictedtothehighest-confidencesubsetof the côtoie(cid:222) côtoyer3 h=’oie’ e(cid:222) re 0.072
vocabulary(5.2%to77.9%inthesedata)forwhichtheassociation aboie(cid:222) aboyer3 v n l s r oie(cid:222) eoir 0.070
exceedssimplefixedprobabilityandfrequencythresholds. When
estimatedusinga1.2millionwordsubsetoftheFrenchHansards,
for example, the MProj measure alone achives 98.5% precision voie9 noie9 octroie(cid:222) octroyer3 rassoie9
oinng3927..76%%oofftthheeitnoflkeenctseidnFthreenccohrpvuesrb).sUinnltihkeectroardpiutison(caolnsstrtiitnugt-- louvoieh=(cid:222)’vlooieu’voir7 edméppllooiiee99 ie(cid:222)h=y’reorie’0.50 assaosisee(cid:222)oir3
tvrearnbssdupcotsieonth-beagseredamteostrpchhoallolegnygiensd,uthcteisoentmypeitchaoldlyswhihgehr-efrierqreugeunlcayr iiee(cid:222)(cid:222) iyrer 00..3448 plpolioey(cid:222)er3 be(cid:222) re 0.4c6 ohi=e’s(cid:222) oieeo’ir
wordsareoftenthebestmodelleddata inthevocabularymaking é e n h=’noie’
these multilingualprojection techniquesa naturalcomplement to h=’loie’ broie9 croie9
ie(cid:222) yer
existingmodels. h=’broie’ h=’croie’
prévoie9 ie(cid:222) yer e(cid:222) re
7.1ThehTigrhiepr-ebciasisoendonMthoerMpPhrooj-lcoogveyreMdsoubdseetlaslsomakethese hr=ev’éovioeie9’ reennvvooiiee99 TraininTge sdta dtaa tfar ocmor rMecPtrlyo ja annaalylyzseids39
h=’evoie’ h=’nvoie’
partialpairingseffective trainingdata forrobustsupervisedalgo- ie(cid:222) ir ie(cid:222) yer Test data incorrectly analyzed7
rithmsthatcangeneralizethestringtransformationbehaviortothe
remaininguncovered vocabulary. Whileany supervised morpho-
Figure9: ExampleofaFrenchMTriebranch,showinginflec-
logicalanalysistechniqueispossiblehere,weemployatrie-based
modelingtechniquewheretheprobabilityofagivenstem-change tion $ root probabilities ((cid:22)(cid:23)(cid:18)4y $ wh(cid:19)(cid:127)k1(cid:128)(cid:20) ) for variable length
(fromtheinventoryobservedintheMProj-pairedtrainingdata)is suffixhistories ((cid:127) 1). MTrie analyseson test data are given in
italics.
modeledhierarchicallyusingvariablesuffixcontext, asdescribed
inYarowskyandWicentowski(2000): Table3 illustratesthe performanceofa variety ofmorphology
induction models. When using the projection-based MProj and
(cid:22)(cid:23)(cid:18)root(cid:19)in{flect1Ui(cid:22)(cid:23)on(cid:18)4(cid:20)Cy Bu$ (cid:22)(cid:23)w~(cid:18)4vx(cid:19)(cid:127)kwh1(cid:128)(cid:19)(cid:20)vzyG(cid:20)CfoBHr(cid:22)(cid:23)(cid:127)k(cid:18)416yB $ suwhffi(cid:19)xvzyG(cid:18)4(cid:129)2(cid:20)C72v(cid:130)By(cid:131)(cid:20) atrgiee-btoasweodrMdsTtrhiaetmmoadyelnsottoegveetnhearp(pweiatrhitnhethleatptearraelxletelncdoirnpgusc)o,vfeurl-l
1}|
verblemmatizationprecisiononthe1.2MwordHansardsubsetex- 7.2 MorphologyInductionviaAlignedBibles
ceed(cid:138)s99.5%(bytype)and99.9%(bytoken)with95.8%coverage Performanceusingevensmallparallelcorpora(e.g.a120Ksub-
bytypeand99.8%coveragebytoken. Abackoffmodelbasedon setoftheFrenchHansards)stillyieldsarespectable93.2%(type)
Levenshtein-distanceanddistributionalcontext similarityhandles and98.9%(token)precisionontheverb-lemmatizationtestsetfor
the relatively small percentage of cases where MProj and MTrie thefullHansards. GiventhattheBibleisactuallylarger(approxi-
together are not sufficiently confident, bringing the system cov- mately300Kwords,dependingonversionandlanguage)andavail-
erage to 100% coverage witha small drop inprecision to 97.9% ableon-lineorviaOCRforvirtuallyalllanguages(Resniketal.,
(by type) and 99.8% (by token) on the unrestricted space of in- 2000),wealsoconductedseveralexperimentsonBible-basedmor-
flected verbs observed in thefull FrenchHansards. Asshown in phologyinduction,furtherdetailedinTable3.
Section7.3,performanceisstronglycorrelatedwithsizeoftheini-
tialalignedbilingualcorpus,withalargerHansardsubsetof12M 7.2.1 BoostingPerformanceviaMultiple
wordsyielding99.4%precision(bytype)and99.9%precision(by ParallelTranslations
token).PerformanceonCzechisdiscussedinSection7.3.
Even though at most one translation of the Bible is typically
availableinagivenforeignlanguage,numerousEnglishBiblever-
Precision Coverage sionsarefreelyavailableandaperformanceincreasecanbeachie-
Model Typ Tok Typ Tok vedbysimultaneouslyutilizingalignmentstoeachEnglishversion.
AsillustratedinFigure10,differentalignedBiblepairsmayexhibit
FRENCHVerbalMorphologyInduction
(orbemissing)differentfullorpartialbridgelinksforagivenword
FrenchHansards(12Mwords):
(duebothtodifferentlexicalusageandpoortextualparallelismin
MProjonly .992 .999 .779 .994
some text-regions or version pairs). However, (cid:22) (cid:16) (cid:18)Uirl2momono(cid:19)= adcfe g(cid:139)(cid:20)
MProj+MTrie .998 .999 .988 .999
and (cid:22)6(cid:16)J(cid:18)U= adcfe gx(cid:19)i infl(cid:20) need not be estimated from the same Bible
MProj+MTrie+BKM .994 .999 1.00 1.00
pair. Evenif onehas onlyoneBibleinagiven sourcelanguage,
FrenchHansards(1.2Mwords): eachalignmentwithadistinctEnglishversiongivesnewbridging
MProjonly .985 .998 .327 .976 opportunities with no additional resources needed on the source
MProj+MTrie .995 .999 .958 .998 languageside.Thebaselineapproach(evaluatedhere)issimplyto
MProj+MTrie+BKM .979 .998 1.00 1.00 concatenatethe different aligned versions together. While word-
FrenchHansards(120Kwords): pairinstancestranslatedthesamewayineachversionwillbere-
MProjonly .962 .931 .095 .901 peated, thisrather reasonablyreflectsthe increasedconfidencein
MProj+MTrie .984 .993 .916 .994 thisparticularalignment.Analternatemodelwouldweightversion
MProj+MTrie+BKM .932 .989 1.00 1.00 pairsdifferentlybasedontheotherwise-measuredtranslationfaith-
FrenchBible(300Kwords)via1EnglishBible: fulnessandalignmentqualitybetweentheversionpairs. Doingso
MProjonly 1.00 1.00 .052 .747 wouldhelpdecreasenoise.Increasingfrom1to3Englishversions
MProj+MTrie .991 .998 .918 .992 reducesthetypeerrorrate(atfullcoverage)by22%onFrenchand
MProj+MTrie+BKM .954 .994 1.00 1.00 28%onSpanishwithnoincreaseinthesourcelanguageresources.
FrenchBible(300Kwords)via3EnglishBibles:
MProjonly .928 .975 .100 .820
BELIEVE English Bridge Lemma
MProj+MTrie .981 .991 .931 .990
SOURCE 1 believed believing (e.g. KJV)
MProj+MTrie+BKM .964 .991 1.00 1.00
SOURCE 2 believed (e.g. NIV)
CZECHVerbalMorphologyInduction
SOURCE 3 believe (e.g. RSV)
CzechReader’sDigest(500Kwords):
MProjonly .915 .993 .152 .805 NPO(cid:25)\AZ]4^_VQWX4YZ[ NPO(cid:25)Q(cid:25)RSUS4T‘V\ Z]4^ [
MProj+MTrie .916 .917 .893 .975
MProj+MTrie+BKM .878 .913 1.00 1.00
croyaient croyant croire croiVtre
SPANISHVerbalMorphologyInduction
SpanishBible(300Kwords)via1EnglishBible: French Inflections NPO(cid:25)QRS4SUTVQWX4YZ[ French Roots
MProjonly .973 .935 .264 .351
MProj+MTrie .988 .998 .971 .967
Figure10:UseofmultipleparallelBibletranslations
MProj+MTrie+BKM .966 .985 1.00 1.00
SpanishBible(300Kwords)viaFrenchBible:
MProjonly .980 .935 .722 .765
MProj+MTrie .983 .974 .986 .993 7.2.2 BoostingPerformanceviaMultiple
MProj+MTrie+BKM .974 .968 1.00 1.00 BridgeLanguages
SpanishBible(300Kwords)via3EnglishBibles:
Oncelemmatizationcapabilitieshavebeensuccessfullyprojected
MProjonly .964 .948 .468 .551
toanewlanguage(suchasFrench),thislanguagecanthenserveas
MProj+MTrie .990 .998 .978 .987
anadditionalbridgingsourceformorphologyinductioninathird
MProj+MTrie .976 .987 1.00 1.00
language(such asSpanish), asillustratedinFigure 11. Thiscan
be particularlyeffective if the two languagesare very similar (as
inSpanish-French)oriftheiravailableBibleversionsareaclose
Table 3: Performance of full verbal morphological analysis,
translationofacommonsource(e.g. theLatinVulgateBible). As
includingprecision/coveragebytype/token
showninTable3,usingthepreviouslyanalyzedFrenchBibleasa
bridgeforSpanishachieves performance(97.4%precision)com-
parabletotheuseof3parallelEnglishBibleversions. morphology. (b) the small size of the Czech parallel cor-
pus (less than twice the length of the Bible). (c) the com-
monoccurrenceinCzechoftwoverysimilarperfectiveand
English Bridge Lemmas French Bridge Lemmas
non-perfectiverootvariants(e.g. odola´vatandodolat, both
BELIEVE CROIRE
ofwhichmeantoresist). Asimplemonolingualdictionary-
believed believing believe croyaient croire
derived list of canonicalroots would resolve ambiguityre-
gardingwhichistheappropriatetarget.
(cid:140)P(cid:141)‘(cid:152) (cid:150)(cid:153)U(cid:154)(cid:155)(cid:146)(cid:142)(cid:147)(cid:148)U(cid:149)(cid:150)(cid:151) (cid:140)(cid:128)(cid:141)(cid:25)(cid:142)(cid:143)(cid:144)4(cid:144)4(cid:145)(cid:146)(cid:152) (cid:150)(cid:153)4(cid:154) (cid:151) (cid:156) Manyoftheerrorsareduetoall(ormost)inflectionsofasin-
gleverbmappingtothesameincorrectroot. Butformany
applicationswherethefunctionoflemmatizationistocluster
creyeron creia creer crear equivalent words(e.g. stemmingforinformationretrieval),
thechoiceoflabelforthelemmaislessimportantthancor-
Spanish Inflections (cid:140)(cid:128)(cid:141)(cid:25)(cid:142):(cid:143)(cid:144)4(cid:144)4(cid:145)(cid:146)(cid:142);(cid:147)(cid:148)4(cid:149)(cid:150)(cid:151) Spanish Roots rectlylinkingthemembersofthelemma.
.10
Figure11:Useofbridgesinmultiplelanguages.
.05
French Bible French Hansards
7.3 MorphologyInduction: Observations .02 (by type) (Error rate by TYPE)
e
t
This section includes additional detail regarding the morphol- a
R .01
ogyinductionexperiments,supplementingthepreviousdetailsand r French Bible
o (by token)
analysesgiveninSection7andTable3. r
r .005
E
French Hansards
(cid:156) PerformanceinductionusingtheFrenchBibleasthebridge
(Error rate by TOKEN)
sourceisevaluatedusingthefulltestverbsetextractedfrom .002
theFrenchHansards. Thestrongperformancewhentrained
onlyusingtheBibleillustratesthatevenasmallsingletext .001 120 K 300 K 1.2 M 12 M
in a very different genre can provide effective transfer to
modern(conversational)French. Whiletheobserved genre Size of Aligned Corpus (words)
andtopic-sensitivevocabularydifferssubstantiallybetween
Figure12:LearningCurvesforFrenchMorphology
the Bible and Hansards, the observed inventories of stem
changesandsuffixationactuallyhavelargeoverlap,asdothe (cid:156) ThelearningcurvesinFigure12showthestrongcorrelation
setofobservedhigh-frequencyirregularverbs. Thusthein- betweenperformanceandsizeofthealignedcorpus. Given
ventoryofmorphologicalphenomenaseemtotranslatebetter thatlargequantitiesofparalleltextcurrentlyexistintrans-
acrossgenrethandolexicalchoiceandcollocationmodels. lationbureauarchivesandOCR-ablebooks,nottomention
(cid:156) Over60%oferrorsareduetogapsinthecandidaterootlists. the increasing online availability of bitext on the web, the
Currentlythecandidaterootlistsarederivedautomaticallyby naturalgrowthofavailablebitextquantitiesshouldcontinue
applyingtheprojectedPOSmodelsandselectinganyword tosupportperformanceimprovement.
withtheprobabilityofbeinganuninflectedverbgreaterthan (cid:156) The system analysis examples shown in Table 4 are repre-
agenerousthresholdandalsoendinginacanonicalverbsuf- sentativeofmodelperformanceandareselectedtoillustrate
fix.Falsepositivesareeasilytolerated(lessthan5%oferrors therangeofencounteredphenomena. Allsystemevaluation
areduetospuriousnon-rootcompetitors),butwithmissing isbasedonthetaskofselectingthecorrectrootforagiven
rootsthealgorithmsareforcedeithertoproposepreviously inflection (which has a long lexicography-basedconsensus
unseenrootsoraligntotheclosestpreviouslyobservedroot regardingthe“truth”).Incontrast,thedescriptiveanalysisof
candidate. Thuswhilenonon-Englishdictionarywasused anysuchpairingisverytheorydependentwithoutstandard
in the computation of these results, it would substantially consensus. The “TopBridge” column shows the strongest
improve performance to have a dictionary-based inventory Englishbridgelemmautilizedinmapping(typicallyoneof
ofpotentialroots,increasingcoverageanddecreasingnoise manypotentialbridgelemmas).
fromcompetingnon-rootsandspellingerrors.
Theseresultsarequiteimpressiveinthattheyarebasedonessen-
(cid:156) Performanceinalllanguageshasbeensignificnatlyhindered tiallynolanguage-specificknowledgeofFrench,SpanishorCzech.
bylow-accuracy parallel-corpusword-alignmentsusingthe In addition, the multilingual bridge algorithm issurface-form in-
original Model-3 GIZA tools. Use of Och and Ney’s re- dependent, and can just as readily handle obscure infixational or
centlyreleasedandenhancedGIZA++word-alignmentmod- reduplicativemorphologicalprocesses.
els(OchandNey,2000)shouldimproveperformanceforall
oftheapplicationsstudiedinthispaper,aswoulditerativere- 8. CONCLUSION
alignmentsusingricheralignmentfeatures(includinglemma
Thispaperhaspresentedadetailedsurveyoforiginalalgorithms
andpart-of-speech)derivedfromthisresearch.
for cross-language annotation projection and noise-robust tagger
(cid:156) The current somewhat lower performance on Czech is due induction, evaluated on four diverse applications. It shows how
to several factors. They include (a) very low accuracy ini- previousmajorinvestmentsinEnglishannotatedcorporaandtool
tial word-alignments due to often non-parallel translations developmentcanbeeffectivelyleveragedacrosslanguages,achiev-
of the Reader’s Digest sample and the failure of the initial ingaccuratestand-alonetooldevelopmentinotherlanguageswith-
word-alignmentmodelstohandlethehighlyinflectedCzech outcomparablehumanannotationefforts.Collectivelythisworkis
the mostcomprehensive existingexplorationof avery promising InducedMorphologicalAnalysesforCZECH
new(cid:157)paradigmforcross-languageresourceprojection. Inflection RootOut Analysis TopBridge
bral bra´t al$ a´t marry
Acknowledgements
brala bra´t ala$ a´t accept
This research has been partially supported by NSF grant IIS- brali bra´t ali$ a´t marry
9985033 and ONR/MURI contract N00014-01-1-0685. The au- byl by´t yl$ y´t be
thorsthankSilviuCucerzan,RaduFlorian,JanHajic,GideonMann byli by´t yli$ y´t be
andCharlesSchaferfortheirvaluablecontributionsandfeedback. bylo by´t ylo$ y´t be
chovala chovat la$ t behave
9. REFERENCES chova´ chovat a´$ at behave
chova´me chovat a´me$ at behave
[1] Y.Al-Onaizan,J.Curin,M.Jahr,K.Knight,J.Lafferty,D. chodila chodit la$ t walk
Melamed,FJOch,D.Purdy,N.SmithandD.Yarowsky.1999. chod´ı chodit ´ı$ it walk
StatisticalMachineTranslation(techreport).JohnsHopkins chodˇte chodit dˇte$ dit swim
University. chra´nila chra´nit la$ t protect
[2] E.Brill.1995.Transformation-basederror-drivenlearningand chra´n´ı chra´nit ´ı$ it protect
naturallanguageprocessing:Acasestudyinpartofspeech couval couvat l$ t back
tagging.ComputationalLinguistics,24(1):543–565. chce cht´ıt ce$ t´ıt want
[3] P.Brown,J.Cocke,S.DellaPietra,V.DellaPietra,F.Jelinek, chcete cht´ıt cete$ t´ıt want
J.Lafferty,R.Mercer,andP.Rossin.1990.Astatistical chcesˇ cht´ıt cesˇ$ t´ıt want
approachtomachinetranslation.ComputationalLinguistics, chci cht´ıt ci$ t´ıt want
16(2):29–85. chteˇj´ı cht´ıt eˇj´ı$ ´ıt want
[4] S.CucerzanandD.Yarowsky.1999.Languageindependent chteˇli cht´ıt eˇli$ ´ıt want
namedentityrecognitioncombiningmorphologicaland chteˇlo cht´ıt eˇlo$ ´ıt want
contextualevidence.”InProceedings,1999JointSIGDAT
ConferenceonEmpiricalMethodsinNLPandVeryLarge InducedMorphologicalAnalysesforSPANISH
Corpora,pp.90-99. Inflection RootOut Analysis TopBridge
[5] P.FungandK.Church.1994.K-vec:anewapproachfor aborrecio´ aborrecer io´$ er hate
aligningparalleltexts.InProceedingsofCOLING-94,pp. aborrec´ıa aborrecer ´ıa$ er hate
1096–1102. aborrezco aborrecer zco$ cer hate
[6] P.FungandK.McKeown.1994.Aligningnoisyparallel abrace abrazar ce$ zar embrace
corporaacrosslanguagegroups:Wordpairfeaturematchingby abrazado abrazar ado$ ar embrace
dynamicwarping.InProceedingsofAMTA-94,pp.81–88. adquiere adquirir ere$ rir get
[7] D.Jones,andR.Havrilla.1998Twistedpairgrammar: andamos andar amos$ ar walk
Supportforrapiddevelopmentofmachinetranslationforlow andando andar ando$ ar walk
densitylanguagesInProcs.ofAMTA’98,pp.318–332. andara´n andar ara´n$ ar wander
[8] D.Melamed.1999.Bitextmapsandalignmentviapattern andara´s andar ara´s$ ar wander
recognition.ComputationalLinguistics,25(1):107–130. andemos andar emos$ ar walk
[9] G.NgaiandR.Florian.2001.Transformation-basedlearning anden andar en$ ar walk
inthefastlane.InProceedingsofNAACL-2001,pp.40-47. anduvo andar uvo$ ar walk
[10] F.J.OchandH.Ney.2000.Improvedstatisticalalignment busca´is buscar a´is$ ar seek
models.InProceedingsofACL-2000,pp.440-447. busco´ buscar o´$ ar seek
busque buscar que$ car seek
[11] L.RamshawandM.Marcus,1999.Textchunkingusing
busque´ buscar que´$ car seek
transformation-basedlearning.InArmstrongetal.(Eds.),
NaturalLanguageProcessingUsingVeryLargeCorpora.
InducedMorphologicalAnalysesforFRENCH
Kluwer,pp.157-176.
Inflection RootOut Analysis TopBridge
[12] P.Resnik,M.Olsen,andM.Diab.2000.TheBibleasa
abre`ge abre´ger e`ge$ e´ger shorten
parallelcorpus:annotatingthe‘Bookof2000Tongues’
abre`gent abre´ger e`gent$ e´ger shorten
ComputersandtheHumanities,33(1-2):129-153.
abre´gerai abre´ger erai$ er curtail
[13] D.Wu.1995.Analgorithmforsimultaneouslybracketing
ache`te acheter e`te$ eter buy
paralleltexts.InProc.ofACL-95,pp.244–251.
ache`tent acheter e`tent$ eter buy
[14] D.Wu.1997.Statisticalinversiontransductiongrammarsan ache`tera acheter e`tera$ eter buy
bilingualparsingofparallelcorpora.Computational advenait advenir ait$ ir happen
Linguistics,23(3):377-404. advenu advenir u$ ir happen
[15] D.YarowskyandG.Ngai.2001.InducingmultilingualPOS adviendrait advenir iendrait$ enir happen
taggersandNPBracketersviarobustprojectionacrossaligned advient advenir ient$ enir happen
corpora.InProceedingsofNAACL-2001,pp.377-404. alie`ne alie´ner e`ne$ e´ner alienate
[16] D.YarowskyandR.Wicentowski.2000.Minimally alie`nent alie´ner e`nent$ e´ner alienate
supervisedmorphologicalanalysisbymultimodalalignment.In conc¸u concevoir c¸u$ cevoir conceive
ProceedingsofACL-2000,pp.207-216. crois croire s$ re believe
croyaient croire yaient$ ire believe
Table4:Sampleofinducedmorphologicalanalyses