Ciaramita and Johnson's 2004 talk at the Senseval workshop at ACL

19 Pages·2004·0.08 MB·English

Senseval 3/ACL’04 July 2004
Multi-Component Word Sense Disambiguation
Massimiliano Ciaramita and Mark Johnson
Brown University
BLLIP: http://www.cog.brown.edu/Research/nlp
Ciaramita and Johnson 1
Senseval 3/ACL’04 July 2004
Outline
• Pattern classification for WSD
– Features
– Flat multiclass averaged perceptron
• Multi-component WSD
– Generating external training data
– Multi-component perceptron
• Experiments and results
Ciaramita and Johnson 2
Senseval 3/ACL’04 July 2004
Pattern classification for WSD
English lexical sample: 57 test words: 32 verbs, 20 nouns, 5 adjec-
tives. For each word w:
1. compile a training set: S(w) = (x , y )n
i i
d
• x ∈ IR a vector of features
i
• y ∈ Y(w), one of the possible senses of w
i
d
2. learn a classifier on S(w): H : IR → Y(w)
3. use the classifier to disambiguate the unseen test data
Ciaramita and Johnson 3
Senseval 3/ACL’04 July 2004
Features
• Standard feature set for wsd (derived from (Yoong and Hwee,
2002))
– “A-DT newspaper-NN and-CC now-RB a-DT bank-NN have-
AUX since-RB taken-VBN over-RB”
• POS of neighboring words - P ; e.g., P = DT,
x,x∈{−3,−2,−1,0,+1,+2,+3} −1
P = NN, P = AUX, ...
0 +1
• Surrounding words - WS; e.g., WS = take , WS = over , WS = newspaper
v r n
• N-grams:
– NG ; e.g., NG = now, NG = have, NG = take
x,x∈{−2,−1,+1,+2} −2 +1 +2
– NG ; e.g., NG = now a,
x,y:(x,y)∈{(−2,−1),(−1,+1),(+1,+2)} −2,−1
NG = have since
+1,+2
Ciaramita and Johnson 4
Senseval 3/ACL’04 July 2004
Syntactic features (Charniak,2000)
• Governing elements under a phrase - G ; e.g., G = take S
1 1
• Governed elements under a phrase - G ; e.g., G = a NP, G = now NP
2 2 2
• Coordinates - OO; e.g., OO = newspaper
S1
S
NP VP .
DT NN CC ADVP DT NN AUX ADVP VP .
A newspaper and RB a bank have RB VBN PRT
now since taken RP
G2
over
G1
OO
Ciaramita and Johnson 5
Senseval 3/ACL’04 July 2004
Multiclass Perceptron (Crammer and Singer, 2003)
• Discriminant function: H(x; V) = arg maxk hv , xi
r=1 r
|Y(w)|×d
• Input: V ∈ IR , d ≈ 200, 000, initialized as V = 0
• Repeat T times - passes over training data or epochs
Multiclass Perceptron((x, y)n, V)
1 for i = 1 to i = n
2 do E = {r : hv , x i > hv , x i}
r i y i
3 if |E| > 0
4 then 1. τ = 1 for r = y
r
5 2. τ = 0 for r ∈/ E ∪ {y}
r
6 3. τ = − 1 for r ∈ E
r
|E|
7 for r = 1 to r = k
8 do v ← v + τ x ;
r r r i
Ciaramita and Johnson 6
Senseval 3/ACL’04 July 2004
Averaged perceptron classifier
• Perceptron’s output: V(0), . . . , V(n)
• V(i) is the weight matrix after the first i training items
• Final model: V = V(n)
• Averaged perceptron: (Collins, 2002)
– final model: V = 1 Pn V(i)
n i=1
– reduces the effect of over-training
Ciaramita and Johnson 7
Senseval 3/ACL’04 July 2004
Outline
• Pattern classification for WSD
– Features
– Flat multiclass perceptron
• Multi-component WSD
– Generating external training data
– Multi-component perceptron
• Experiments and results
Ciaramita and Johnson 8
Senseval 3/ACL’04 July 2004
Sparse data problem in WSD
• Thousands of word senses - 120,000 in Wordnet 2.0
• Very specific classes - 50% of noun synsets contain one noun
• Problem: training instances often too few for fine-grained se-
mantic distinctions
• Solution:
1. use the hierarchy of Wordnet to find similar word senses and
generate external training data for these senses
2. integrate task-specific and external data with perceptron
• Intuition - to classify an instance of the noun disk additional
knowledge about concepts such as other “audio” or “computer
memory” devices could be helpful
Ciaramita and Johnson 9
Senseval 3/ACL’04 July 2004
Finding neighbor senses
• disc = memory device for information storing
1
• disc = phonograph record
2
MEMORY_DEVICE
... RECORDING MAGNETIC_DISC
AUDIO_RECORDING
... DISC AUDIOTAPE magnetic_diskdisk FLOPPY HARD_DISK
DIGITAL_AUDIOTAPE
audiotape
floppy_disk diskette floppy hard_disk fixed_disk
disc record platter LP
digital_audiotape dat
l.p. lp
Ciaramita and Johnson 10