Difference between revisions of "WordSimilarity-353 Test Collection (State of the art)"
Jump to navigation
Jump to search
Line 8: | Line 8: | ||
* introduced by [http://www.cs.technion.ac.il/~gabr/papers/tois_context.pdf Finkelstein et al. (2002)] | * introduced by [http://www.cs.technion.ac.il/~gabr/papers/tois_context.pdf Finkelstein et al. (2002)] | ||
* subsequently used by many other researchers | * subsequently used by many other researchers | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
Line 28: | Line 19: | ||
! Algorithm | ! Algorithm | ||
! Reference | ! Reference | ||
+ | ! Type | ||
! Spearman's rho | ! Spearman's rho | ||
|- | |- | ||
| HSMN+csmRNN | | HSMN+csmRNN | ||
| Luong et al. (2013) | | Luong et al. (2013) | ||
− | | 0. | + | | Corpus-based |
+ | | 0.65 | ||
+ | |- | ||
+ | | ESA-Wikipedia | ||
+ | | Gabrilovich and Markovitch | ||
+ | | Hybrid | ||
+ | | 0.75 | ||
|} | |} | ||
+ | |||
== References == | == References == | ||
− | Luong, Minh-Thang, Richard Socher, and Christopher D. Manning. (2013) [http://nlp.stanford.edu/~lmthang/data/papers/conll13_morpho.pdf Better word representations with recursive neural networks for morphology]. CoNLL-2013: 104. | + | Gabrilovich, Evgeniy, and Shaul Markovitch. (2007). [http://www.cs.technion.ac.il/~gabr/papers/ijcai-2007-sim.pdf Computing Semantic Relatedness Using Wikipedia-based Explicit Semantic Analysis]. In IJCAI, vol. 7, pp. 1606-1611. |
+ | |||
+ | Luong, Minh-Thang, Richard Socher, and Christopher D. Manning. (2013). [http://nlp.stanford.edu/~lmthang/data/papers/conll13_morpho.pdf Better word representations with recursive neural networks for morphology]. CoNLL-2013: 104. | ||
Revision as of 12:48, 3 October 2013
- WordSimilarity-353 Test Collection
- contains two sets of English word pairs along with human-assigned similarity judgements
- first set (set1) contains 153 word pairs along with their similarity scores assigned by 13 subjects
- second set (set2) contains 200 word pairs with similarity assessed by 16 subjects
- WordSimilarity-353 dataset is available here
- performance is measured by Spearman's rank correlation coefficient
- introduced by Finkelstein et al. (2002)
- subsequently used by many other researchers
Table of results
- Listed in order of increasing Spearman's rho.
Algorithm | Reference | Type | Spearman's rho |
---|---|---|---|
HSMN+csmRNN | Luong et al. (2013) | Corpus-based | 0.65 |
ESA-Wikipedia | Gabrilovich and Markovitch | Hybrid | 0.75 |
References
Gabrilovich, Evgeniy, and Shaul Markovitch. (2007). Computing Semantic Relatedness Using Wikipedia-based Explicit Semantic Analysis. In IJCAI, vol. 7, pp. 1606-1611.
Luong, Minh-Thang, Richard Socher, and Christopher D. Manning. (2013). Better word representations with recursive neural networks for morphology. CoNLL-2013: 104.