WordSimilarity-353 Test Collection (State of the art)

• WordSimilarity-353 Test Collection
• contains two sets of English word pairs along with human-assigned similarity judgements
• first set (set1) contains 153 word pairs along with their similarity scores assigned by 13 subjects
• second set (set2) contains 200 word pairs with similarity assessed by 16 subjects
• WordSimilarity-353 dataset is available here
• performance is measured by Spearman's rank correlation coefficient
• introduced by Finkelstein et al. (2002)
• subsequently used by many other researchers

Table of results

• Listed in order of increasing Spearman's rho.

Algorithm	Reference	Type	Spearman's rho
HSMN+csmRNN	Luong et al. (2013)	Corpus-based	0.65
ESA-Wikipedia	Gabrilovich and Markovitch (2007)	Hybrid	0.75

References

Gabrilovich, Evgeniy, and Shaul Markovitch. (2007). Computing Semantic Relatedness Using Wikipedia-based Explicit Semantic Analysis. In IJCAI, vol. 7, pp. 1606-1611.

Luong, Minh-Thang, Richard Socher, and Christopher D. Manning. (2013). Better word representations with recursive neural networks for morphology. CoNLL-2013: 104.

See also

• Attributional and Relational Similarity (State of the art)
• ESL Synonym Questions
• SAT Analogy Questions
• TOEFL Synonym Questions
• State of the art