SemEval-2012 Task 2 (State of the art)
- SemEval-2012 Task 2: Measuring Degrees of Relational Similarity
- graded relational similarity ratings across 79 relation categories
- relation taxonomy of Bejar et al. (1991)
Table of results
- Listed in order of increasing Spearman's rho.
| Algorithm | Reference | MaxDiff | Spearman |
|---|---|---|---|
| Random | Jurgens et al. (2012) | 31.2 | 0.018 |
| BUAP | Tovar et al. (2012) | 31.7 | 0.014 |
| Duluth-V2 | Pedersen (2012) | 31.1 | 0.038 |
| Duluth-V1 | Pedersen (2012) | 31.5 | 0.039 |
| Duluth-V0 | Pedersen (2012) | 32.4 | 0.050 |
| PMI | Jurgens et al. (2012) | 33.9 | 0.112 |
| UTD-SVM | Rink & Harabagiu (2012) | 34.7 | 0.116 |
| UTD-NB | Rink & Harabagiu (2012) | 39.4 | 0.229 |
| RNN-1600 | Mikolov et al. (2013) | 0.275 | |
| UTD-LDA | Rink & Harabagiu (2013) | 0.334 | |
| Com | Zhila et al. (2013) | 0.353 | |
| SuperSim | Turney (2013) | 0.408 |
References
Isaac I. Bejar, Roger Chaffin, and Susan E. Embretson. (1991). Cognitive and Psychometric Analysis of Analogical Problem Solving. Springer-Verlag.
David A. Jurgens, Saif M. Mohammad, Peter D. Turney, and Keith J. Holyoak. (2012). SemEval-2012 Task 2: Measuring degrees of relational similarity. In Proceedings of the First Joint Conference on Lexical and Computational Semantics (*SEM), pages 356-364, Montreal, Canada.
Ted Pedersen (2012), Duluth: Measuring Degrees of Relational Similarity with the Gloss Vector Measure of Semantic Relatedness, In Proceedings of the First Joint Conference on Lexical and Computational Semantics (*SEM), Montreal, Canada, pp. 497-501.
Bryan Rink and Sanda Harabagiu (2012), UTD: Determining Relational Similarity Using Lexical Patterns, In Proceedings of the First Joint Conference on Lexical and Computational Semantics (*SEM), Montreal, Canada, pp. 413-418.
Mireya Tovar, J. Alejandro Reyes, Azucena Montes, Darnes Vilariño, David Pinto, and Saul León (2012), BUAP: A First Approximation to Relational Similarity Measuring, In Proceedings of the First Joint Conference on Lexical and Computational Semantics (*SEM), Montreal, Canada, pp. 502-505.