Difference between revisions of "SemEval-2012 Task 2 (State of the art)"
(Created page with "* [https://sites.google.com/site/semeval2012task2/home SemEval-2012 Task 2]: Measuring Degrees of Relational Similarity * graded relational similarity ratings across 79 relati...") |
|||
Line 5: | Line 5: | ||
== Table of results == | == Table of results == | ||
+ | |||
+ | * '''Listed in order of increasing [http://en.wikipedia.org/wiki/Spearman_rank_correlation Spearman's rho].''' | ||
+ | |||
{| border="1" cellpadding="5" cellspacing="1" | {| border="1" cellpadding="5" cellspacing="1" | ||
Line 10: | Line 13: | ||
! Algorithm | ! Algorithm | ||
! Reference | ! Reference | ||
− | ! MaxDiff | + | ! [http://en.wikipedia.org/wiki/MaxDiff MaxDiff] |
− | ! Spearman | + | ! [http://en.wikipedia.org/wiki/Spearman%27s_rank_correlation_coefficient Spearman] |
|- | |- | ||
| Random | | Random | ||
Line 76: | Line 79: | ||
== References == | == References == | ||
+ | |||
+ | Isaac I. Bejar, Roger Chaffin, and Susan E. Embretson. (1991). ''[http://www.amazon.com/Cognitive-Psychometric-Analysis-Analogical-Psychology/dp/0387973214/ Cognitive and Psychometric Analysis of Analogical Problem Solving]''. Springer-Verlag. | ||
David A. Jurgens, Saif M. Mohammad, Peter D. Turney, and Keith J. Holyoak. (2012). [http://ixa2.si.ehu.es/starsem/proc/pdf/STARSEM-SEMEVAL047.pdf SemEval-2012 Task 2: Measuring degrees of relational similarity]. In ''Proceedings of the First Joint Conference on Lexical and Computational Semantics (*SEM)'', pages 356-364, Montreal, Canada. | David A. Jurgens, Saif M. Mohammad, Peter D. Turney, and Keith J. Holyoak. (2012). [http://ixa2.si.ehu.es/starsem/proc/pdf/STARSEM-SEMEVAL047.pdf SemEval-2012 Task 2: Measuring degrees of relational similarity]. In ''Proceedings of the First Joint Conference on Lexical and Computational Semantics (*SEM)'', pages 356-364, Montreal, Canada. |
Revision as of 08:17, 15 October 2013
- SemEval-2012 Task 2: Measuring Degrees of Relational Similarity
- graded relational similarity ratings across 79 relation categories
- relation taxonomy of Bejar et al. (1991)
Table of results
- Listed in order of increasing Spearman's rho.
Algorithm | Reference | MaxDiff | Spearman |
---|---|---|---|
Random | Jurgens et al. (2012) | 31.2 | 0.018 |
BUAP | Tovar et al. (2012) | 31.7 | 0.014 |
Duluth-V2 | Pedersen (2012) | 31.1 | 0.038 |
Duluth-V1 | Pedersen (2012) | 31.5 | 0.039 |
Duluth-V0 | Pedersen (2012) | 32.4 | 0.050 |
PMI | Jurgens et al. (2012) | 33.9 | 0.112 |
UTD-SVM | Rink & Harabagiu (2012) | 34.7 | 0.116 |
UTD-NB | Rink & Harabagiu (2012) | 39.4 | 0.229 |
RNN-1600 | Mikolov et al. (2013) | 0.275 | |
UTD-LDA | Rink & Harabagiu (2013) | 0.334 | |
Com | Zhila et al. (2013) | 0.353 | |
SuperSim | Turney (2013) | 0.408 |
References
Isaac I. Bejar, Roger Chaffin, and Susan E. Embretson. (1991). Cognitive and Psychometric Analysis of Analogical Problem Solving. Springer-Verlag.
David A. Jurgens, Saif M. Mohammad, Peter D. Turney, and Keith J. Holyoak. (2012). SemEval-2012 Task 2: Measuring degrees of relational similarity. In Proceedings of the First Joint Conference on Lexical and Computational Semantics (*SEM), pages 356-364, Montreal, Canada.
Ted Pedersen (2012), Duluth: Measuring Degrees of Relational Similarity with the Gloss Vector Measure of Semantic Relatedness, In Proceedings of the First Joint Conference on Lexical and Computational Semantics (*SEM), Montreal, Canada, pp. 497-501.
Bryan Rink and Sanda Harabagiu (2012), UTD: Determining Relational Similarity Using Lexical Patterns, In Proceedings of the First Joint Conference on Lexical and Computational Semantics (*SEM), Montreal, Canada, pp. 413-418.
Mireya Tovar, J. Alejandro Reyes, Azucena Montes, Darnes Vilariño, David Pinto, and Saul León (2012), BUAP: A First Approximation to Relational Similarity Measuring, In Proceedings of the First Joint Conference on Lexical and Computational Semantics (*SEM), Montreal, Canada, pp. 502-505.