Difference between revisions of "Similar-Associated-Both Test Collection (State of the art)"
Jump to navigation
Jump to search
Line 6: | Line 6: | ||
* dataset is provided in the Appendix of Chiarello et al. (1990); also available on request from [http://www.apperceptual.com/ Peter Turney] | * dataset is provided in the Appendix of Chiarello et al. (1990); also available on request from [http://www.apperceptual.com/ Peter Turney] | ||
* see also: [[Similarity (State of the art)]], [[SimLex-999 (State of the art)]] | * see also: [[Similarity (State of the art)]], [[SimLex-999 (State of the art)]] | ||
+ | |||
+ | == Samples == | ||
+ | |||
+ | {| border="1" cellpadding="5" cellspacing="1" | ||
+ | |- | ||
+ | !Word pair | ||
+ | !Class label | ||
+ | |- | ||
+ | | table:bed | ||
+ | | similar | ||
+ | |- | ||
+ | | music:art | ||
+ | | similar | ||
+ | |- | ||
+ | | hair:fur | ||
+ | | similar | ||
+ | |- | ||
+ | | house:cabin | ||
+ | | similar | ||
+ | |- | ||
+ | | cradle:baby | ||
+ | | associated | ||
+ | |- | ||
+ | | mug:beer | ||
+ | | associated | ||
+ | |- | ||
+ | | camel:hump | ||
+ | | associated | ||
+ | |- | ||
+ | | cheese:mouse | ||
+ | | associated | ||
+ | |- | ||
+ | | ale:beer | ||
+ | | both | ||
+ | |- | ||
+ | | uncle:aunt | ||
+ | | both | ||
+ | |- | ||
+ | | pepper:salt | ||
+ | | both | ||
+ | |- | ||
+ | | frown:smile | ||
+ | | both | ||
+ | |} | ||
== Table of results == | == Table of results == |
Revision as of 12:29, 28 June 2015
- the contrast between taxonomical similarity (co-hyponymy) and association (co-occurrence)
- 144 word pairs labeled similar-only, associated-only, or similar+associated
- 48 pairs in each of the three classes
- test collection created by Chiarello et al. (1990)
- Chiarello et al. (1990) used the dataset in human priming experiments; they did not measure classification accuracy
- dataset is provided in the Appendix of Chiarello et al. (1990); also available on request from Peter Turney
- see also: Similarity (State of the art), SimLex-999 (State of the art)
Samples
Word pair | Class label |
---|---|
table:bed | similar |
music:art | similar |
hair:fur | similar |
house:cabin | similar |
cradle:baby | associated |
mug:beer | associated |
camel:hump | associated |
cheese:mouse | associated |
ale:beer | both |
uncle:aunt | both |
pepper:salt | both |
frown:smile | both |
Table of results
Algorithm | Reference | Accuracy | 95% confidence |
---|---|---|---|
Dual-Space | Turney (2012) | 61.1% | 52.6-69.1% |
PairClass | Turney (2008) | 77.1% | 70.1-84.3% |
Notes
- 95% confidence = confidence interval calculated using the Binomial Exact Test
References
Chiarello, C., Burgess, C., Richards, L., & Pollock, A. (1990). Semantic and associative priming in the cerebral hemispheres: Some words do, some words don't . . . sometimes, some places. Brain and Language, 38, 75{104.
Turney, P.D. (2008). A uniform approach to analogies, synonyms, antonyms, and associations. Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), Manchester, UK, pp. 905-912.
Turney, P.D. (2012). Domain and function: A dual-space model of semantic relations and compositions, Journal of Artificial Intelligence Research (JAIR), 44, 533-585.