SimLex-999 (State of the art) - Revision history

Doboandris at 00:34, 16 September 2019

2019-09-16T00:34:47Z

← Older revision		Revision as of 00:34, 16 September 2019
Line 19:		Line 19:
	\|-		\|-
	\| Do19-hybrid		\| Do19-hybrid
	\| Dobó (2019)<ref name=dobo19>Dobó, A. (2019). [http://doktori.bibl.u-szeged.hu/10120/1/AndrasDoboThesis2019.pdf A comprehensive analysis of the parameters in the creation and comparison of feature vectors in distributional semantic models for multiple languages]. University of Szeged.</ref>		\| Dobó (2019)<ref name=dobo19>Dobó, A. (2019). [http://doktori.bibl.u-szeged.hu/10120/1/AndrasDoboThesis2019.pdf A comprehensive analysis of the parameters in the creation and comparison of feature vectors in distributional semantic models for multiple languages]. University of Szeged. [https://github.com/doboandras/dsm-parameter-analysis GitHub repository]</ref>
	\| Dobó (2019)<ref name=dobo19/>		\| Dobó (2019)<ref name=dobo19/>
	\| Hybrid \|\| 0.621 \|\| 0.481		\| Hybrid \|\| 0.621 \|\| 0.481

Doboandris at 23:00, 12 August 2019

2019-08-12T23:00:55Z

Show changes

Roys02 at 16:21, 4 March 2016

2016-03-04T16:21:52Z

@@ Line 47: / Line 47: @@
 | Hill et al. (2014b)<ref name=rnnenc/>
 | Distributional, multilingual || 0.52 || -
 |}

Pdturney at 18:45, 28 June 2015

2015-06-28T18:45:24Z

← Older revision		Revision as of 18:45, 28 June 2015
Line 2:		Line 2:

	See also: [[Similarity (State of the art)]], [[Similar-Associated-Both Test Collection (State of the art)]].		See also: [[Similarity (State of the art)]], [[Similar-Associated-Both Test Collection (State of the art)]].


	{\|		{\|
Line 47:		Line 48:
	\| Distributional, multilingual \|\| 0.52 \|\| -		\| Distributional, multilingual \|\| 0.52 \|\| -
	\|}		\|}


	== References ==		== References ==

Pdturney at 18:43, 28 June 2015

2015-06-28T18:43:35Z

← Older revision		Revision as of 18:43, 28 June 2015
Line 1:		Line 1:
	[http://www.cl.cam.ac.uk/~fh295/simlex.html SimLex-999] aims at a cleaner benchmark of similarity (but not relatedness). Pairs of words were chosen to represent different ranges of similarity and with either high or low association. Subjects were instructed to differentiate between similarity and relatedness and rate regarding the former only.		[http://www.cl.cam.ac.uk/~fh295/simlex.html SimLex-999] aims at a cleaner benchmark of similarity (but not relatedness). Pairs of words were chosen to represent different ranges of similarity and with either high or low association. Subjects were instructed to differentiate between similarity and relatedness and rate regarding the former only.

	See also [[Similar-Associated-Both Test Collection (State of the art)]].		See also: [[Similarity (State of the art)]], [[Similar-Associated-Both Test Collection (State of the art)]].

	{\|		{\|

Pdturney at 18:40, 28 June 2015

2015-06-28T18:40:45Z

← Older revision		Revision as of 18:40, 28 June 2015
Line 1:		Line 1:
	[http://www.cl.cam.ac.uk/~fh295/simlex.html SimLex-999] aims at a cleaner benchmark of similarity (but not relatedness). Pairs of words were chosen to represent different ranges of similarity and with either high or low association. Subjects were instructed to differentiate between similarity and relatedness and rate regarding the former only.		[http://www.cl.cam.ac.uk/~fh295/simlex.html SimLex-999] aims at a cleaner benchmark of similarity (but not relatedness). Pairs of words were chosen to represent different ranges of similarity and with either high or low association. Subjects were instructed to differentiate between similarity and relatedness and rate regarding the former only.

			See also [[Similar-Associated-Both Test Collection (State of the art)]].

	{\|		{\|

Minhle: Hill et al. (2014) --> 2014a & 2014b

2015-06-25T12:58:49Z

Hill et al. (2014) --> 2014a & 2014b

← Older revision		Revision as of 12:58, 25 June 2015
Line 7:		Line 7:
	\| Neural language model		\| Neural language model
	\| Collobert & Weston (2008)<ref>R. Collobert and J. Weston. 2008. A unified architecture for natural language pro- cessing: Deep neural networks with multitask learning. In International Conference on Machine Learn- ing, ICML.</ref>		\| Collobert & Weston (2008)<ref>R. Collobert and J. Weston. 2008. A unified architecture for natural language pro- cessing: Deep neural networks with multitask learning. In International Conference on Machine Learn- ing, ICML.</ref>
	\| Hill et al. (~~2014~~)<ref name=simlex/>		\| Hill et al. (2014a)<ref name=simlex/>
	\| Distributional \|\| 0.268 \|\| - \|\| Trained on Wikipedia		\| Distributional \|\| 0.268 \|\| - \|\| Trained on Wikipedia
	\|-		\|-
	\| Neural language model with global context		\| Neural language model with global context
	\| Huang et al. (2012)<ref>Eric H Huang, Richard Socher, Christopher D Manning, and Andrew Y Ng. 2012. Improving word representations via global context and multiple word prototypes. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers-Volume 1, pages 873–882. Association for Computational Linguistics.</ref>		\| Huang et al. (2012)<ref>Eric H Huang, Richard Socher, Christopher D Manning, and Andrew Y Ng. 2012. Improving word representations via global context and multiple word prototypes. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers-Volume 1, pages 873–882. Association for Computational Linguistics.</ref>
	\| Hill et al. (~~2014~~)<ref name=simlex/>		\| Hill et al. (2014a)<ref name=simlex/>
	\| Distributional \|\| 0.098 \|\| - \|\| Trained on Wikipedia		\| Distributional \|\| 0.098 \|\| - \|\| Trained on Wikipedia
	\|-		\|-
	\| Word2vec		\| Word2vec
	\| Mikolov et al. (2013)<ref>Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. In Proceedings of International Conference of Learning Representations, Scottsdale, Arizona, USA.</ref>		\| Mikolov et al. (2013)<ref>Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. In Proceedings of International Conference of Learning Representations, Scottsdale, Arizona, USA.</ref>
	\| Hill et al. (~~2014~~)<ref name=simlex>Hill, F., Reichart, R., & Korhonen, A. (~~2014~~). SimLex-999: Evaluating Semantic Models with (Genuine) Similarity Estimation. Computation and Language.</ref>		\| Hill et al. (2014a)<ref name=simlex>Hill, F., Reichart, R., & Korhonen, A. (2014a). SimLex-999: Evaluating Semantic Models with (Genuine) Similarity Estimation. Computation and Language.</ref>
	\| Distributional \|\| 0.414 \|\| - \|\| Trained on Wikipedia		\| Distributional \|\| 0.414 \|\| - \|\| Trained on Wikipedia
	\|-		\|-
Line 41:		Line 41:
	\|-		\|-
	\| RNNenc		\| RNNenc
	\| Hill et al. (~~2014~~)<ref name=rnnenc>Hill, F., Cho, K., Jean, S., Devin, C., & Bengio, Y. (~~2014~~). Not All Neural Embeddings are Born Equal, 1–5.</ref>		\| Hill et al. (2014b)<ref name=rnnenc>Hill, F., Cho, K., Jean, S., Devin, C., & Bengio, Y. (2014b). Not All Neural Embeddings are Born Equal, 1–5.</ref>
	\| Hill et al. (~~2014~~)<ref name=rnnenc/>		\| Hill et al. (2014b)<ref name=rnnenc/>
	\| Distributional, multilingual \|\| 0.52 \|\| -		\| Distributional, multilingual \|\| 0.52 \|\| -
	\|}		\|}

Minhle: Created page with "[http://www.cl.cam.ac.uk/~fh295/simlex.html SimLex-999] aims at a cleaner benchmark of similarity (but not relatedness). Pairs of words were chosen to represent different rang..."

2015-06-25T12:47:04Z

Created page with "[http://www.cl.cam.ac.uk/~fh295/simlex.html SimLex-999] aims at a cleaner benchmark of similarity (but not relatedness). Pairs of words were chosen to represent different rang..."

New page

[http://www.cl.cam.ac.uk/~fh295/simlex.html SimLex-999] aims at a cleaner benchmark of similarity (but not relatedness). Pairs of words were chosen to represent different ranges of similarity and with either high or low association. Subjects were instructed to differentiate between similarity and relatedness and rate regarding the former only.

{|
|-
! Algorithm !! Reference for algorithm !! Reference for reported results !! Type !! Spearman's rho !! Pearson's r !! Notes
|-
| Neural language model
| Collobert & Weston (2008)<ref>R. Collobert and J. Weston. 2008. A unified architecture for natural language pro- cessing: Deep neural networks with multitask learning. In International Conference on Machine Learn- ing, ICML.</ref>
| Hill et al. (2014)<ref name=simlex/>
| Distributional || 0.268 || - || Trained on Wikipedia
|-
| Neural language model with global context
| Huang et al. (2012)<ref>Eric H Huang, Richard Socher, Christopher D Manning, and Andrew Y Ng. 2012. Improving word representations via global context and multiple word prototypes. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers-Volume 1, pages 873–882. Association for Computational Linguistics.</ref>
| Hill et al. (2014)<ref name=simlex/>
| Distributional || 0.098 || - || Trained on Wikipedia
|-
| Word2vec
| Mikolov et al. (2013)<ref>Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. In Proceedings of International Conference of Learning Representations, Scottsdale, Arizona, USA.</ref>
| Hill et al. (2014)<ref name=simlex>Hill, F., Reichart, R., & Korhonen, A. (2014). SimLex-999: Evaluating Semantic Models with (Genuine) Similarity Estimation. Computation and Language.</ref>
| Distributional || 0.414 || - || Trained on Wikipedia
|-
| Lesk
|
| Banjade et al. (2015)<ref name=lemontea>Banjade, R., Maharjan, N., Niraula, N., Rus, V., & Gautam, D. (2015). Lemon and Tea Are Not Similar: Measuring Word-to-Word Similarity by Combining Different Methods. Computational Linguistics and Intelligent Text Processing, 9041, 335–346. doi:10.1007/978-3-319-18111-0_25</ref>
| || 0.404 || 0.347
|-
| UMBC
| Han et al. (2013)<ref>Han, L., Kashyap, A., Finin, T., Mayfield, J., Weese, J.: UMBC EBIQUITY-CORE: Semantic textual similarity systems. In: Proceedings of the Second Joint Conference on Lexical and Computational Semantics, vol. 1, pp. 44–52 (2013)</ref>
| Banjade et al. (2015)<ref name=lemontea/>
| || 0.558 || 0.557 || without using POS information
|-
| SVR4
| Banjade et al. (2015)<ref name=lemontea/>
| Banjade et al. (2015)<ref name=lemontea/>
| Combined || 0.642 || 0.658
|-
| ESA
|
| Banjade et al. (2015)<ref name=lemontea/>
| || 0.271 || 0.145
|-
| RNNenc
| Hill et al. (2014)<ref name=rnnenc>Hill, F., Cho, K., Jean, S., Devin, C., & Bengio, Y. (2014). Not All Neural Embeddings are Born Equal, 1–5.</ref>
| Hill et al. (2014)<ref name=rnnenc/>
| Distributional, multilingual || 0.52 || -
|}

== References ==
<references/>

[[Category:State of the art]]
[[Category:Similarity]]