ACL Wiki - User contributions [en]

User:Fnielsen

2020-06-16T15:40:17Z

Fnielsen: Update URL to Scholia

Associate Professor at Technical University of Denmark interest in natural language processing, machine learning, Wikipedia, Wikidata and other social collaborative platforms.
My homepage is http://people.compute.dtu.dk/faan/

I am currently working on Wikidata as a knowledge graph and ressource and have implement the Scholia web service that present information from Wikidata. It is running from https://scholia.toolforge.org

WordSimilarity-353 Test Collection (State of the art)

2020-06-16T15:37:06Z

Fnielsen: Fix paper link

* [http://www.cs.technion.ac.il/~gabr/resources/data/wordsim353/ WordSimilarity-353 Test Collection]
* contains two sets of English word pairs along with human-assigned similarity judgements
* first set (set1) contains 153 word pairs along with their similarity scores assigned by 13 subjects
* second set (set2) contains 200 word pairs with similarity assessed by 16 subjects
* WordSimilarity-353 dataset is available [http://www.cs.technion.ac.il/~gabr/resources/data/wordsim353/ here]
* performance is measured by [http://en.wikipedia.org/wiki/Spearman_rank_correlation Spearman's rank correlation coefficient]
* introduced by [http://gabrilovich.com/papers/context_search.pdf Finkelstein et al. (2002)]
* subsequently used by many other researchers
* [https://www.wikidata.org/wiki/Q31845205 Wikidata] and [https://scholia.toolforge.org/use/Q31845205 Scholia]
* see also: [[Similarity (State of the art)]]

== Table of results ==

* '''Listed in order of increasing [http://en.wikipedia.org/wiki/Spearman_rank_correlation Spearman's rho].'''

{| border="1" cellpadding="5" cellspacing="1"
|-
! Algorithm
! Reference for algorithm
! Reference for reported results
! Type
! Spearman's rho
! Pearson's r
|-
| L&C
| Leacock and Chodorow (1998)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.302
| 0.356
|-
| WNE
| Jarmasz (2003)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.305
| 0.271
|-
| J&C
| Jiang and Conrath 1997
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.318
| 0.354
|-
| L&C
| Leacock and Chodorow (1998)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.348
| 0.341
|-
| H&S
| Hirst and St-Onge (1998)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.302
| 0.356
|-
| Lin
| Lin (1998)
| Hassan and Mihalcea (2011)
| Corpus-based
| 0.348
| 0.357
|-
| Resnik
| Resnik (1995)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.353
| 0.365
|-
| ROGET
| Jarmasz (2003)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.415
| 0.536
|-
| C&W
| Collobert and Weston (2008)
| Collobert and Weston (2008)
| Corpus-based
| 0.5
| N/A
|-
| WikiRelate
| Strube and Ponzetto (2006)
| Strube and Ponzetto (2006)
| Corpus-based
| N/A
| 0.48
|-
| Do19-corpus
| Dobó (2019)
| Dobó (2019)
| Corpus-based
| 0.579
| 0.577
|-
| LSA
| Landauer et al. (1997)
| Hassan and Mihalcea (2011)
| Corpus-based
| 0.581
| 0.492
|-
| LSA
| Landauer et al. (1997)
| Hassan and Mihalcea (2011)
| Corpus-based
| 0.581
| 0.563
|-
| simVB+simWN
| Finkelstein et al. (2002)
| Finkelstein et al. (2002)
| Hybrid
| N/A
| 0.55
|-
| SSA
| Hassan and Mihalcea (2011)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.622
| 0.629
|-
| HSMN+csmRNN
| Luong et al. (2013)
| Luong et al. (2013)
| Corpus-based
| 0.65
| N/A
|-
| Pe14
| Pennington (2014)
| Dobó (2019)
| Corpus-based
| 0.706
| 0.705
|-
| Multi-prototype
| Huang et al. (2012)
| Huang et al. (2012)
| Corpus-based
| 0.71
| N/A
|-
| Multi-lingual SSA
| Hassan et al. (2011)
| Hassan et al. (2011)
| Corpus-based
| 0.713
| 0.674
|-
| DSG
| Song et al. (2018)
| Song et al. (2018)
| Corpus-based
| 0.726
| N/A
|-
| Sa18
| Salle et al. (2018)
| Dobó (2019)
| Corpus-based
| 0.733
| 0.704
|-
| ESA
| Gabrilovich and Markovitch (2007)
| Gabrilovich and Markovitch (2007)
| Corpus-based
| 0.748
| 0.503
|-
| Do19-hybrid
| Dobó (2019)
| Dobó (2019)
| Hybrid
| 0.795
| 0.276
|-
| TSA
| Radinsky et al. (2011)
| Radinsky et al. (2011)
| Hybrid
| 0.80
| N/A
|-
| CLEAR
| Halawi et al. (2012)
| Halawi et al. (2012)
| Corpus-based
| 0.81
| N/A
|-
| Y&Q
| Yih and Qazvinian (2012)
| Yih and Qazvinian (2012)
| Hybrid
| 0.81
| N/A
|-
| ConceptNet Numberbatch
| Speer et al. (2017)
| Speer et al. (2017)
| Hybrid
| 0.828
| N/A
|}

== References ==

* '''Listed in alphabetical order.'''

Dobó, A. (2019). [http://doktori.bibl.u-szeged.hu/10120/1/AndrasDoboThesis2019.pdf A comprehensive analysis of the parameters in the creation and comparison of feature vectors in distributional semantic models for multiple languages]. University of Szeged. [https://github.com/doboandras/dsm-parameter-analysis GitHub repository]

Finkelstein, Lev, Evgeniy Gabrilovich, Yossi Matias, Ehud Rivlin, Zach Solan, Gadi Wolfman, and Eytan Ruppin. (2002) [http://www.cs.technion.ac.il/~gabr/papers/tois_context.pdf Placing Search in Context: The Concept Revisited]. ACM Transactions on Information Systems, 20(1):116-131.

Gabrilovich, Evgeniy, and Shaul Markovitch, [http://www.cs.technion.ac.il/~gabr/papers/ijcai-2007-sim.pdf Computing Semantic Relatedness using Wikipedia-based Explicit Semantic Analysis], Proceedings of The 20th International Joint Conference on Artificial Intelligence (IJCAI), Hyderabad, India, 2007.

Halawi, Guy, Gideon Dror, Evgeniy Gabrilovich, and Yehuda Koren. (2012). [http://gabrilovich.com/publications/papers/Halawi2012LSL.pdf Large-scale learning of word relatedness with constraints]. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 1406-1414. ACM.

Hassan, Samer, and Rada Mihalcea: [http://www.aaai.org/ocs/index.php/AAAI/AAAI11/paper/download/3616/3972/ Semantic Relatedness Using Salient Semantic Analysis]. AAAI 2011

Hirst, Graeme and David St-Onge. Lexical chains as representations of context for the detection and correction of malapropisms. In Christiane Fellbaum, editor, WordNet: An Electronic Lexical Database. The MIT Press, Cambridge, MA, pages 305–332, 1998.

Huang, Eric H., Richard Socher, Christopher D. Manning, and Andrew Y. Ng. 2012. Improving word representations via global context and multiple word prototypes. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers - Volume 1 (ACL '12), Vol. 1. Association for Computational Linguistics, Stroudsburg, PA, USA, 873-882.

Islam, A., and Inkpen, D. 2006. [http://www.site.uottawa.ca/~mdislam/publications/LREC_06_242.pdf Second order co-occurrence pmi for determining the semantic similarity of words]. Proceedings of the International Conference on Language Resources and Evaluation (LREC 2006) 1033–1038.

Jarmasz, M. 2003. [http://www.arxiv.org/pdf/1204.0140 Roget’s thesaurus as a Lexical Resource for Natural Language Processing]. Ph.D. Dissertation, Ottawa Carleton Institute for Computer Science, School of Information Technology and Engineering, University of Ottawa.

Jiang, Jay J. and David W. Conrath. Semantic similarity based on corpus statistics and lexical taxonomy. In Proceedings of International Conference on Research in Computational Linguistics (ROCLING X), Taiwan, pages 19–33, 1997.

Landauer, T. K.; L, T. K.; Laham, D.; Rehder, B.; and Schreiner, M. E. 1997. How well can passage meaning be derived without using word order? a comparison of latent semantic analysis and humans.

Leacock, Claudia and Martin Chodorow. Combining local context and WordNet similarity for word sense identification. In Christiane Fellbaum, editor, WordNet: An Electronic Lexical Database. The MIT Press, Cambridge, MA, pages 265–283, 1998.

Lin, Dekang. An information-theoretic definition of similarity. In Proceedings of the 15th International Conference on Machine Learning, Madison,WI, pages 296–304, 1998.

Luong, Minh-Thang, Richard Socher, and Christopher D. Manning. (2013). [http://nlp.stanford.edu/~lmthang/data/papers/conll13_morpho.pdf Better word representations with recursive neural networks for morphology]. CoNLL-2013: 104.

Pennington, J., Socher, R., and Manning, C. (2014). [https://www.aclweb.org/anthology/D14-1162 Glove: Global vectors for word representation]. ''EMNLP 2014'', pp. 1532-1543.

Pilehvar, M.T., D. Jurgens and R. Navigli. [http://wwwusers.di.uniroma1.it/~navigli/pubs/ACL_2013_Pilehvar_Jurgens_Navigli.pdf Align, Disambiguate and Walk: A Unified Approach for Measuring Semantic Similarity]. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (ACL 2013), Sofia, Bulgaria, August 4-9, 2013, pp. 1341-1351.

Radinsky, Kira, Eugene Agichtein, Evgeniy Gabrilovich, and Shaul Markovitch. (2011). [http://gabrilovich.com/publications/papers/Radinsky2011WTS.pdf A word at a time: computing word relatedness using temporal semantic analysis]. In Proceedings of the 20th international conference on World wide web, pp. 337-346. ACM.

Resnik, Philip. Using information content to evaluate semantic similarity. In Proceedings of the 14th International Joint Conference on Artificial Intelligence, pages 448–453, Montreal, Canada, 1995.

Salle A., Idiart M., and Villavicencio A. (2018) [https://github.com/alexandres/lexvec/blob/master/README.md LexVec]

Song, Yan, Shuming Shi, Jing Li, and Haisong Zhang. 2018. [https://www.aclweb.org/anthology/N18-2028.pdf Directional skip-gram: Explicitly distinguish-ing left and right context for word embeddings]. NAACL-2018, pages 175–180.

Speer, Rob, Joshua Chin and Catherine Havasi. (2017). [http://aaai.org/ocs/index.php/AAAI/AAAI17/paper/view/14972 ConceptNet 5.5: An Open Multilingual Graph of General Knowledge]. Proceedings of The 31st AAAI Conference on Artificial Intelligence, San Francisco, CA.

Strube, Michael and Simone Paolo Ponzetto. (2006). [http://www.aaai.org/Papers/AAAI/2006/AAAI06-223.pdf WikiRelate! Computing Semantic Relatedness Using Wikipedia]. Proceedings of The 21st National Conference on Artificial Intelligence (AAAI), Boston, MA.

Yih, W. and Qazvinian, V. (2012). [http://aclweb.org/anthology/N/N12/N12-1077.pdf Measuring Word Relatedness Using Heterogeneous Vector Space Models]. Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2012).

[[Category:State of the art]]
[[Category:Similarity]]

WordSimilarity-353 Test Collection (State of the art)

2020-06-16T15:32:08Z

Fnielsen: Update URL to Scholia

* [http://www.cs.technion.ac.il/~gabr/resources/data/wordsim353/ WordSimilarity-353 Test Collection]
* contains two sets of English word pairs along with human-assigned similarity judgements
* first set (set1) contains 153 word pairs along with their similarity scores assigned by 13 subjects
* second set (set2) contains 200 word pairs with similarity assessed by 16 subjects
* WordSimilarity-353 dataset is available [http://www.cs.technion.ac.il/~gabr/resources/data/wordsim353/ here]
* performance is measured by [http://en.wikipedia.org/wiki/Spearman_rank_correlation Spearman's rank correlation coefficient]
* introduced by [http://www.cs.technion.ac.il/~gabr/papers/tois_context.pdf Finkelstein et al. (2002)]
* subsequently used by many other researchers
* [https://www.wikidata.org/wiki/Q31845205 Wikidata] and [https://scholia.toolforge.org/use/Q31845205 Scholia]
* see also: [[Similarity (State of the art)]]

== Table of results ==

* '''Listed in order of increasing [http://en.wikipedia.org/wiki/Spearman_rank_correlation Spearman's rho].'''

{| border="1" cellpadding="5" cellspacing="1"
|-
! Algorithm
! Reference for algorithm
! Reference for reported results
! Type
! Spearman's rho
! Pearson's r
|-
| L&C
| Leacock and Chodorow (1998)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.302
| 0.356
|-
| WNE
| Jarmasz (2003)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.305
| 0.271
|-
| J&C
| Jiang and Conrath 1997
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.318
| 0.354
|-
| L&C
| Leacock and Chodorow (1998)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.348
| 0.341
|-
| H&S
| Hirst and St-Onge (1998)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.302
| 0.356
|-
| Lin
| Lin (1998)
| Hassan and Mihalcea (2011)
| Corpus-based
| 0.348
| 0.357
|-
| Resnik
| Resnik (1995)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.353
| 0.365
|-
| ROGET
| Jarmasz (2003)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.415
| 0.536
|-
| C&W
| Collobert and Weston (2008)
| Collobert and Weston (2008)
| Corpus-based
| 0.5
| N/A
|-
| WikiRelate
| Strube and Ponzetto (2006)
| Strube and Ponzetto (2006)
| Corpus-based
| N/A
| 0.48
|-
| Do19-corpus
| Dobó (2019)
| Dobó (2019)
| Corpus-based
| 0.579
| 0.577
|-
| LSA
| Landauer et al. (1997)
| Hassan and Mihalcea (2011)
| Corpus-based
| 0.581
| 0.492
|-
| LSA
| Landauer et al. (1997)
| Hassan and Mihalcea (2011)
| Corpus-based
| 0.581
| 0.563
|-
| simVB+simWN
| Finkelstein et al. (2002)
| Finkelstein et al. (2002)
| Hybrid
| N/A
| 0.55
|-
| SSA
| Hassan and Mihalcea (2011)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.622
| 0.629
|-
| HSMN+csmRNN
| Luong et al. (2013)
| Luong et al. (2013)
| Corpus-based
| 0.65
| N/A
|-
| Pe14
| Pennington (2014)
| Dobó (2019)
| Corpus-based
| 0.706
| 0.705
|-
| Multi-prototype
| Huang et al. (2012)
| Huang et al. (2012)
| Corpus-based
| 0.71
| N/A
|-
| Multi-lingual SSA
| Hassan et al. (2011)
| Hassan et al. (2011)
| Corpus-based
| 0.713
| 0.674
|-
| DSG
| Song et al. (2018)
| Song et al. (2018)
| Corpus-based
| 0.726
| N/A
|-
| Sa18
| Salle et al. (2018)
| Dobó (2019)
| Corpus-based
| 0.733
| 0.704
|-
| ESA
| Gabrilovich and Markovitch (2007)
| Gabrilovich and Markovitch (2007)
| Corpus-based
| 0.748
| 0.503
|-
| Do19-hybrid
| Dobó (2019)
| Dobó (2019)
| Hybrid
| 0.795
| 0.276
|-
| TSA
| Radinsky et al. (2011)
| Radinsky et al. (2011)
| Hybrid
| 0.80
| N/A
|-
| CLEAR
| Halawi et al. (2012)
| Halawi et al. (2012)
| Corpus-based
| 0.81
| N/A
|-
| Y&Q
| Yih and Qazvinian (2012)
| Yih and Qazvinian (2012)
| Hybrid
| 0.81
| N/A
|-
| ConceptNet Numberbatch
| Speer et al. (2017)
| Speer et al. (2017)
| Hybrid
| 0.828
| N/A
|}

== References ==

* '''Listed in alphabetical order.'''

Dobó, A. (2019). [http://doktori.bibl.u-szeged.hu/10120/1/AndrasDoboThesis2019.pdf A comprehensive analysis of the parameters in the creation and comparison of feature vectors in distributional semantic models for multiple languages]. University of Szeged. [https://github.com/doboandras/dsm-parameter-analysis GitHub repository]

Finkelstein, Lev, Evgeniy Gabrilovich, Yossi Matias, Ehud Rivlin, Zach Solan, Gadi Wolfman, and Eytan Ruppin. (2002) [http://www.cs.technion.ac.il/~gabr/papers/tois_context.pdf Placing Search in Context: The Concept Revisited]. ACM Transactions on Information Systems, 20(1):116-131.

Gabrilovich, Evgeniy, and Shaul Markovitch, [http://www.cs.technion.ac.il/~gabr/papers/ijcai-2007-sim.pdf Computing Semantic Relatedness using Wikipedia-based Explicit Semantic Analysis], Proceedings of The 20th International Joint Conference on Artificial Intelligence (IJCAI), Hyderabad, India, 2007.

Halawi, Guy, Gideon Dror, Evgeniy Gabrilovich, and Yehuda Koren. (2012). [http://gabrilovich.com/publications/papers/Halawi2012LSL.pdf Large-scale learning of word relatedness with constraints]. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 1406-1414. ACM.

Hassan, Samer, and Rada Mihalcea: [http://www.aaai.org/ocs/index.php/AAAI/AAAI11/paper/download/3616/3972/ Semantic Relatedness Using Salient Semantic Analysis]. AAAI 2011

Hirst, Graeme and David St-Onge. Lexical chains as representations of context for the detection and correction of malapropisms. In Christiane Fellbaum, editor, WordNet: An Electronic Lexical Database. The MIT Press, Cambridge, MA, pages 305–332, 1998.

Huang, Eric H., Richard Socher, Christopher D. Manning, and Andrew Y. Ng. 2012. Improving word representations via global context and multiple word prototypes. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers - Volume 1 (ACL '12), Vol. 1. Association for Computational Linguistics, Stroudsburg, PA, USA, 873-882.

Islam, A., and Inkpen, D. 2006. [http://www.site.uottawa.ca/~mdislam/publications/LREC_06_242.pdf Second order co-occurrence pmi for determining the semantic similarity of words]. Proceedings of the International Conference on Language Resources and Evaluation (LREC 2006) 1033–1038.

Jarmasz, M. 2003. [http://www.arxiv.org/pdf/1204.0140 Roget’s thesaurus as a Lexical Resource for Natural Language Processing]. Ph.D. Dissertation, Ottawa Carleton Institute for Computer Science, School of Information Technology and Engineering, University of Ottawa.

Jiang, Jay J. and David W. Conrath. Semantic similarity based on corpus statistics and lexical taxonomy. In Proceedings of International Conference on Research in Computational Linguistics (ROCLING X), Taiwan, pages 19–33, 1997.

Landauer, T. K.; L, T. K.; Laham, D.; Rehder, B.; and Schreiner, M. E. 1997. How well can passage meaning be derived without using word order? a comparison of latent semantic analysis and humans.

Leacock, Claudia and Martin Chodorow. Combining local context and WordNet similarity for word sense identification. In Christiane Fellbaum, editor, WordNet: An Electronic Lexical Database. The MIT Press, Cambridge, MA, pages 265–283, 1998.

Lin, Dekang. An information-theoretic definition of similarity. In Proceedings of the 15th International Conference on Machine Learning, Madison,WI, pages 296–304, 1998.

Luong, Minh-Thang, Richard Socher, and Christopher D. Manning. (2013). [http://nlp.stanford.edu/~lmthang/data/papers/conll13_morpho.pdf Better word representations with recursive neural networks for morphology]. CoNLL-2013: 104.

Pennington, J., Socher, R., and Manning, C. (2014). [https://www.aclweb.org/anthology/D14-1162 Glove: Global vectors for word representation]. ''EMNLP 2014'', pp. 1532-1543.

Pilehvar, M.T., D. Jurgens and R. Navigli. [http://wwwusers.di.uniroma1.it/~navigli/pubs/ACL_2013_Pilehvar_Jurgens_Navigli.pdf Align, Disambiguate and Walk: A Unified Approach for Measuring Semantic Similarity]. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (ACL 2013), Sofia, Bulgaria, August 4-9, 2013, pp. 1341-1351.

Radinsky, Kira, Eugene Agichtein, Evgeniy Gabrilovich, and Shaul Markovitch. (2011). [http://gabrilovich.com/publications/papers/Radinsky2011WTS.pdf A word at a time: computing word relatedness using temporal semantic analysis]. In Proceedings of the 20th international conference on World wide web, pp. 337-346. ACM.

Resnik, Philip. Using information content to evaluate semantic similarity. In Proceedings of the 14th International Joint Conference on Artificial Intelligence, pages 448–453, Montreal, Canada, 1995.

Salle A., Idiart M., and Villavicencio A. (2018) [https://github.com/alexandres/lexvec/blob/master/README.md LexVec]

Song, Yan, Shuming Shi, Jing Li, and Haisong Zhang. 2018. [https://www.aclweb.org/anthology/N18-2028.pdf Directional skip-gram: Explicitly distinguish-ing left and right context for word embeddings]. NAACL-2018, pages 175–180.

Speer, Rob, Joshua Chin and Catherine Havasi. (2017). [http://aaai.org/ocs/index.php/AAAI/AAAI17/paper/view/14972 ConceptNet 5.5: An Open Multilingual Graph of General Knowledge]. Proceedings of The 31st AAAI Conference on Artificial Intelligence, San Francisco, CA.

Strube, Michael and Simone Paolo Ponzetto. (2006). [http://www.aaai.org/Papers/AAAI/2006/AAAI06-223.pdf WikiRelate! Computing Semantic Relatedness Using Wikipedia]. Proceedings of The 21st National Conference on Artificial Intelligence (AAAI), Boston, MA.

Yih, W. and Qazvinian, V. (2012). [http://aclweb.org/anthology/N/N12/N12-1077.pdf Measuring Word Relatedness Using Heterogeneous Vector Space Models]. Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2012).

[[Category:State of the art]]
[[Category:Similarity]]

Automatic Text Summarization (State of the art)

2019-12-02T15:43:50Z

Fnielsen: References

== "Standard" measure: ==

== Available summmarization datasets: ==
{| border="1" cellpadding="5" cellspacing="1"
|-
! Dataset
! Reference
! Number of texts
! Dataset Link
! List of state-of-the-art results
!Comments
|-
| Newsroom
| Grusky et al. (2018)<ref>[https://doi.org/10.18653/V1/N18-1065 Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies]</ref>
|
| https://summari.es/
|-
|}

== "Standard" datasets: ==
{{StateOfTheArtTable}}

| SystemName || How does it work? || Author and Article [http://www.example.com] || Software? || 98% according to... || Any extra comments?
|-

|-

|}

== References ==
<references/>

[[Category:State of the art]]

Automatic Text Summarization (State of the art)

2019-12-02T15:41:33Z

Fnielsen: Newsroom dataset

Talk:Main Page

2018-07-05T13:31:38Z

Fnielsen: /* Wikidata and Scholia links */ Wordsim-353 edit

== SIGs? ==

I couldnt find links to the ACL special interest groups.
I guess a good place is the first page under topics. {{unsigned|Ioan|17 December 2006}}

::I added a link on the main page, under Organizations, departments, ... --[[User:Pdturney|Pdturney]] 09:31, 17 December 2006 (EST)

I am not sure that's the appropriate section. There is only about two dozens of SIGs, and they are not
classified geographically like the other organisations. I think they should go straight on the first page,
because they are an important part of ACL. --[[User:Ioan|Ioan]] 09:53, 17 December 2006 (EST)

::There's no requirement to classify them geographically like the other organisations. But if you want to put them somewhere else, go ahead. Be bold. --[[User:Pdturney|Pdturney]] 10:27, 17 December 2006 (EST)

== Companies? ==

How about a bullet for commercial companies handling NLP tasks?

::I added "companies" to "Organizations, departments, institutions, groups". By the way, please sign your messages by clicking on the button in edit mode that looks like a signature. The editor automatically expands it to your user name and the current date and time. --[[User:Pdturney|Pdturney]] 08:38, 31 October 2006 (EST)

== Favicon? ==

The ACL favicon is not being displayed. Is this on purpose? --[[User:Grano|Grano]] 15:19, 31 October 2006 (EST)

::It's working now. Thanks for bringing it to our attention. --[[User:Pdturney|Pdturney]] 19:39, 31 October 2006 (EST)

== Wikidata and Scholia links ==

I have taken the liberty of including links to Wikidata and (our) Scholia webservices, see [https://aclweb.org/w/index.php?title=Syntactic_Analogies_(State_of_the_art)&diff=12266&oldid=11763 here] for an example. Please advise on whether this is regarded as inappropriately spamming this wiki. — [[User:Fnielsen|Fnielsen]] ([[User talk:Fnielsen|talk]]) 07:26, 5 July 2018 (MDT)
: Regarding [https://aclweb.org/w/index.php?title=WordSimilarity-353_Test_Collection_(State_of_the_art)&diff=12268&oldid=11817]: the [https://tools.wmflabs.org/scholia/use/Q31845205 Scholia page for Wordsim-353] is more elaborate. — [[User:Fnielsen|Fnielsen]] ([[User talk:Fnielsen|talk]]) 07:31, 5 July 2018 (MDT)

WordSimilarity-353 Test Collection (State of the art)

2018-07-05T13:29:51Z

Fnielsen: Wikidata and Scholia links

* [http://www.cs.technion.ac.il/~gabr/resources/data/wordsim353/ WordSimilarity-353 Test Collection]
* contains two sets of English word pairs along with human-assigned similarity judgements
* first set (set1) contains 153 word pairs along with their similarity scores assigned by 13 subjects
* second set (set2) contains 200 word pairs with similarity assessed by 16 subjects
* WordSimilarity-353 dataset is available [http://www.cs.technion.ac.il/~gabr/resources/data/wordsim353/ here]
* performance is measured by [http://en.wikipedia.org/wiki/Spearman_rank_correlation Spearman's rank correlation coefficient]
* introduced by [http://www.cs.technion.ac.il/~gabr/papers/tois_context.pdf Finkelstein et al. (2002)]
* subsequently used by many other researchers
* [https://www.wikidata.org/wiki/Q31845205 Wikidata] and [https://tools.wmflabs.org/scholia/use/Q31845205 Scholia]
* see also: [[Similarity (State of the art)]]

== Table of results ==

* '''Listed in order of increasing [http://en.wikipedia.org/wiki/Spearman_rank_correlation Spearman's rho].'''

{| border="1" cellpadding="5" cellspacing="1"
|-
! Algorithm
! Reference for algorithm
! Reference for reported results
! Type
! Spearman's rho
! Pearson's r
|-
| L&C
| Leacock and Chodorow (1998)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.302
| 0.356
|-
| WNE
| Jarmasz (2003)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.305
| 0.271
|-
| J&C
| Jiang and Conrath 1997
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.318
| 0.354
|-
| L&C
| Leacock and Chodorow (1998)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.348
| 0.341
|-
| H&S
| Hirst and St-Onge (1998)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.302
| 0.356
|-
| Lin
| Lin (1998)
| Hassan and Mihalcea (2011)
| Corpus-based
| 0.348
| 0.357
|-
| Resnik
| Resnik (1995)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.353
| 0.365
|-
| ROGET
| Jarmasz (2003)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.415
| 0.536
|-
| C&W
| Collobert and Weston (2008)
| Collobert and Weston (2008)
| Corpus-based
| 0.5
| N/A
|-
| WikiRelate
| Strube and Ponzetto (2006)
| Strube and Ponzetto (2006)
| Corpus-based
| N/A
| 0.48
|-
| LSA
| Landauer et al. (1997)
| Hassan and Mihalcea (2011)
| Corpus-based
| 0.581
| 0.492
|-
| LSA
| Landauer et al. (1997)
| Hassan and Mihalcea (2011)
| Corpus-based
| 0.581
| 0.563
|-
| simVB+simWN
| Finkelstein et al. (2002)
| Finkelstein et al. (2002)
| Hybrid
| N/A
| 0.55
|-
| SSA
| Hassan and Mihalcea (2011)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.622
| 0.629
|-
| HSMN+csmRNN
| Luong et al. (2013)
| Luong et al. (2013)
| Corpus-based
| 0.65
| N/A
|-
| Multi-prototype
| Huang et al. (2012)
| Huang et al. (2012)
| Corpus-based
| 0.71
| N/A
|-
| Multi-lingual SSA
| Hassan et al. (2011)
| Hassan et al. (2011)
| Corpus-based
| 0.713
| 0.674
|-
| ESA
| Gabrilovich and Markovitch (2007)
| Gabrilovich and Markovitch (2007)
| Corpus-based
| 0.748
| 0.503
|-
| TSA
| Radinsky et al. (2011)
| Radinsky et al. (2011)
| Hybrid
| 0.80
| N/A
|-
| CLEAR
| Halawi et al. (2012)
| Halawi et al. (2012)
| Corpus-based
| 0.81
| N/A
|-
| Y&Q
| Yih and Qazvinian (2012)
| Yih and Qazvinian (2012)
| Hybrid
| 0.81
| N/A
|-
| ConceptNet Numberbatch
| Speer et al. (2017)
| Speer et al. (2017)
| Hybrid
| 0.828
| N/A
|}

== References ==

* '''Listed in alphabetical order.'''

Finkelstein, Lev, Evgeniy Gabrilovich, Yossi Matias, Ehud Rivlin, Zach Solan, Gadi Wolfman, and Eytan Ruppin. (2002) [http://www.cs.technion.ac.il/~gabr/papers/tois_context.pdf Placing Search in Context: The Concept Revisited]. ACM Transactions on Information Systems, 20(1):116-131.

Gabrilovich, Evgeniy, and Shaul Markovitch, [http://www.cs.technion.ac.il/~gabr/papers/ijcai-2007-sim.pdf Computing Semantic Relatedness using Wikipedia-based Explicit Semantic Analysis], Proceedings of The 20th International Joint Conference on Artificial Intelligence (IJCAI), Hyderabad, India, 2007.

Halawi, Guy, Gideon Dror, Evgeniy Gabrilovich, and Yehuda Koren. (2012). [http://gabrilovich.com/publications/papers/Halawi2012LSL.pdf Large-scale learning of word relatedness with constraints]. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 1406-1414. ACM.

Hassan, Samer, and Rada Mihalcea: [http://www.aaai.org/ocs/index.php/AAAI/AAAI11/paper/download/3616/3972/ Semantic Relatedness Using Salient Semantic Analysis]. AAAI 2011

Hirst, Graeme and David St-Onge. Lexical chains as representations of context for the detection and correction of malapropisms. In Christiane Fellbaum, editor, WordNet: An Electronic Lexical Database. The MIT Press, Cambridge, MA, pages 305–332, 1998.

Huang, Eric H., Richard Socher, Christopher D. Manning, and Andrew Y. Ng. 2012. Improving word representations via global context and multiple word prototypes. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers - Volume 1 (ACL '12), Vol. 1. Association for Computational Linguistics, Stroudsburg, PA, USA, 873-882.

Islam, A., and Inkpen, D. 2006. [http://www.site.uottawa.ca/~mdislam/publications/LREC_06_242.pdf Second order co-occurrence pmi for determining the semantic similarity of words]. Proceedings of the International Conference on Language Resources and Evaluation (LREC 2006) 1033–1038.

Jarmasz, M. 2003. [http://www.arxiv.org/pdf/1204.0140 Roget’s thesaurus as a Lexical Resource for Natural Language Processing]. Ph.D. Dissertation, Ottawa Carleton Institute for Computer Science, School of Information Technology and Engineering, University of Ottawa.

Jiang, Jay J. and David W. Conrath. Semantic similarity based on corpus statistics and lexical taxonomy. In Proceedings of International Conference on Research in Computational Linguistics (ROCLING X), Taiwan, pages 19–33, 1997.

Landauer, T. K.; L, T. K.; Laham, D.; Rehder, B.; and Schreiner, M. E. 1997. How well can passage meaning be derived without using word order? a comparison of latent semantic analysis and humans.

Leacock, Claudia and Martin Chodorow. Combining local context and WordNet similarity for word sense identification. In Christiane Fellbaum, editor, WordNet: An Electronic Lexical Database. The MIT Press, Cambridge, MA, pages 265–283, 1998.

Lin, Dekang. An information-theoretic definition of similarity. In Proceedings of the 15th International Conference on Machine Learning, Madison,WI, pages 296–304, 1998.

Luong, Minh-Thang, Richard Socher, and Christopher D. Manning. (2013). [http://nlp.stanford.edu/~lmthang/data/papers/conll13_morpho.pdf Better word representations with recursive neural networks for morphology]. CoNLL-2013: 104.

Pilehvar, M.T., D. Jurgens and R. Navigli. [http://wwwusers.di.uniroma1.it/~navigli/pubs/ACL_2013_Pilehvar_Jurgens_Navigli.pdf Align, Disambiguate and Walk: A Unified Approach for Measuring Semantic Similarity]. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (ACL 2013), Sofia, Bulgaria, August 4-9, 2013, pp. 1341-1351.

Radinsky, Kira, Eugene Agichtein, Evgeniy Gabrilovich, and Shaul Markovitch. (2011). [http://gabrilovich.com/publications/papers/Radinsky2011WTS.pdf A word at a time: computing word relatedness using temporal semantic analysis]. In Proceedings of the 20th international conference on World wide web, pp. 337-346. ACM.

Resnik, Philip. Using information content to evaluate semantic similarity. In Proceedings of the 14th International Joint Conference on Artificial Intelligence, pages 448–453, Montreal, Canada, 1995.

Speer, Rob, Joshua Chin and Catherine Havasi. (2017). [http://aaai.org/ocs/index.php/AAAI/AAAI17/paper/view/14972 ConceptNet 5.5: An Open Multilingual Graph of General Knowledge]. Proceedings of The 31st AAAI Conference on Artificial Intelligence, San Francisco, CA.

Strube, Michael and Simone Paolo Ponzetto. (2006). [http://www.aaai.org/Papers/AAAI/2006/AAAI06-223.pdf WikiRelate! Computing Semantic Relatedness Using Wikipedia]. Proceedings of The 21st National Conference on Artificial Intelligence (AAAI), Boston, MA.

Yih, W. and Qazvinian, V. (2012). [http://aclweb.org/anthology/N/N12/N12-1077.pdf Measuring Word Relatedness Using Heterogeneous Vector Space Models]. Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2012).

[[Category:State of the art]]
[[Category:Similarity]]

Talk:Main Page

2018-07-05T13:26:41Z

Fnielsen: /* Wikidata and Scholia links */ new section

Syntactic Analogies (State of the art)

2018-07-05T13:23:47Z

Fnielsen: Wikidata and Scholia

* [http://research.microsoft.com/en-us/um/people/gzweig/Pubs/myz_naacl13_test_set.tgz Microsoft Research Syntactic Analogies Dataset]
* A test set of analogy questions of the form "a is to b as c is to" testing
** base/comparative/superlative forms of adjectives
** singular/plural forms of common nouns
** possessive/non-possessive forms of common nouns
** base, past and 3rd person present tense forms of verbs
* Originally proposed in [http://aclweb.org/anthology//N/N13/N13-1090.pdf Mikolov et al. (2013)]
* [https://www.wikidata.org/wiki/Q55387870 Wikidata] and [https://tools.wmflabs.org/scholia/use/Q55387870 Scholia]
* see also: [[Similarity (State of the art)]]

== Table of results ==

* '''Listed in order of increasing accuracy'''

{| border="1" cellpadding="5" cellspacing="1"
|-
! Algorithm
! Reference
! Accuracy (%)
|-
| CW-100
| Mikolov et al. (2013)
| 5.0
|-
| HLBL-100
| Mikolov et al. (2013)
| 18.7
|-
| RNN-1600
| Mikolov et al. (2013)
| 39.6
|-
| vLBL+NCE5
| Mnih and Kavukcuoglu (2013)
| 60.8
|-
|}

== References ==

* '''Listed alphabetically.'''

Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. (2013). [http://aclweb.org/anthology//N/N13/N13-1090.pdf Linguistic regularities in continuous space word representations]. In ''Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL 2013)'', Atlanta, Georgia.

Mnih, A. and Kavukcuoglu, K. (2013). [http://machinelearning.wustl.edu/mlpapers/paper_files/NIPS2013_5165.pdf Learning word embeddings efficiently with noise-contrastive estimation]. In Advances in Neural Information Processing Systems (pp. 2265-2273).

[[Category:State of the art]]
[[Category:Similarity]]
[[Category:Analogy]]