ACL Wiki - User contributions [en]

Question Answering (State of the art)

2015-05-11T22:15:14Z

Scottyih:

== Answer Sentence Selection ==

The task of answer sentence selection is designed for the open-domain question answering setting. Given a question and a set of candidate sentences, the task is to choose the correct sentence that contains the exact answer and can sufficiently support the answer choice.

* [http://cs.stanford.edu/people/mengqiu/data/qg-emnlp07-data.tgz QA Answer Sentence Selection Dataset]: labeled sentences using TREC QA track data, provided by [http://cs.stanford.edu/people/mengqiu/ Mengqiu Wang] and first used in [http://www.aclweb.org/anthology/D/D07/D07-1003.pdf Wang et al. (2007)].

{| border="1" cellpadding="5" cellspacing="1"
|-
! Algorithm
! Reference
! [http://en.wikipedia.org/wiki/Mean_average_precision MAP]
! [http://en.wikipedia.org/wiki/Mean_reciprocal_rank MRR]
|-
| Punyakanok (2004)
| Wang et al. (2007)
| 0.419
| 0.494
|-
| Cui (2005)
| Wang et al. (2007)
| 0.427
| 0.526
|-
| Wang (2007)
| Wang et al. (2007)
| 0.603
| 0.685
|-
| H&S (2010)
| Heilman and Smith (2010)
| 0.609
| 0.692
|-
| W&M (2010)
| Wang and Manning (2010)
| 0.595
| 0.695
|-
| Yao (2013)
| Yao et al. (2013)
| 0.631
| 0.748
|-
| S&M (2013)
| Severyn and Moschitti (2013)
| 0.678
| 0.736
|-
| Shnarch (2013) - Backward
| Shnarch (2013)
| 0.686
| 0.754
|-
| Yih (2013) - LCLR
| Yih et al. (2013)
| 0.709
| 0.770
|-
| Yu (2014) - TRAIN-ALL bigram+count
| Yu et al. (2014)
| 0.711
| 0.785
|-
|}

== References ==
* Vasin Punyakanok, Dan Roth, and Wen-Tau Yih. 2004. [http://cogcomp.cs.illinois.edu/papers/PunyakanokRoYi04a.pdf Mapping dependencies trees: An application to question answering]. In Proceedings of the 8th International Symposium on Artificial Intelligence and Mathematics, Fort Lauderdale, FL, USA.
* Hang Cui, Renxu Sun, Keya Li, Min-Yen Kan, and Tat-Seng Chua. 2005. [http://ws.csie.ncku.edu.tw/login/upload/2005/paper/Question%20answering%20Question%20answering%20passage%20retrieval%20using%20dependency%20relations.pdf Question answering passage retrieval using dependency relations]. In Proceedings of the 28th ACM-SIGIR International Conference on Research and Development in Information Retrieval, Salvador, Brazil.
* Wang, Mengqiu and Smith, Noah A. and Mitamura, Teruko. 2007. [http://www.aclweb.org/anthology/D/D07/D07-1003.pdf What is the Jeopardy Model? A Quasi-Synchronous Grammar for QA]. In EMNLP-CoNLL 2007.
* Heilman, Michael and Smith, Noah A. 2010. [http://www.aclweb.org/anthology/N10-1145 Tree Edit Models for Recognizing Textual Entailments, Paraphrases, and Answers to Questions]. In NAACL-HLT 2010.
* Wang, Mengqiu and Manning, Christopher. 2010. [http://aclweb.org/anthology//C/C10/C10-1131.pdf Probabilistic Tree-Edit Models with Structured Latent Variables for Textual Entailment and Question Answering]. In COLING 2010.
* E. Shnarch. 2013. Probabilistic Models for Lexical Inference. Ph.D. thesis, Bar Ilan University.
* Yao, Xuchen and Van Durme, Benjamin and Callison-Burch, Chris and Clark, Peter. 2013. [http://www.aclweb.org/anthology/N13-1106.pdf Answer Extraction as Sequence Tagging with Tree Edit Distance]. In NAACL-HLT 2013.
* Yih, Wen-tau and Chang, Ming-Wei and Meek, Christopher and Pastusiak, Andrzej. 2013. [http://research.microsoft.com/pubs/192357/QA-SentSel-Updated-PostACL.pdf Question Answering Using Enhanced Lexical Semantic Models]. In ACL 2013.
* Severyn, Aliaksei and Moschitti, Alessandro. 2013. [http://www.aclweb.org/anthology/D13-1044.pdf Automatic Feature Engineering for Answer Selection and Extraction]. In EMNLP 2013.
* Lei Yu, Karl Moritz Hermann, Phil Blunsom, and Stephen Pulman. 2014 [http://arxiv.org/pdf/1412.1632v1.pdf Deep Learning for Answer Sentence Selection]. In NIPS deep learning workshop.

[[Category:State of the art]]

State of the art

2014-01-21T22:34:49Z

Scottyih:

The purpose of this section of the ACL wiki is to be a repository of ''k''-best state-of-the-art results (i.e., methods and software) for various core natural language processing tasks.

As a side effect, this should hopefully evolve into a knowledge base of standard evaluation methods and datasets for various tasks, as well as encourage more effort into reproducibility of results. This will help newcomers to a field appreciate what has been done so far and what the main tasks are, and will help keep active researchers informed on fields other than their specific research. The next time you need a system for PP attachment, or wonder what is the current state of word sense disambiguation, this will be the place to visit.

Please contribute! (This is also a good place for you to display your results!)

As a historical point of reference, you may want to refer to the [http://web.archive.org/web/20100325144600/http://cslu.cse.ogi.edu/HLTsurvey/ Survey of the State of the Art in Human Language Technology] ([http://www.lt-world.org/hlt_survey/master.pdf also available as PDF]), edited by R. Cole, J. Mariani, H. Uszkoreit, G. B. Varile, A. Zaenen, A. Zampolli, V. Zue, 1996.


* [[Anaphora Resolution (State of the art)|Anaphora Resolution]] (stub)
* [[Automatic Summarization (State of the art)|Automatic Summarization]]
* [[Chunking (State of the art)|Chunking]] (stub)
* [[Dependency Parsing (State of the art)|Dependency Parsing]] (stub)
* [[Document Classification (State of the art)|Document Classification]] (stub)
* [[Language Identification (State of the art)|Language Identification]] (stub)
* [[Named Entity Recognition (State of the art)|Named Entity Recognition]]
* [[Noun-Modifier Semantic Relations (State of the art)|Noun-Modifier Semantic Relations]]
* [[NP Chunking (State of the art)|NP Chunking]]
* [[Paraphrase Identification (State of the art)|Paraphrase Identification]]
* [[Parsing (State of the art)|Parsing]]
* [[POS Induction (State of the art) |POS Induction]]
* [[POS Tagging (State of the art) |POS Tagging]]
* [[PP Attachment (State of the art)|PP Attachment]] (stub)
* [[Question Answering (State of the art)|Question Answering]]
* [[Semantic Role Labeling (State of the art)|Semantic Role Labeling]] (stub)
* [[Sentiment Analysis (State of the art)|Sentiment Analysis]] (stub)
* [[Similarity (State of the art)|Similarity]] -- [[ESL Synonym Questions (State of the art)|ESL]], [[SAT Analogy Questions (State of the art)|SAT]], [[TOEFL Synonym Questions (State of the art)|TOEFL]], [[RG-65 Test Collection (State of the art)|RG-65 Test Collection]], [[WordSimilarity-353 Test Collection (State of the art)|WordSimilarity-353]], [[SemEval-2012 Task 2 (State of the art)|SemEval-2012 Task 2]]
* [[Speech Recognition (State of the art)|Speech Recognition]] (article request)
* [[Temporal Expression Recognition and Normalisation (State of the art)|Temporal Expression Recognition and Normalisation]]
* [[Cleaneval (State of the art)| Web Corpus Cleaning]] (stub)
* [[Word Segmentation (State of the art)|Word Segmentation]] (stub)
* [[Word Sense Disambiguation (State of the art)|Word Sense Disambiguation]] (stub)


[[Category:State of the art]]

Question Answering (State of the art)

2014-01-21T22:34:03Z

Scottyih:

== Answer Sentence Selection ==

The task of answer sentence selection is designed for the open-domain question answering setting. Given a question and a set of candidate sentences, the task is to choose the correct sentence that contains the exact answer and can sufficiently support the answer choice.

* [http://cs.stanford.edu/people/mengqiu/data/qg-emnlp07-data.tgz QA Answer Sentence Selection Dataset]: labeled sentences using TREC QA track data, provided by [http://cs.stanford.edu/people/mengqiu/ Mengqiu Wang] and first used in [http://www.aclweb.org/anthology/D/D07/D07-1003.pdf Wang et al. (2007)].

{| border="1" cellpadding="5" cellspacing="1"
|-
! Algorithm
! Reference
! [http://en.wikipedia.org/wiki/Mean_average_precision MAP]
! [http://en.wikipedia.org/wiki/Mean_reciprocal_rank MRR]
|-
| Punyakanok (2004)
| Wang et al. (2007)
| 0.419
| 0.494
|-
| Cui (2005)
| Wang et al. (2007)
| 0.427
| 0.526
|-
| Wang (2007)
| Wang et al. (2007)
| 0.603
| 0.685
|-
| H&S (2010)
| Heilman and Smith (2010)
| 0.609
| 0.692
|-
| W&M (2010)
| Wang and Manning (2010)
| 0.595
| 0.695
|-
| Yao (2013)
| Yao et al. (2013)
| 0.631
| 0.748
|-
| S&M (2013)
| Severyn and Moschitti (2013)
| 0.678
| 0.736
|-
| Shnarch (2013) - Backward
| Shnarch (2013)
| 0.686
| 0.754
|-
| Yih (2013) - LCLR
| Yih et al. (2013)
| 0.709
| 0.770
|-
|}

== References ==
* Vasin Punyakanok, Dan Roth, and Wen-Tau Yih. 2004. [http://cogcomp.cs.illinois.edu/papers/PunyakanokRoYi04a.pdf Mapping dependencies trees: An application to question answering]. In Proceedings of the 8th International Symposium on Artificial Intelligence and Mathematics, Fort Lauderdale, FL, USA.
* Hang Cui, Renxu Sun, Keya Li, Min-Yen Kan, and Tat-Seng Chua. 2005. [http://ws.csie.ncku.edu.tw/login/upload/2005/paper/Question%20answering%20Question%20answering%20passage%20retrieval%20using%20dependency%20relations.pdf Question answering passage retrieval using dependency relations]. In Proceedings of the 28th ACM-SIGIR International Conference on Research and Development in Information Retrieval, Salvador, Brazil.
* Wang, Mengqiu and Smith, Noah A. and Mitamura, Teruko. 2007. [http://www.aclweb.org/anthology/D/D07/D07-1003.pdf What is the Jeopardy Model? A Quasi-Synchronous Grammar for QA]. In EMNLP-CoNLL 2007.
* Heilman, Michael and Smith, Noah A. 2010. [http://www.aclweb.org/anthology/N10-1145 Tree Edit Models for Recognizing Textual Entailments, Paraphrases, and Answers to Questions]. In NAACL-HLT 2010.
* Wang, Mengqiu and Manning, Christopher. 2010. [http://aclweb.org/anthology//C/C10/C10-1131.pdf Probabilistic Tree-Edit Models with Structured Latent Variables for Textual Entailment and Question Answering]. In COLING 2010.
* E. Shnarch. 2013. Probabilistic Models for Lexical Inference. Ph.D. thesis, Bar Ilan University.
* Yao, Xuchen and Van Durme, Benjamin and Callison-Burch, Chris and Clark, Peter. 2013. [http://www.aclweb.org/anthology/N13-1106.pdf Answer Extraction as Sequence Tagging with Tree Edit Distance]. In NAACL-HLT 2013.
* Yih, Wen-tau and Chang, Ming-Wei and Meek, Christopher and Pastusiak, Andrzej. 2013. [http://research.microsoft.com/pubs/192357/QA-SentSel-Updated-PostACL.pdf Question Answering Using Enhanced Lexical Semantic Models]. In ACL 2013.
* Severyn, Aliaksei and Moschitti, Alessandro. 2013. [http://www.aclweb.org/anthology/D13-1044.pdf Automatic Feature Engineering for Answer Selection and Extraction]. In EMNLP 2013.

[[Category:State of the art]]

Question Answering (State of the art)

2014-01-21T22:33:20Z

Scottyih: /* Answer Sentence Selection */

== Answer Sentence Selection ==

The task of answer sentence selection is designed for the open-domain question answering setting. Given a question and a set of candidate sentences, the task is to choose the correct sentence that contains the exact answer and can sufficiently support the answer choice.

* [http://cs.stanford.edu/people/mengqiu/data/qg-emnlp07-data.tgz QA Answer Sentence Selection Dataset]: labeled sentences using TREC QA track data, provided by [http://cs.stanford.edu/people/mengqiu/ Mengqiu Wang] and first used in [http://www.aclweb.org/anthology/D/D07/D07-1003.pdf Wang et al. (2007)].

{| border="1" cellpadding="5" cellspacing="1"
|-
! Algorithm
! Reference
! [http://en.wikipedia.org/wiki/Mean_average_precision MAP]
! [http://en.wikipedia.org/wiki/Mean_reciprocal_rank MRR]
|-
| Punyakanok (2004)
| Wang et al. (2007)
| 0.419
| 0.494
|-
| Cui (2005)
| Wang et al. (2007)
| 0.427
| 0.526
|-
| Wang (2007)
| Wang et al. (2007)
| 0.603
| 0.685
|-
| H&S (2010)
| Heilman and Smith (2010)
| 0.609
| 0.692
|-
| W&M (2010)
| Wang and Manning (2010)
| 0.595
| 0.695
|-
| Yao (2013)
| Yao et al. (2013)
| 0.631
| 0.748
|-
| S&M (2013)
| Severyn and Moschitti (2013)
| 0.678
| 0.736
|-
| Shnarch (2013) - Backward
| Shnarch (2013)
| 0.686
| 0.754
|-
| Yih (2013) - LCLR
| Yih et al. (2013)
| 0.709
| 0.770
|-
|}

== References ==
* Vasin Punyakanok, Dan Roth, and Wen-Tau Yih. 2004. [http://cogcomp.cs.illinois.edu/papers/PunyakanokRoYi04a.pdf Mapping dependencies trees: An application to question answering]. In Proceedings of the 8th International Symposium on Artificial Intelligence and Mathematics, Fort Lauderdale, FL, USA.
* Hang Cui, Renxu Sun, Keya Li, Min-Yen Kan, and Tat-Seng Chua. 2005. [http://ws.csie.ncku.edu.tw/login/upload/2005/paper/Question%20answering%20Question%20answering%20passage%20retrieval%20using%20dependency%20relations.pdf Question answering passage retrieval using dependency relations]. In Proceedings of the 28th ACM-SIGIR International Conference on Research and Development in Information Retrieval, Salvador, Brazil.
* Wang, Mengqiu and Smith, Noah A. and Mitamura, Teruko. 2007. [http://www.aclweb.org/anthology/D/D07/D07-1003.pdf What is the Jeopardy Model? A Quasi-Synchronous Grammar for QA]. In EMNLP-CoNLL 2007.
* Heilman, Michael and Smith, Noah A. 2010. [http://www.aclweb.org/anthology/N10-1145 Tree Edit Models for Recognizing Textual Entailments, Paraphrases, and Answers to Questions]. In NAACL-HLT 2010.
* Wang, Mengqiu and Manning, Christopher. 2010. [http://aclweb.org/anthology//C/C10/C10-1131.pdf Probabilistic Tree-Edit Models with Structured Latent Variables for Textual Entailment and Question Answering]. In COLING 2010.
* E. Shnarch. 2013. Probabilistic Models for Lexical Inference. Ph.D. thesis, Bar Ilan University.
* Yao, Xuchen and Van Durme, Benjamin and Callison-Burch, Chris and Clark, Peter. 2013. [http://www.aclweb.org/anthology/N13-1106.pdf Answer Extraction as Sequence Tagging with Tree Edit Distance]. In NAACL-HLT 2013.
* Yih, Wen-tau and Chang, Ming-Wei and Meek, Christopher and Pastusiak, Andrzej. 2013. [http://research.microsoft.com/pubs/192357/QA-SentSel-Updated-PostACL.pdf Question Answering Using Enhanced Lexical Semantic Models]. In ACL 2013.
* Severyn, Aliaksei and Moschitti, Alessandro. 2013. [http://www.aclweb.org/anthology/D13-1044.pdf Automatic Feature Engineering for Answer Selection and Extraction]. In EMNLP 2013.

[[Category:State of the art]]

Question Answering (State of the art)

2014-01-21T22:32:49Z

Scottyih: /* Answer Sentence Selection */

== Answer Sentence Selection ==

The task of answer sentence selection is designed for the open-domain question answering setting. Given a question and a set of candidate sentences, the task is to choose the correct sentence that contains the exact answer and can sufficiently support the answer choice.

* [http://cs.stanford.edu/people/mengqiu/data/qg-emnlp07-data.tgz QA Answer Sentence Selection Dataset]: labeled sentences using TREC QA track data, provided by [http://cs.stanford.edu/people/mengqiu/ Mengqiu Wang] and first used in Wang et al. (2007).

{| border="1" cellpadding="5" cellspacing="1"
|-
! Algorithm
! Reference
! [http://en.wikipedia.org/wiki/Mean_average_precision MAP]
! [http://en.wikipedia.org/wiki/Mean_reciprocal_rank MRR]
|-
| Punyakanok (2004)
| Wang et al. (2007)
| 0.419
| 0.494
|-
| Cui (2005)
| Wang et al. (2007)
| 0.427
| 0.526
|-
| Wang (2007)
| Wang et al. (2007)
| 0.603
| 0.685
|-
| H&S (2010)
| Heilman and Smith (2010)
| 0.609
| 0.692
|-
| W&M (2010)
| Wang and Manning (2010)
| 0.595
| 0.695
|-
| Yao (2013)
| Yao et al. (2013)
| 0.631
| 0.748
|-
| S&M (2013)
| Severyn and Moschitti (2013)
| 0.678
| 0.736
|-
| Shnarch (2013) - Backward
| Shnarch (2013)
| 0.686
| 0.754
|-
| Yih (2013) - LCLR
| Yih et al. (2013)
| 0.709
| 0.770
|-
|}

== References ==
* Vasin Punyakanok, Dan Roth, and Wen-Tau Yih. 2004. [http://cogcomp.cs.illinois.edu/papers/PunyakanokRoYi04a.pdf Mapping dependencies trees: An application to question answering]. In Proceedings of the 8th International Symposium on Artificial Intelligence and Mathematics, Fort Lauderdale, FL, USA.
* Hang Cui, Renxu Sun, Keya Li, Min-Yen Kan, and Tat-Seng Chua. 2005. [http://ws.csie.ncku.edu.tw/login/upload/2005/paper/Question%20answering%20Question%20answering%20passage%20retrieval%20using%20dependency%20relations.pdf Question answering passage retrieval using dependency relations]. In Proceedings of the 28th ACM-SIGIR International Conference on Research and Development in Information Retrieval, Salvador, Brazil.
* Wang, Mengqiu and Smith, Noah A. and Mitamura, Teruko. 2007. [http://www.aclweb.org/anthology/D/D07/D07-1003.pdf What is the Jeopardy Model? A Quasi-Synchronous Grammar for QA]. In EMNLP-CoNLL 2007.
* Heilman, Michael and Smith, Noah A. 2010. [http://www.aclweb.org/anthology/N10-1145 Tree Edit Models for Recognizing Textual Entailments, Paraphrases, and Answers to Questions]. In NAACL-HLT 2010.
* Wang, Mengqiu and Manning, Christopher. 2010. [http://aclweb.org/anthology//C/C10/C10-1131.pdf Probabilistic Tree-Edit Models with Structured Latent Variables for Textual Entailment and Question Answering]. In COLING 2010.
* E. Shnarch. 2013. Probabilistic Models for Lexical Inference. Ph.D. thesis, Bar Ilan University.
* Yao, Xuchen and Van Durme, Benjamin and Callison-Burch, Chris and Clark, Peter. 2013. [http://www.aclweb.org/anthology/N13-1106.pdf Answer Extraction as Sequence Tagging with Tree Edit Distance]. In NAACL-HLT 2013.
* Yih, Wen-tau and Chang, Ming-Wei and Meek, Christopher and Pastusiak, Andrzej. 2013. [http://research.microsoft.com/pubs/192357/QA-SentSel-Updated-PostACL.pdf Question Answering Using Enhanced Lexical Semantic Models]. In ACL 2013.
* Severyn, Aliaksei and Moschitti, Alessandro. 2013. [http://www.aclweb.org/anthology/D13-1044.pdf Automatic Feature Engineering for Answer Selection and Extraction]. In EMNLP 2013.

[[Category:State of the art]]

Question Answering (State of the art)

2014-01-21T22:26:49Z

Scottyih: /* Answer Sentence Selection */

== Answer Sentence Selection ==

The task of answer sentence selection is designed for the open-domain question answering setting. Given a question and a set of candidate sentences, the task is to choose the correct sentence that contains the exact answer and can sufficiently support the answer choice.

{| border="1" cellpadding="5" cellspacing="1"
|-
! Algorithm
! Reference
! [http://en.wikipedia.org/wiki/Mean_average_precision MAP]
! [http://en.wikipedia.org/wiki/Mean_reciprocal_rank MRR]
|-
| Punyakanok (2004)
| Wang et al. (2007)
| 0.419
| 0.494
|-
| Cui (2005)
| Wang et al. (2007)
| 0.427
| 0.526
|-
| Wang (2007)
| Wang et al. (2007)
| 0.603
| 0.685
|-
| H&S (2010)
| Heilman and Smith (2010)
| 0.609
| 0.692
|-
| W&M (2010)
| Wang and Manning (2010)
| 0.595
| 0.695
|-
| Yao (2013)
| Yao et al. (2013)
| 0.631
| 0.748
|-
| S&M (2013)
| Severyn and Moschitti (2013)
| 0.678
| 0.736
|-
| Shnarch (2013) - Backward
| Shnarch (2013)
| 0.686
| 0.754
|-
| Yih (2013) - LCLR
| Yih et al. (2013)
| 0.709
| 0.770
|-
|}

== References ==
* Vasin Punyakanok, Dan Roth, and Wen-Tau Yih. 2004. [http://cogcomp.cs.illinois.edu/papers/PunyakanokRoYi04a.pdf Mapping dependencies trees: An application to question answering]. In Proceedings of the 8th International Symposium on Artificial Intelligence and Mathematics, Fort Lauderdale, FL, USA.
* Hang Cui, Renxu Sun, Keya Li, Min-Yen Kan, and Tat-Seng Chua. 2005. [http://ws.csie.ncku.edu.tw/login/upload/2005/paper/Question%20answering%20Question%20answering%20passage%20retrieval%20using%20dependency%20relations.pdf Question answering passage retrieval using dependency relations]. In Proceedings of the 28th ACM-SIGIR International Conference on Research and Development in Information Retrieval, Salvador, Brazil.
* Wang, Mengqiu and Smith, Noah A. and Mitamura, Teruko. 2007. [http://www.aclweb.org/anthology/D/D07/D07-1003.pdf What is the Jeopardy Model? A Quasi-Synchronous Grammar for QA]. In EMNLP-CoNLL 2007.
* Heilman, Michael and Smith, Noah A. 2010. [http://www.aclweb.org/anthology/N10-1145 Tree Edit Models for Recognizing Textual Entailments, Paraphrases, and Answers to Questions]. In NAACL-HLT 2010.
* Wang, Mengqiu and Manning, Christopher. 2010. [http://aclweb.org/anthology//C/C10/C10-1131.pdf Probabilistic Tree-Edit Models with Structured Latent Variables for Textual Entailment and Question Answering]. In COLING 2010.
* E. Shnarch. 2013. Probabilistic Models for Lexical Inference. Ph.D. thesis, Bar Ilan University.
* Yao, Xuchen and Van Durme, Benjamin and Callison-Burch, Chris and Clark, Peter. 2013. [http://www.aclweb.org/anthology/N13-1106.pdf Answer Extraction as Sequence Tagging with Tree Edit Distance]. In NAACL-HLT 2013.
* Yih, Wen-tau and Chang, Ming-Wei and Meek, Christopher and Pastusiak, Andrzej. 2013. [http://research.microsoft.com/pubs/192357/QA-SentSel-Updated-PostACL.pdf Question Answering Using Enhanced Lexical Semantic Models]. In ACL 2013.
* Severyn, Aliaksei and Moschitti, Alessandro. 2013. [http://www.aclweb.org/anthology/D13-1044.pdf Automatic Feature Engineering for Answer Selection and Extraction]. In EMNLP 2013.

[[Category:State of the art]]

Question Answering (State of the art)

2014-01-21T22:23:48Z

Scottyih: /* References */

== Answer Sentence Selection ==

The task of answer sentence selection is designed for the open-domain question answering setting. Given a question and a set of candidate sentences, the task is to choose the correct sentence that contains the exact answer and can sufficiently support the answer choice.

{| border="1" cellpadding="5" cellspacing="1"
|-
! Algorithm
! Reference
! [http://en.wikipedia.org/wiki/Mean_average_precision MAP]
! [http://en.wikipedia.org/wiki/Mean_reciprocal_rank MRR]
|-
| Punyakanok (2004)
| Wang et al. (2007)
| 0.419
| 0.494
|-
| Cui (2005)
| Wang et al. (2007)
| 0.427
| 0.526
|-
| Wang (2007)
| Wang et al. (2007)
| 0.603
| 0.685
|-
| H&S (2010)
| Heilman and Smith (2010)
| 0.609
| 0.692
|-
| W&M (2010)
| Wang and Manning (2010)
| 0.595
| 0.695
|-
| Yao (2013)
| Yao et al. (2013)
| 0.631
| 0.748
|-
| Shnarch (2013) - Backward
| Shnarch (2013)
| 0.686
| 0.754
|-
| Yih (2013) - LCLR
| Yih et al. (2013)
| 0.709
| 0.770
|-
|}

== References ==
* Vasin Punyakanok, Dan Roth, and Wen-Tau Yih. 2004. [http://cogcomp.cs.illinois.edu/papers/PunyakanokRoYi04a.pdf Mapping dependencies trees: An application to question answering]. In Proceedings of the 8th International Symposium on Artificial Intelligence and Mathematics, Fort Lauderdale, FL, USA.
* Hang Cui, Renxu Sun, Keya Li, Min-Yen Kan, and Tat-Seng Chua. 2005. [http://ws.csie.ncku.edu.tw/login/upload/2005/paper/Question%20answering%20Question%20answering%20passage%20retrieval%20using%20dependency%20relations.pdf Question answering passage retrieval using dependency relations]. In Proceedings of the 28th ACM-SIGIR International Conference on Research and Development in Information Retrieval, Salvador, Brazil.
* Wang, Mengqiu and Smith, Noah A. and Mitamura, Teruko. 2007. [http://www.aclweb.org/anthology/D/D07/D07-1003.pdf What is the Jeopardy Model? A Quasi-Synchronous Grammar for QA]. In EMNLP-CoNLL 2007.
* Heilman, Michael and Smith, Noah A. 2010. [http://www.aclweb.org/anthology/N10-1145 Tree Edit Models for Recognizing Textual Entailments, Paraphrases, and Answers to Questions]. In NAACL-HLT 2010.
* Wang, Mengqiu and Manning, Christopher. 2010. [http://aclweb.org/anthology//C/C10/C10-1131.pdf Probabilistic Tree-Edit Models with Structured Latent Variables for Textual Entailment and Question Answering]. In COLING 2010.
* E. Shnarch. 2013. Probabilistic Models for Lexical Inference. Ph.D. thesis, Bar Ilan University.
* Yao, Xuchen and Van Durme, Benjamin and Callison-Burch, Chris and Clark, Peter. 2013. [http://www.aclweb.org/anthology/N13-1106.pdf Answer Extraction as Sequence Tagging with Tree Edit Distance]. In NAACL-HLT 2013.
* Yih, Wen-tau and Chang, Ming-Wei and Meek, Christopher and Pastusiak, Andrzej. 2013. [http://research.microsoft.com/pubs/192357/QA-SentSel-Updated-PostACL.pdf Question Answering Using Enhanced Lexical Semantic Models]. In ACL 2013.
* Severyn, Aliaksei and Moschitti, Alessandro. 2013. [http://www.aclweb.org/anthology/D13-1044.pdf Automatic Feature Engineering for Answer Selection and Extraction]. In EMNLP 2013.

[[Category:State of the art]]

Question Answering (State of the art)

2014-01-21T22:19:37Z

Scottyih: /* References */

== Answer Sentence Selection ==

The task of answer sentence selection is designed for the open-domain question answering setting. Given a question and a set of candidate sentences, the task is to choose the correct sentence that contains the exact answer and can sufficiently support the answer choice.

{| border="1" cellpadding="5" cellspacing="1"
|-
! Algorithm
! Reference
! [http://en.wikipedia.org/wiki/Mean_average_precision MAP]
! [http://en.wikipedia.org/wiki/Mean_reciprocal_rank MRR]
|-
| Punyakanok (2004)
| Wang et al. (2007)
| 0.419
| 0.494
|-
| Cui (2005)
| Wang et al. (2007)
| 0.427
| 0.526
|-
| Wang (2007)
| Wang et al. (2007)
| 0.603
| 0.685
|-
| H&S (2010)
| Heilman and Smith (2010)
| 0.609
| 0.692
|-
| W&M (2010)
| Wang and Manning (2010)
| 0.595
| 0.695
|-
| Yao (2013)
| Yao et al. (2013)
| 0.631
| 0.748
|-
| Shnarch (2013) - Backward
| Shnarch (2013)
| 0.686
| 0.754
|-
| Yih (2013) - LCLR
| Yih et al. (2013)
| 0.709
| 0.770
|-
|}

== References ==
* Vasin Punyakanok, Dan Roth, and Wen-Tau Yih. 2004. [http://cogcomp.cs.illinois.edu/papers/PunyakanokRoYi04a.pdf Mapping dependencies trees: An application to question answering]. In Proceedings of the 8th International Symposium on Artificial Intelligence and Mathematics, Fort Lauderdale, FL, USA.
* Hang Cui, Renxu Sun, Keya Li, Min-Yen Kan, and Tat-Seng Chua. 2005. [http://ws.csie.ncku.edu.tw/login/upload/2005/paper/Question%20answering%20Question%20answering%20passage%20retrieval%20using%20dependency%20relations.pdf Question answering passage retrieval using dependency relations]. In Proceedings of the 28th ACM-SIGIR International Conference on Research and Development in Information Retrieval, Salvador, Brazil.
* Wang, Mengqiu and Smith, Noah A. and Mitamura, Teruko. 2007. [http://www.aclweb.org/anthology/D/D07/D07-1003.pdf What is the Jeopardy Model? A Quasi-Synchronous Grammar for QA]. In EMNLP-CoNLL 2007.
* Heilman, Michael and Smith, Noah A. 2010. [http://www.aclweb.org/anthology/N10-1145 Tree Edit Models for Recognizing Textual Entailments, Paraphrases, and Answers to Questions]. In NAACL-HLT 2010.
* E. Shnarch. Probabilistic Models for Lexical Inference. Ph.D. thesis, Bar Ilan University. 2013.
* Yao, Xuchen and Van Durme, Benjamin and Callison-Burch, Chris and Clark, Peter. [http://www.aclweb.org/anthology/N13-1106 Answer Extraction as Sequence Tagging with Tree Edit Distance]. In NAACL-HLT 2013.
* Yih, Wen-tau and Chang, Ming-Wei and Meek, Christopher and Pastusiak, Andrzej. [http://research.microsoft.com/pubs/192357/QA-SentSel-Updated-PostACL.pdf Question Answering Using Enhanced Lexical Semantic Models]. In ACL 2013.
* Severyn, Aliaksei and Moschitti, Alessandro. [http://www.aclweb.org/anthology/D13-1044 Automatic Feature Engineering for Answer Selection and Extraction]. In EMNLP 2013.

[[Category:State of the art]]

Question Answering (State of the art)

2014-01-21T22:15:21Z

Scottyih: /* Answer Sentence Selection */

Question Answering (State of the art)

2014-01-21T22:09:53Z

Scottyih: /* References */

Question Answering (State of the art)

2014-01-21T21:55:17Z

Scottyih: Created page with "== Answer Sentence Selection == The task of answer sentence selection is designed for the open-domain question answering setting. Given a question and a set of candidate sent..."

State of the art

2014-01-21T21:32:30Z

Scottyih:

The purpose of this section of the ACL wiki is to be a repository of ''k''-best state-of-the-art results (i.e., methods and software) for various core natural language processing tasks.

As a side effect, this should hopefully evolve into a knowledge base of standard evaluation methods and datasets for various tasks, as well as encourage more effort into reproducibility of results. This will help newcomers to a field appreciate what has been done so far and what the main tasks are, and will help keep active researchers informed on fields other than their specific research. The next time you need a system for PP attachment, or wonder what is the current state of word sense disambiguation, this will be the place to visit.

Please contribute! (This is also a good place for you to display your results!)

As a historical point of reference, you may want to refer to the [http://web.archive.org/web/20100325144600/http://cslu.cse.ogi.edu/HLTsurvey/ Survey of the State of the Art in Human Language Technology] ([http://www.lt-world.org/hlt_survey/master.pdf also available as PDF]), edited by R. Cole, J. Mariani, H. Uszkoreit, G. B. Varile, A. Zaenen, A. Zampolli, V. Zue, 1996.


* [[Anaphora Resolution (State of the art)|Anaphora Resolution]] (stub)
* [[Automatic Summarization (State of the art)|Automatic Summarization]]
* [[Chunking (State of the art)|Chunking]] (stub)
* [[Dependency Parsing (State of the art)|Dependency Parsing]] (stub)
* [[Document Classification (State of the art)|Document Classification]] (stub)
* [[Language Identification (State of the art)|Language Identification]] (stub)
* [[Named Entity Recognition (State of the art)|Named Entity Recognition]]
* [[Noun-Modifier Semantic Relations (State of the art)|Noun-Modifier Semantic Relations]]
* [[NP Chunking (State of the art)|NP Chunking]]
* [[Paraphrase Identification (State of the art)|Paraphrase Identification]]
* [[Parsing (State of the art)|Parsing]]
* [[POS Induction (State of the art) |POS Induction]]
* [[POS Tagging (State of the art) |POS Tagging]]
* [[PP Attachment (State of the art)|PP Attachment]] (stub)
* [[Question Answering (State of the art)|Question Answering]] (stub)
* [[Semantic Role Labeling (State of the art)|Semantic Role Labeling]] (stub)
* [[Sentiment Analysis (State of the art)|Sentiment Analysis]] (stub)
* [[Similarity (State of the art)|Similarity]] -- [[ESL Synonym Questions (State of the art)|ESL]], [[SAT Analogy Questions (State of the art)|SAT]], [[TOEFL Synonym Questions (State of the art)|TOEFL]], [[RG-65 Test Collection (State of the art)|RG-65 Test Collection]], [[WordSimilarity-353 Test Collection (State of the art)|WordSimilarity-353]], [[SemEval-2012 Task 2 (State of the art)|SemEval-2012 Task 2]]
* [[Speech Recognition (State of the art)|Speech Recognition]] (article request)
* [[Temporal Expression Recognition and Normalisation (State of the art)|Temporal Expression Recognition and Normalisation]]
* [[Cleaneval (State of the art)| Web Corpus Cleaning]] (stub)
* [[Word Segmentation (State of the art)|Word Segmentation]] (stub)
* [[Word Sense Disambiguation (State of the art)|Word Sense Disambiguation]] (stub)


[[Category:State of the art]]

Syntactic Analogies (State of the art)

2014-01-15T20:01:19Z

Scottyih: Created page with "* [http://research.microsoft.com/en-us/um/people/gzweig/Pubs/myz_naacl13_test_set.tgz Microsoft Research Syntactic Analogies Dataset] * A test set of analogy questions of the ..."

* [http://research.microsoft.com/en-us/um/people/gzweig/Pubs/myz_naacl13_test_set.tgz Microsoft Research Syntactic Analogies Dataset]
* A test set of analogy questions of the form "a is to b as c is to" testing
** base/comparative/superlative forms of adjectives
** singular/plural forms of common nouns
** possessive/non-possessive forms of common nouns
** base, past and 3rd person present tense forms of verbs
* Originally proposed in [http://aclweb.org/anthology//N/N13/N13-1090.pdf Mikolov et al. (2013)]
* see also: [[Similarity (State of the art)]]

== Table of results ==

* '''Listed in order of increasing accuracy'''

{| border="1" cellpadding="5" cellspacing="1"
|-
! Algorithm
! Reference
! Accuracy (%)
|-
| CW-100
| Mikolov et al. (2013)
| 5.0
|-
| HLBL-100
| Mikolov et al. (2013)
| 18.7
|-
| RNN-1600
| Mikolov et al. (2013)
| 39.6
|-
| vLBL+NCE5
| Mnih and Kavukcuoglu (2013)
| 60.8
|-
|}

== References ==

* '''Listed alphabetically.'''

Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. (2013). [http://aclweb.org/anthology//N/N13/N13-1090.pdf Linguistic regularities in continuous space word representations]. In ''Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL 2013)'', Atlanta, Georgia.

Mnih, A. and Kavukcuoglu, K. (2013). [http://machinelearning.wustl.edu/mlpapers/paper_files/NIPS2013_5165.pdf Learning word embeddings efficiently with noise-contrastive estimation]. In Advances in Neural Information Processing Systems (pp. 2265-2273).

[[Category:State of the art]]

Similarity (State of the art)

2014-01-15T19:39:12Z

Scottyih: /* Relational similarity */

* see also: [[State of the art]]

== Attributional similarity ==

* '''attributional similarity:''' the degree to which two words are synonymous
* state-of-the-art results for:
** [[TOEFL Synonym Questions (State of the art)|TOEFL Synonym Questions]]
** [[ESL Synonym Questions (State of the art)|ESL Synonym Questions]]
** [[RG-65 Test Collection (State of the art)|RG-65 Test Collection]]
** [[WordSimilarity-353 Test Collection (State of the art)|WordSimilarity-353 Test Collection]]

== Relational similarity ==

* '''relational similarity:''' the degree to which two relations are analogous
* state-of-the-art results for:
** [[SAT Analogy Questions (State of the art)|SAT Analogy Questions]]
** [[SemEval-2012 Task 2 (State of the art)|SemEval-2012 Task 2: Measuring Degrees of Relational Similarity]]
** [[Syntactic Analogies|Microsoft Research Syntactic Analogies Dataset]]

== Sentence similarity ==

* '''sentence similarity:''' sentence paraphrase, paraphrase identification, paraphrase recognition
* state-of-the-art results for:
** [[Paraphrase Identification (State of the art)|Microsoft Research Paraphrase Corpus]]

== External links ==

* SemEval-2012 Task 2: [https://sites.google.com/site/semeval2012task2/ Measuring Degrees of Relational Similarity]
* SemEval-2012 Task 6: [http://www.cs.york.ac.uk/semeval-2012/task6/ Semantic Textual Similarity]
* *SEM 2013 Shared Task: [http://ixa2.si.ehu.es/sts/ Semantic Textual Similarity]

[[Category:State of the art]]

RG-65 Test Collection (State of the art)

2014-01-15T19:27:34Z

Scottyih: /* Table of results */

* state of the art in Rubenstein & Goodenough (RG-65) dataset
* 65 word pairs;
* Similarity of each pair is scored according to a scale from 0 to 4 (the higher the "similarity of meaning," the higher the number);
* The similarity values in the dataset are the means of judgments made by 51 subjects [Rubenstein and Goodenough, 1965].
* see also: [[Similarity (State of the art)]]

== Table of results ==

* '''Listed in order of decreasing [http://en.wikipedia.org/wiki/Spearman_rank_correlation Spearman's rho].'''

{| border="1" cellpadding="5" cellspacing="1" width="100%"
|-
! Algorithm
! Reference for algorithm
! Reference for reported results
! Type
! [http://en.wikipedia.org/wiki/Spearman%27s_rank_correlation_coefficient Spearman correlation] (ρ)
! [http://en.wikipedia.org/wiki/Pearson_product-moment_correlation_coefficient Pearson correlation] (r)
|-
| Y&Q
| Yih and Qazvinian (2012)
| Yih and Qazvinian (2012)
| Hybrid
| 0.890
| -
|-
| ADW
| Pilehvar et al. (2013)
| Pilehvar et al. (2013)
| Knowledge-based
| 0.868
| 0.810
|-
| PPR
| Hughes and Ramage (2007)
| Hughes and Ramage (2007)
| Knowledge-based
| 0.838
| -
|-
| SSA
| Hassan and Mihalcea (2011)
| Hassan and Mihalcea (2011)
| Corpus-based
| 0.833
| 0.861
|-
| PPR
| Agirre et al. (2009)
| Agirre et al. (2009)
| Knowledge-based
| 0.830
| -
|-
| H&S
| Hirst and St-Onge (1998)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.813
| 0.732
|-
| Roget
| Jarmasz (2003)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.804
| 0.818
|-
| J&C
| Jiang and Conrath (1997)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.804
| 0.731
|-
| WNE
| Jarmasz (2003)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.801
| 0.787
|-
| L&C
| Leacock and Chodorow (1998)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.797
| 0.852
|-
| Lin
| Lin (1998)
| Hassan and Mihalcea (2011)
| Corpus-based
| 0.788
| 0.834
|-
| ESA*
| Gabrilovich and Markovitch (2007)
| Hassan and Mihalcea (2011)
| Corpus-based
| 0.749
| 0.716
|-
| SOCPMI*
| Islam and Inkpen (2006)
| Hassan and Mihalcea (2011)
| Corpus-based
| 0.741
| 0.729
|-
| Resnik
| Resnik (1995)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.731
| 0.800
|-
| WLM
| Milne and Witten (2008)
| Milne and Witten (2008)
| Knowledge-based
| 0.640
| -
|-
| LSA*
| Landauer et al. (1997)
| Hassan and Mihalcea (2011)
| Corpus-based
| 0.609
| 0.644
|-
| WikiRelate
| Strube and Ponzetto (2006)
| Strube and Ponzetto (2006)
| Knowledge-based
| -
| 0.530
|}

Note: values reported by (Hassan and Mihalcea, 2011) are "based on the collected raw data from the respective authors", and those highlighted by (*) are re-implementations.

== References ==

* '''Listed alphabetically.'''

Agirre, Eneko, Enrique Alfonseca, Keith Hall, Jana Kravalova, Marius Pasca, Aitor Soroa: [http://www.aclweb.org/anthology/N09-1003 A Study on Similarity and Relatedness Using Distributional and WordNet-based Approaches]. HLT-NAACL 2009: 19-27

Gabrilovich, Evgeniy, and Shaul Markovitch, [http://www.cs.technion.ac.il/~gabr/papers/ijcai-2007-sim.pdf Computing Semantic Relatedness using Wikipedia-based Explicit Semantic Analysis], Proceedings of The 20th International Joint Conference on Artificial Intelligence (IJCAI), Hyderabad, India, 2007.

Hassan, Samer, and Rada Mihalcea: [http://www.cse.unt.edu/~rada/papers/hassan.aaai11.pdf‎ Semantic Relatedness Using Salient Semantic Analysis]. AAAI 2011

Hirst, Graeme and David St-Onge. Lexical chains as representations of context for the detection and correction of malapropisms. In Christiane Fellbaum, editor, WordNet: An Electronic Lexical Database. The MIT Press, Cambridge, MA, pages 305–332, 1998.

Hughes, Thad, Daniel Ramage, Lexical Semantic Relatedness with Random Graph Walks. EMNLP-CoNLL 2007: 581-589.

Islam, A., and Inkpen, D. 2006. [http://www.site.uottawa.ca/~mdislam/publications/LREC_06_242.pdf Second order co-occurrence pmi for determining the semantic similarity of words]. Proceedings of the International Conference on Language Resources and Evaluation (LREC 2006) 1033–1038.

Jarmasz, M. 2003. [http://www.arxiv.org/pdf/1204.0140 Roget’s thesaurus as a Lexical Resource for Natural Language Processing]. Ph.D. Dissertation, Ottawa Carleton Institute for Computer Science, School of Information Technology and Engineering, University of Ottawa.

Jiang, Jay J. and David W. Conrath. Semantic similarity based on corpus statistics and lexical taxonomy. In Proceedings of International Conference on Research in Computational Linguistics (ROCLING X), Taiwan, pages 19–33, 1997.

Landauer, T. K.; L, T. K.; Laham, D.; Rehder, B.; and Schreiner, M. E. 1997. How well can passage meaning be derived without using word order? a comparison of latent semantic analysis and humans.

Leacock, Claudia and Martin Chodorow. Combining local context and WordNet similarity for word sense identification. In Christiane Fellbaum, editor, WordNet: An Electronic Lexical Database. The MIT Press, Cambridge, MA, pages 265–283, 1998.

Lin, Dekang. An information-theoretic definition of similarity. In Proceedings of the 15th International Conference on Machine Learning, Madison,WI, pages 296–304, 1998.

Milne, David, and Ian H. Witten, An Effective, Low-Cost Measure of Semantic Relatedness Obtained from Wikipedia Links, In Proceedings of AAAI 2008.

Pilehvar, M.T., D. Jurgens and R. Navigli. [http://wwwusers.di.uniroma1.it/~navigli/pubs/ACL_2013_Pilehvar_Jurgens_Navigli.pdf Align, Disambiguate and Walk: A Unified Approach for Measuring Semantic Similarity]. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (ACL 2013), Sofia, Bulgaria, August 4-9, 2013, pp. 1341-1351.

Resnik, Philip. Using information content to evaluate semantic similarity. In Proceedings of the 14th International Joint Conference on Artificial Intelligence, pages 448–453, Montreal, Canada, 1995.

Rubenstein, Herbert, and John B. Goodenough. Contextual correlates of synonymy. Communications of the ACM, 8(10):627–633, 1965.

Strube, Michael, Simone Paolo Ponzetto: WikiRelate! Computing Semantic Relatedness Using Wikipedia. AAAI 2006: 1419-1424

Yih, W. and Qazvinian, V. (2012). [http://aclweb.org/anthology/N/N12/N12-1077.pdf Measuring Word Relatedness Using Heterogeneous Vector Space Models]. Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2012).

[[Category:State of the art]]

RG-65 Test Collection (State of the art)

2014-01-15T19:25:24Z

Scottyih: /* References */

* state of the art in Rubenstein & Goodenough (RG-65) dataset
* 65 word pairs;
* Similarity of each pair is scored according to a scale from 0 to 4 (the higher the "similarity of meaning," the higher the number);
* The similarity values in the dataset are the means of judgments made by 51 subjects [Rubenstein and Goodenough, 1965].
* see also: [[Similarity (State of the art)]]

== Table of results ==

* '''Listed in order of decreasing [http://en.wikipedia.org/wiki/Spearman_rank_correlation Spearman's rho].'''

{| border="1" cellpadding="5" cellspacing="1" width="100%"
|-
! Algorithm
! Reference for algorithm
! Reference for reported results
! Type
! [http://en.wikipedia.org/wiki/Spearman%27s_rank_correlation_coefficient Spearman correlation] (ρ)
! [http://en.wikipedia.org/wiki/Pearson_product-moment_correlation_coefficient Pearson correlation] (r)
|-
| ADW
| Pilehvar et al. (2013)
| Pilehvar et al. (2013)
| Knowledge-based
| 0.868
| 0.810
|-
| PPR
| Hughes and Ramage (2007)
| Hughes and Ramage (2007)
| Knowledge-based
| 0.838
| -
|-
| SSA
| Hassan and Mihalcea (2011)
| Hassan and Mihalcea (2011)
| Corpus-based
| 0.833
| 0.861
|-
| PPR
| Agirre et al. (2009)
| Agirre et al. (2009)
| Knowledge-based
| 0.830
| -
|-
| H&S
| Hirst and St-Onge (1998)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.813
| 0.732
|-
| Roget
| Jarmasz (2003)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.804
| 0.818
|-
| J&C
| Jiang and Conrath (1997)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.804
| 0.731
|-
| WNE
| Jarmasz (2003)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.801
| 0.787
|-
| L&C
| Leacock and Chodorow (1998)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.797
| 0.852
|-
| Lin
| Lin (1998)
| Hassan and Mihalcea (2011)
| Corpus-based
| 0.788
| 0.834
|-
| ESA*
| Gabrilovich and Markovitch (2007)
| Hassan and Mihalcea (2011)
| Corpus-based
| 0.749
| 0.716
|-
| SOCPMI*
| Islam and Inkpen (2006)
| Hassan and Mihalcea (2011)
| Corpus-based
| 0.741
| 0.729
|-
| Resnik
| Resnik (1995)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.731
| 0.800
|-
| WLM
| Milne and Witten (2008)
| Milne and Witten (2008)
| Knowledge-based
| 0.640
| -
|-
| LSA*
| Landauer et al. (1997)
| Hassan and Mihalcea (2011)
| Corpus-based
| 0.609
| 0.644
|-
| WikiRelate
| Strube and Ponzetto (2006)
| Strube and Ponzetto (2006)
| Knowledge-based
| -
| 0.530
|}

Note: values reported by (Hassan and Mihalcea, 2011) are "based on the collected raw data from the respective authors", and those highlighted by (*) are re-implementations.

== References ==

* '''Listed alphabetically.'''

Agirre, Eneko, Enrique Alfonseca, Keith Hall, Jana Kravalova, Marius Pasca, Aitor Soroa: [http://www.aclweb.org/anthology/N09-1003 A Study on Similarity and Relatedness Using Distributional and WordNet-based Approaches]. HLT-NAACL 2009: 19-27

Gabrilovich, Evgeniy, and Shaul Markovitch, [http://www.cs.technion.ac.il/~gabr/papers/ijcai-2007-sim.pdf Computing Semantic Relatedness using Wikipedia-based Explicit Semantic Analysis], Proceedings of The 20th International Joint Conference on Artificial Intelligence (IJCAI), Hyderabad, India, 2007.

Hassan, Samer, and Rada Mihalcea: [http://www.cse.unt.edu/~rada/papers/hassan.aaai11.pdf‎ Semantic Relatedness Using Salient Semantic Analysis]. AAAI 2011

Hirst, Graeme and David St-Onge. Lexical chains as representations of context for the detection and correction of malapropisms. In Christiane Fellbaum, editor, WordNet: An Electronic Lexical Database. The MIT Press, Cambridge, MA, pages 305–332, 1998.

Hughes, Thad, Daniel Ramage, Lexical Semantic Relatedness with Random Graph Walks. EMNLP-CoNLL 2007: 581-589.

Islam, A., and Inkpen, D. 2006. [http://www.site.uottawa.ca/~mdislam/publications/LREC_06_242.pdf Second order co-occurrence pmi for determining the semantic similarity of words]. Proceedings of the International Conference on Language Resources and Evaluation (LREC 2006) 1033–1038.

Jarmasz, M. 2003. [http://www.arxiv.org/pdf/1204.0140 Roget’s thesaurus as a Lexical Resource for Natural Language Processing]. Ph.D. Dissertation, Ottawa Carleton Institute for Computer Science, School of Information Technology and Engineering, University of Ottawa.

Jiang, Jay J. and David W. Conrath. Semantic similarity based on corpus statistics and lexical taxonomy. In Proceedings of International Conference on Research in Computational Linguistics (ROCLING X), Taiwan, pages 19–33, 1997.

Landauer, T. K.; L, T. K.; Laham, D.; Rehder, B.; and Schreiner, M. E. 1997. How well can passage meaning be derived without using word order? a comparison of latent semantic analysis and humans.

Leacock, Claudia and Martin Chodorow. Combining local context and WordNet similarity for word sense identification. In Christiane Fellbaum, editor, WordNet: An Electronic Lexical Database. The MIT Press, Cambridge, MA, pages 265–283, 1998.

Lin, Dekang. An information-theoretic definition of similarity. In Proceedings of the 15th International Conference on Machine Learning, Madison,WI, pages 296–304, 1998.

Milne, David, and Ian H. Witten, An Effective, Low-Cost Measure of Semantic Relatedness Obtained from Wikipedia Links, In Proceedings of AAAI 2008.

Pilehvar, M.T., D. Jurgens and R. Navigli. [http://wwwusers.di.uniroma1.it/~navigli/pubs/ACL_2013_Pilehvar_Jurgens_Navigli.pdf Align, Disambiguate and Walk: A Unified Approach for Measuring Semantic Similarity]. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (ACL 2013), Sofia, Bulgaria, August 4-9, 2013, pp. 1341-1351.

Resnik, Philip. Using information content to evaluate semantic similarity. In Proceedings of the 14th International Joint Conference on Artificial Intelligence, pages 448–453, Montreal, Canada, 1995.

Rubenstein, Herbert, and John B. Goodenough. Contextual correlates of synonymy. Communications of the ACM, 8(10):627–633, 1965.

Strube, Michael, Simone Paolo Ponzetto: WikiRelate! Computing Semantic Relatedness Using Wikipedia. AAAI 2006: 1419-1424

Yih, W. and Qazvinian, V. (2012). [http://aclweb.org/anthology/N/N12/N12-1077.pdf Measuring Word Relatedness Using Heterogeneous Vector Space Models]. Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2012).

[[Category:State of the art]]

WordSimilarity-353 Test Collection (State of the art)

2014-01-15T19:22:14Z

Scottyih: /* Table of results */

* [http://www.cs.technion.ac.il/~gabr/resources/data/wordsim353/ WordSimilarity-353 Test Collection]
* contains two sets of English word pairs along with human-assigned similarity judgements
* first set (set1) contains 153 word pairs along with their similarity scores assigned by 13 subjects
* second set (set2) contains 200 word pairs with similarity assessed by 16 subjects
* WordSimilarity-353 dataset is available [http://www.cs.technion.ac.il/~gabr/resources/data/wordsim353/ here]
* performance is measured by [http://en.wikipedia.org/wiki/Spearman_rank_correlation Spearman's rank correlation coefficient]
* introduced by [http://www.cs.technion.ac.il/~gabr/papers/tois_context.pdf Finkelstein et al. (2002)]
* subsequently used by many other researchers
* see also: [[Similarity (State of the art)]]

== Table of results ==

* '''Listed in order of increasing [http://en.wikipedia.org/wiki/Spearman_rank_correlation Spearman's rho].'''

{| border="1" cellpadding="5" cellspacing="1"
|-
! Algorithm
! Reference for algorithm
! Reference for reported results
! Type
! Spearman's rho
! Pearson's r
|-
| L&C
| Leacock and Chodorow (1998)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.302
| 0.356
|-
| WNE
| Jarmasz (2003)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.305
| 0.271
|-
| J&C
| Jiang and Conrath 1997
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.318
| 0.354
|-
| L&C
| Leacock and Chodorow (1998)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.348
| 0.341
|-
| H&S
| Hirst and St-Onge (1998)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.302
| 0.356
|-
| Lin
| Lin (1998)
| Hassan and Mihalcea (2011)
| Corpus-based
| 0.348
| 0.357
|-
| Resnik
| Resnik (1995)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.353
| 0.365
|-
| ROGET
| Jarmasz (2003)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.415
| 0.536
|-
| C&W
| Collobert and Weston (2008)
| Collobert and Weston (2008)
| Corpus-based
| 0.5
| N/A
|-
| WikiRelate
| Strube and Ponzetto (2006)
| Strube and Ponzetto (2006)
| Corpus-based
| N/A
| 0.48
|-
| LSA
| Landauer et al. (1997)
| Hassan and Mihalcea (2011)
| Corpus-based
| 0.581
| 0.492
|-
| LSA
| Landauer et al. (1997)
| Hassan and Mihalcea (2011)
| Corpus-based
| 0.581
| 0.563
|-
| simVB+simWN
| Finkelstein et al. (2002)
| Finkelstein et al. (2002)
| Hybrid
| N/A
| 0.55
|-
| SSA
| Hassan and Mihalcea (2011)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.622
| 0.629
|-
| HSMN+csmRNN
| Luong et al. (2013)
| Luong et al. (2013)
| Corpus-based
| 0.65
| N/A
|-
| Multi-prototype
| Huang et al. (2012)
| Huang et al. (2012)
| Corpus-based
| 0.71
| N/A
|-
| Multi-lingual SSA
| Hassan et al. (2011)
| Hassan et al. (2011)
| Corpus-based
| 0.713
| 0.674
|-
| ESA
| Gabrilovich and Markovitch (2007)
| Gabrilovich and Markovitch (2007)
| Corpus-based
| 0.748
| 0.503
|-
| TSA
| Radinsky et al. (2011)
| Radinsky et al. (2011)
| Hybrid
| 0.80
| N/A
|-
| CLEAR
| Halawi et al. (2012)
| Halawi et al. (2012)
| Corpus-based
| 0.81
| N/A
|-
| Y&Q
| Yih and Qazvinian (2012)
| Yih and Qazvinian (2012)
| Hybrid
| 0.81
| N/A
|-
|}

== References ==

Finkelstein, Lev, Evgeniy Gabrilovich, Yossi Matias, Ehud Rivlin, Zach Solan, Gadi Wolfman, and Eytan Ruppin. (2002) [http://www.cs.technion.ac.il/~gabr/papers/tois_context.pdf Placing Search in Context: The Concept Revisited]. ACM Transactions on Information Systems, 20(1):116-131.

Gabrilovich, Evgeniy, and Shaul Markovitch, [http://www.cs.technion.ac.il/~gabr/papers/ijcai-2007-sim.pdf Computing Semantic Relatedness using Wikipedia-based Explicit Semantic Analysis], Proceedings of The 20th International Joint Conference on Artificial Intelligence (IJCAI), Hyderabad, India, 2007.

Hassan, Samer, and Rada Mihalcea: [http://www.cse.unt.edu/~rada/papers/hassan.aaai11.pdf‎ Semantic Relatedness Using Salient Semantic Analysis]. AAAI 2011

Hirst, Graeme and David St-Onge. Lexical chains as representations of context for the detection and correction of malapropisms. In Christiane Fellbaum, editor, WordNet: An Electronic Lexical Database. The MIT Press, Cambridge, MA, pages 305–332, 1998.

Islam, A., and Inkpen, D. 2006. [http://www.site.uottawa.ca/~mdislam/publications/LREC_06_242.pdf Second order co-occurrence pmi for determining the semantic similarity of words]. Proceedings of the International Conference on Language Resources and Evaluation (LREC 2006) 1033–1038.

Jarmasz, M. 2003. [http://www.arxiv.org/pdf/1204.0140 Roget’s thesaurus as a Lexical Resource for Natural Language Processing]. Ph.D. Dissertation, Ottawa Carleton Institute for Computer Science, School of Information Technology and Engineering, University of Ottawa.

Jiang, Jay J. and David W. Conrath. Semantic similarity based on corpus statistics and lexical taxonomy. In Proceedings of International Conference on Research in Computational Linguistics (ROCLING X), Taiwan, pages 19–33, 1997.

Landauer, T. K.; L, T. K.; Laham, D.; Rehder, B.; and Schreiner, M. E. 1997. How well can passage meaning be derived without using word order? a comparison of latent semantic analysis and humans.

Leacock, Claudia and Martin Chodorow. Combining local context and WordNet similarity for word sense identification. In Christiane Fellbaum, editor, WordNet: An Electronic Lexical Database. The MIT Press, Cambridge, MA, pages 265–283, 1998.

Lin, Dekang. An information-theoretic definition of similarity. In Proceedings of the 15th International Conference on Machine Learning, Madison,WI, pages 296–304, 1998.

Pilehvar, M.T., D. Jurgens and R. Navigli. [http://wwwusers.di.uniroma1.it/~navigli/pubs/ACL_2013_Pilehvar_Jurgens_Navigli.pdf Align, Disambiguate and Walk: A Unified Approach for Measuring Semantic Similarity]. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (ACL 2013), Sofia, Bulgaria, August 4-9, 2013, pp. 1341-1351.

Resnik, Philip. Using information content to evaluate semantic similarity. In Proceedings of the 14th International Joint Conference on Artificial Intelligence, pages 448–453, Montreal, Canada, 1995.

Halawi, Guy, Gideon Dror, Evgeniy Gabrilovich, and Yehuda Koren. (2012). [http://gabrilovich.com/publications/papers/Halawi2012LSL.pdf Large-scale learning of word relatedness with constraints]. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 1406-1414. ACM.

Luong, Minh-Thang, Richard Socher, and Christopher D. Manning. (2013). [http://nlp.stanford.edu/~lmthang/data/papers/conll13_morpho.pdf Better word representations with recursive neural networks for morphology]. CoNLL-2013: 104.

Radinsky, Kira, Eugene Agichtein, Evgeniy Gabrilovich, and Shaul Markovitch. (2011). [http://gabrilovich.com/publications/papers/Radinsky2011WTS.pdf A word at a time: computing word relatedness using temporal semantic analysis]. In Proceedings of the 20th international conference on World wide web, pp. 337-346. ACM.

Strube, Michael and Simone Paolo Ponzetto. (2006). [http://www.aaai.org/Papers/AAAI/2006/AAAI06-223.pdf WikiRelate! Computing Semantic Relatedness Using Wikipedia]. Proceedings of The 21st National Conference on Artificial Intelligence (AAAI), Boston, MA.

Yih, W. and Qazvinian, V. (2012). [http://aclweb.org/anthology/N/N12/N12-1077.pdf Measuring Word Relatedness Using Heterogeneous Vector Space Models]. Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2012).

[[Category:State of the art]]

WordSimilarity-353 Test Collection (State of the art)

2014-01-15T19:19:49Z

Scottyih: /* References */

* [http://www.cs.technion.ac.il/~gabr/resources/data/wordsim353/ WordSimilarity-353 Test Collection]
* contains two sets of English word pairs along with human-assigned similarity judgements
* first set (set1) contains 153 word pairs along with their similarity scores assigned by 13 subjects
* second set (set2) contains 200 word pairs with similarity assessed by 16 subjects
* WordSimilarity-353 dataset is available [http://www.cs.technion.ac.il/~gabr/resources/data/wordsim353/ here]
* performance is measured by [http://en.wikipedia.org/wiki/Spearman_rank_correlation Spearman's rank correlation coefficient]
* introduced by [http://www.cs.technion.ac.il/~gabr/papers/tois_context.pdf Finkelstein et al. (2002)]
* subsequently used by many other researchers
* see also: [[Similarity (State of the art)]]

== Table of results ==

* '''Listed in order of increasing [http://en.wikipedia.org/wiki/Spearman_rank_correlation Spearman's rho].'''

{| border="1" cellpadding="5" cellspacing="1"
|-
! Algorithm
! Reference for algorithm
! Reference for reported results
! Type
! Spearman's rho
! Pearson's r
|-
| L&C
| Leacock and Chodorow (1998)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.302
| 0.356
|-
| WNE
| Jarmasz (2003)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.305
| 0.271
|-
| J&C
| Jiang and Conrath 1997
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.318
| 0.354
|-
| L&C
| Leacock and Chodorow (1998)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.348
| 0.341
|-
| H&S
| Hirst and St-Onge (1998)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.302
| 0.356
|-
| Lin
| Lin (1998)
| Hassan and Mihalcea (2011)
| Corpus-based
| 0.348
| 0.357
|-
| Resnik
| Resnik (1995)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.353
| 0.365
|-
| ROGET
| Jarmasz (2003)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.415
| 0.536
|-
| C&W
| Collobert and Weston (2008)
| Collobert and Weston (2008)
| Corpus-based
| 0.5
| N/A
|-
| WikiRelate
| Strube and Ponzetto (2006)
| Strube and Ponzetto (2006)
| Corpus-based
| N/A
| 0.48
|-
| LSA
| Landauer et al. (1997)
| Hassan and Mihalcea (2011)
| Corpus-based
| 0.581
| 0.492
|-
| LSA
| Landauer et al. (1997)
| Hassan and Mihalcea (2011)
| Corpus-based
| 0.581
| 0.563
|-
| simVB+simWN
| Finkelstein et al. (2002)
| Finkelstein et al. (2002)
| Hybrid
| N/A
| 0.55
|-
| SSA
| Hassan and Mihalcea (2011)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.622
| 0.629
|-
| HSMN+csmRNN
| Luong et al. (2013)
| Luong et al. (2013)
| Corpus-based
| 0.65
| N/A
|-
| Multi-prototype
| Huang et al. (2012)
| Huang et al. (2012)
| Corpus-based
| 0.71
| N/A
|-
| Multi-lingual SSA
| Hassan et al. (2011)
| Hassan et al. (2011)
| Corpus-based
| 0.713
| 0.674
|-
| ESA
| Gabrilovich and Markovitch (2007)
| Gabrilovich and Markovitch (2007)
| Corpus-based
| 0.748
| 0.503
|-
| TSA
| Radinsky et al. (2011)
| Radinsky et al. (2011)
| Hybrid
| 0.80
| N/A
|-
| CLEAR
| Halawi et al. (2012)
| Halawi et al. (2012)
| Corpus-based
| 0.81
| N/A
|-
|}

== References ==

Finkelstein, Lev, Evgeniy Gabrilovich, Yossi Matias, Ehud Rivlin, Zach Solan, Gadi Wolfman, and Eytan Ruppin. (2002) [http://www.cs.technion.ac.il/~gabr/papers/tois_context.pdf Placing Search in Context: The Concept Revisited]. ACM Transactions on Information Systems, 20(1):116-131.

Gabrilovich, Evgeniy, and Shaul Markovitch, [http://www.cs.technion.ac.il/~gabr/papers/ijcai-2007-sim.pdf Computing Semantic Relatedness using Wikipedia-based Explicit Semantic Analysis], Proceedings of The 20th International Joint Conference on Artificial Intelligence (IJCAI), Hyderabad, India, 2007.

Hassan, Samer, and Rada Mihalcea: [http://www.cse.unt.edu/~rada/papers/hassan.aaai11.pdf‎ Semantic Relatedness Using Salient Semantic Analysis]. AAAI 2011

Hirst, Graeme and David St-Onge. Lexical chains as representations of context for the detection and correction of malapropisms. In Christiane Fellbaum, editor, WordNet: An Electronic Lexical Database. The MIT Press, Cambridge, MA, pages 305–332, 1998.

Islam, A., and Inkpen, D. 2006. [http://www.site.uottawa.ca/~mdislam/publications/LREC_06_242.pdf Second order co-occurrence pmi for determining the semantic similarity of words]. Proceedings of the International Conference on Language Resources and Evaluation (LREC 2006) 1033–1038.

Jarmasz, M. 2003. [http://www.arxiv.org/pdf/1204.0140 Roget’s thesaurus as a Lexical Resource for Natural Language Processing]. Ph.D. Dissertation, Ottawa Carleton Institute for Computer Science, School of Information Technology and Engineering, University of Ottawa.

Jiang, Jay J. and David W. Conrath. Semantic similarity based on corpus statistics and lexical taxonomy. In Proceedings of International Conference on Research in Computational Linguistics (ROCLING X), Taiwan, pages 19–33, 1997.

Landauer, T. K.; L, T. K.; Laham, D.; Rehder, B.; and Schreiner, M. E. 1997. How well can passage meaning be derived without using word order? a comparison of latent semantic analysis and humans.

Leacock, Claudia and Martin Chodorow. Combining local context and WordNet similarity for word sense identification. In Christiane Fellbaum, editor, WordNet: An Electronic Lexical Database. The MIT Press, Cambridge, MA, pages 265–283, 1998.

Lin, Dekang. An information-theoretic definition of similarity. In Proceedings of the 15th International Conference on Machine Learning, Madison,WI, pages 296–304, 1998.

Pilehvar, M.T., D. Jurgens and R. Navigli. [http://wwwusers.di.uniroma1.it/~navigli/pubs/ACL_2013_Pilehvar_Jurgens_Navigli.pdf Align, Disambiguate and Walk: A Unified Approach for Measuring Semantic Similarity]. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (ACL 2013), Sofia, Bulgaria, August 4-9, 2013, pp. 1341-1351.

Resnik, Philip. Using information content to evaluate semantic similarity. In Proceedings of the 14th International Joint Conference on Artificial Intelligence, pages 448–453, Montreal, Canada, 1995.

Halawi, Guy, Gideon Dror, Evgeniy Gabrilovich, and Yehuda Koren. (2012). [http://gabrilovich.com/publications/papers/Halawi2012LSL.pdf Large-scale learning of word relatedness with constraints]. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 1406-1414. ACM.

Luong, Minh-Thang, Richard Socher, and Christopher D. Manning. (2013). [http://nlp.stanford.edu/~lmthang/data/papers/conll13_morpho.pdf Better word representations with recursive neural networks for morphology]. CoNLL-2013: 104.

Radinsky, Kira, Eugene Agichtein, Evgeniy Gabrilovich, and Shaul Markovitch. (2011). [http://gabrilovich.com/publications/papers/Radinsky2011WTS.pdf A word at a time: computing word relatedness using temporal semantic analysis]. In Proceedings of the 20th international conference on World wide web, pp. 337-346. ACM.

Strube, Michael and Simone Paolo Ponzetto. (2006). [http://www.aaai.org/Papers/AAAI/2006/AAAI06-223.pdf WikiRelate! Computing Semantic Relatedness Using Wikipedia]. Proceedings of The 21st National Conference on Artificial Intelligence (AAAI), Boston, MA.

Yih, W. and Qazvinian, V. (2012). [http://aclweb.org/anthology/N/N12/N12-1077.pdf Measuring Word Relatedness Using Heterogeneous Vector Space Models]. Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2012).

[[Category:State of the art]]

WordSimilarity-353 Test Collection (State of the art)

2014-01-15T19:19:20Z

Scottyih: /* References */

* [http://www.cs.technion.ac.il/~gabr/resources/data/wordsim353/ WordSimilarity-353 Test Collection]
* contains two sets of English word pairs along with human-assigned similarity judgements
* first set (set1) contains 153 word pairs along with their similarity scores assigned by 13 subjects
* second set (set2) contains 200 word pairs with similarity assessed by 16 subjects
* WordSimilarity-353 dataset is available [http://www.cs.technion.ac.il/~gabr/resources/data/wordsim353/ here]
* performance is measured by [http://en.wikipedia.org/wiki/Spearman_rank_correlation Spearman's rank correlation coefficient]
* introduced by [http://www.cs.technion.ac.il/~gabr/papers/tois_context.pdf Finkelstein et al. (2002)]
* subsequently used by many other researchers
* see also: [[Similarity (State of the art)]]

== Table of results ==

* '''Listed in order of increasing [http://en.wikipedia.org/wiki/Spearman_rank_correlation Spearman's rho].'''

{| border="1" cellpadding="5" cellspacing="1"
|-
! Algorithm
! Reference for algorithm
! Reference for reported results
! Type
! Spearman's rho
! Pearson's r
|-
| L&C
| Leacock and Chodorow (1998)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.302
| 0.356
|-
| WNE
| Jarmasz (2003)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.305
| 0.271
|-
| J&C
| Jiang and Conrath 1997
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.318
| 0.354
|-
| L&C
| Leacock and Chodorow (1998)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.348
| 0.341
|-
| H&S
| Hirst and St-Onge (1998)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.302
| 0.356
|-
| Lin
| Lin (1998)
| Hassan and Mihalcea (2011)
| Corpus-based
| 0.348
| 0.357
|-
| Resnik
| Resnik (1995)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.353
| 0.365
|-
| ROGET
| Jarmasz (2003)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.415
| 0.536
|-
| C&W
| Collobert and Weston (2008)
| Collobert and Weston (2008)
| Corpus-based
| 0.5
| N/A
|-
| WikiRelate
| Strube and Ponzetto (2006)
| Strube and Ponzetto (2006)
| Corpus-based
| N/A
| 0.48
|-
| LSA
| Landauer et al. (1997)
| Hassan and Mihalcea (2011)
| Corpus-based
| 0.581
| 0.492
|-
| LSA
| Landauer et al. (1997)
| Hassan and Mihalcea (2011)
| Corpus-based
| 0.581
| 0.563
|-
| simVB+simWN
| Finkelstein et al. (2002)
| Finkelstein et al. (2002)
| Hybrid
| N/A
| 0.55
|-
| SSA
| Hassan and Mihalcea (2011)
| Hassan and Mihalcea (2011)
| Knowledge-based
| 0.622
| 0.629
|-
| HSMN+csmRNN
| Luong et al. (2013)
| Luong et al. (2013)
| Corpus-based
| 0.65
| N/A
|-
| Multi-prototype
| Huang et al. (2012)
| Huang et al. (2012)
| Corpus-based
| 0.71
| N/A
|-
| Multi-lingual SSA
| Hassan et al. (2011)
| Hassan et al. (2011)
| Corpus-based
| 0.713
| 0.674
|-
| ESA
| Gabrilovich and Markovitch (2007)
| Gabrilovich and Markovitch (2007)
| Corpus-based
| 0.748
| 0.503
|-
| TSA
| Radinsky et al. (2011)
| Radinsky et al. (2011)
| Hybrid
| 0.80
| N/A
|-
| CLEAR
| Halawi et al. (2012)
| Halawi et al. (2012)
| Corpus-based
| 0.81
| N/A
|-
|}

== References ==

Finkelstein, Lev, Evgeniy Gabrilovich, Yossi Matias, Ehud Rivlin, Zach Solan, Gadi Wolfman, and Eytan Ruppin. (2002) [http://www.cs.technion.ac.il/~gabr/papers/tois_context.pdf Placing Search in Context: The Concept Revisited]. ACM Transactions on Information Systems, 20(1):116-131.

Gabrilovich, Evgeniy, and Shaul Markovitch, [http://www.cs.technion.ac.il/~gabr/papers/ijcai-2007-sim.pdf Computing Semantic Relatedness using Wikipedia-based Explicit Semantic Analysis], Proceedings of The 20th International Joint Conference on Artificial Intelligence (IJCAI), Hyderabad, India, 2007.

Hassan, Samer, and Rada Mihalcea: [http://www.cse.unt.edu/~rada/papers/hassan.aaai11.pdf‎ Semantic Relatedness Using Salient Semantic Analysis]. AAAI 2011

Hirst, Graeme and David St-Onge. Lexical chains as representations of context for the detection and correction of malapropisms. In Christiane Fellbaum, editor, WordNet: An Electronic Lexical Database. The MIT Press, Cambridge, MA, pages 305–332, 1998.

Islam, A., and Inkpen, D. 2006. [http://www.site.uottawa.ca/~mdislam/publications/LREC_06_242.pdf Second order co-occurrence pmi for determining the semantic similarity of words]. Proceedings of the International Conference on Language Resources and Evaluation (LREC 2006) 1033–1038.

Jarmasz, M. 2003. [http://www.arxiv.org/pdf/1204.0140 Roget’s thesaurus as a Lexical Resource for Natural Language Processing]. Ph.D. Dissertation, Ottawa Carleton Institute for Computer Science, School of Information Technology and Engineering, University of Ottawa.

Jiang, Jay J. and David W. Conrath. Semantic similarity based on corpus statistics and lexical taxonomy. In Proceedings of International Conference on Research in Computational Linguistics (ROCLING X), Taiwan, pages 19–33, 1997.

Landauer, T. K.; L, T. K.; Laham, D.; Rehder, B.; and Schreiner, M. E. 1997. How well can passage meaning be derived without using word order? a comparison of latent semantic analysis and humans.

Leacock, Claudia and Martin Chodorow. Combining local context and WordNet similarity for word sense identification. In Christiane Fellbaum, editor, WordNet: An Electronic Lexical Database. The MIT Press, Cambridge, MA, pages 265–283, 1998.

Lin, Dekang. An information-theoretic definition of similarity. In Proceedings of the 15th International Conference on Machine Learning, Madison,WI, pages 296–304, 1998.

Pilehvar, M.T., D. Jurgens and R. Navigli. [http://wwwusers.di.uniroma1.it/~navigli/pubs/ACL_2013_Pilehvar_Jurgens_Navigli.pdf Align, Disambiguate and Walk: A Unified Approach for Measuring Semantic Similarity]. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (ACL 2013), Sofia, Bulgaria, August 4-9, 2013, pp. 1341-1351.

Resnik, Philip. Using information content to evaluate semantic similarity. In Proceedings of the 14th International Joint Conference on Artificial Intelligence, pages 448–453, Montreal, Canada, 1995.

Halawi, Guy, Gideon Dror, Evgeniy Gabrilovich, and Yehuda Koren. (2012). [http://gabrilovich.com/publications/papers/Halawi2012LSL.pdf Large-scale learning of word relatedness with constraints]. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 1406-1414. ACM.

Luong, Minh-Thang, Richard Socher, and Christopher D. Manning. (2013). [http://nlp.stanford.edu/~lmthang/data/papers/conll13_morpho.pdf Better word representations with recursive neural networks for morphology]. CoNLL-2013: 104.

Radinsky, Kira, Eugene Agichtein, Evgeniy Gabrilovich, and Shaul Markovitch. (2011). [http://gabrilovich.com/publications/papers/Radinsky2011WTS.pdf A word at a time: computing word relatedness using temporal semantic analysis]. In Proceedings of the 20th international conference on World wide web, pp. 337-346. ACM.

Strube, Michael and Simone Paolo Ponzetto. (2006). [http://www.aaai.org/Papers/AAAI/2006/AAAI06-223.pdf WikiRelate! Computing Semantic Relatedness Using Wikipedia]. Proceedings of The 21st National Conference on Artificial Intelligence (AAAI), Boston, MA.

Yih, W. and Qazvinian, V. (2012). [http://aclweb.org/anthology/N/N12/N12-1077.pdf Measuring Word Relatedness Using Heterogeneous Vector Space Models]. Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT).

[[Category:State of the art]]