ACL Wiki - User contributions [en]

https://www.aclweb.org/aclwiki/api.php?action=feedcontributions&feedformat=atom&user=Amarchetti ACL Wiki - User contributions [en] 2026-04-27T18:43:00Z User contributions MediaWiki 1.43.6 https://www.aclweb.org/aclwiki/index.php?title=RTE7_-_Ablation_Tests&diff=9286 RTE7 - Ablation Tests 2012-03-27T14:40:07Z

<p>Amarchetti: </p> <hr /> <div>The following table lists the results of the ablation tests submitted by participants to RTE7 .<br><br /> The exploratory effort about knowledge resources, started in RTE5 and extended to tools in RTE-6, was proposed also in RTE-7.<br /> <br><br /> <br><br /> <br /> In the table below, the first column contains the specific resources which have been ablated.<br /> <br>The second column lists the Team Run in the form ''[name_of_the_Team][number_of_the_submitted_run].[submission_task]'' (e.g. BIU1.2way, Boeing3.3way).<br>The third column presents the normalized difference between the accuracy of the complete system run and the accuracy of the ablation run (i.e. the output of the complete system without the ablated resource), showing the impact of the resource on the performance of the system.<br><br /> The fourth column contains a brief description of the specific usage of the resource. It is based on the information provided both in the "readme" files submitted together with the ablation tests and in the system reports published in the RTE7 proceedings.<br><br /> If the ablated resource is highlighted in yellow, it is a tool, otherwise is a knowledge resource.<br><br /> <br /> Participants are kindly invited to check if all the inserted information is correct and complete.<br /> <br /> <br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! Ablated Component<br /> ! Ablation Run<ref>For further information about participants, click here: [[RTE Challenges - Data about participants]]</ref><br /> ! <small>Resource impact - F1 </small><br /> ! Resource Usage Description<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | BIU2_abl-1<br /> | style="text-align: center;"| -0.05<br /> | style="text-align: center;"| Without WordNet, which is used as a lexical rulebase resource<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |Direct<br /> | BIU2_abl-2<br /> | style="text-align: center;"| 0.94<br /> | style="text-align: center;"| Without Bap (AKA "Direct"), which is used as a lexical rulebase resource<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | BIU2_abl-3<br /> | style="text-align: center;"| 1.56<br /> | style="text-align: center;"| Without Wikipedia, which is used as a lexical rulebase resource<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Coreference resolver<br /> | BIU2_abl-4<br /> | style="text-align: center;"| 0.69<br /> | style="text-align: center;"| Without any coreference resolution engine, instead of sing ArkRef to obtain coref information from the text, when preprocessing it<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | DFKI1_abl-1<br /> | style="text-align: center;"| -0.14<br /> | style="text-align: center;"| Features based on WordNet similarity measures (JWNL).<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entity Recognition<br /> | DFKI1_abl-2<br /> | style="text-align: center;"| 2.08<br /> | style="text-align: center;"| Features based on WordNet similarity measures (JWNL).<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | FBK_irst3_abl-2<br /> | style="text-align: center;"| -2.64<br /> | style="text-align: center;"| Ablating wikipedia LSA similarity scores.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entity Recognition<br /> | FBK_irst3_abl-3<br /> | style="text-align: center;"| -0.89<br /> | style="text-align: center;"| Ablating named entities matching module.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Paraphrase Table<br /> | FBK_irst3_abl-4<br /> | style="text-align: center;"| -1.43<br /> | style="text-align: center;"| Ablating paraphrase matching module. The paraphrases were extracted from parallel corpora.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Acronym List<br /> | IKOMA3_abl-1<br /> | style="text-align: center;"| -0.16<br /> | style="text-align: center;"| No acronyms of organization names extracted from the corpus.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | CatVar<br /> | IKOMA3_abl-2<br /> | style="text-align: center;"| 0.84<br /> | style="text-align: center;"| No CatVar.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | IKOMA3_abl-3<br /> | style="text-align: center;"| 0.85<br /> | style="text-align: center;"| No WordNet.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | JU_CSE_TAC1_abl-1<br /> | style="text-align: center;"| 9.81<br /> | style="text-align: center;"| WordNet Ablated<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entity Recognition<br /> | JU_CSE_TAC1_abl-2<br /> | style="text-align: center;"| 7.97<br /> | style="text-align: center;"| NER Ablated<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | SINAI1_abl-1<br /> | style="text-align: center;"| -0.12<br /> | style="text-align: center;"| Resource ablated: lexical similarity module based on Personalized Page Rank vectors over WordNet 3.0<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | SJTU_CIT1_abl-1<br /> | style="text-align: center;"| 8.89<br /> | style="text-align: center;"| we removed wikipedia resouce<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | SJTU_CIT1_abl-2<br /> | style="text-align: center;"| 5.93<br /> | style="text-align: center;"| we removed verbocern resource<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | u_tokyo1_abl-1<br /> | style="text-align: center;"| 0.83<br /> | style="text-align: center;"| Ablated resource is WordNet<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | u_tokyo2_abl-1<br /> | style="text-align: center;"| 0.64<br /> | style="text-align: center;"| Ablated resource is WordNet<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | u_tokyo3_abl-1<br /> | style="text-align: center;"| 0.99<br /> | style="text-align: center;"| Ablated resource is WordNet<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Knowledge Resource<br /> | UAIC20112_abl-1<br /> | style="text-align: center;"| 0<br /> | style="text-align: center;"| Ablation of the BK (acronym database and world knowledge component)<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entity Recognition<br /> | UAIC20112_abl-3<br /> | style="text-align: center;"| -8.29<br /> | style="text-align: center;"| Ablation of the NE resources.<br /> <br /> |}<br /> <br><br /> ==Footnotes==<br /> <references /><br /> <br /> <br /> Return to [[RTE Knowledge Resources]]</div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE7_-_Ablation_Tests&diff=9285 RTE7 - Ablation Tests 2012-03-27T14:31:13Z

<p>Amarchetti: </p> <hr /> <div>The following table lists the results of the ablation tests submitted by participants to RTE7 .<br><br /> The exploratory effort about knowledge resources, started in RTE5 and extended to tools in RTE-6, was proposed also in RTE-7.<br /> <br><br /> <br><br /> <br /> In the table below, the first column contains the specific resources which have been ablated.<br /> <br>The second column lists the Team Run in the form ''[name_of_the_Team][number_of_the_submitted_run].[submission_task]'' (e.g. BIU1.2way, Boeing3.3way).<br>The third column presents the normalized difference between the accuracy of the complete system run and the accuracy of the ablation run (i.e. the output of the complete system without the ablated resource), showing the impact of the resource on the performance of the system.<br><br /> The fourth column contains a brief description of the specific usage of the resource. It is based on the information provided both in the "readme" files submitted together with the ablation tests and in the system reports published in the RTE7 proceedings.<br><br /> If the ablated resource is highlighted in yellow, it is a tool, otherwise is a knowledge resource.<br><br /> <br /> Participants are kindly invited to check if all the inserted information is correct and complete.<br /> <br /> <br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! Ablated Component<br /> ! Ablation Run<ref>For further information about participants, click here: [[RTE Challenges - Data about participants]]</ref><br /> ! <small>Resource impact - F1 </small><br /> ! Resource Usage Description<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | BIU2_abl-1<br /> | style="text-align: center;"| -0.05<br /> | style="text-align: center;"| Without WordNet, which is used as a lexical rulebase resource<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |Direct<br /> | BIU2_abl-2<br /> | style="text-align: center;"| 0.94<br /> | style="text-align: center;"| Without Bap (AKA "Direct"), which is used as a lexical rulebase resource<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | BIU2_abl-3<br /> | style="text-align: center;"| 1.56<br /> | style="text-align: center;"| Without Wikipedia, which is used as a lexical rulebase resource<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Coreference resolver<br /> | BIU2_abl-4<br /> | style="text-align: center;"| 0.69<br /> | style="text-align: center;"| Without any coreference resolution engine, instead of sing ArkRef to obtain coref information from the text, when preprocessing it<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | DFKI_abl-1<br /> | style="text-align: center;"| -0.14<br /> | style="text-align: center;"| Features based on WordNet similarity measures (JWNL).<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entity Recognition<br /> | DFKI_abl-2<br /> | style="text-align: center;"| 2.08<br /> | style="text-align: center;"| Features based on WordNet similarity measures (JWNL).<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | FBK_irst3_abl-2<br /> | style="text-align: center;"| -2.64<br /> | style="text-align: center;"| Ablating wikipedia LSA similarity scores.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entity Recognition<br /> | FBK_irst3_abl-3<br /> | style="text-align: center;"| -0.89<br /> | style="text-align: center;"| Ablating named entities matching module.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Paraphrase Table<br /> | FBK_irst3_abl-4<br /> | style="text-align: center;"| -1.43<br /> | style="text-align: center;"| Ablating paraphrase matching module. The paraphrases were extracted from parallel corpora.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Acronym List<br /> | IKOMA3_abl-1<br /> | style="text-align: center;"| -0.16<br /> | style="text-align: center;"| No acronyms of organization names extracted from the corpus.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | CatVar<br /> | IKOMA3_abl-2<br /> | style="text-align: center;"| 0.84<br /> | style="text-align: center;"| No CatVar.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | IKOMA3_abl-3<br /> | style="text-align: center;"| 0.85<br /> | style="text-align: center;"| No WordNet.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | JU_CSE_TAC1_abl-1<br /> | style="text-align: center;"| 9.81<br /> | style="text-align: center;"| WordNet Ablated<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entity Recognition<br /> | JU_CSE_TAC1_abl-2<br /> | style="text-align: center;"| 7.97<br /> | style="text-align: center;"| NER Ablated<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | SINAI1_abl-1<br /> | style="text-align: center;"| -0.12<br /> | style="text-align: center;"| Resource ablated: lexical similarity module based on Personalized Page Rank vectors over WordNet 3.0<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | SJTU_CIT1_abl-1<br /> | style="text-align: center;"| 8.89<br /> | style="text-align: center;"| we removed wikipedia resouce<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | SJTU_CIT1_abl-2<br /> | style="text-align: center;"| 5.93<br /> | style="text-align: center;"| we removed verbocern resource<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | u_tokyo1_abl-1<br /> | style="text-align: center;"| 0.83<br /> | style="text-align: center;"| Ablated resource is WordNet<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | u_tokyo2_abl-2<br /> | style="text-align: center;"| 0.64<br /> | style="text-align: center;"| Ablated resource is WordNet<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | u_tokyo2_abl-3<br /> | style="text-align: center;"| 0.99<br /> | style="text-align: center;"| Ablated resource is WordNet<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Knowledge Resource<br /> | UAIC20112_abl-1<br /> | style="text-align: center;"| 0<br /> | style="text-align: center;"| Ablation of the BK (acronym database and world knowledge component)<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entity Recognition<br /> | UAIC20112_abl-3<br /> | style="text-align: center;"| -8.29<br /> | style="text-align: center;"| Ablation of the NE resources.<br /> <br /> |}<br /> <br><br /> ==Footnotes==<br /> <references /><br /> <br /> <br /> Return to [[RTE Knowledge Resources]]</div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE7_-_Ablation_Tests&diff=9284 RTE7 - Ablation Tests 2012-03-27T14:30:46Z

<p>Amarchetti: </p> <hr /> <div>The following table lists the results of the ablation tests submitted by participants to RTE7 .<br><br /> The exploratory effort about knowledge resources, started in RTE5 and extended to tools in RTE-6, was proposed also in RTE-7.<br /> <br><br /> <br><br /> <br /> In the table below, the first column contains the specific resources which have been ablated.<br /> <br>The second column lists the Team Run in the form ''[name_of_the_Team][number_of_the_submitted_run].[submission_task]'' (e.g. BIU1.2way, Boeing3.3way).<br>The third column presents the normalized difference between the accuracy of the complete system run and the accuracy of the ablation run (i.e. the output of the complete system without the ablated resource), showing the impact of the resource on the performance of the system.<br><br /> The fourth column contains a brief description of the specific usage of the resource. It is based on the information provided both in the "readme" files submitted together with the ablation tests and in the system reports published in the RTE7 proceedings.<br><br /> If the ablated resource is highlighted in yellow, it is a tool, otherwise is a knowledge resource.<br><br /> <br /> Participants are kindly invited to check if all the inserted information is correct and complete.<br /> <br /> <br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! Ablated Component<br /> ! Ablation Run<ref>For further information about participants, click here: [[RTE Challenges - Data about participants]]</ref><br /> ! <small>Resource impact - F1 </small><br /> ! Resource Usage Description<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | BIU2_abl-1<br /> | style="text-align: center;"| -0.05<br /> | style="text-align: center;"| Without WordNet, which is used as a lexical rulebase resource<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |Direct<br /> | BIU1_abl-2<br /> | style="text-align: center;"| 0.94<br /> | style="text-align: center;"| Without Bap (AKA "Direct"), which is used as a lexical rulebase resource<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | BIU1_abl-3<br /> | style="text-align: center;"| 1.56<br /> | style="text-align: center;"| Without Wikipedia, which is used as a lexical rulebase resource<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Coreference resolver<br /> | BIU2_abl-4<br /> | style="text-align: center;"| 0.69<br /> | style="text-align: center;"| Without any coreference resolution engine, instead of sing ArkRef to obtain coref information from the text, when preprocessing it<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | DFKI_abl-1<br /> | style="text-align: center;"| -0.14<br /> | style="text-align: center;"| Features based on WordNet similarity measures (JWNL).<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entity Recognition<br /> | DFKI_abl-2<br /> | style="text-align: center;"| 2.08<br /> | style="text-align: center;"| Features based on WordNet similarity measures (JWNL).<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | FBK_irst3_abl-2<br /> | style="text-align: center;"| -2.64<br /> | style="text-align: center;"| Ablating wikipedia LSA similarity scores.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entity Recognition<br /> | FBK_irst3_abl-3<br /> | style="text-align: center;"| -0.89<br /> | style="text-align: center;"| Ablating named entities matching module.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Paraphrase Table<br /> | FBK_irst3_abl-4<br /> | style="text-align: center;"| -1.43<br /> | style="text-align: center;"| Ablating paraphrase matching module. The paraphrases were extracted from parallel corpora.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Acronym List<br /> | IKOMA3_abl-1<br /> | style="text-align: center;"| -0.16<br /> | style="text-align: center;"| No acronyms of organization names extracted from the corpus.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | CatVar<br /> | IKOMA3_abl-2<br /> | style="text-align: center;"| 0.84<br /> | style="text-align: center;"| No CatVar.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | IKOMA3_abl-3<br /> | style="text-align: center;"| 0.85<br /> | style="text-align: center;"| No WordNet.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | JU_CSE_TAC1_abl-1<br /> | style="text-align: center;"| 9.81<br /> | style="text-align: center;"| WordNet Ablated<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entity Recognition<br /> | JU_CSE_TAC1_abl-2<br /> | style="text-align: center;"| 7.97<br /> | style="text-align: center;"| NER Ablated<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | SINAI1_abl-1<br /> | style="text-align: center;"| -0.12<br /> | style="text-align: center;"| Resource ablated: lexical similarity module based on Personalized Page Rank vectors over WordNet 3.0<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | SJTU_CIT1_abl-1<br /> | style="text-align: center;"| 8.89<br /> | style="text-align: center;"| we removed wikipedia resouce<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | SJTU_CIT1_abl-2<br /> | style="text-align: center;"| 5.93<br /> | style="text-align: center;"| we removed verbocern resource<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | u_tokyo1_abl-1<br /> | style="text-align: center;"| 0.83<br /> | style="text-align: center;"| Ablated resource is WordNet<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | u_tokyo2_abl-2<br /> | style="text-align: center;"| 0.64<br /> | style="text-align: center;"| Ablated resource is WordNet<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | u_tokyo2_abl-3<br /> | style="text-align: center;"| 0.99<br /> | style="text-align: center;"| Ablated resource is WordNet<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Knowledge Resource<br /> | UAIC20112_abl-1<br /> | style="text-align: center;"| 0<br /> | style="text-align: center;"| Ablation of the BK (acronym database and world knowledge component)<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entity Recognition<br /> | UAIC20112_abl-3<br /> | style="text-align: center;"| -8.29<br /> | style="text-align: center;"| Ablation of the NE resources.<br /> <br /> |}<br /> <br><br /> ==Footnotes==<br /> <references /><br /> <br /> <br /> Return to [[RTE Knowledge Resources]]</div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE7_-_Ablation_Tests&diff=9283 RTE7 - Ablation Tests 2012-03-27T14:29:01Z

<p>Amarchetti: </p> <hr /> <div>The following table lists the results of the ablation tests submitted by participants to RTE7 .<br><br /> The exploratory effort about knowledge resources, started in RTE5 and extended to tools in RTE-6, was proposed also in RTE-7.<br /> <br><br /> <br><br /> <br /> In the table below, the first column contains the specific resources which have been ablated.<br /> <br>The second column lists the Team Run in the form ''[name_of_the_Team][number_of_the_submitted_run].[submission_task]'' (e.g. BIU1.2way, Boeing3.3way).<br>The third column presents the normalized difference between the accuracy of the complete system run and the accuracy of the ablation run (i.e. the output of the complete system without the ablated resource), showing the impact of the resource on the performance of the system.<br><br /> The fourth column contains a brief description of the specific usage of the resource. It is based on the information provided both in the "readme" files submitted together with the ablation tests and in the system reports published in the RTE7 proceedings.<br><br /> If the ablated resource is highlighted in yellow, it is a tool, otherwise is a knowledge resource.<br><br /> <br /> Participants are kindly invited to check if all the inserted information is correct and complete.<br /> <br /> <br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! Ablated Component<br /> ! Ablation Run<ref>For further information about participants, click here: [[RTE Challenges - Data about participants]]</ref><br /> ! <small>Resource impact - F1 </small><br /> ! Resource Usage Description<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | BIU2_abl-1<br /> | style="text-align: center;"| -0.05<br /> | style="text-align: center;"| Without WordNet, which is used as a lexical rulebase resource<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |Direct<br /> | BIU1_abl-2<br /> | style="text-align: center;"| 0.94<br /> | style="text-align: center;"| Without Bap (AKA "Direct"), which is used as a lexical rulebase resource<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | BIU1_abl-3<br /> | style="text-align: center;"| 1.56<br /> | style="text-align: center;"| Without Wikipedia, which is used as a lexical rulebase resource<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Coreference resolver<br /> | BIU1_abl-4<br /> | style="text-align: center;"| 0.69<br /> | style="text-align: center;"| Without any coreference resolution engine, instead of sing ArkRef to obtain coref information from the text, when preprocessing it<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | DFKI_abl-1<br /> | style="text-align: center;"| -0.14<br /> | style="text-align: center;"| Features based on WordNet similarity measures (JWNL).<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entity Recognition<br /> | DFKI_abl-2<br /> | style="text-align: center;"| 2.08<br /> | style="text-align: center;"| Features based on WordNet similarity measures (JWNL).<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | FBK_irst3_abl-2<br /> | style="text-align: center;"| -2.64<br /> | style="text-align: center;"| Ablating wikipedia LSA similarity scores.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entity Recognition<br /> | FBK_irst3_abl-3<br /> | style="text-align: center;"| -0.89<br /> | style="text-align: center;"| Ablating named entities matching module.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Paraphrase Table<br /> | FBK_irst3_abl-4<br /> | style="text-align: center;"| -1.43<br /> | style="text-align: center;"| Ablating paraphrase matching module. The paraphrases were extracted from parallel corpora.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Acronym List<br /> | IKOMA3_abl-1<br /> | style="text-align: center;"| -0.16<br /> | style="text-align: center;"| No acronyms of organization names extracted from the corpus.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | CatVar<br /> | IKOMA3_abl-2<br /> | style="text-align: center;"| 0.84<br /> | style="text-align: center;"| No CatVar.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | IKOMA3_abl-3<br /> | style="text-align: center;"| 0.85<br /> | style="text-align: center;"| No WordNet.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | JU_CSE_TAC1_abl-1<br /> | style="text-align: center;"| 9.81<br /> | style="text-align: center;"| WordNet Ablated<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entity Recognition<br /> | JU_CSE_TAC1_abl-2<br /> | style="text-align: center;"| 7.97<br /> | style="text-align: center;"| NER Ablated<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | SINAI1_abl-1<br /> | style="text-align: center;"| -0.12<br /> | style="text-align: center;"| Resource ablated: lexical similarity module based on Personalized Page Rank vectors over WordNet 3.0<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | SJTU_CIT1_abl-1<br /> | style="text-align: center;"| 8.89<br /> | style="text-align: center;"| we removed wikipedia resouce<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | SJTU_CIT1_abl-2<br /> | style="text-align: center;"| 5.93<br /> | style="text-align: center;"| we removed verbocern resource<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | u_tokyo1_abl-1<br /> | style="text-align: center;"| 0.83<br /> | style="text-align: center;"| Ablated resource is WordNet<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | u_tokyo2_abl-2<br /> | style="text-align: center;"| 0.64<br /> | style="text-align: center;"| Ablated resource is WordNet<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | u_tokyo2_abl-3<br /> | style="text-align: center;"| 0.99<br /> | style="text-align: center;"| Ablated resource is WordNet<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Knowledge Resource<br /> | UAIC20112_abl-1<br /> | style="text-align: center;"| 0<br /> | style="text-align: center;"| Ablation of the BK (acronym database and world knowledge component)<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entity Recognition<br /> | UAIC20112_abl-3<br /> | style="text-align: center;"| -8.29<br /> | style="text-align: center;"| Ablation of the NE resources.<br /> <br /> |}<br /> <br><br /> ==Footnotes==<br /> <references /><br /> <br /> <br /> Return to [[RTE Knowledge Resources]]</div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE7_-_Ablation_Tests&diff=9282 RTE7 - Ablation Tests 2012-03-27T14:27:21Z

<p>Amarchetti: </p> <hr /> <div>The following table lists the results of the ablation tests submitted by participants to RTE7 .<br><br /> The exploratory effort about knowledge resources, started in RTE5 and extended to tools in RTE-6, was proposed also in RTE-7.<br /> <br><br /> <br><br /> <br /> In the table below, the first column contains the specific resources which have been ablated.<br /> <br>The second column lists the Team Run in the form ''[name_of_the_Team][number_of_the_submitted_run].[submission_task]'' (e.g. BIU1.2way, Boeing3.3way).<br>The third column presents the normalized difference between the accuracy of the complete system run and the accuracy of the ablation run (i.e. the output of the complete system without the ablated resource), showing the impact of the resource on the performance of the system.<br><br /> The fourth column contains a brief description of the specific usage of the resource. It is based on the information provided both in the "readme" files submitted together with the ablation tests and in the system reports published in the RTE7 proceedings.<br><br /> If the ablated resource is highlighted in yellow, it is a tool, otherwise is a knowledge resource.<br><br /> <br /> Participants are kindly invited to check if all the inserted information is correct and complete.<br /> <br /> <br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! Ablated Component<br /> ! Ablation Run<ref>For further information about participants, click here: [[RTE Challenges - Data about participants]]</ref><br /> ! <small>Resource impact - F1 </small><br /> ! Resource Usage Description<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | BIU2_abl-1<br /> | style="text-align: center;"| -0.05<br /> | style="text-align: center;"| Without WordNet, which is used as a lexical rulebase resource<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |Direct<br /> | BIU1_abl-2<br /> | style="text-align: center;"| 0.94<br /> | style="text-align: center;"| Without Bap (AKA "Direct"), which is used as a lexical rulebase resource<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | BIU1_abl-3<br /> | style="text-align: center;"| 1.56<br /> | style="text-align: center;"| Without Wikipedia, which is used as a lexical rulebase resource<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Coreference resolver<br /> | BIU1_abl-4<br /> | style="text-align: center;"| 0.69<br /> | style="text-align: center;"| Without any coreference resolution engine, instead of sing ArkRef to obtain coref information from the text, when preprocessing it<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | DFKI_abl-1<br /> | style="text-align: center;"| -0.14<br /> | style="text-align: center;"| Features based on WordNet similarity measures (JWNL).<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entity Recognition<br /> | DFKI_abl-2<br /> | style="text-align: center;"| 2.08<br /> | style="text-align: center;"| Features based on WordNet similarity measures (JWNL).<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | FBK_irst3_abl-2<br /> | style="text-align: center;"| -2.64<br /> | style="text-align: center;"| Ablating wikipedia LSA similarity scores.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entity Recognition<br /> | FBK_irst3_abl-3<br /> | style="text-align: center;"| -0.89<br /> | style="text-align: center;"| Ablating named entities matching module.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Paraphrase Table<br /> | FBK_irst3_abl-4<br /> | style="text-align: center;"| -1.43<br /> | style="text-align: center;"| Ablating paraphrase matching module. The paraphrases were extracted from parallel corpora.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Acronym List<br /> | IKOMA3_abl-1<br /> | style="text-align: center;"| -0.16<br /> | style="text-align: center;"| No acronyms of organization names extracted from the corpus.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | CatVar<br /> | IKOMA3_abl-2<br /> | style="text-align: center;"| 0.84<br /> | style="text-align: center;"| No CatVar.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | IKOMA3_abl-3<br /> | style="text-align: center;"| 0.85<br /> | style="text-align: center;"| No WordNet.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | JU_CSE_TAC1_abl-1<br /> | style="text-align: center;"| 9.81<br /> | style="text-align: center;"| WordNet Ablated<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entity Recognition<br /> | JU_CSE_TAC1_abl-2<br /> | style="text-align: center;"| 7.97<br /> | style="text-align: center;"| NER Ablated<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | SINAI1_abl-1<br /> | style="text-align: center;"| -0.12<br /> | style="text-align: center;"| Resource ablated: lexical similarity module based on Personalized Page Rank vectors over WordNet 3.0<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | SJTU_CIT1_abl-1<br /> | style="text-align: center;"| 8.89<br /> | style="text-align: center;"| we removed wikipedia resouce<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | SJTU_CIT1_abl-2<br /> | style="text-align: center;"| 5.93<br /> | style="text-align: center;"| we removed verbocern resource<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | u_tokyo1_abl-1<br /> | style="text-align: center;"| 0.83<br /> | style="text-align: center;"| Ablated resource is WordNet<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | u_tokyo2_abl-2<br /> | style="text-align: center;"| 0.64<br /> | style="text-align: center;"| Ablated resource is WordNet<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | u_tokyo2_abl-3<br /> | style="text-align: center;"| 0.99<br /> | style="text-align: center;"| Ablated resource is WordNet<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | UAIC20112_abl-1<br /> | style="text-align: center;"| 0<br /> | style="text-align: center;"| Ablation of the BK (acronym database and world knowledge component)<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entity Recognition<br /> | UAIC20112_abl-3<br /> | style="text-align: center;"| -8.29<br /> | style="text-align: center;"| Ablation of the NE resources.<br /> <br /> |}<br /> <br><br /> ==Footnotes==<br /> <references /><br /> <br /> <br /> Return to [[RTE Knowledge Resources]]</div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE_Knowledge_Resources&diff=8776 RTE Knowledge Resources 2011-04-04T15:21:04Z

<p>Amarchetti: </p> <hr /> <div>Knowledge resources have shown their relevance for applied semantic inference, and are extensively used by applied inference systems, such as those developed within the Textual Entailment framework.<br><br /> <br><br /> This page presents a list of the knowledge resources used by systems that have participated in the last RTE challenges. The first table lists the publicly available resources, the second one lists unpublished resources. Both tables are sortable by Resource name, type, author and number of users.<br><br /> <br><br /> RTE Participants are encouraged to add information about all kind of knowledge resources used, from standard existing resources (e.g. WordNet) to knowledge collections created for specific purposes, which can be made available to the community.<br><br /> <br><br /> __TOC__<br /> <br><br /> === Call for Resources ===<br /> <br /> In order to help the research, all the participants are invited to contribute, sharing their own resources with the RTE community. <br /> <br><br /> Making the resources available to be used by other systems has several advantages. On the one hand, it helps improve the TE technology; on the other hand, it offers an opportunity to further test and evaluate the resource.<br /> <br><br /> <br><br /> [[RTE6 - Call for Resources]]<br /> <br><br /> <br><br /> [[RTE7 - Call for Resources]]<br /> <br><br /> === Ablation Tests ===<br /> An ablation test consists of removing one module at a time from a system, and rerunning the system on the test set with the other modules, except the one tested. <br /> <br><br /> Ablation test are meant to help better understand the relevance of the knowledge resources used by RTE systems, and evaluate the contribution of each of them to the systems' performances. In fact, comparing the results achieved in the ablation tests to those obtained by the systems as a whole allows assessing the contribution given by each single resource.<br /> <br><br /> <br><br /> *[[RTE5 - Ablation Tests]]<br /> *[[RTE6 - Ablation Tests]]<br /> <br><br /> <br /> === Publicly available Resources ===<br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! width="80"|Resource<br /> ! width="50"|Type<br /> ! width="150"|Author<br /> ! class="unsortable" width="300"|Brief description<br /> ! width="45"|<small>PAST Users <ref name:"rtethree">RTE-3 data have been provided by participants by means of a questionnaire.</ref></small><br /> ! width="45"|<small>RTE4 Users<ref name:"rtefour">RTE-4 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! width="45"|<small>RTE5 Users<ref name:"rtefive">RTE-5 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! width="45"|<small>RTE6 Users<ref name:"rtesix">RTE-6 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! class="unsortable" width="30"|<small>Usage info</small><br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[WordNet]]<br /> | Lexical DB<br /> | Princeton University<br /> | Lexical database of English nouns, verbs, adjectives and adverbs<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|21<br /> | style="text-align: center;"|18<br /> |style="text-align: center;"| 22<br /> | [[WordNet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://xwn.hlt.utdallas.edu/index.html eXtended Wordnet]<br /> | Lexical DB<br /> | Human Language Technology Research Institute, University of Texas at Dallas<br /> | Extension of WordNet based on the exploitation of the information contained in WordNet definitional glosses: the glosses are syntactically parsed, transformed into logic forms and content words are semantically disambiguated. The Extended Wordnet is an ongoing project. <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|2<br /> |style="text-align: center;"| 0<br /> | [[eXtended WordNet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://ai.stanford.edu/~rion/swn/ Augmented Wordnet]<br /> | Lexical DB<br /> | Stanford University<br /> | The resource is the result of the application of a learning algorithm for inducing semantic taxonomies from parsed text. The algorithm automatically acquires items of world knowledge, and uses these to produce significantly enhanced versions of WordNet (up to 40,000 synsets more).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Augmented Wordnet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://verbs.colorado.edu/~mpalmer/projects/verbnet.html Verbnet]<br /> | Lexical DB<br /> | University of Colorado Boulder<br /> | Lexicon for English verbs organized into classes extending Levin (1993) classes through refinement and addition of subclasses to achieve syntactic and semantic coherence among members of a class<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Verbnet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[VerbOcean]]<br /> | Lexical DB<br /> | Information Sciences Institute, University of Southern California<br /> | Broad-coverage semantic network of verbs<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|6<br /> |style="text-align: center;"| 7<br /> | [[VerbOcean - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://framenet.icsi.berkeley.edu/ FrameNet]<br /> | Lexical DB<br /> | ICSI (International Computer Science Institute) - Berkley University<br /> | Lexical resource for English words, based on frame semantics (valences) and supported by corpus evidence<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|2<br /> |style="text-align: center;"| 3<br /> | [[Framenet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/meyers/NomBank.html NomBank]<br /> | Lexical DB<br /> | New York University<br /> | Lexical resource containing syntactic frames for nouns, extracted from annotated corpora <br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[NomBank Resource - RTE Users|Users]] <br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://verbs.colorado.edu/~mpalmer/projects/ace.html PropBank]<br /> | Lexical DB<br /> | University of Colorado Boulder <br /> | Lexical resource containing syntactic frames for verbs, extracted from annotated corpora <br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[PropBank Resource - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/nomlex/index.html Nomlex] Plus<br /> | Lexical DB<br /> | New York University<br /> | Dictionary of English nominalizations: it describes the allowed complements for a nominalization and relates the nominal complements to the arguments of the corresponding verb<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Nomlex Plus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.cs.ualberta.ca/~lindek/downloads.htm Dekang Lin’s Thesaurus]<br /> | Thesaurus<br /> | University of Alberta<br /> | Thesaurus automatically constructed using a parsed corpus, based on distributional similarity scores<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 2<br /> | [[Dekang Lin’s Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://icon.shef.ac.uk/Moby/mthes.html Grady Ward's Moby Thesaurus]<br /> | Thesaurus<br /> | University of Sheffield<br /> | Thesaurus containing 30,260 root words, with 2,520,264 synonyms and related terms. Grady Ward placed this thesaurus in the public domain in 1996.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Grady Ward's Moby Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://en.wikipedia.org/wiki/Roget%27s_Thesaurus Roget's Thesaurus]<br /> | Thesaurus<br /> | Peter Mark Roget (Electronic version distributed by University of Chicago)<br /> | Roget's Thesaurus is a widely-used English thesaurus, created by Dr. Peter Mark Roget in 1805. The original edition had 15,000 words, and each new edition has been larger. The electronic edition ([http://machaut.uchicago.edu/rogets version 1.02]) is made available by University of Chicago.<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Roget's Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.wikipedia.org/ Wikipedia]<br /> | Encyclopedia<br /> | <br /> | Free encyclopedia. Used for extraction of lexical-semantic rules (from its more structured parts), named entity recognition, geographical information etc.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|6<br /> |style="text-align: center;"| 6<br /> | [[Wikipedia - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.umbel.org/ Umbel]<br /> | Ontology<br /> | Structured Dynamics LLC, Coralville, IA<br /> | UMBEL stands for Upper Mapping and Binding Exchange Layer and is a lightweight ontology structure for relating Web content and data to a standard set of subject concepts<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Umbel - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.mpi-inf.mpg.de/yago-naga/yago/ YAGO]<br /> | Ontology<br /> | Max-Planck Institute for Informatics, Saarbrücken, Germany<br /> | Light-weight and extensible ontology. It contains more than 2 million entities and 20 million facts about these entities. The facts have been automatically extracted from Wikipedia and unified with WordNet.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[YAGO - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://dbpedia.org/About DBpedia]<br /> | Ontology<br /> | Open community project<br /> | DBpedia is a community effort to extract structured information from Wikipedia and to make this information available on the Web. The DBpedia knowledge base currently describes more than 2.9 million things in 91 different languages and consists of 479 million pieces of information.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[DBpedia - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[DIRT Paraphrase Collection]]<br /> | Collection of paraphrases<br /> | University of Alberta<br /> | DIRT (Discovery of Inference Rules from Text) is both an algorithm and a resulting knowledge collection. The DIRT knowledge collection is the output of the DIRT algorithm over a 1GB set of newspaper text.<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|4<br /> | style="text-align: center;"|3<br /> |style="text-align: center;"| 4<br /> | [[DIRT Paraphrase Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[TEASE]] Collection<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University<br /> | Output of the TEASE algorithm <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Tease Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://badc.nerc.ac.uk/help/abbrevs.html BADC Acronym and Abbreviation List]<br /> | Word List<br /> | BADC (British Atmospheric Data Centre)<br /> | Acronym and Abbreviation List<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[BADC Acronym and Abbreviation List - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://clipdemos.umiacs.umd.edu/catvar/ Catvar The Categorial Variation Database (English)]<br /> | Word List<br /> | University of Maryland<br /> | A Categorial-Variation Database (or Catvar) is a database of clusters of uninflected words (lexemes) and their categorial (i.e. part-of-speech) variants. The database was developed for English using a combination of resources and algorithms, including the LCS Verb and Preposition Databases (Dorr 2001), the Brown Corpus section of the Penn Treebank (Marcus et al. 1994), an English morphological analysis lexicon developed for PC-Kimmo (ENGLEX) (Antworth 1990), WordNet1.6 (Fellbaum 1998), an English Verb-Noun list extracted from Nomlex (Macleod et al. 1998), a similar list extracted from LDOCE (Procter 1978) and the Porter stemmer (Porter 1980).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 1<br /> | [[Categorial-Variation Database - RTE Users|Users]]<br /> <br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.acronym-guide.com/ Acronym Guide]<br /> | Word List<br /> | Acronym-Guide.com <br /> | Acronym and Abbreviation Lists for English, branched in thematic directories <br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|3<br /> |style="text-align: center;"| 0<br /> | [[Acronym Guide - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.ldc.upenn.edu/Catalog/CatalogEntry.jsp?catalogId=LDC2006T13 Web1T 5-grams]<br /> | Word list<br /> | Linguistic Data Consortium, University of Pennsylvania; Google Inc.<br /> | Data set containing English word n-grams and their observed frequency counts. The n-gram counts were generated from approximately 1 trillion word tokens of text from publicly accessible Web pages<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Web1T - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.coli.uni-saarland.de/%7Erwang/resources/RTE3_RTE4_NGD.zip Normalized Google Distance (RTE3&RTE4)]<br /> | Word Pair Co-occurrence<br /> | Saarland University<br /> | Co-occurrence of the word pairs in RTE3 and RTE4 using Normalized Google Distance (Cilibrasi and Vitanyi, 2004). The word pairs are all the possible combinations of content words in T and H. In practice, we used [http://m.www.yahoo.com/ Yahoo!] as the search engine.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Normalized Google Distance (RTE3&RTE4)- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.coli.uni-saarland.de/%7Erwang/resources/RTE5_NGD.zip Normalized Google Distance (RTE5)]<br /> | Word Pair Co-occurrence<br /> | Saarland University<br /> | Co-occurrence of the word pairs in RTE3 and RTE4 using Normalized Google Distance (Cilibrasi and Vitanyi, 2004). The word pairs are all the possible combinations of content words in T and H. In practice, we used [http://m.www.yahoo.com/ Yahoo!] as the search engine.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Normalized Google Distance (RTE5)- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://geonames.usgs.gov/index.html GNIS - Geographic Names Information System]<br /> | Gazetteer<br /> | USGS (United States Geological Survey)<br /> | Database containing the Federal and national standard toponyms for USA, associated areas and Antarctica<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[GNIS - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.geonames.org/ Geonames]<br /> | Gazetteer<br /> | <br /> | Database containing eight million geographical names. It is integrating geographical data such as names of places in various languages, elevation, population and others from various sources.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Geonames - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/paraphrase/ Sekine's Paraphrase Database]<br /> | Collection of paraphrases<br /> | Department of Computer Science, New York University <br /> | Data-base created using Sekine's method, NOT cleaned up by human. It includes 19,975 sets of paraphrases with 191,572 phrases.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Sekine's Paraphrase Database - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://research.microsoft.com/research/downloads/Details/607D14D9-20CD-47E3-85BC-A2F65CD28042/Details.aspx Microsoft Research Paraphrase Corpus]<br /> | Collection of paraphrases<br /> | Microsoft Research<br /> | Text file containing 5800 pairs of sentences which have been extracted from news sources on the web, along with human annotations indicating whether each pair captures a paraphrase/semantic equivalence relationship.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Microsoft Research Paraphrase Corpus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.cs.cornell.edu/~cristian/Without_a_doubt_-_Data.html Downward entailing operators] <br /> | Collection of entailing operators<br /> | Department of Computer Science, Cornell University, Ithaca NY<br /> | System output of an unsupervised algorithm recovering many Downward Entailing operators, like 'doubt'.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Downward entailing operators - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/WikiRules.html WikiRules!]<br /> | Lexical Reference rule-base<br /> | Bar-Ilan University<br /> | Extraction of about 8 million lexical reference rules from the text body (first sentence) and from metadata (links, redirects, parentheses) of Wikipedia. Provides better performance than other automatically constructed resources and comparable performance to WordNet. Offers complementary knowledge to WordNet. <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[WikiRules! - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://www.cs.utexas.edu/users/pclark/dart/ DART]<br /> | Collection of "world knowledge" propositions<br /> | Boeing Research and Technology<br /> | 23 million tuples such as "airplanes can fly to airports", "rivers can flood" collected from abstracted parse trees.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[DART - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://cs.biu.ac.il/~nlp/download/FRED/latest/FRED.zip FRED]<br /> | FrameNet-derived entailment rule-base<br /> | Bar-Ilan University<br /> | This package contains the outputs of the FRED algorithm, an algorithm which extracts entailment rules from FrameNet.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[FRED - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/DIRECT.html DIRECT]<br /> | Directional Distributional Term-Similarity Resource<br /> | Bar-Ilan University<br /> | This is a resource of directional distributional term-similarity rules (mostly lexical entailment rules) automatically extracted using the inclusion relation as described in (Kotlerman et.al., ACL-09).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[DIRECT - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/binaryDirt.html binaryDIRT]<br /> | Entailment rules between binary templates using DIRT algorithm<br /> | Bar-Ilan University<br /> | This resource contains entailment rules over binary templates learned over the Reuters corpus using<br /> the DIRT algorithm of Lin and Pantel.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[BinaryDIRT- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/unaryBInc.html unaryBInc]<br /> | Entailment rules between unary templates using BInc algorithm<br /> | Bar-Ilan University<br /> | This resource contains entailment rules over unary templates learned over the Reuters corpus using<br /> the BInc algorithm of Szpektor and Dagan (2008).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[unaryBInc- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''New resource''<br /> | <br /> | <br /> | ''Participants are encouraged to contribute''<br /> | style="text-align: center;"| <br /> | style="text-align: center;"| <br /> | style="text-align: center;"| <br /> |style="text-align: center;"| <br /> | [[New Resource2 - RTE Users|Users]]<br /> <br /> |}<br /> <br><br /> <br><br /> <br /> === Not available Resources ===<br /> The following table lists the unpublished resources used by RTE participants. Some of them have been developed by Users themselves specifically for RTE. Interested people may turn to authors to obtain further information.<br /> <br><br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! width="80"|Resource<br /> ! width="50"|Type<br /> ! width="180"|Author<br /> ! class="unsortable" width="500"|Brief description<br /> ! width="30"|<small>PAST Users</small><br /> ! width="30"|<small>RTE4 Users</small><br /> ! width="30"|<small>RTE5 Users</small><br /> ! width="30"|<small>RTE6 Users</small><br /> ! class="unsortable" width="30"|<small>Usage info</small><br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.parc.com/ PARC] Polarity Lexicon<br /> | Lexical DB<br /> | PARC - Palo Alto Research Center<br /> | Verbs classification with respect to semantic polarity<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | [[Parc Polarity Lexicon - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Gazetteer from [http://trec.nist.gov/ TREC]<br /> | Gazetteer<br /> | NIST - National Institute of Standards and Technology<br /> | Cities and other geographical names<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | [[Gazetteer from TREC - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DFKI Geographic Ontology<br>''(to be released)''<br /> | Ontology<br /> | DFKI - German Research Center for Artificial Intelligence<br /> | Ontology containing geographic terms and two kinds of relations: the directional ''part-of'' relation, and the ''equal'' relation for synonyms and abbreviations of the same geographic area (e.g ''the United Kingdom'', ''the UK'', ''Great Britain'', etc.) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | [[Geographic Ontology - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Geo''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | Meronymy entailment rules, based on TREC’s TIPSTER gazetteer.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | [[Geo - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Regex''<br /> | Collection of Entailment rules <br /> | Bar-Ilan University; Tel-Aviv University<br /> | Small set of entailment rules based on regular expressions, intended to address lexical variability involving temporal phrases<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | [[Regex - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Syntactic rule base <br>''(to be released)''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | A manually-composed collection of entailment rules which define parse tree transformations. The rules cover generic syntactic phenomena such as appositions, conjunctions, passive, relative clause, etc. (Bar-Haim et al., AAAI-07) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | [[Syntactic rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Polarity rule base <br>''(to be released)''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | A manually-composed collection of entailment rules which detect predicates whose polarity is negative (e.g. didn't dance) or unknown (e.g. plans to dance). The rules capture diverse phenomena that affect polarity, e.g. verbal negation, modal verbs, conditionals, and certain verbs that induce negative or "unknown" polarity context. The latter were taken mainly from VerbNet. Extends a resource described in (Bar-Haim et al., AAAI-07) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0 <br /> | style="text-align: center;"|0<br /> | [[Polarity rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Lexical-Syntactic rule base<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | Extract lexical-syntactic entailment rules for predicates (verbal and nominal), including argument mapping. The resource is based on WordNet, Nomlex-Plus and Unary DIRT (Szpektor and Dagan, Coling 08)<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | [[Lexical-Syntactic rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | OPENU Collection<br /> | Collection of Entailment Rules and Patterns<br /> | Open University<br /> | Collections of rules, patterns etc. for RTE purpose, extracted from Reuter corpus parsed using Minipar.<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0 <br /> | [[OPENU Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Abbr''<br /> | Collection of rules for abbreviation<br /> | Bar-Ilan University; Tel-Aviv University<br /> | 2000 Abbreviation rules, extracted from [http://badc.nerc.ac.uk/help/abbrevs.html BADC] and [http://www.acronym-guide.com/ Acronym Guide]<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | [[Abbr - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Negation_list<br /> | Negation rules<br /> | „Al. I. Cuza“ University, Iasi, Romania <br /> | List of negative terms and words (verbs, adjectives, nouns) affecting modality or factuality of a infinitive verb preceded by the particle "to" (e.g. "believe","necessary", "attempt")<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1 <br /> | style="text-align: center;"|0<br /> | [[UAIC Negation_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DLSIUAES Negation_list<br /> | List of negative terms<br /> | University of Alicante<br /> | Basic list of negative terms.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0 <br /> | [[DLSIUAES Negation_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Quantifier_list<br /> | List of quantifiers<br /> | „Al. I. Cuza“ University, Iasi, Romania <br /> | List of quantifiers affecting entailment judgment. The quantifiers are taken from a list which contains expressions like “more than”, “less than”, or words such as “over”, “under”, etc.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0 <br /> | [[UAIC Quantifier_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | FBKirst StopWord list<br /> | List of frequent words<br /> | FBK-Irst;<br/>University of Trento - Italy <br /> | A list of the 572 most frequent English words.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0 <br /> | [[FBKirst StopWord list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | IKOMA Dictionary of Named Entities Acronyms and Synonyms<br /> | Dictionary of Named Entities Acronyms and Synonyms<br /> | IKOMA; NEC Corporation, Takayama, Ikoma, Nara, Japan<br /> | Acronym dictionary constructed automatically from the corpus and a synonym dictionary that contains geographical terms.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | [[IKOMA2 - RTE Users|Users]]<br /> <br /> |}<br /> <br><br /> <br /> ==Footnotes==<br /> <references/><br /> <br /> Return to: [[Textual_Entailment_Resource_Pool]]</div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE_Knowledge_Resources&diff=8775 RTE Knowledge Resources 2011-04-04T15:18:10Z

<p>Amarchetti: </p> <hr /> <div>Knowledge resources have shown their relevance for applied semantic inference, and are extensively used by applied inference systems, such as those developed within the Textual Entailment framework.<br><br /> <br><br /> This page presents a list of the knowledge resources used by systems that have participated in the last RTE challenges. The first table lists the publicly available resources, the second one lists unpublished resources. Both tables are sortable by Resource name, type, author and number of users.<br><br /> <br><br /> RTE Participants are encouraged to add information about all kind of knowledge resources used, from standard existing resources (e.g. WordNet) to knowledge collections created for specific purposes, which can be made available to the community.<br><br /> <br><br /> __TOC__<br /> <br><br /> === Call for Resources ===<br /> <br /> In order to help the research, all the participants are invited to contribute, sharing their own resources with the RTE community. <br /> <br><br /> Making the resources available to be used by other systems has several advantages. On the one hand, it helps improve the TE technology; on the other hand, it offers an opportunity to further test and evaluate the resource.<br /> <br><br /> <br><br /> [[RTE6 - Call for Resources]]<br /> <br><br /> <br><br /> [[RTE7 - Call for Resources]]<br /> <br><br /> === Ablation Tests ===<br /> An ablation test consists of removing one module at a time from a system, and rerunning the system on the test set with the other modules, except the one tested. <br /> <br><br /> Ablation test are meant to help better understand the relevance of the knowledge resources used by RTE systems, and evaluate the contribution of each of them to the systems' performances. In fact, comparing the results achieved in the ablation tests to those obtained by the systems as a whole allows assessing the contribution given by each single resource.<br /> <br><br /> <br><br /> *[[RTE5 - Ablation Tests]]<br /> *[[RTE6 - Ablation Tests]]<br /> <br><br /> <br /> === Publicly available Resources ===<br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! width="80"|Resource<br /> ! width="50"|Type<br /> ! width="150"|Author<br /> ! class="unsortable" width="300"|Brief description<br /> ! width="45"|<small>PAST Users <ref name:"rtethree">RTE-3 data have been provided by participants by means of a questionnaire.</ref></small><br /> ! width="45"|<small>RTE4 Users<ref name:"rtefour">RTE-4 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! width="45"|<small>RTE5 Users<ref name:"rtefive">RTE-5 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! width="45"|<small>RTE6 Users<ref name:"rtesix">RTE-6 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! class="unsortable" width="30"|<small>Usage info</small><br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[WordNet]]<br /> | Lexical DB<br /> | Princeton University<br /> | Lexical database of English nouns, verbs, adjectives and adverbs<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|21<br /> | style="text-align: center;"|18<br /> |style="text-align: center;"| 22<br /> | [[WordNet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://xwn.hlt.utdallas.edu/index.html eXtended Wordnet]<br /> | Lexical DB<br /> | Human Language Technology Research Institute, University of Texas at Dallas<br /> | Extension of WordNet based on the exploitation of the information contained in WordNet definitional glosses: the glosses are syntactically parsed, transformed into logic forms and content words are semantically disambiguated. The Extended Wordnet is an ongoing project. <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|2<br /> |style="text-align: center;"| 0<br /> | [[eXtended WordNet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://ai.stanford.edu/~rion/swn/ Augmented Wordnet]<br /> | Lexical DB<br /> | Stanford University<br /> | The resource is the result of the application of a learning algorithm for inducing semantic taxonomies from parsed text. The algorithm automatically acquires items of world knowledge, and uses these to produce significantly enhanced versions of WordNet (up to 40,000 synsets more).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Augmented Wordnet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://verbs.colorado.edu/~mpalmer/projects/verbnet.html Verbnet]<br /> | Lexical DB<br /> | University of Colorado Boulder<br /> | Lexicon for English verbs organized into classes extending Levin (1993) classes through refinement and addition of subclasses to achieve syntactic and semantic coherence among members of a class<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Verbnet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[VerbOcean]]<br /> | Lexical DB<br /> | Information Sciences Institute, University of Southern California<br /> | Broad-coverage semantic network of verbs<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|6<br /> |style="text-align: center;"| 7<br /> | [[VerbOcean - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://framenet.icsi.berkeley.edu/ FrameNet]<br /> | Lexical DB<br /> | ICSI (International Computer Science Institute) - Berkley University<br /> | Lexical resource for English words, based on frame semantics (valences) and supported by corpus evidence<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|2<br /> |style="text-align: center;"| 3<br /> | [[Framenet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/meyers/NomBank.html NomBank]<br /> | Lexical DB<br /> | New York University<br /> | Lexical resource containing syntactic frames for nouns, extracted from annotated corpora <br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[NomBank Resource - RTE Users|Users]] <br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://verbs.colorado.edu/~mpalmer/projects/ace.html PropBank]<br /> | Lexical DB<br /> | University of Colorado Boulder <br /> | Lexical resource containing syntactic frames for verbs, extracted from annotated corpora <br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[PropBank Resource - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/nomlex/index.html Nomlex] Plus<br /> | Lexical DB<br /> | New York University<br /> | Dictionary of English nominalizations: it describes the allowed complements for a nominalization and relates the nominal complements to the arguments of the corresponding verb<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Nomlex Plus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.cs.ualberta.ca/~lindek/downloads.htm Dekang Lin’s Thesaurus]<br /> | Thesaurus<br /> | University of Alberta<br /> | Thesaurus automatically constructed using a parsed corpus, based on distributional similarity scores<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 2<br /> | [[Dekang Lin’s Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://icon.shef.ac.uk/Moby/mthes.html Grady Ward's Moby Thesaurus]<br /> | Thesaurus<br /> | University of Sheffield<br /> | Thesaurus containing 30,260 root words, with 2,520,264 synonyms and related terms. Grady Ward placed this thesaurus in the public domain in 1996.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Grady Ward's Moby Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://en.wikipedia.org/wiki/Roget%27s_Thesaurus Roget's Thesaurus]<br /> | Thesaurus<br /> | Peter Mark Roget (Electronic version distributed by University of Chicago)<br /> | Roget's Thesaurus is a widely-used English thesaurus, created by Dr. Peter Mark Roget in 1805. The original edition had 15,000 words, and each new edition has been larger. The electronic edition ([http://machaut.uchicago.edu/rogets version 1.02]) is made available by University of Chicago.<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Roget's Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.wikipedia.org/ Wikipedia]<br /> | Encyclopedia<br /> | <br /> | Free encyclopedia. Used for extraction of lexical-semantic rules (from its more structured parts), named entity recognition, geographical information etc.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|6<br /> |style="text-align: center;"| 6<br /> | [[Wikipedia - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.umbel.org/ Umbel]<br /> | Ontology<br /> | Structured Dynamics LLC, Coralville, IA<br /> | UMBEL stands for Upper Mapping and Binding Exchange Layer and is a lightweight ontology structure for relating Web content and data to a standard set of subject concepts<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Umbel - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.mpi-inf.mpg.de/yago-naga/yago/ YAGO]<br /> | Ontology<br /> | Max-Planck Institute for Informatics, Saarbrücken, Germany<br /> | Light-weight and extensible ontology. It contains more than 2 million entities and 20 million facts about these entities. The facts have been automatically extracted from Wikipedia and unified with WordNet.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[YAGO - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://dbpedia.org/About DBpedia]<br /> | Ontology<br /> | Open community project<br /> | DBpedia is a community effort to extract structured information from Wikipedia and to make this information available on the Web. The DBpedia knowledge base currently describes more than 2.9 million things in 91 different languages and consists of 479 million pieces of information.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[DBpedia - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[DIRT Paraphrase Collection]]<br /> | Collection of paraphrases<br /> | University of Alberta<br /> | DIRT (Discovery of Inference Rules from Text) is both an algorithm and a resulting knowledge collection. The DIRT knowledge collection is the output of the DIRT algorithm over a 1GB set of newspaper text.<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|4<br /> | style="text-align: center;"|3<br /> |style="text-align: center;"| 4<br /> | [[DIRT Paraphrase Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[TEASE]] Collection<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University<br /> | Output of the TEASE algorithm <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Tease Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://badc.nerc.ac.uk/help/abbrevs.html BADC Acronym and Abbreviation List]<br /> | Word List<br /> | BADC (British Atmospheric Data Centre)<br /> | Acronym and Abbreviation List<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[BADC Acronym and Abbreviation List - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://clipdemos.umiacs.umd.edu/catvar/ Catvar The Categorial Variation Database (English)]<br /> | Word List<br /> | University of Maryland<br /> | A Categorial-Variation Database (or Catvar) is a database of clusters of uninflected words (lexemes) and their categorial (i.e. part-of-speech) variants. The database was developed for English using a combination of resources and algorithms, including the LCS Verb and Preposition Databases (Dorr 2001), the Brown Corpus section of the Penn Treebank (Marcus et al. 1994), an English morphological analysis lexicon developed for PC-Kimmo (ENGLEX) (Antworth 1990), WordNet1.6 (Fellbaum 1998), an English Verb-Noun list extracted from Nomlex (Macleod et al. 1998), a similar list extracted from LDOCE (Procter 1978) and the Porter stemmer (Porter 1980).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 1<br /> | [[Categorial-Variation Database - RTE Users|Users]]<br /> <br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.acronym-guide.com/ Acronym Guide]<br /> | Word List<br /> | Acronym-Guide.com <br /> | Acronym and Abbreviation Lists for English, branched in thematic directories <br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|3<br /> |style="text-align: center;"| 0<br /> | [[Acronym Guide - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.ldc.upenn.edu/Catalog/CatalogEntry.jsp?catalogId=LDC2006T13 Web1T 5-grams]<br /> | Word list<br /> | Linguistic Data Consortium, University of Pennsylvania; Google Inc.<br /> | Data set containing English word n-grams and their observed frequency counts. The n-gram counts were generated from approximately 1 trillion word tokens of text from publicly accessible Web pages<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Web1T - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.coli.uni-saarland.de/%7Erwang/resources/RTE3_RTE4_NGD.zip Normalized Google Distance (RTE3&RTE4)]<br /> | Word Pair Co-occurrence<br /> | Saarland University<br /> | Co-occurrence of the word pairs in RTE3 and RTE4 using Normalized Google Distance (Cilibrasi and Vitanyi, 2004). The word pairs are all the possible combinations of content words in T and H. In practice, we used [http://m.www.yahoo.com/ Yahoo!] as the search engine.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Normalized Google Distance (RTE3&RTE4)- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.coli.uni-saarland.de/%7Erwang/resources/RTE5_NGD.zip Normalized Google Distance (RTE5)]<br /> | Word Pair Co-occurrence<br /> | Saarland University<br /> | Co-occurrence of the word pairs in RTE3 and RTE4 using Normalized Google Distance (Cilibrasi and Vitanyi, 2004). The word pairs are all the possible combinations of content words in T and H. In practice, we used [http://m.www.yahoo.com/ Yahoo!] as the search engine.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Normalized Google Distance (RTE5)- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://geonames.usgs.gov/index.html GNIS - Geographic Names Information System]<br /> | Gazetteer<br /> | USGS (United States Geological Survey)<br /> | Database containing the Federal and national standard toponyms for USA, associated areas and Antarctica<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[GNIS - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.geonames.org/ Geonames]<br /> | Gazetteer<br /> | <br /> | Database containing eight million geographical names. It is integrating geographical data such as names of places in various languages, elevation, population and others from various sources.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Geonames - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/paraphrase/ Sekine's Paraphrase Database]<br /> | Collection of paraphrases<br /> | Department of Computer Science, New York University <br /> | Data-base created using Sekine's method, NOT cleaned up by human. It includes 19,975 sets of paraphrases with 191,572 phrases.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Sekine's Paraphrase Database - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://research.microsoft.com/research/downloads/Details/607D14D9-20CD-47E3-85BC-A2F65CD28042/Details.aspx Microsoft Research Paraphrase Corpus]<br /> | Collection of paraphrases<br /> | Microsoft Research<br /> | Text file containing 5800 pairs of sentences which have been extracted from news sources on the web, along with human annotations indicating whether each pair captures a paraphrase/semantic equivalence relationship.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Microsoft Research Paraphrase Corpus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.cs.cornell.edu/~cristian/Without_a_doubt_-_Data.html Downward entailing operators] <br /> | Collection of entailing operators<br /> | Department of Computer Science, Cornell University, Ithaca NY<br /> | System output of an unsupervised algorithm recovering many Downward Entailing operators, like 'doubt'.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Downward entailing operators - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/WikiRules.html WikiRules!]<br /> | Lexical Reference rule-base<br /> | Bar-Ilan University<br /> | Extraction of about 8 million lexical reference rules from the text body (first sentence) and from metadata (links, redirects, parentheses) of Wikipedia. Provides better performance than other automatically constructed resources and comparable performance to WordNet. Offers complementary knowledge to WordNet. <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[WikiRules! - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://www.cs.utexas.edu/users/pclark/dart/ DART]<br /> | Collection of "world knowledge" propositions<br /> | Boeing Research and Technology<br /> | 23 million tuples such as "airplanes can fly to airports", "rivers can flood" collected from abstracted parse trees.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[DART - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://cs.biu.ac.il/~nlp/download/FRED/latest/FRED.zip FRED]<br /> | FrameNet-derived entailment rule-base<br /> | Bar-Ilan University<br /> | This package contains the outputs of the FRED algorithm, an algorithm which extracts entailment rules from FrameNet.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[FRED - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/DIRECT.html DIRECT]<br /> | Directional Distributional Term-Similarity Resource<br /> | Bar-Ilan University<br /> | This is a resource of directional distributional term-similarity rules (mostly lexical entailment rules) automatically extracted using the inclusion relation as described in (Kotlerman et.al., ACL-09).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[DIRECT - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/binaryDirt.html binaryDIRT]<br /> | Entailment rules between binary templates using DIRT algorithm<br /> | Bar-Ilan University<br /> | This resource contains entailment rules over binary templates learned over the Reuters corpus using<br /> the DIRT algorithm of Lin and Pantel.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[BinaryDIRT- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/unaryBInc.html unaryBInc]<br /> | Entailment rules between unary templates using BInc algorithm<br /> | Bar-Ilan University<br /> | This resource contains entailment rules over unary templates learned over the Reuters corpus using<br /> the BInc algorithm of Szpektor and Dagan (2008).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[unaryBInc- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''New resource''<br /> | <br /> | <br /> | ''Participants are encouraged to contribute''<br /> | style="text-align: center;"| <br /> | style="text-align: center;"| <br /> | style="text-align: center;"| <br /> |style="text-align: center;"| <br /> | [[New Resource2 - RTE Users|Users]]<br /> <br /> |}<br /> <br><br /> <br><br /> <br /> === Not available Resources ===<br /> The following table lists the unpublished resources used by RTE participants. Some of them have been developed by Users themselves specifically for RTE. Interested people may turn to authors to obtain further information.<br /> <br><br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! width="80"|Resource<br /> ! width="50"|Type<br /> ! width="180"|Author<br /> ! class="unsortable" width="500"|Brief description<br /> ! width="30"|<small>PAST Users</small><br /> ! width="30"|<small>RTE4 Users</small><br /> ! width="30"|<small>RTE5 Users</small><br /> ! width="30"|<small>RTE6 Users</small><br /> ! class="unsortable" width="30"|<small>Usage info</small><br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.parc.com/ PARC] Polarity Lexicon<br /> | Lexical DB<br /> | PARC - Palo Alto Research Center<br /> | Verbs classification with respect to semantic polarity<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | [[Parc Polarity Lexicon - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Gazetteer from [http://trec.nist.gov/ TREC]<br /> | Gazetteer<br /> | NIST - National Institute of Standards and Technology<br /> | Cities and other geographical names<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | [[Gazetteer from TREC - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DFKI Geographic Ontology<br>''(to be released)''<br /> | Ontology<br /> | DFKI - German Research Center for Artificial Intelligence<br /> | Ontology containing geographic terms and two kinds of relations: the directional ''part-of'' relation, and the ''equal'' relation for synonyms and abbreviations of the same geographic area (e.g ''the United Kingdom'', ''the UK'', ''Great Britain'', etc.) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | [[Geographic Ontology - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Geo''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | Meronymy entailment rules, based on TREC’s TIPSTER gazetteer.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | [[Geo - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Regex''<br /> | Collection of Entailment rules <br /> | Bar-Ilan University; Tel-Aviv University<br /> | Small set of entailment rules based on regular expressions, intended to address lexical variability involving temporal phrases<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | [[Regex - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Syntactic rule base <br>''(to be released)''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | A manually-composed collection of entailment rules which define parse tree transformations. The rules cover generic syntactic phenomena such as appositions, conjunctions, passive, relative clause, etc. (Bar-Haim et al., AAAI-07) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | [[Syntactic rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Polarity rule base <br>''(to be released)''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | A manually-composed collection of entailment rules which detect predicates whose polarity is negative (e.g. didn't dance) or unknown (e.g. plans to dance). The rules capture diverse phenomena that affect polarity, e.g. verbal negation, modal verbs, conditionals, and certain verbs that induce negative or "unknown" polarity context. The latter were taken mainly from VerbNet. Extends a resource described in (Bar-Haim et al., AAAI-07) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0 <br /> | style="text-align: center;"|0<br /> | [[Polarity rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Lexical-Syntactic rule base<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | Extract lexical-syntactic entailment rules for predicates (verbal and nominal), including argument mapping. The resource is based on WordNet, Nomlex-Plus and Unary DIRT (Szpektor and Dagan, Coling 08)<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | [[Lexical-Syntactic rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | OPENU Collection<br /> | Collection of Entailment Rules and Patterns<br /> | Open University<br /> | Collections of rules, patterns etc. for RTE purpose, extracted from Reuter corpus parsed using Minipar.<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0 <br /> | [[OPENU Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Abbr''<br /> | Collection of rules for abbreviation<br /> | Bar-Ilan University; Tel-Aviv University<br /> | 2000 Abbreviation rules, extracted from [http://badc.nerc.ac.uk/help/abbrevs.html BADC] and [http://www.acronym-guide.com/ Acronym Guide]<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | [[Abbr - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Negation_list<br /> | Negation rules<br /> | „Al. I. Cuza“ University, Iasi, Romania <br /> | List of negative terms and words (verbs, adjectives, nouns) affecting modality or factuality of a infinitive verb preceded by the particle "to" (e.g. "believe","necessary", "attempt")<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1 <br /> | style="text-align: center;"|0<br /> | [[UAIC Negation_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DLSIUAES Negation_list<br /> | List of negative terms<br /> | University of Alicante<br /> | Basic list of negative terms.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0 <br /> | [[DLSIUAES Negation_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Quantifier_list<br /> | List of quantifiers<br /> | „Al. I. Cuza“ University, Iasi, Romania <br /> | List of quantifiers affecting entailment judgment. The quantifiers are taken from a list which contains expressions like “more than”, “less than”, or words such as “over”, “under”, etc.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0 <br /> | [[UAIC Quantifier_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | FBKirst StopWord list<br /> | List of frequent words<br /> | FBK-Irst;<br/>University of Trento - Italy <br /> | A list of the 572 most frequent English words.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0 <br /> | [[FBKirst StopWord list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | IKOMA Dictionary of Named Entities Acronyms and Synonyms<br /> | Dictionary of Named Entities Acronyms and Synonyms<br /> | IKOMA; NEC Corporation, Takayama, Ikoma, Nara, Japan<br /> | Acronym dictionary constructed automatically from the corpus and a synonym dictionary that contains geographical terms.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | [[IKOMA2 - RTE Users|Users]]<br /> <br /> |}<br /> <br><br /> <br /> ==Footnotes==<br /> <references/><br /> <br /> <b>Return to [[Textual_Entailment_Resource_Pool]]</b></div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE_Knowledge_Resources&diff=8774 RTE Knowledge Resources 2011-04-04T15:12:54Z

<p>Amarchetti: </p> <hr /> <div>Knowledge resources have shown their relevance for applied semantic inference, and are extensively used by applied inference systems, such as those developed within the Textual Entailment framework.<br><br /> <br><br /> This page presents a list of the knowledge resources used by systems that have participated in the last RTE challenges. The first table lists the publicly available resources, the second one lists unpublished resources. Both tables are sortable by Resource name, type, author and number of users.<br><br /> <br><br /> RTE Participants are encouraged to add information about all kind of knowledge resources used, from standard existing resources (e.g. WordNet) to knowledge collections created for specific purposes, which can be made available to the community.<br><br /> <br><br /> __TOC__<br /> <br><br /> === Call for Resources ===<br /> <br /> In order to help the research, all the participants are invited to contribute, sharing their own resources with the RTE community. <br /> <br><br /> Making the resources available to be used by other systems has several advantages. On the one hand, it helps improve the TE technology; on the other hand, it offers an opportunity to further test and evaluate the resource.<br /> <br><br /> <br><br /> [[RTE6 - Call for Resources]]<br /> <br><br /> <br><br /> [[RTE7 - Call for Resources]]<br /> <br><br /> === Ablation Tests ===<br /> An ablation test consists of removing one module at a time from a system, and rerunning the system on the test set with the other modules, except the one tested. <br /> <br><br /> Ablation test are meant to help better understand the relevance of the knowledge resources used by RTE systems, and evaluate the contribution of each of them to the systems' performances. In fact, comparing the results achieved in the ablation tests to those obtained by the systems as a whole allows assessing the contribution given by each single resource.<br /> <br><br /> <br><br /> *[[RTE5 - Ablation Tests]]<br /> *[[RTE6 - Ablation Tests]]<br /> <br><br /> <br /> === Publicly available Resources ===<br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! width="80"|Resource<br /> ! width="50"|Type<br /> ! width="150"|Author<br /> ! class="unsortable" width="300"|Brief description<br /> ! width="45"|<small>PAST Users <ref name:"rtethree">RTE-3 data have been provided by participants by means of a questionnaire.</ref></small><br /> ! width="45"|<small>RTE4 Users<ref name:"rtefour">RTE-4 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! width="45"|<small>RTE5 Users<ref name:"rtefive">RTE-5 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! width="45"|<small>RTE6 Users<ref name:"rtesix">RTE-6 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! class="unsortable" width="30"|<small>Usage info</small><br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[WordNet]]<br /> | Lexical DB<br /> | Princeton University<br /> | Lexical database of English nouns, verbs, adjectives and adverbs<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|21<br /> | style="text-align: center;"|18<br /> |style="text-align: center;"| 22<br /> | [[WordNet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://xwn.hlt.utdallas.edu/index.html eXtended Wordnet]<br /> | Lexical DB<br /> | Human Language Technology Research Institute, University of Texas at Dallas<br /> | Extension of WordNet based on the exploitation of the information contained in WordNet definitional glosses: the glosses are syntactically parsed, transformed into logic forms and content words are semantically disambiguated. The Extended Wordnet is an ongoing project. <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|2<br /> |style="text-align: center;"| 0<br /> | [[eXtended WordNet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://ai.stanford.edu/~rion/swn/ Augmented Wordnet]<br /> | Lexical DB<br /> | Stanford University<br /> | The resource is the result of the application of a learning algorithm for inducing semantic taxonomies from parsed text. The algorithm automatically acquires items of world knowledge, and uses these to produce significantly enhanced versions of WordNet (up to 40,000 synsets more).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Augmented Wordnet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://verbs.colorado.edu/~mpalmer/projects/verbnet.html Verbnet]<br /> | Lexical DB<br /> | University of Colorado Boulder<br /> | Lexicon for English verbs organized into classes extending Levin (1993) classes through refinement and addition of subclasses to achieve syntactic and semantic coherence among members of a class<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Verbnet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[VerbOcean]]<br /> | Lexical DB<br /> | Information Sciences Institute, University of Southern California<br /> | Broad-coverage semantic network of verbs<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|6<br /> |style="text-align: center;"| 7<br /> | [[VerbOcean - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://framenet.icsi.berkeley.edu/ FrameNet]<br /> | Lexical DB<br /> | ICSI (International Computer Science Institute) - Berkley University<br /> | Lexical resource for English words, based on frame semantics (valences) and supported by corpus evidence<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|2<br /> |style="text-align: center;"| 3<br /> | [[Framenet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/meyers/NomBank.html NomBank]<br /> | Lexical DB<br /> | New York University<br /> | Lexical resource containing syntactic frames for nouns, extracted from annotated corpora <br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[NomBank Resource - RTE Users|Users]] <br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://verbs.colorado.edu/~mpalmer/projects/ace.html PropBank]<br /> | Lexical DB<br /> | University of Colorado Boulder <br /> | Lexical resource containing syntactic frames for verbs, extracted from annotated corpora <br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[PropBank Resource - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/nomlex/index.html Nomlex] Plus<br /> | Lexical DB<br /> | New York University<br /> | Dictionary of English nominalizations: it describes the allowed complements for a nominalization and relates the nominal complements to the arguments of the corresponding verb<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Nomlex Plus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.cs.ualberta.ca/~lindek/downloads.htm Dekang Lin’s Thesaurus]<br /> | Thesaurus<br /> | University of Alberta<br /> | Thesaurus automatically constructed using a parsed corpus, based on distributional similarity scores<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 2<br /> | [[Dekang Lin’s Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://icon.shef.ac.uk/Moby/mthes.html Grady Ward's Moby Thesaurus]<br /> | Thesaurus<br /> | University of Sheffield<br /> | Thesaurus containing 30,260 root words, with 2,520,264 synonyms and related terms. Grady Ward placed this thesaurus in the public domain in 1996.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Grady Ward's Moby Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://en.wikipedia.org/wiki/Roget%27s_Thesaurus Roget's Thesaurus]<br /> | Thesaurus<br /> | Peter Mark Roget (Electronic version distributed by University of Chicago)<br /> | Roget's Thesaurus is a widely-used English thesaurus, created by Dr. Peter Mark Roget in 1805. The original edition had 15,000 words, and each new edition has been larger. The electronic edition ([http://machaut.uchicago.edu/rogets version 1.02]) is made available by University of Chicago.<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Roget's Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.wikipedia.org/ Wikipedia]<br /> | Encyclopedia<br /> | <br /> | Free encyclopedia. Used for extraction of lexical-semantic rules (from its more structured parts), named entity recognition, geographical information etc.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|6<br /> |style="text-align: center;"| 6<br /> | [[Wikipedia - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.umbel.org/ Umbel]<br /> | Ontology<br /> | Structured Dynamics LLC, Coralville, IA<br /> | UMBEL stands for Upper Mapping and Binding Exchange Layer and is a lightweight ontology structure for relating Web content and data to a standard set of subject concepts<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Umbel - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.mpi-inf.mpg.de/yago-naga/yago/ YAGO]<br /> | Ontology<br /> | Max-Planck Institute for Informatics, Saarbrücken, Germany<br /> | Light-weight and extensible ontology. It contains more than 2 million entities and 20 million facts about these entities. The facts have been automatically extracted from Wikipedia and unified with WordNet.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[YAGO - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://dbpedia.org/About DBpedia]<br /> | Ontology<br /> | Open community project<br /> | DBpedia is a community effort to extract structured information from Wikipedia and to make this information available on the Web. The DBpedia knowledge base currently describes more than 2.9 million things in 91 different languages and consists of 479 million pieces of information.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[DBpedia - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[DIRT Paraphrase Collection]]<br /> | Collection of paraphrases<br /> | University of Alberta<br /> | DIRT (Discovery of Inference Rules from Text) is both an algorithm and a resulting knowledge collection. The DIRT knowledge collection is the output of the DIRT algorithm over a 1GB set of newspaper text.<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|4<br /> | style="text-align: center;"|3<br /> |style="text-align: center;"| 4<br /> | [[DIRT Paraphrase Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[TEASE]] Collection<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University<br /> | Output of the TEASE algorithm <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Tease Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://badc.nerc.ac.uk/help/abbrevs.html BADC Acronym and Abbreviation List]<br /> | Word List<br /> | BADC (British Atmospheric Data Centre)<br /> | Acronym and Abbreviation List<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[BADC Acronym and Abbreviation List - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://clipdemos.umiacs.umd.edu/catvar/ Catvar The Categorial Variation Database (English)]<br /> | Word List<br /> | University of Maryland<br /> | A Categorial-Variation Database (or Catvar) is a database of clusters of uninflected words (lexemes) and their categorial (i.e. part-of-speech) variants. The database was developed for English using a combination of resources and algorithms, including the LCS Verb and Preposition Databases (Dorr 2001), the Brown Corpus section of the Penn Treebank (Marcus et al. 1994), an English morphological analysis lexicon developed for PC-Kimmo (ENGLEX) (Antworth 1990), WordNet1.6 (Fellbaum 1998), an English Verb-Noun list extracted from Nomlex (Macleod et al. 1998), a similar list extracted from LDOCE (Procter 1978) and the Porter stemmer (Porter 1980).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 1<br /> | [[Categorial-Variation Database - RTE Users|Users]]<br /> <br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.acronym-guide.com/ Acronym Guide]<br /> | Word List<br /> | Acronym-Guide.com <br /> | Acronym and Abbreviation Lists for English, branched in thematic directories <br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|3<br /> |style="text-align: center;"| 0<br /> | [[Acronym Guide - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.ldc.upenn.edu/Catalog/CatalogEntry.jsp?catalogId=LDC2006T13 Web1T 5-grams]<br /> | Word list<br /> | Linguistic Data Consortium, University of Pennsylvania; Google Inc.<br /> | Data set containing English word n-grams and their observed frequency counts. The n-gram counts were generated from approximately 1 trillion word tokens of text from publicly accessible Web pages<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Web1T - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.coli.uni-saarland.de/%7Erwang/resources/RTE3_RTE4_NGD.zip Normalized Google Distance (RTE3&RTE4)]<br /> | Word Pair Co-occurrence<br /> | Saarland University<br /> | Co-occurrence of the word pairs in RTE3 and RTE4 using Normalized Google Distance (Cilibrasi and Vitanyi, 2004). The word pairs are all the possible combinations of content words in T and H. In practice, we used [http://m.www.yahoo.com/ Yahoo!] as the search engine.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Normalized Google Distance (RTE3&RTE4)- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.coli.uni-saarland.de/%7Erwang/resources/RTE5_NGD.zip Normalized Google Distance (RTE5)]<br /> | Word Pair Co-occurrence<br /> | Saarland University<br /> | Co-occurrence of the word pairs in RTE3 and RTE4 using Normalized Google Distance (Cilibrasi and Vitanyi, 2004). The word pairs are all the possible combinations of content words in T and H. In practice, we used [http://m.www.yahoo.com/ Yahoo!] as the search engine.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Normalized Google Distance (RTE5)- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://geonames.usgs.gov/index.html GNIS - Geographic Names Information System]<br /> | Gazetteer<br /> | USGS (United States Geological Survey)<br /> | Database containing the Federal and national standard toponyms for USA, associated areas and Antarctica<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[GNIS - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.geonames.org/ Geonames]<br /> | Gazetteer<br /> | <br /> | Database containing eight million geographical names. It is integrating geographical data such as names of places in various languages, elevation, population and others from various sources.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Geonames - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/paraphrase/ Sekine's Paraphrase Database]<br /> | Collection of paraphrases<br /> | Department of Computer Science, New York University <br /> | Data-base created using Sekine's method, NOT cleaned up by human. It includes 19,975 sets of paraphrases with 191,572 phrases.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Sekine's Paraphrase Database - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://research.microsoft.com/research/downloads/Details/607D14D9-20CD-47E3-85BC-A2F65CD28042/Details.aspx Microsoft Research Paraphrase Corpus]<br /> | Collection of paraphrases<br /> | Microsoft Research<br /> | Text file containing 5800 pairs of sentences which have been extracted from news sources on the web, along with human annotations indicating whether each pair captures a paraphrase/semantic equivalence relationship.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Microsoft Research Paraphrase Corpus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.cs.cornell.edu/~cristian/Without_a_doubt_-_Data.html Downward entailing operators] <br /> | Collection of entailing operators<br /> | Department of Computer Science, Cornell University, Ithaca NY<br /> | System output of an unsupervised algorithm recovering many Downward Entailing operators, like 'doubt'.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Downward entailing operators - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/WikiRules.html WikiRules!]<br /> | Lexical Reference rule-base<br /> | Bar-Ilan University<br /> | Extraction of about 8 million lexical reference rules from the text body (first sentence) and from metadata (links, redirects, parentheses) of Wikipedia. Provides better performance than other automatically constructed resources and comparable performance to WordNet. Offers complementary knowledge to WordNet. <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[WikiRules! - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://www.cs.utexas.edu/users/pclark/dart/ DART]<br /> | Collection of "world knowledge" propositions<br /> | Boeing Research and Technology<br /> | 23 million tuples such as "airplanes can fly to airports", "rivers can flood" collected from abstracted parse trees.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[DART - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://cs.biu.ac.il/~nlp/download/FRED/latest/FRED.zip FRED]<br /> | FrameNet-derived entailment rule-base<br /> | Bar-Ilan University<br /> | This package contains the outputs of the FRED algorithm, an algorithm which extracts entailment rules from FrameNet.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[FRED - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/DIRECT.html DIRECT]<br /> | Directional Distributional Term-Similarity Resource<br /> | Bar-Ilan University<br /> | This is a resource of directional distributional term-similarity rules (mostly lexical entailment rules) automatically extracted using the inclusion relation as described in (Kotlerman et.al., ACL-09).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[DIRECT - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/binaryDirt.html binaryDIRT]<br /> | Entailment rules between binary templates using DIRT algorithm<br /> | Bar-Ilan University<br /> | This resource contains entailment rules over binary templates learned over the Reuters corpus using<br /> the DIRT algorithm of Lin and Pantel.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[BinaryDIRT- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/unaryBInc.html unaryBInc]<br /> | Entailment rules between unary templates using BInc algorithm<br /> | Bar-Ilan University<br /> | This resource contains entailment rules over unary templates learned over the Reuters corpus using<br /> the BInc algorithm of Szpektor and Dagan (2008).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[unaryBInc- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''New resource''<br /> | <br /> | <br /> | ''Participants are encouraged to contribute''<br /> | style="text-align: center;"| <br /> | style="text-align: center;"| <br /> | style="text-align: center;"| <br /> |style="text-align: center;"| <br /> | [[New Resource2 - RTE Users|Users]]<br /> <br /> |}<br /> <br><br /> <br><br /> <br /> === Not available Resources ===<br /> The following table lists the unpublished resources used by RTE participants. Some of them have been developed by Users themselves specifically for RTE. Interested people may turn to authors to obtain further information.<br /> <br><br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! width="80"|Resource<br /> ! width="50"|Type<br /> ! width="180"|Author<br /> ! class="unsortable" width="500"|Brief description<br /> ! width="30"|<small>PAST Users</small><br /> ! width="30"|<small>RTE4 Users</small><br /> ! width="30"|<small>RTE5 Users</small><br /> ! width="30"|<small>RTE6 Users</small><br /> ! class="unsortable" width="30"|<small>Usage info</small><br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.parc.com/ PARC] Polarity Lexicon<br /> | Lexical DB<br /> | PARC - Palo Alto Research Center<br /> | Verbs classification with respect to semantic polarity<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | [[Parc Polarity Lexicon - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Gazetteer from [http://trec.nist.gov/ TREC]<br /> | Gazetteer<br /> | NIST - National Institute of Standards and Technology<br /> | Cities and other geographical names<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | [[Gazetteer from TREC - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DFKI Geographic Ontology<br>''(to be released)''<br /> | Ontology<br /> | DFKI - German Research Center for Artificial Intelligence<br /> | Ontology containing geographic terms and two kinds of relations: the directional ''part-of'' relation, and the ''equal'' relation for synonyms and abbreviations of the same geographic area (e.g ''the United Kingdom'', ''the UK'', ''Great Britain'', etc.) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | [[Geographic Ontology - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Geo''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | Meronymy entailment rules, based on TREC’s TIPSTER gazetteer.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | [[Geo - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Regex''<br /> | Collection of Entailment rules <br /> | Bar-Ilan University; Tel-Aviv University<br /> | Small set of entailment rules based on regular expressions, intended to address lexical variability involving temporal phrases<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | [[Regex - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Syntactic rule base <br>''(to be released)''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | A manually-composed collection of entailment rules which define parse tree transformations. The rules cover generic syntactic phenomena such as appositions, conjunctions, passive, relative clause, etc. (Bar-Haim et al., AAAI-07) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | [[Syntactic rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Polarity rule base <br>''(to be released)''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | A manually-composed collection of entailment rules which detect predicates whose polarity is negative (e.g. didn't dance) or unknown (e.g. plans to dance). The rules capture diverse phenomena that affect polarity, e.g. verbal negation, modal verbs, conditionals, and certain verbs that induce negative or "unknown" polarity context. The latter were taken mainly from VerbNet. Extends a resource described in (Bar-Haim et al., AAAI-07) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0 <br /> | style="text-align: center;"|0<br /> | [[Polarity rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Lexical-Syntactic rule base<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | Extract lexical-syntactic entailment rules for predicates (verbal and nominal), including argument mapping. The resource is based on WordNet, Nomlex-Plus and Unary DIRT (Szpektor and Dagan, Coling 08)<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | [[Lexical-Syntactic rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | OPENU Collection<br /> | Collection of Entailment Rules and Patterns<br /> | Open University<br /> | Collections of rules, patterns etc. for RTE purpose, extracted from Reuter corpus parsed using Minipar.<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0 <br /> | [[OPENU Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Abbr''<br /> | Collection of rules for abbreviation<br /> | Bar-Ilan University; Tel-Aviv University<br /> | 2000 Abbreviation rules, extracted from [http://badc.nerc.ac.uk/help/abbrevs.html BADC] and [http://www.acronym-guide.com/ Acronym Guide]<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | [[Abbr - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Negation_list<br /> | Negation rules<br /> | „Al. I. Cuza“ University, Iasi, Romania <br /> | List of negative terms and words (verbs, adjectives, nouns) affecting modality or factuality of a infinitive verb preceded by the particle "to" (e.g. "believe","necessary", "attempt")<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1 <br /> | style="text-align: center;"|0<br /> | [[UAIC Negation_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DLSIUAES Negation_list<br /> | List of negative terms<br /> | University of Alicante<br /> | Basic list of negative terms.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0 <br /> | [[DLSIUAES Negation_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Quantifier_list<br /> | List of quantifiers<br /> | „Al. I. Cuza“ University, Iasi, Romania <br /> | List of quantifiers affecting entailment judgment. The quantifiers are taken from a list which contains expressions like “more than”, “less than”, or words such as “over”, “under”, etc.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0 <br /> | [[UAIC Quantifier_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | FBKirst StopWord list<br /> | List of frequent words<br /> | FBK-Irst;<br/>University of Trento - Italy <br /> | A list of the 572 most frequent English words.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0 <br /> | [[FBKirst StopWord list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | IKOMA Dictionary of Named Entities Acronyms and Synonyms<br /> | Dictionary of Named Entities Acronyms and Synonyms<br /> | IKOMA; NEC Corporation, Takayama, Ikoma, Nara, Japan<br /> | Acronym dictionary constructed automatically from the corpus and a synonym dictionary that contains geographical terms.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | [[IKOMA2 - RTE Users|Users]]<br /> <br /> |}<br /> <br><br /> <br /> ==Footnotes==<br /> <references/></div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE_Knowledge_Resources&diff=8773 RTE Knowledge Resources 2011-04-04T15:11:14Z

<p>Amarchetti: /* Footnotes */</p> <hr /> <div>Knowledge resources have shown their relevance for applied semantic inference, and are extensively used by applied inference systems, such as those developed within the Textual Entailment framework.<br><br /> <br><br /> This page presents a list of the knowledge resources used by systems that have participated in the last RTE challenges. The first table lists the publicly available resources, the second one lists unpublished resources. Both tables are sortable by Resource name, type, author and number of users.<br><br /> <br><br /> RTE Participants are encouraged to add information about all kind of knowledge resources used, from standard existing resources (e.g. WordNet) to knowledge collections created for specific purposes, which can be made available to the community.<br><br /> <br><br /> __TOC__<br /> <br><br /> === Call for Resources ===<br /> <br /> In order to help the research, all the participants are invited to contribute, sharing their own resources with the RTE community. <br /> <br><br /> Making the resources available to be used by other systems has several advantages. On the one hand, it helps improve the TE technology; on the other hand, it offers an opportunity to further test and evaluate the resource.<br /> <br><br /> <br><br /> [[RTE6 - Call for Resources]]<br /> <br><br /> <br><br /> [[RTE7 - Call for Resources]]<br /> <br><br /> === Ablation Tests ===<br /> An ablation test consists of removing one module at a time from a system, and rerunning the system on the test set with the other modules, except the one tested. <br /> <br><br /> Ablation test are meant to help better understand the relevance of the knowledge resources used by RTE systems, and evaluate the contribution of each of them to the systems' performances. In fact, comparing the results achieved in the ablation tests to those obtained by the systems as a whole allows assessing the contribution given by each single resource.<br /> <br><br /> <br><br /> *[[RTE5 - Ablation Tests]]<br /> *[[RTE6 - Ablation Tests]]<br /> <br><br /> <br /> === Publicly available Resources ===<br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! width="80"|Resource<br /> ! width="50"|Type<br /> ! width="150"|Author<br /> ! class="unsortable" width="300"|Brief description<br /> ! width="45"|<small>PAST Users <ref name:"rtethree">RTE-3 data have been provided by participants by means of a questionnaire.</ref></small><br /> ! width="45"|<small>RTE4 Users<ref name:"rtefour">RTE-4 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! width="45"|<small>RTE5 Users<ref name:"rtefive">RTE-5 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! width="45"|<small>RTE6 Users<ref name:"rtesix">RTE-6 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! class="unsortable" width="30"|<small>Usage info</small><br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[WordNet]]<br /> | Lexical DB<br /> | Princeton University<br /> | Lexical database of English nouns, verbs, adjectives and adverbs<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|21<br /> | style="text-align: center;"|18<br /> |style="text-align: center;"| 22<br /> | [[WordNet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://xwn.hlt.utdallas.edu/index.html eXtended Wordnet]<br /> | Lexical DB<br /> | Human Language Technology Research Institute, University of Texas at Dallas<br /> | Extension of WordNet based on the exploitation of the information contained in WordNet definitional glosses: the glosses are syntactically parsed, transformed into logic forms and content words are semantically disambiguated. The Extended Wordnet is an ongoing project. <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|2<br /> |style="text-align: center;"| 0<br /> | [[eXtended WordNet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://ai.stanford.edu/~rion/swn/ Augmented Wordnet]<br /> | Lexical DB<br /> | Stanford University<br /> | The resource is the result of the application of a learning algorithm for inducing semantic taxonomies from parsed text. The algorithm automatically acquires items of world knowledge, and uses these to produce significantly enhanced versions of WordNet (up to 40,000 synsets more).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Augmented Wordnet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://verbs.colorado.edu/~mpalmer/projects/verbnet.html Verbnet]<br /> | Lexical DB<br /> | University of Colorado Boulder<br /> | Lexicon for English verbs organized into classes extending Levin (1993) classes through refinement and addition of subclasses to achieve syntactic and semantic coherence among members of a class<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Verbnet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[VerbOcean]]<br /> | Lexical DB<br /> | Information Sciences Institute, University of Southern California<br /> | Broad-coverage semantic network of verbs<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|6<br /> |style="text-align: center;"| 7<br /> | [[VerbOcean - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://framenet.icsi.berkeley.edu/ FrameNet]<br /> | Lexical DB<br /> | ICSI (International Computer Science Institute) - Berkley University<br /> | Lexical resource for English words, based on frame semantics (valences) and supported by corpus evidence<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|2<br /> |style="text-align: center;"| 3<br /> | [[Framenet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/meyers/NomBank.html NomBank]<br /> | Lexical DB<br /> | New York University<br /> | Lexical resource containing syntactic frames for nouns, extracted from annotated corpora <br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[NomBank Resource - RTE Users|Users]] <br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://verbs.colorado.edu/~mpalmer/projects/ace.html PropBank]<br /> | Lexical DB<br /> | University of Colorado Boulder <br /> | Lexical resource containing syntactic frames for verbs, extracted from annotated corpora <br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[PropBank Resource - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/nomlex/index.html Nomlex] Plus<br /> | Lexical DB<br /> | New York University<br /> | Dictionary of English nominalizations: it describes the allowed complements for a nominalization and relates the nominal complements to the arguments of the corresponding verb<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Nomlex Plus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.cs.ualberta.ca/~lindek/downloads.htm Dekang Lin’s Thesaurus]<br /> | Thesaurus<br /> | University of Alberta<br /> | Thesaurus automatically constructed using a parsed corpus, based on distributional similarity scores<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 2<br /> | [[Dekang Lin’s Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://icon.shef.ac.uk/Moby/mthes.html Grady Ward's Moby Thesaurus]<br /> | Thesaurus<br /> | University of Sheffield<br /> | Thesaurus containing 30,260 root words, with 2,520,264 synonyms and related terms. Grady Ward placed this thesaurus in the public domain in 1996.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Grady Ward's Moby Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://en.wikipedia.org/wiki/Roget%27s_Thesaurus Roget's Thesaurus]<br /> | Thesaurus<br /> | Peter Mark Roget (Electronic version distributed by University of Chicago)<br /> | Roget's Thesaurus is a widely-used English thesaurus, created by Dr. Peter Mark Roget in 1805. The original edition had 15,000 words, and each new edition has been larger. The electronic edition ([http://machaut.uchicago.edu/rogets version 1.02]) is made available by University of Chicago.<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Roget's Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.wikipedia.org/ Wikipedia]<br /> | Encyclopedia<br /> | <br /> | Free encyclopedia. Used for extraction of lexical-semantic rules (from its more structured parts), named entity recognition, geographical information etc.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|6<br /> |style="text-align: center;"| 6<br /> | [[Wikipedia - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.umbel.org/ Umbel]<br /> | Ontology<br /> | Structured Dynamics LLC, Coralville, IA<br /> | UMBEL stands for Upper Mapping and Binding Exchange Layer and is a lightweight ontology structure for relating Web content and data to a standard set of subject concepts<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Umbel - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.mpi-inf.mpg.de/yago-naga/yago/ YAGO]<br /> | Ontology<br /> | Max-Planck Institute for Informatics, Saarbrücken, Germany<br /> | Light-weight and extensible ontology. It contains more than 2 million entities and 20 million facts about these entities. The facts have been automatically extracted from Wikipedia and unified with WordNet.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[YAGO - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://dbpedia.org/About DBpedia]<br /> | Ontology<br /> | Open community project<br /> | DBpedia is a community effort to extract structured information from Wikipedia and to make this information available on the Web. The DBpedia knowledge base currently describes more than 2.9 million things in 91 different languages and consists of 479 million pieces of information.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[DBpedia - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[DIRT Paraphrase Collection]]<br /> | Collection of paraphrases<br /> | University of Alberta<br /> | DIRT (Discovery of Inference Rules from Text) is both an algorithm and a resulting knowledge collection. The DIRT knowledge collection is the output of the DIRT algorithm over a 1GB set of newspaper text.<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|4<br /> | style="text-align: center;"|3<br /> |style="text-align: center;"| 4<br /> | [[DIRT Paraphrase Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[TEASE]] Collection<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University<br /> | Output of the TEASE algorithm <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Tease Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://badc.nerc.ac.uk/help/abbrevs.html BADC Acronym and Abbreviation List]<br /> | Word List<br /> | BADC (British Atmospheric Data Centre)<br /> | Acronym and Abbreviation List<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[BADC Acronym and Abbreviation List - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://clipdemos.umiacs.umd.edu/catvar/ Catvar The Categorial Variation Database (English)]<br /> | Word List<br /> | University of Maryland<br /> | A Categorial-Variation Database (or Catvar) is a database of clusters of uninflected words (lexemes) and their categorial (i.e. part-of-speech) variants. The database was developed for English using a combination of resources and algorithms, including the LCS Verb and Preposition Databases (Dorr 2001), the Brown Corpus section of the Penn Treebank (Marcus et al. 1994), an English morphological analysis lexicon developed for PC-Kimmo (ENGLEX) (Antworth 1990), WordNet1.6 (Fellbaum 1998), an English Verb-Noun list extracted from Nomlex (Macleod et al. 1998), a similar list extracted from LDOCE (Procter 1978) and the Porter stemmer (Porter 1980).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 1<br /> | [[Categorial-Variation Database - RTE Users|Users]]<br /> <br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.acronym-guide.com/ Acronym Guide]<br /> | Word List<br /> | Acronym-Guide.com <br /> | Acronym and Abbreviation Lists for English, branched in thematic directories <br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|3<br /> |style="text-align: center;"| 0<br /> | [[Acronym Guide - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.ldc.upenn.edu/Catalog/CatalogEntry.jsp?catalogId=LDC2006T13 Web1T 5-grams]<br /> | Word list<br /> | Linguistic Data Consortium, University of Pennsylvania; Google Inc.<br /> | Data set containing English word n-grams and their observed frequency counts. The n-gram counts were generated from approximately 1 trillion word tokens of text from publicly accessible Web pages<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Web1T - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.coli.uni-saarland.de/%7Erwang/resources/RTE3_RTE4_NGD.zip Normalized Google Distance (RTE3&RTE4)]<br /> | Word Pair Co-occurrence<br /> | Saarland University<br /> | Co-occurrence of the word pairs in RTE3 and RTE4 using Normalized Google Distance (Cilibrasi and Vitanyi, 2004). The word pairs are all the possible combinations of content words in T and H. In practice, we used [http://m.www.yahoo.com/ Yahoo!] as the search engine.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Normalized Google Distance (RTE3&RTE4)- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.coli.uni-saarland.de/%7Erwang/resources/RTE5_NGD.zip Normalized Google Distance (RTE5)]<br /> | Word Pair Co-occurrence<br /> | Saarland University<br /> | Co-occurrence of the word pairs in RTE3 and RTE4 using Normalized Google Distance (Cilibrasi and Vitanyi, 2004). The word pairs are all the possible combinations of content words in T and H. In practice, we used [http://m.www.yahoo.com/ Yahoo!] as the search engine.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Normalized Google Distance (RTE5)- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://geonames.usgs.gov/index.html GNIS - Geographic Names Information System]<br /> | Gazetteer<br /> | USGS (United States Geological Survey)<br /> | Database containing the Federal and national standard toponyms for USA, associated areas and Antarctica<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[GNIS - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.geonames.org/ Geonames]<br /> | Gazetteer<br /> | <br /> | Database containing eight million geographical names. It is integrating geographical data such as names of places in various languages, elevation, population and others from various sources.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Geonames - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/paraphrase/ Sekine's Paraphrase Database]<br /> | Collection of paraphrases<br /> | Department of Computer Science, New York University <br /> | Data-base created using Sekine's method, NOT cleaned up by human. It includes 19,975 sets of paraphrases with 191,572 phrases.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Sekine's Paraphrase Database - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://research.microsoft.com/research/downloads/Details/607D14D9-20CD-47E3-85BC-A2F65CD28042/Details.aspx Microsoft Research Paraphrase Corpus]<br /> | Collection of paraphrases<br /> | Microsoft Research<br /> | Text file containing 5800 pairs of sentences which have been extracted from news sources on the web, along with human annotations indicating whether each pair captures a paraphrase/semantic equivalence relationship.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Microsoft Research Paraphrase Corpus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.cs.cornell.edu/~cristian/Without_a_doubt_-_Data.html Downward entailing operators] <br /> | Collection of entailing operators<br /> | Department of Computer Science, Cornell University, Ithaca NY<br /> | System output of an unsupervised algorithm recovering many Downward Entailing operators, like 'doubt'.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Downward entailing operators - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/WikiRules.html WikiRules!]<br /> | Lexical Reference rule-base<br /> | Bar-Ilan University<br /> | Extraction of about 8 million lexical reference rules from the text body (first sentence) and from metadata (links, redirects, parentheses) of Wikipedia. Provides better performance than other automatically constructed resources and comparable performance to WordNet. Offers complementary knowledge to WordNet. <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[WikiRules! - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://www.cs.utexas.edu/users/pclark/dart/ DART]<br /> | Collection of "world knowledge" propositions<br /> | Boeing Research and Technology<br /> | 23 million tuples such as "airplanes can fly to airports", "rivers can flood" collected from abstracted parse trees.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[DART - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://cs.biu.ac.il/~nlp/download/FRED/latest/FRED.zip FRED]<br /> | FrameNet-derived entailment rule-base<br /> | Bar-Ilan University<br /> | This package contains the outputs of the FRED algorithm, an algorithm which extracts entailment rules from FrameNet.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[FRED - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/DIRECT.html DIRECT]<br /> | Directional Distributional Term-Similarity Resource<br /> | Bar-Ilan University<br /> | This is a resource of directional distributional term-similarity rules (mostly lexical entailment rules) automatically extracted using the inclusion relation as described in (Kotlerman et.al., ACL-09).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[DIRECT - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/binaryDirt.html binaryDIRT]<br /> | Entailment rules between binary templates using DIRT algorithm<br /> | Bar-Ilan University<br /> | This resource contains entailment rules over binary templates learned over the Reuters corpus using<br /> the DIRT algorithm of Lin and Pantel.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[BinaryDIRT- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/unaryBInc.html unaryBInc]<br /> | Entailment rules between unary templates using BInc algorithm<br /> | Bar-Ilan University<br /> | This resource contains entailment rules over unary templates learned over the Reuters corpus using<br /> the BInc algorithm of Szpektor and Dagan (2008).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[unaryBInc- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''New resource''<br /> | <br /> | <br /> | ''Participants are encouraged to contribute''<br /> | style="text-align: center;"| <br /> | style="text-align: center;"| <br /> | style="text-align: center;"| <br /> |style="text-align: center;"| <br /> | [[New Resource2 - RTE Users|Users]]<br /> <br /> |}<br /> <br><br /> <br><br /> <br /> === Not available Resources ===<br /> The following table lists the unpublished resources used by RTE participants. Some of them have been developed by Users themselves specifically for RTE. Interested people may turn to authors to obtain further information.<br /> <br><br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! width="80"|Resource<br /> ! width="50"|Type<br /> ! width="180"|Author<br /> ! class="unsortable" width="500"|Brief description<br /> ! width="30"|<small>PAST Users</small><br /> ! width="30"|<small>RTE4 Users</small><br /> ! width="30"|<small>RTE5 Users</small><br /> ! width="30"|<small>RTE6 Users</small><br /> ! class="unsortable" width="30"|<small>Usage info</small><br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.parc.com/ PARC] Polarity Lexicon<br /> | Lexical DB<br /> | PARC - Palo Alto Research Center<br /> | Verbs classification with respect to semantic polarity<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | [[Parc Polarity Lexicon - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Gazetteer from [http://trec.nist.gov/ TREC]<br /> | Gazetteer<br /> | NIST - National Institute of Standards and Technology<br /> | Cities and other geographical names<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | [[Gazetteer from TREC - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DFKI Geographic Ontology<br>''(to be released)''<br /> | Ontology<br /> | DFKI - German Research Center for Artificial Intelligence<br /> | Ontology containing geographic terms and two kinds of relations: the directional ''part-of'' relation, and the ''equal'' relation for synonyms and abbreviations of the same geographic area (e.g ''the United Kingdom'', ''the UK'', ''Great Britain'', etc.) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | [[Geographic Ontology - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Geo''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | Meronymy entailment rules, based on TREC’s TIPSTER gazetteer.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | [[Geo - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Regex''<br /> | Collection of Entailment rules <br /> | Bar-Ilan University; Tel-Aviv University<br /> | Small set of entailment rules based on regular expressions, intended to address lexical variability involving temporal phrases<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | [[Regex - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Syntactic rule base <br>''(to be released)''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | A manually-composed collection of entailment rules which define parse tree transformations. The rules cover generic syntactic phenomena such as appositions, conjunctions, passive, relative clause, etc. (Bar-Haim et al., AAAI-07) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | [[Syntactic rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Polarity rule base <br>''(to be released)''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | A manually-composed collection of entailment rules which detect predicates whose polarity is negative (e.g. didn't dance) or unknown (e.g. plans to dance). The rules capture diverse phenomena that affect polarity, e.g. verbal negation, modal verbs, conditionals, and certain verbs that induce negative or "unknown" polarity context. The latter were taken mainly from VerbNet. Extends a resource described in (Bar-Haim et al., AAAI-07) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0 <br /> | style="text-align: center;"|0<br /> | [[Polarity rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Lexical-Syntactic rule base<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | Extract lexical-syntactic entailment rules for predicates (verbal and nominal), including argument mapping. The resource is based on WordNet, Nomlex-Plus and Unary DIRT (Szpektor and Dagan, Coling 08)<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | [[Lexical-Syntactic rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | OPENU Collection<br /> | Collection of Entailment Rules and Patterns<br /> | Open University<br /> | Collections of rules, patterns etc. for RTE purpose, extracted from Reuter corpus parsed using Minipar.<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0 <br /> | [[OPENU Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Abbr''<br /> | Collection of rules for abbreviation<br /> | Bar-Ilan University; Tel-Aviv University<br /> | 2000 Abbreviation rules, extracted from [http://badc.nerc.ac.uk/help/abbrevs.html BADC] and [http://www.acronym-guide.com/ Acronym Guide]<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | [[Abbr - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Negation_list<br /> | Negation rules<br /> | „Al. I. Cuza“ University, Iasi, Romania <br /> | List of negative terms and words (verbs, adjectives, nouns) affecting modality or factuality of a infinitive verb preceded by the particle "to" (e.g. "believe","necessary", "attempt")<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1 <br /> | style="text-align: center;"|0<br /> | [[UAIC Negation_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DLSIUAES Negation_list<br /> | List of negative terms<br /> | University of Alicante<br /> | Basic list of negative terms.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0 <br /> | [[DLSIUAES Negation_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Quantifier_list<br /> | List of quantifiers<br /> | „Al. I. Cuza“ University, Iasi, Romania <br /> | List of quantifiers affecting entailment judgment. The quantifiers are taken from a list which contains expressions like “more than”, “less than”, or words such as “over”, “under”, etc.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0 <br /> | [[UAIC Quantifier_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | FBKirst StopWord list<br /> | List of frequent words<br /> | FBK-Irst;<br/>University of Trento - Italy <br /> | A list of the 572 most frequent English words.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0 <br /> | [[FBKirst StopWord list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | IKOMA Dictionary of Named Entities Acronyms and Synonyms<br /> | Dictionary of Named Entities Acronyms and Synonyms<br /> | IKOMA; NEC Corporation, Takayama, Ikoma, Nara, Japan<br /> | Acronym dictionary constructed automatically from the corpus and a synonym dictionary that contains geographical terms.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | [[IKOMA2 - RTE Users|Users]]<br /> <br /> |}<br /> <br><br /> <br /> ==Footnotes==<br /> <references/><br /> <br /> Return to [[RTE Knowledge Resources]]</div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE7_-_Call_for_Resources&diff=8772 RTE7 - Call for Resources 2011-04-04T15:03:34Z

<p>Amarchetti: </p> <hr /> <div>A study on the knowledge resources used in Textual Entailment was started with the ablation tests carried out on the RTE-5 Main task submissions, which gave a hint about the impact of knowledge resources on TE system performances. Given the interest raised by this new initiative, the investigation of knowledge resources, which in RTE-6 was extended to tools, will be also continued in the RTE-7 challenge.<p><br /> <br /> Making the resources available to be used by other systems has several advantages: <br /> <br /> <li>It helps improve the TE technology; <br /> <li>it offers an opportunity to further test and evaluate the resource.</p><br /> <br /> In order to help the research, all the participants are invited to contribute and share their own resources with the RTE community, adding them to the table below.<br /> <br /> <br /> <br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! Resource<br /> ! Type<br /> ! Author<br /> ! class="unsortable"|Brief description<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |''New resource''<br /> | <br /> | <br /> | ''People are encouraged to contribute''<br /> |}<br /> <br><br /> <br /> Return to [[RTE Knowledge Resources]]</div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE7_-_Call_for_Resources&diff=8771 RTE7 - Call for Resources 2011-04-04T14:42:40Z

<p>Amarchetti: New page: A study on the knowledge resources used in Textual Entailment was started with the ablation tests carried out on the RTE-5 Main task submissions, which gave a hint about the impact of know...</p> <hr /> <div>A study on the knowledge resources used in Textual Entailment was started with the ablation tests carried out on the RTE-5 Main task submissions, which gave a hint about the impact of knowledge resources on TE system performances. Given the interest raised by this new initiative, the investigation of knowledge resources, which in RTE-6 was extended to tools, will be also continued in the RTE-7 challenge.<p><br /> In order to help the research, all the participants are invited to contribute, sharing their own resources with the RTE community.<br /> Making the resources available to be used by other systems has several advantages: <br /> <br /> <li>It helps improve the TE technology; <br /> <li>it offers an opportunity to further test and evaluate the resource.</p><br /> <br /> <br /> <br /> <br /> <br /> ==New resources available!==<br /> <br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! Resource<br /> ! Type<br /> ! Author<br /> ! class="unsortable"|Brief description<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |''New resource''<br /> | <br /> | <br /> | ''People are encouraged to contribute''<br /> |}<br /> <br><br /> <br /> Return to [[RTE Knowledge Resources]]</div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE_Knowledge_Resources&diff=8770 RTE Knowledge Resources 2011-04-04T14:27:30Z

<p>Amarchetti: </p> <hr /> <div>Knowledge resources have shown their relevance for applied semantic inference, and are extensively used by applied inference systems, such as those developed within the Textual Entailment framework.<br><br /> <br><br /> This page presents a list of the knowledge resources used by systems that have participated in the last RTE challenges. The first table lists the publicly available resources, the second one lists unpublished resources. Both tables are sortable by Resource name, type, author and number of users.<br><br /> <br><br /> RTE Participants are encouraged to add information about all kind of knowledge resources used, from standard existing resources (e.g. WordNet) to knowledge collections created for specific purposes, which can be made available to the community.<br><br /> <br><br /> __TOC__<br /> <br><br /> === Call for Resources ===<br /> <br /> In order to help the research, all the participants are invited to contribute, sharing their own resources with the RTE community. <br /> <br><br /> Making the resources available to be used by other systems has several advantages. On the one hand, it helps improve the TE technology; on the other hand, it offers an opportunity to further test and evaluate the resource.<br /> <br><br /> <br><br /> [[RTE6 - Call for Resources]]<br /> <br><br /> <br><br /> [[RTE7 - Call for Resources]]<br /> <br><br /> === Ablation Tests ===<br /> An ablation test consists of removing one module at a time from a system, and rerunning the system on the test set with the other modules, except the one tested. <br /> <br><br /> Ablation test are meant to help better understand the relevance of the knowledge resources used by RTE systems, and evaluate the contribution of each of them to the systems' performances. In fact, comparing the results achieved in the ablation tests to those obtained by the systems as a whole allows assessing the contribution given by each single resource.<br /> <br><br /> <br><br /> *[[RTE5 - Ablation Tests]]<br /> *[[RTE6 - Ablation Tests]]<br /> <br><br /> <br /> === Publicly available Resources ===<br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! width="80"|Resource<br /> ! width="50"|Type<br /> ! width="150"|Author<br /> ! class="unsortable" width="300"|Brief description<br /> ! width="45"|<small>PAST Users <ref name:"rtethree">RTE-3 data have been provided by participants by means of a questionnaire.</ref></small><br /> ! width="45"|<small>RTE4 Users<ref name:"rtefour">RTE-4 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! width="45"|<small>RTE5 Users<ref name:"rtefive">RTE-5 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! width="45"|<small>RTE6 Users<ref name:"rtesix">RTE-6 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! class="unsortable" width="30"|<small>Usage info</small><br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[WordNet]]<br /> | Lexical DB<br /> | Princeton University<br /> | Lexical database of English nouns, verbs, adjectives and adverbs<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|21<br /> | style="text-align: center;"|18<br /> |style="text-align: center;"| 22<br /> | [[WordNet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://xwn.hlt.utdallas.edu/index.html eXtended Wordnet]<br /> | Lexical DB<br /> | Human Language Technology Research Institute, University of Texas at Dallas<br /> | Extension of WordNet based on the exploitation of the information contained in WordNet definitional glosses: the glosses are syntactically parsed, transformed into logic forms and content words are semantically disambiguated. The Extended Wordnet is an ongoing project. <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|2<br /> |style="text-align: center;"| 0<br /> | [[eXtended WordNet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://ai.stanford.edu/~rion/swn/ Augmented Wordnet]<br /> | Lexical DB<br /> | Stanford University<br /> | The resource is the result of the application of a learning algorithm for inducing semantic taxonomies from parsed text. The algorithm automatically acquires items of world knowledge, and uses these to produce significantly enhanced versions of WordNet (up to 40,000 synsets more).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Augmented Wordnet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://verbs.colorado.edu/~mpalmer/projects/verbnet.html Verbnet]<br /> | Lexical DB<br /> | University of Colorado Boulder<br /> | Lexicon for English verbs organized into classes extending Levin (1993) classes through refinement and addition of subclasses to achieve syntactic and semantic coherence among members of a class<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Verbnet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[VerbOcean]]<br /> | Lexical DB<br /> | Information Sciences Institute, University of Southern California<br /> | Broad-coverage semantic network of verbs<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|6<br /> |style="text-align: center;"| 7<br /> | [[VerbOcean - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://framenet.icsi.berkeley.edu/ FrameNet]<br /> | Lexical DB<br /> | ICSI (International Computer Science Institute) - Berkley University<br /> | Lexical resource for English words, based on frame semantics (valences) and supported by corpus evidence<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|2<br /> |style="text-align: center;"| 3<br /> | [[Framenet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/meyers/NomBank.html NomBank]<br /> | Lexical DB<br /> | New York University<br /> | Lexical resource containing syntactic frames for nouns, extracted from annotated corpora <br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[NomBank Resource - RTE Users|Users]] <br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://verbs.colorado.edu/~mpalmer/projects/ace.html PropBank]<br /> | Lexical DB<br /> | University of Colorado Boulder <br /> | Lexical resource containing syntactic frames for verbs, extracted from annotated corpora <br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[PropBank Resource - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/nomlex/index.html Nomlex] Plus<br /> | Lexical DB<br /> | New York University<br /> | Dictionary of English nominalizations: it describes the allowed complements for a nominalization and relates the nominal complements to the arguments of the corresponding verb<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Nomlex Plus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.cs.ualberta.ca/~lindek/downloads.htm Dekang Lin’s Thesaurus]<br /> | Thesaurus<br /> | University of Alberta<br /> | Thesaurus automatically constructed using a parsed corpus, based on distributional similarity scores<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 2<br /> | [[Dekang Lin’s Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://icon.shef.ac.uk/Moby/mthes.html Grady Ward's Moby Thesaurus]<br /> | Thesaurus<br /> | University of Sheffield<br /> | Thesaurus containing 30,260 root words, with 2,520,264 synonyms and related terms. Grady Ward placed this thesaurus in the public domain in 1996.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Grady Ward's Moby Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://en.wikipedia.org/wiki/Roget%27s_Thesaurus Roget's Thesaurus]<br /> | Thesaurus<br /> | Peter Mark Roget (Electronic version distributed by University of Chicago)<br /> | Roget's Thesaurus is a widely-used English thesaurus, created by Dr. Peter Mark Roget in 1805. The original edition had 15,000 words, and each new edition has been larger. The electronic edition ([http://machaut.uchicago.edu/rogets version 1.02]) is made available by University of Chicago.<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Roget's Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.wikipedia.org/ Wikipedia]<br /> | Encyclopedia<br /> | <br /> | Free encyclopedia. Used for extraction of lexical-semantic rules (from its more structured parts), named entity recognition, geographical information etc.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|6<br /> |style="text-align: center;"| 6<br /> | [[Wikipedia - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.umbel.org/ Umbel]<br /> | Ontology<br /> | Structured Dynamics LLC, Coralville, IA<br /> | UMBEL stands for Upper Mapping and Binding Exchange Layer and is a lightweight ontology structure for relating Web content and data to a standard set of subject concepts<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Umbel - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.mpi-inf.mpg.de/yago-naga/yago/ YAGO]<br /> | Ontology<br /> | Max-Planck Institute for Informatics, Saarbrücken, Germany<br /> | Light-weight and extensible ontology. It contains more than 2 million entities and 20 million facts about these entities. The facts have been automatically extracted from Wikipedia and unified with WordNet.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[YAGO - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://dbpedia.org/About DBpedia]<br /> | Ontology<br /> | Open community project<br /> | DBpedia is a community effort to extract structured information from Wikipedia and to make this information available on the Web. The DBpedia knowledge base currently describes more than 2.9 million things in 91 different languages and consists of 479 million pieces of information.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[DBpedia - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[DIRT Paraphrase Collection]]<br /> | Collection of paraphrases<br /> | University of Alberta<br /> | DIRT (Discovery of Inference Rules from Text) is both an algorithm and a resulting knowledge collection. The DIRT knowledge collection is the output of the DIRT algorithm over a 1GB set of newspaper text.<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|4<br /> | style="text-align: center;"|3<br /> |style="text-align: center;"| 4<br /> | [[DIRT Paraphrase Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[TEASE]] Collection<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University<br /> | Output of the TEASE algorithm <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Tease Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://badc.nerc.ac.uk/help/abbrevs.html BADC Acronym and Abbreviation List]<br /> | Word List<br /> | BADC (British Atmospheric Data Centre)<br /> | Acronym and Abbreviation List<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[BADC Acronym and Abbreviation List - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://clipdemos.umiacs.umd.edu/catvar/ Catvar The Categorial Variation Database (English)]<br /> | Word List<br /> | University of Maryland<br /> | A Categorial-Variation Database (or Catvar) is a database of clusters of uninflected words (lexemes) and their categorial (i.e. part-of-speech) variants. The database was developed for English using a combination of resources and algorithms, including the LCS Verb and Preposition Databases (Dorr 2001), the Brown Corpus section of the Penn Treebank (Marcus et al. 1994), an English morphological analysis lexicon developed for PC-Kimmo (ENGLEX) (Antworth 1990), WordNet1.6 (Fellbaum 1998), an English Verb-Noun list extracted from Nomlex (Macleod et al. 1998), a similar list extracted from LDOCE (Procter 1978) and the Porter stemmer (Porter 1980).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 1<br /> | [[Categorial-Variation Database - RTE Users|Users]]<br /> <br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.acronym-guide.com/ Acronym Guide]<br /> | Word List<br /> | Acronym-Guide.com <br /> | Acronym and Abbreviation Lists for English, branched in thematic directories <br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|3<br /> |style="text-align: center;"| 0<br /> | [[Acronym Guide - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.ldc.upenn.edu/Catalog/CatalogEntry.jsp?catalogId=LDC2006T13 Web1T 5-grams]<br /> | Word list<br /> | Linguistic Data Consortium, University of Pennsylvania; Google Inc.<br /> | Data set containing English word n-grams and their observed frequency counts. The n-gram counts were generated from approximately 1 trillion word tokens of text from publicly accessible Web pages<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Web1T - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.coli.uni-saarland.de/%7Erwang/resources/RTE3_RTE4_NGD.zip Normalized Google Distance (RTE3&RTE4)]<br /> | Word Pair Co-occurrence<br /> | Saarland University<br /> | Co-occurrence of the word pairs in RTE3 and RTE4 using Normalized Google Distance (Cilibrasi and Vitanyi, 2004). The word pairs are all the possible combinations of content words in T and H. In practice, we used [http://m.www.yahoo.com/ Yahoo!] as the search engine.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Normalized Google Distance (RTE3&RTE4)- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.coli.uni-saarland.de/%7Erwang/resources/RTE5_NGD.zip Normalized Google Distance (RTE5)]<br /> | Word Pair Co-occurrence<br /> | Saarland University<br /> | Co-occurrence of the word pairs in RTE3 and RTE4 using Normalized Google Distance (Cilibrasi and Vitanyi, 2004). The word pairs are all the possible combinations of content words in T and H. In practice, we used [http://m.www.yahoo.com/ Yahoo!] as the search engine.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Normalized Google Distance (RTE5)- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://geonames.usgs.gov/index.html GNIS - Geographic Names Information System]<br /> | Gazetteer<br /> | USGS (United States Geological Survey)<br /> | Database containing the Federal and national standard toponyms for USA, associated areas and Antarctica<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[GNIS - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.geonames.org/ Geonames]<br /> | Gazetteer<br /> | <br /> | Database containing eight million geographical names. It is integrating geographical data such as names of places in various languages, elevation, population and others from various sources.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Geonames - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/paraphrase/ Sekine's Paraphrase Database]<br /> | Collection of paraphrases<br /> | Department of Computer Science, New York University <br /> | Data-base created using Sekine's method, NOT cleaned up by human. It includes 19,975 sets of paraphrases with 191,572 phrases.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Sekine's Paraphrase Database - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://research.microsoft.com/research/downloads/Details/607D14D9-20CD-47E3-85BC-A2F65CD28042/Details.aspx Microsoft Research Paraphrase Corpus]<br /> | Collection of paraphrases<br /> | Microsoft Research<br /> | Text file containing 5800 pairs of sentences which have been extracted from news sources on the web, along with human annotations indicating whether each pair captures a paraphrase/semantic equivalence relationship.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Microsoft Research Paraphrase Corpus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.cs.cornell.edu/~cristian/Without_a_doubt_-_Data.html Downward entailing operators] <br /> | Collection of entailing operators<br /> | Department of Computer Science, Cornell University, Ithaca NY<br /> | System output of an unsupervised algorithm recovering many Downward Entailing operators, like 'doubt'.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Downward entailing operators - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/WikiRules.html WikiRules!]<br /> | Lexical Reference rule-base<br /> | Bar-Ilan University<br /> | Extraction of about 8 million lexical reference rules from the text body (first sentence) and from metadata (links, redirects, parentheses) of Wikipedia. Provides better performance than other automatically constructed resources and comparable performance to WordNet. Offers complementary knowledge to WordNet. <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[WikiRules! - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://www.cs.utexas.edu/users/pclark/dart/ DART]<br /> | Collection of "world knowledge" propositions<br /> | Boeing Research and Technology<br /> | 23 million tuples such as "airplanes can fly to airports", "rivers can flood" collected from abstracted parse trees.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[DART - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://cs.biu.ac.il/~nlp/download/FRED/latest/FRED.zip FRED]<br /> | FrameNet-derived entailment rule-base<br /> | Bar-Ilan University<br /> | This package contains the outputs of the FRED algorithm, an algorithm which extracts entailment rules from FrameNet.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[FRED - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/DIRECT.html DIRECT]<br /> | Directional Distributional Term-Similarity Resource<br /> | Bar-Ilan University<br /> | This is a resource of directional distributional term-similarity rules (mostly lexical entailment rules) automatically extracted using the inclusion relation as described in (Kotlerman et.al., ACL-09).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[DIRECT - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/binaryDirt.html binaryDIRT]<br /> | Entailment rules between binary templates using DIRT algorithm<br /> | Bar-Ilan University<br /> | This resource contains entailment rules over binary templates learned over the Reuters corpus using<br /> the DIRT algorithm of Lin and Pantel.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[BinaryDIRT- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/unaryBInc.html unaryBInc]<br /> | Entailment rules between unary templates using BInc algorithm<br /> | Bar-Ilan University<br /> | This resource contains entailment rules over unary templates learned over the Reuters corpus using<br /> the BInc algorithm of Szpektor and Dagan (2008).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[unaryBInc- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''New resource''<br /> | <br /> | <br /> | ''Participants are encouraged to contribute''<br /> | style="text-align: center;"| <br /> | style="text-align: center;"| <br /> | style="text-align: center;"| <br /> |style="text-align: center;"| <br /> | [[New Resource2 - RTE Users|Users]]<br /> <br /> |}<br /> <br><br /> <br><br /> <br /> === Not available Resources ===<br /> The following table lists the unpublished resources used by RTE participants. Some of them have been developed by Users themselves specifically for RTE. Interested people may turn to authors to obtain further information.<br /> <br><br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! width="80"|Resource<br /> ! width="50"|Type<br /> ! width="180"|Author<br /> ! class="unsortable" width="500"|Brief description<br /> ! width="30"|<small>PAST Users</small><br /> ! width="30"|<small>RTE4 Users</small><br /> ! width="30"|<small>RTE5 Users</small><br /> ! width="30"|<small>RTE6 Users</small><br /> ! class="unsortable" width="30"|<small>Usage info</small><br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.parc.com/ PARC] Polarity Lexicon<br /> | Lexical DB<br /> | PARC - Palo Alto Research Center<br /> | Verbs classification with respect to semantic polarity<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | [[Parc Polarity Lexicon - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Gazetteer from [http://trec.nist.gov/ TREC]<br /> | Gazetteer<br /> | NIST - National Institute of Standards and Technology<br /> | Cities and other geographical names<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | [[Gazetteer from TREC - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DFKI Geographic Ontology<br>''(to be released)''<br /> | Ontology<br /> | DFKI - German Research Center for Artificial Intelligence<br /> | Ontology containing geographic terms and two kinds of relations: the directional ''part-of'' relation, and the ''equal'' relation for synonyms and abbreviations of the same geographic area (e.g ''the United Kingdom'', ''the UK'', ''Great Britain'', etc.) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | [[Geographic Ontology - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Geo''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | Meronymy entailment rules, based on TREC’s TIPSTER gazetteer.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | [[Geo - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Regex''<br /> | Collection of Entailment rules <br /> | Bar-Ilan University; Tel-Aviv University<br /> | Small set of entailment rules based on regular expressions, intended to address lexical variability involving temporal phrases<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | [[Regex - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Syntactic rule base <br>''(to be released)''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | A manually-composed collection of entailment rules which define parse tree transformations. The rules cover generic syntactic phenomena such as appositions, conjunctions, passive, relative clause, etc. (Bar-Haim et al., AAAI-07) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | [[Syntactic rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Polarity rule base <br>''(to be released)''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | A manually-composed collection of entailment rules which detect predicates whose polarity is negative (e.g. didn't dance) or unknown (e.g. plans to dance). The rules capture diverse phenomena that affect polarity, e.g. verbal negation, modal verbs, conditionals, and certain verbs that induce negative or "unknown" polarity context. The latter were taken mainly from VerbNet. Extends a resource described in (Bar-Haim et al., AAAI-07) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0 <br /> | style="text-align: center;"|0<br /> | [[Polarity rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Lexical-Syntactic rule base<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | Extract lexical-syntactic entailment rules for predicates (verbal and nominal), including argument mapping. The resource is based on WordNet, Nomlex-Plus and Unary DIRT (Szpektor and Dagan, Coling 08)<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | [[Lexical-Syntactic rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | OPENU Collection<br /> | Collection of Entailment Rules and Patterns<br /> | Open University<br /> | Collections of rules, patterns etc. for RTE purpose, extracted from Reuter corpus parsed using Minipar.<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0 <br /> | [[OPENU Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Abbr''<br /> | Collection of rules for abbreviation<br /> | Bar-Ilan University; Tel-Aviv University<br /> | 2000 Abbreviation rules, extracted from [http://badc.nerc.ac.uk/help/abbrevs.html BADC] and [http://www.acronym-guide.com/ Acronym Guide]<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | [[Abbr - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Negation_list<br /> | Negation rules<br /> | „Al. I. Cuza“ University, Iasi, Romania <br /> | List of negative terms and words (verbs, adjectives, nouns) affecting modality or factuality of a infinitive verb preceded by the particle "to" (e.g. "believe","necessary", "attempt")<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1 <br /> | style="text-align: center;"|0<br /> | [[UAIC Negation_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DLSIUAES Negation_list<br /> | List of negative terms<br /> | University of Alicante<br /> | Basic list of negative terms.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0 <br /> | [[DLSIUAES Negation_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Quantifier_list<br /> | List of quantifiers<br /> | „Al. I. Cuza“ University, Iasi, Romania <br /> | List of quantifiers affecting entailment judgment. The quantifiers are taken from a list which contains expressions like “more than”, “less than”, or words such as “over”, “under”, etc.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0 <br /> | [[UAIC Quantifier_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | FBKirst StopWord list<br /> | List of frequent words<br /> | FBK-Irst;<br/>University of Trento - Italy <br /> | A list of the 572 most frequent English words.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0 <br /> | [[FBKirst StopWord list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | IKOMA Dictionary of Named Entities Acronyms and Synonyms<br /> | Dictionary of Named Entities Acronyms and Synonyms<br /> | IKOMA; NEC Corporation, Takayama, Ikoma, Nara, Japan<br /> | Acronym dictionary constructed automatically from the corpus and a synonym dictionary that contains geographical terms.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | [[IKOMA2 - RTE Users|Users]]<br /> <br /> |}<br /> <br><br /> <br /> ==Footnotes==<br /> <references/></div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=Textual_Entailment_Resource_Pool&diff=8681 Textual Entailment Resource Pool 2011-02-09T09:14:24Z

<p>Amarchetti: </p> <hr /> <div>[[Textual Entailment]] &gt; '''Resources''':<br /> ----<br /> <br /> [[Textual Entailment|Textual entailment]] systems rely on many different types of [[Natural Language Processing|NLP]] resources, including term banks, paraphrase lists, parsers, named-entity recognizers, etc. With so many resources being continuously released and improved, it can be difficult to know which particular resource to use when developing a system.<br /> <br /> In response, the [[Recognizing Textual Entailment|Recognizing Textual Entailment (RTE)]] shared task community initiated a new activity for building this ''Textual Entailment Resource Pool''. RTE participants and any other member of the NLP community are encouraged to contribute to the pool.<br /> <br /> In an effort to determine the relative impact of the resources, RTE participants are strongly encouraged to report, whenever possible, the contribution to the overall performance of each utilized resource. Formal qualitative and quantitative results should be included in a separate section of the system report as well as posted on the talk pages of this Textual Entailment Resource Pool.<br /> <br /> '''Adding''' a new resource is very easy. See how to '''use existing templates''' to do this in [[Help:Using Templates]].<br /> <br /> == Complete RTE Systems ==<br /> <br /> * [http://project.cgm.unive.it/html/venses.html VENSES] (from Ca' Foscari University of Venice, Italy)<br /> * [http://svn.ask.it.usyd.edu.au/trac/candc/wiki/nutcracker Nutcracker] (available for download)<br /> * [http://l2r.cs.uiuc.edu/~cogcomp/kindleDemo.php Entailment Demo] (from the University of Illinois at Urbana-Champaign) - INACTIVE (as of 2010-12-22)<br /> * [http://edits.fbk.eu/ EDITS - Edit Distance Textual Entailment Suite] (open source software developed by [http://hlt.fbk.eu/ Human Language Technology (HLT) group at FBK-Irst])<br /> <br /> == RTE data sets ==<br /> === Past campaigns data sets ===<br /> * [http://pascallin.ecs.soton.ac.uk/Challenges/RTE/Datasets RTE1 dataset] - provided by [http://pascallin.ecs.soton.ac.uk PASCAL]<br /> * [http://pascallin.ecs.soton.ac.uk/Challenges/RTE2/Datasets RTE2 dataset] - provided by [http://pascallin.ecs.soton.ac.uk PASCAL]<br /> * [http://pascallin.ecs.soton.ac.uk/Challenges/RTE3/Datasets RTE3 dataset] - provided by [http://pascallin.ecs.soton.ac.uk PASCAL]<br /> * [http://www.nist.gov/tac/data/past/2008/RTE-4.html RTE4 dataset] - provided by [http://www.nist.gov/index.html NIST] - freely available upon request. For details see [http://www.nist.gov/tac/data/forms/index.html TAC User Agreements]<br /> * [http://www.nist.gov/tac/data/past/2009/RTE-5.html RTE5 dataset] - provided by [http://www.nist.gov/index.html NIST] - freely available upon request. For details see [http://www.nist.gov/tac/data/forms/index.html TAC User Agreements]<br /> * [http://www.nist.gov/tac/2010/RTE/index.html RTE6 dataset] - provided by [http://www.nist.gov/index.html NIST] - freely available upon request. For details see [http://www.nist.gov/tac/data/forms/index.html TAC User Agreements]<br /> <br /> === Other data sets ===<br /> * [http://www.coli.uni-saarland.de/projects/salsa/fate FrameNet manually annotated RTE 2006 Test Set.] Provided by [http://www.coli.uni-saarland.de/projects/salsa/ SALSA project, Saarland University.]<br /> * [http://www.cs.biu.ac.il/~nlp/files/RTE_2006_Aligned.zip Manually Word Aligned RTE 2006 Data Sets.] Provided by [http://research.microsoft.com/nlp/ the Natural Language Processing Group, Microsoft Research.]<br /> * [http://www-nlp.stanford.edu/projects/contradiction/ RTE data sets annotated for a 3-way decision: entails, contradicts, unknown.] Provided by Stanford NLP Group.<br /> * [http://www.cs.utexas.edu/~pclark/bpi-test-suite/ BPI RTE data set] - 250 pairs, focusing on world knowledge. Provided jointly by [http://www.boeing.com/phantom/math_ct/index.html Boeing], [http://wordnet.cs.princeton.edu/ Princeton], and [http://www.isi.edu ISI].<br /> * [http://hlt.fbk.eu/en/Technology/TE_Specialized_Data Textual Entailment Specialized Data Sets] - 90 RTE-5 Test Set pairs annotated with linguistic phenomena + 203 monothematic pairs (i.e. pairs where only one linguistic phenomenon is relevant to the entailment relation) created from the 90 annotated pairs. Provided jointly by [http://hlt.fbk.eu/en/home FBK-Irst], and [http://www.celct.it/ CELCT].<br /> * [http://www.nist.gov/tac/data/ RTE-5 Search Pilot Data Set annotated with anaphora and coreference information] - RTE-5 Search Data Set annotated with anaphora/coreference information + Augmented RTE-5 Search Data Set, where all the referring expressions which need to be resolved in the entailing sentences are substituted by explicit expressions on the basis of the anaphora/coreference annotation. Provided by [http://www.celct.it/ CELCT] and distributed by [http://www.nist.gov/index.html NIST] at the [http://www.nist.gov/tac/data/ Past TAC Data] web page (2009 Search Pilot, annotated test/dev data).<br /> * [http://www.investigacion.frc.utn.edu.ar/mslabs/~jcastillo/Sagan-test-suite/ RTE-3-Expanded, RTE-4-Expanded, RTE-5-Expanded.] RTE data set expanded in the two and three way task, at least 2000 pairs in each data set.<br /> * [https://agora.cs.illinois.edu/display/rtedata/Explanation+Based+Analysis+of+RTE+Data Explanation-Based Analysis annotation of RTE 5 Main Task subset] described in [http://l2r.cs.uiuc.edu/~danr/Papers/SammonsVyRo10.pdf this ACL 2010 paper]<br /> <br /> == Knowledge Resources ==<br /> The [[RTE Knowledge Resources]] page presents: <br /> <br /> * a [[RTE Knowledge Resources#Call for Resources|call for resources]], inviting system developers to share the resources used by their own TE engines, to both help improve the TE technology and further test and evaluate such resources;<br /> * [[RTE Knowledge Resources#Ablation tests|the ablation tests]] carried out in the RTE challenges in order to evaluate the impact of knowledge resources and tools on TE system performances;<br /> * [[RTE Knowledge Resources#Publicly available Resources|lists of knowledge resources]], both publically available and unpublished, used by systems participating in the last RTE challenges.<br /> <br /> <br /> == Tools ==<br /> <br /> === Parsers ===<br /> * [http://svn.ask.it.usyd.edu.au/trac/candc C&C parser for Combinatory Categorial Grammar]<br /> * [[Minipar]]<br /> * [http://l2r.cs.uiuc.edu/~cogcomp/asoftware.php?skey=SP Shallow Parser] - from the University of Illinois at Urbana-Champaign, see a [http://l2r.cs.uiuc.edu/~cogcomp/shallow_parse_demo.php web demo] of this tool<br /> <br /> === Role Labelling ===<br /> * [http://cemantix.org/assert.html ASSERT]<br /> * [http://www.coli.uni-saarland.de/projects/salsa/shal/ Shalmaneser]<br /> * [http://l2r.cs.uiuc.edu/~cogcomp/asoftware.php?skey=SRL Semantic Role Labeler] - from the University of Illinois at Urbana-Champaign, see a [http://l2r.cs.uiuc.edu/~cogcomp/srl-demo.php web demo] of this tool<br /> <br /> === Entity Recognition Tools ===<br /> * [http://l2r.cs.uiuc.edu/~cogcomp/asoftware.php?skey=NE Illinois Named Entity Tagger] - see a [http://l2r.cs.uiuc.edu/~cogcomp/ne_demo.php web demo] of this tool<br /> * [http://l2r.cs.uiuc.edu/~cogcomp/asoftware.php?skey=CORANKER Illinois Multi-lingual Named Entity Discovery Tool] - see a [http://l2r.cs.uiuc.edu/~cogcomp/ne_matcher_demo.php web demo] of this tool<br /> <br /> === Similarity / Relatedness Tools ===<br /> * [http://ixa2.si.ehu.es/ukb UKB]: Open source WordNet-based similarity/relatedness tool, includes also pre-computed semantic vectors for all words<br /> <br /> === Corpus Readers ===<br /> * [http://nltk.org NLTK] provides a corpus reader for the data from RTE Challenges 1, 2, and 3 - see the [http://nltk.org/doc/guides/corpus.html#rte Corpus Readers] Guide for more information.<br /> <br /> === Related Libraries ===<br /> <br /> * [http://www.semantilog.org/pypes.html PyPES] general purpose library containing evaluation environment for RTE and McPIET text inference engine based on the ERG (English Resource Grammar)<br /> <br /> == References ==<br /> <br /> *[[Textual Entailment References#Workshops and Tutorials | Workshops and Tutorials ]]<br /> *[[Textual Entailment References#Papers in recent conferences and other workshops | Papers in recent conferences and other workshops ]]<br /> *[[Textual Entailment References#Journal papers | Journal papers ]]<br /> <br /> == Links ==<br /> * [http://homepages.inf.ed.ac.uk/jbos/rte/ Textual Entailment site by Johan Bos]<br /> * [http://ai-nlp.info.uniroma2.it/research/te/ Textual Entailment at the University of Rome "Tor Vergata"]<br /> [[Category:Textual Entailment Portal]]<br /> * [http://cogcomp.cs.illinois.edu/page/demo_view/18 Illinois Textual Entailment System Component demos]</div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=IKOMA2_-_RTE_Users&diff=8674 IKOMA2 - RTE Users 2011-02-04T14:36:33Z

<p>Amarchetti: New page: {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1" |- bgcolor="#CDCDCD" ! nowrap="nowrap"|Participants* ! nowrap="nowrap"|Campaign ! nowrap="nowrap"|Version ! nowrap=...</p> <hr /> <div>{|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1" <br /> |- bgcolor="#CDCDCD"<br /> ! nowrap="nowrap"|Participants*<br /> ! nowrap="nowrap"|Campaign<br /> ! nowrap="nowrap"|Version<br /> ! nowrap="nowrap"|Specific usage description<br /> ! nowrap="nowrap"|Evaluations / Comments<br /> <br /> |- bgcolor="#ECECEC" align="left"<br /> | IKOMA<br /> | RTE6<br /> | <br /> | Remove synonym dictionaries: as acronym dictionary constructed automatically from the corpus and a synonym dictionary that contains geographical terms.<br /> <br /> | Ablation test performed. Negative impact of the resource: -0.76% F1 impact.<br /> <br /> |- bgcolor="#ECECEC" align="left"<br /> | ''New user''<br /> | <br /> | <br /> | <br /> | ''Participants are encouraged to contribute.''<br /> |}<br /> {|class="wikitable" cellpadding="3" cellspacing="0" border="0" style="margin-left: 20px;"<br /> |-<br /> ! align="left"|Total: 1<br /> |}<br /> <br><br /> <br /> [*] For further information about participants, click here: [[RTE Challenges - Data about participants]]<br /> <br /> Return to [[RTE Knowledge Resources]]</div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE_Knowledge_Resources&diff=8673 RTE Knowledge Resources 2011-02-04T14:18:22Z

<p>Amarchetti: /* Not available Resources */</p> <hr /> <div>Knowledge resources have shown their relevance for applied semantic inference, and are extensively used by applied inference systems, such as those developed within the Textual Entailment framework.<br><br /> <br><br /> This page presents a list of the knowledge resources used by systems that have participated in the last RTE challenges. The first table lists the publicly available resources, the second one lists unpublished resources. Both tables are sortable by Resource name, type, author and number of users.<br><br /> <br><br /> RTE Participants are encouraged to add information about all kind of knowledge resources used, from standard existing resources (e.g. WordNet) to knowledge collections created for specific purposes, which can be made available to the community.<br><br /> <br><br /> __TOC__<br /> <br><br /> === Call for Resources ===<br /> <br /> In order to help the research, all the participants are invited to contribute, sharing their own resources with the RTE community. <br /> <br><br /> Making the resources available to be used by other systems has several advantages. On the one hand, it helps improve the TE technology; on the other hand, it offers an opportunity to further test and evaluate the resource.<br /> <br><br /> <br><br /> [[RTE6 - Call for Resources]]<br /> <br><br /> === Ablation Tests ===<br /> An ablation test consists of removing one module at a time from a system, and rerunning the system on the test set with the other modules, except the one tested. <br /> <br><br /> Ablation test are meant to help better understand the relevance of the knowledge resources used by RTE systems, and evaluate the contribution of each of them to the systems' performances. In fact, comparing the results achieved in the ablation tests to those obtained by the systems as a whole allows assessing the contribution given by each single resource.<br /> <br><br /> <br><br /> *[[RTE5 - Ablation Tests]]<br /> *[[RTE6 - Ablation Tests]]<br /> <br><br /> <br /> === Publicly available Resources ===<br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! width="80"|Resource<br /> ! width="50"|Type<br /> ! width="150"|Author<br /> ! class="unsortable" width="300"|Brief description<br /> ! width="45"|<small>PAST Users <ref name:"rtethree">RTE-3 data have been provided by participants by means of a questionnaire.</ref></small><br /> ! width="45"|<small>RTE4 Users<ref name:"rtefour">RTE-4 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! width="45"|<small>RTE5 Users<ref name:"rtefive">RTE-5 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! width="45"|<small>RTE6 Users<ref name:"rtesix">RTE-6 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! class="unsortable" width="30"|<small>Usage info</small><br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[WordNet]]<br /> | Lexical DB<br /> | Princeton University<br /> | Lexical database of English nouns, verbs, adjectives and adverbs<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|21<br /> | style="text-align: center;"|18<br /> |style="text-align: center;"| 22<br /> | [[WordNet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://xwn.hlt.utdallas.edu/index.html eXtended Wordnet]<br /> | Lexical DB<br /> | Human Language Technology Research Institute, University of Texas at Dallas<br /> | Extension of WordNet based on the exploitation of the information contained in WordNet definitional glosses: the glosses are syntactically parsed, transformed into logic forms and content words are semantically disambiguated. The Extended Wordnet is an ongoing project. <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|2<br /> |style="text-align: center;"| 0<br /> | [[eXtended WordNet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://ai.stanford.edu/~rion/swn/ Augmented Wordnet]<br /> | Lexical DB<br /> | Stanford University<br /> | The resource is the result of the application of a learning algorithm for inducing semantic taxonomies from parsed text. The algorithm automatically acquires items of world knowledge, and uses these to produce significantly enhanced versions of WordNet (up to 40,000 synsets more).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Augmented Wordnet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://verbs.colorado.edu/~mpalmer/projects/verbnet.html Verbnet]<br /> | Lexical DB<br /> | University of Colorado Boulder<br /> | Lexicon for English verbs organized into classes extending Levin (1993) classes through refinement and addition of subclasses to achieve syntactic and semantic coherence among members of a class<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Verbnet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[VerbOcean]]<br /> | Lexical DB<br /> | Information Sciences Institute, University of Southern California<br /> | Broad-coverage semantic network of verbs<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|6<br /> |style="text-align: center;"| 7<br /> | [[VerbOcean - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://framenet.icsi.berkeley.edu/ FrameNet]<br /> | Lexical DB<br /> | ICSI (International Computer Science Institute) - Berkley University<br /> | Lexical resource for English words, based on frame semantics (valences) and supported by corpus evidence<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|2<br /> |style="text-align: center;"| 3<br /> | [[Framenet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/meyers/NomBank.html NomBank]<br /> | Lexical DB<br /> | New York University<br /> | Lexical resource containing syntactic frames for nouns, extracted from annotated corpora <br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[NomBank Resource - RTE Users|Users]] <br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://verbs.colorado.edu/~mpalmer/projects/ace.html PropBank]<br /> | Lexical DB<br /> | University of Colorado Boulder <br /> | Lexical resource containing syntactic frames for verbs, extracted from annotated corpora <br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[PropBank Resource - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/nomlex/index.html Nomlex] Plus<br /> | Lexical DB<br /> | New York University<br /> | Dictionary of English nominalizations: it describes the allowed complements for a nominalization and relates the nominal complements to the arguments of the corresponding verb<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Nomlex Plus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.cs.ualberta.ca/~lindek/downloads.htm Dekang Lin’s Thesaurus]<br /> | Thesaurus<br /> | University of Alberta<br /> | Thesaurus automatically constructed using a parsed corpus, based on distributional similarity scores<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 2<br /> | [[Dekang Lin’s Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://icon.shef.ac.uk/Moby/mthes.html Grady Ward's Moby Thesaurus]<br /> | Thesaurus<br /> | University of Sheffield<br /> | Thesaurus containing 30,260 root words, with 2,520,264 synonyms and related terms. Grady Ward placed this thesaurus in the public domain in 1996.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Grady Ward's Moby Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://en.wikipedia.org/wiki/Roget%27s_Thesaurus Roget's Thesaurus]<br /> | Thesaurus<br /> | Peter Mark Roget (Electronic version distributed by University of Chicago)<br /> | Roget's Thesaurus is a widely-used English thesaurus, created by Dr. Peter Mark Roget in 1805. The original edition had 15,000 words, and each new edition has been larger. The electronic edition ([http://machaut.uchicago.edu/rogets version 1.02]) is made available by University of Chicago.<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Roget's Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.wikipedia.org/ Wikipedia]<br /> | Encyclopedia<br /> | <br /> | Free encyclopedia. Used for extraction of lexical-semantic rules (from its more structured parts), named entity recognition, geographical information etc.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|6<br /> |style="text-align: center;"| 6<br /> | [[Wikipedia - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.umbel.org/ Umbel]<br /> | Ontology<br /> | Structured Dynamics LLC, Coralville, IA<br /> | UMBEL stands for Upper Mapping and Binding Exchange Layer and is a lightweight ontology structure for relating Web content and data to a standard set of subject concepts<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Umbel - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.mpi-inf.mpg.de/yago-naga/yago/ YAGO]<br /> | Ontology<br /> | Max-Planck Institute for Informatics, Saarbrücken, Germany<br /> | Light-weight and extensible ontology. It contains more than 2 million entities and 20 million facts about these entities. The facts have been automatically extracted from Wikipedia and unified with WordNet.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[YAGO - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://dbpedia.org/About DBpedia]<br /> | Ontology<br /> | Open community project<br /> | DBpedia is a community effort to extract structured information from Wikipedia and to make this information available on the Web. The DBpedia knowledge base currently describes more than 2.9 million things in 91 different languages and consists of 479 million pieces of information.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[DBpedia - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[DIRT Paraphrase Collection]]<br /> | Collection of paraphrases<br /> | University of Alberta<br /> | DIRT (Discovery of Inference Rules from Text) is both an algorithm and a resulting knowledge collection. The DIRT knowledge collection is the output of the DIRT algorithm over a 1GB set of newspaper text.<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|4<br /> | style="text-align: center;"|3<br /> |style="text-align: center;"| 4<br /> | [[DIRT Paraphrase Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[TEASE]] Collection<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University<br /> | Output of the TEASE algorithm <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Tease Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://badc.nerc.ac.uk/help/abbrevs.html BADC Acronym and Abbreviation List]<br /> | Word List<br /> | BADC (British Atmospheric Data Centre)<br /> | Acronym and Abbreviation List<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[BADC Acronym and Abbreviation List - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://clipdemos.umiacs.umd.edu/catvar/ Catvar The Categorial Variation Database (English)]<br /> | Word List<br /> | University of Maryland<br /> | A Categorial-Variation Database (or Catvar) is a database of clusters of uninflected words (lexemes) and their categorial (i.e. part-of-speech) variants. The database was developed for English using a combination of resources and algorithms, including the LCS Verb and Preposition Databases (Dorr 2001), the Brown Corpus section of the Penn Treebank (Marcus et al. 1994), an English morphological analysis lexicon developed for PC-Kimmo (ENGLEX) (Antworth 1990), WordNet1.6 (Fellbaum 1998), an English Verb-Noun list extracted from Nomlex (Macleod et al. 1998), a similar list extracted from LDOCE (Procter 1978) and the Porter stemmer (Porter 1980).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 1<br /> | [[Categorial-Variation Database - RTE Users|Users]]<br /> <br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.acronym-guide.com/ Acronym Guide]<br /> | Word List<br /> | Acronym-Guide.com <br /> | Acronym and Abbreviation Lists for English, branched in thematic directories <br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|3<br /> |style="text-align: center;"| 0<br /> | [[Acronym Guide - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.ldc.upenn.edu/Catalog/CatalogEntry.jsp?catalogId=LDC2006T13 Web1T 5-grams]<br /> | Word list<br /> | Linguistic Data Consortium, University of Pennsylvania; Google Inc.<br /> | Data set containing English word n-grams and their observed frequency counts. The n-gram counts were generated from approximately 1 trillion word tokens of text from publicly accessible Web pages<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Web1T - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.coli.uni-saarland.de/%7Erwang/resources/RTE3_RTE4_NGD.zip Normalized Google Distance (RTE3&RTE4)]<br /> | Word Pair Co-occurrence<br /> | Saarland University<br /> | Co-occurrence of the word pairs in RTE3 and RTE4 using Normalized Google Distance (Cilibrasi and Vitanyi, 2004). The word pairs are all the possible combinations of content words in T and H. In practice, we used [http://m.www.yahoo.com/ Yahoo!] as the search engine.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Normalized Google Distance (RTE3&RTE4)- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.coli.uni-saarland.de/%7Erwang/resources/RTE5_NGD.zip Normalized Google Distance (RTE5)]<br /> | Word Pair Co-occurrence<br /> | Saarland University<br /> | Co-occurrence of the word pairs in RTE3 and RTE4 using Normalized Google Distance (Cilibrasi and Vitanyi, 2004). The word pairs are all the possible combinations of content words in T and H. In practice, we used [http://m.www.yahoo.com/ Yahoo!] as the search engine.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Normalized Google Distance (RTE5)- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://geonames.usgs.gov/index.html GNIS - Geographic Names Information System]<br /> | Gazetteer<br /> | USGS (United States Geological Survey)<br /> | Database containing the Federal and national standard toponyms for USA, associated areas and Antarctica<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[GNIS - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.geonames.org/ Geonames]<br /> | Gazetteer<br /> | <br /> | Database containing eight million geographical names. It is integrating geographical data such as names of places in various languages, elevation, population and others from various sources.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Geonames - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/paraphrase/ Sekine's Paraphrase Database]<br /> | Collection of paraphrases<br /> | Department of Computer Science, New York University <br /> | Data-base created using Sekine's method, NOT cleaned up by human. It includes 19,975 sets of paraphrases with 191,572 phrases.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Sekine's Paraphrase Database - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://research.microsoft.com/research/downloads/Details/607D14D9-20CD-47E3-85BC-A2F65CD28042/Details.aspx Microsoft Research Paraphrase Corpus]<br /> | Collection of paraphrases<br /> | Microsoft Research<br /> | Text file containing 5800 pairs of sentences which have been extracted from news sources on the web, along with human annotations indicating whether each pair captures a paraphrase/semantic equivalence relationship.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Microsoft Research Paraphrase Corpus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.cs.cornell.edu/~cristian/Without_a_doubt_-_Data.html Downward entailing operators] <br /> | Collection of entailing operators<br /> | Department of Computer Science, Cornell University, Ithaca NY<br /> | System output of an unsupervised algorithm recovering many Downward Entailing operators, like 'doubt'.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Downward entailing operators - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/WikiRules.html WikiRules!]<br /> | Lexical Reference rule-base<br /> | Bar-Ilan University<br /> | Extraction of about 8 million lexical reference rules from the text body (first sentence) and from metadata (links, redirects, parentheses) of Wikipedia. Provides better performance than other automatically constructed resources and comparable performance to WordNet. Offers complementary knowledge to WordNet. <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[WikiRules! - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://www.cs.utexas.edu/users/pclark/dart/ DART]<br /> | Collection of "world knowledge" propositions<br /> | Boeing Research and Technology<br /> | 23 million tuples such as "airplanes can fly to airports", "rivers can flood" collected from abstracted parse trees.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[DART - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://cs.biu.ac.il/~nlp/download/FRED/latest/FRED.zip FRED]<br /> | FrameNet-derived entailment rule-base<br /> | Bar-Ilan University<br /> | This package contains the outputs of the FRED algorithm, an algorithm which extracts entailment rules from FrameNet.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[FRED - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/DIRECT.html DIRECT]<br /> | Directional Distributional Term-Similarity Resource<br /> | Bar-Ilan University<br /> | This is a resource of directional distributional term-similarity rules (mostly lexical entailment rules) automatically extracted using the inclusion relation as described in (Kotlerman et.al., ACL-09).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[DIRECT - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/binaryDirt.html binaryDIRT]<br /> | Entailment rules between binary templates using DIRT algorithm<br /> | Bar-Ilan University<br /> | This resource contains entailment rules over binary templates learned over the Reuters corpus using<br /> the DIRT algorithm of Lin and Pantel.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[BinaryDIRT- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/unaryBInc.html unaryBInc]<br /> | Entailment rules between unary templates using BInc algorithm<br /> | Bar-Ilan University<br /> | This resource contains entailment rules over unary templates learned over the Reuters corpus using<br /> the BInc algorithm of Szpektor and Dagan (2008).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[unaryBInc- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''New resource''<br /> | <br /> | <br /> | ''Participants are encouraged to contribute''<br /> | style="text-align: center;"| <br /> | style="text-align: center;"| <br /> | style="text-align: center;"| <br /> |style="text-align: center;"| <br /> | [[New Resource2 - RTE Users|Users]]<br /> <br /> |}<br /> <br><br /> <br><br /> <br /> === Not available Resources ===<br /> The following table lists the unpublished resources used by RTE participants. Some of them have been developed by Users themselves specifically for RTE. Interested people may turn to authors to obtain further information.<br /> <br><br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! width="80"|Resource<br /> ! width="50"|Type<br /> ! width="180"|Author<br /> ! class="unsortable" width="500"|Brief description<br /> ! width="30"|<small>PAST Users</small><br /> ! width="30"|<small>RTE4 Users</small><br /> ! width="30"|<small>RTE5 Users</small><br /> ! width="30"|<small>RTE6 Users</small><br /> ! class="unsortable" width="30"|<small>Usage info</small><br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.parc.com/ PARC] Polarity Lexicon<br /> | Lexical DB<br /> | PARC - Palo Alto Research Center<br /> | Verbs classification with respect to semantic polarity<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | [[Parc Polarity Lexicon - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Gazetteer from [http://trec.nist.gov/ TREC]<br /> | Gazetteer<br /> | NIST - National Institute of Standards and Technology<br /> | Cities and other geographical names<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | [[Gazetteer from TREC - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DFKI Geographic Ontology<br>''(to be released)''<br /> | Ontology<br /> | DFKI - German Research Center for Artificial Intelligence<br /> | Ontology containing geographic terms and two kinds of relations: the directional ''part-of'' relation, and the ''equal'' relation for synonyms and abbreviations of the same geographic area (e.g ''the United Kingdom'', ''the UK'', ''Great Britain'', etc.) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | [[Geographic Ontology - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Geo''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | Meronymy entailment rules, based on TREC’s TIPSTER gazetteer.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | [[Geo - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Regex''<br /> | Collection of Entailment rules <br /> | Bar-Ilan University; Tel-Aviv University<br /> | Small set of entailment rules based on regular expressions, intended to address lexical variability involving temporal phrases<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | [[Regex - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Syntactic rule base <br>''(to be released)''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | A manually-composed collection of entailment rules which define parse tree transformations. The rules cover generic syntactic phenomena such as appositions, conjunctions, passive, relative clause, etc. (Bar-Haim et al., AAAI-07) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | [[Syntactic rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Polarity rule base <br>''(to be released)''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | A manually-composed collection of entailment rules which detect predicates whose polarity is negative (e.g. didn't dance) or unknown (e.g. plans to dance). The rules capture diverse phenomena that affect polarity, e.g. verbal negation, modal verbs, conditionals, and certain verbs that induce negative or "unknown" polarity context. The latter were taken mainly from VerbNet. Extends a resource described in (Bar-Haim et al., AAAI-07) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0 <br /> | style="text-align: center;"|0<br /> | [[Polarity rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Lexical-Syntactic rule base<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | Extract lexical-syntactic entailment rules for predicates (verbal and nominal), including argument mapping. The resource is based on WordNet, Nomlex-Plus and Unary DIRT (Szpektor and Dagan, Coling 08)<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | [[Lexical-Syntactic rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | OPENU Collection<br /> | Collection of Entailment Rules and Patterns<br /> | Open University<br /> | Collections of rules, patterns etc. for RTE purpose, extracted from Reuter corpus parsed using Minipar.<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0 <br /> | [[OPENU Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Abbr''<br /> | Collection of rules for abbreviation<br /> | Bar-Ilan University; Tel-Aviv University<br /> | 2000 Abbreviation rules, extracted from [http://badc.nerc.ac.uk/help/abbrevs.html BADC] and [http://www.acronym-guide.com/ Acronym Guide]<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | [[Abbr - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Negation_list<br /> | Negation rules<br /> | „Al. I. Cuza“ University, Iasi, Romania <br /> | List of negative terms and words (verbs, adjectives, nouns) affecting modality or factuality of a infinitive verb preceded by the particle "to" (e.g. "believe","necessary", "attempt")<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1 <br /> | style="text-align: center;"|0<br /> | [[UAIC Negation_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DLSIUAES Negation_list<br /> | List of negative terms<br /> | University of Alicante<br /> | Basic list of negative terms.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0 <br /> | [[DLSIUAES Negation_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Quantifier_list<br /> | List of quantifiers<br /> | „Al. I. Cuza“ University, Iasi, Romania <br /> | List of quantifiers affecting entailment judgment. The quantifiers are taken from a list which contains expressions like “more than”, “less than”, or words such as “over”, “under”, etc.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0 <br /> | [[UAIC Quantifier_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | FBKirst StopWord list<br /> | List of frequent words<br /> | FBK-Irst;<br/>University of Trento - Italy <br /> | A list of the 572 most frequent English words.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0 <br /> | [[FBKirst StopWord list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | IKOMA Dictionary of Named Entities Acronyms and Synonyms<br /> | Dictionary of Named Entities Acronyms and Synonyms<br /> | IKOMA; NEC Corporation, Takayama, Ikoma, Nara, Japan<br /> | Acronym dictionary constructed automatically from the corpus and a synonym dictionary that contains geographical terms.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | [[IKOMA2 - RTE Users|Users]]<br /> <br /> |}<br /> <br><br /> <br /> ==Footnotes==<br /> <references/></div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE_Knowledge_Resources&diff=8672 RTE Knowledge Resources 2011-02-04T13:34:33Z

<p>Amarchetti: /* Not available Resources */</p> <hr /> <div>Knowledge resources have shown their relevance for applied semantic inference, and are extensively used by applied inference systems, such as those developed within the Textual Entailment framework.<br><br /> <br><br /> This page presents a list of the knowledge resources used by systems that have participated in the last RTE challenges. The first table lists the publicly available resources, the second one lists unpublished resources. Both tables are sortable by Resource name, type, author and number of users.<br><br /> <br><br /> RTE Participants are encouraged to add information about all kind of knowledge resources used, from standard existing resources (e.g. WordNet) to knowledge collections created for specific purposes, which can be made available to the community.<br><br /> <br><br /> __TOC__<br /> <br><br /> === Call for Resources ===<br /> <br /> In order to help the research, all the participants are invited to contribute, sharing their own resources with the RTE community. <br /> <br><br /> Making the resources available to be used by other systems has several advantages. On the one hand, it helps improve the TE technology; on the other hand, it offers an opportunity to further test and evaluate the resource.<br /> <br><br /> <br><br /> [[RTE6 - Call for Resources]]<br /> <br><br /> === Ablation Tests ===<br /> An ablation test consists of removing one module at a time from a system, and rerunning the system on the test set with the other modules, except the one tested. <br /> <br><br /> Ablation test are meant to help better understand the relevance of the knowledge resources used by RTE systems, and evaluate the contribution of each of them to the systems' performances. In fact, comparing the results achieved in the ablation tests to those obtained by the systems as a whole allows assessing the contribution given by each single resource.<br /> <br><br /> <br><br /> *[[RTE5 - Ablation Tests]]<br /> *[[RTE6 - Ablation Tests]]<br /> <br><br /> <br /> === Publicly available Resources ===<br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! width="80"|Resource<br /> ! width="50"|Type<br /> ! width="150"|Author<br /> ! class="unsortable" width="300"|Brief description<br /> ! width="45"|<small>PAST Users <ref name:"rtethree">RTE-3 data have been provided by participants by means of a questionnaire.</ref></small><br /> ! width="45"|<small>RTE4 Users<ref name:"rtefour">RTE-4 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! width="45"|<small>RTE5 Users<ref name:"rtefive">RTE-5 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! width="45"|<small>RTE6 Users<ref name:"rtesix">RTE-6 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! class="unsortable" width="30"|<small>Usage info</small><br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[WordNet]]<br /> | Lexical DB<br /> | Princeton University<br /> | Lexical database of English nouns, verbs, adjectives and adverbs<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|21<br /> | style="text-align: center;"|18<br /> |style="text-align: center;"| 22<br /> | [[WordNet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://xwn.hlt.utdallas.edu/index.html eXtended Wordnet]<br /> | Lexical DB<br /> | Human Language Technology Research Institute, University of Texas at Dallas<br /> | Extension of WordNet based on the exploitation of the information contained in WordNet definitional glosses: the glosses are syntactically parsed, transformed into logic forms and content words are semantically disambiguated. The Extended Wordnet is an ongoing project. <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|2<br /> |style="text-align: center;"| 0<br /> | [[eXtended WordNet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://ai.stanford.edu/~rion/swn/ Augmented Wordnet]<br /> | Lexical DB<br /> | Stanford University<br /> | The resource is the result of the application of a learning algorithm for inducing semantic taxonomies from parsed text. The algorithm automatically acquires items of world knowledge, and uses these to produce significantly enhanced versions of WordNet (up to 40,000 synsets more).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Augmented Wordnet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://verbs.colorado.edu/~mpalmer/projects/verbnet.html Verbnet]<br /> | Lexical DB<br /> | University of Colorado Boulder<br /> | Lexicon for English verbs organized into classes extending Levin (1993) classes through refinement and addition of subclasses to achieve syntactic and semantic coherence among members of a class<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Verbnet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[VerbOcean]]<br /> | Lexical DB<br /> | Information Sciences Institute, University of Southern California<br /> | Broad-coverage semantic network of verbs<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|6<br /> |style="text-align: center;"| 7<br /> | [[VerbOcean - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://framenet.icsi.berkeley.edu/ FrameNet]<br /> | Lexical DB<br /> | ICSI (International Computer Science Institute) - Berkley University<br /> | Lexical resource for English words, based on frame semantics (valences) and supported by corpus evidence<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|2<br /> |style="text-align: center;"| 3<br /> | [[Framenet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/meyers/NomBank.html NomBank]<br /> | Lexical DB<br /> | New York University<br /> | Lexical resource containing syntactic frames for nouns, extracted from annotated corpora <br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[NomBank Resource - RTE Users|Users]] <br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://verbs.colorado.edu/~mpalmer/projects/ace.html PropBank]<br /> | Lexical DB<br /> | University of Colorado Boulder <br /> | Lexical resource containing syntactic frames for verbs, extracted from annotated corpora <br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[PropBank Resource - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/nomlex/index.html Nomlex] Plus<br /> | Lexical DB<br /> | New York University<br /> | Dictionary of English nominalizations: it describes the allowed complements for a nominalization and relates the nominal complements to the arguments of the corresponding verb<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Nomlex Plus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.cs.ualberta.ca/~lindek/downloads.htm Dekang Lin’s Thesaurus]<br /> | Thesaurus<br /> | University of Alberta<br /> | Thesaurus automatically constructed using a parsed corpus, based on distributional similarity scores<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 2<br /> | [[Dekang Lin’s Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://icon.shef.ac.uk/Moby/mthes.html Grady Ward's Moby Thesaurus]<br /> | Thesaurus<br /> | University of Sheffield<br /> | Thesaurus containing 30,260 root words, with 2,520,264 synonyms and related terms. Grady Ward placed this thesaurus in the public domain in 1996.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Grady Ward's Moby Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://en.wikipedia.org/wiki/Roget%27s_Thesaurus Roget's Thesaurus]<br /> | Thesaurus<br /> | Peter Mark Roget (Electronic version distributed by University of Chicago)<br /> | Roget's Thesaurus is a widely-used English thesaurus, created by Dr. Peter Mark Roget in 1805. The original edition had 15,000 words, and each new edition has been larger. The electronic edition ([http://machaut.uchicago.edu/rogets version 1.02]) is made available by University of Chicago.<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Roget's Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.wikipedia.org/ Wikipedia]<br /> | Encyclopedia<br /> | <br /> | Free encyclopedia. Used for extraction of lexical-semantic rules (from its more structured parts), named entity recognition, geographical information etc.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|6<br /> |style="text-align: center;"| 6<br /> | [[Wikipedia - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.umbel.org/ Umbel]<br /> | Ontology<br /> | Structured Dynamics LLC, Coralville, IA<br /> | UMBEL stands for Upper Mapping and Binding Exchange Layer and is a lightweight ontology structure for relating Web content and data to a standard set of subject concepts<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Umbel - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.mpi-inf.mpg.de/yago-naga/yago/ YAGO]<br /> | Ontology<br /> | Max-Planck Institute for Informatics, Saarbrücken, Germany<br /> | Light-weight and extensible ontology. It contains more than 2 million entities and 20 million facts about these entities. The facts have been automatically extracted from Wikipedia and unified with WordNet.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[YAGO - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://dbpedia.org/About DBpedia]<br /> | Ontology<br /> | Open community project<br /> | DBpedia is a community effort to extract structured information from Wikipedia and to make this information available on the Web. The DBpedia knowledge base currently describes more than 2.9 million things in 91 different languages and consists of 479 million pieces of information.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[DBpedia - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[DIRT Paraphrase Collection]]<br /> | Collection of paraphrases<br /> | University of Alberta<br /> | DIRT (Discovery of Inference Rules from Text) is both an algorithm and a resulting knowledge collection. The DIRT knowledge collection is the output of the DIRT algorithm over a 1GB set of newspaper text.<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|4<br /> | style="text-align: center;"|3<br /> |style="text-align: center;"| 4<br /> | [[DIRT Paraphrase Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[TEASE]] Collection<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University<br /> | Output of the TEASE algorithm <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Tease Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://badc.nerc.ac.uk/help/abbrevs.html BADC Acronym and Abbreviation List]<br /> | Word List<br /> | BADC (British Atmospheric Data Centre)<br /> | Acronym and Abbreviation List<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[BADC Acronym and Abbreviation List - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://clipdemos.umiacs.umd.edu/catvar/ Catvar The Categorial Variation Database (English)]<br /> | Word List<br /> | University of Maryland<br /> | A Categorial-Variation Database (or Catvar) is a database of clusters of uninflected words (lexemes) and their categorial (i.e. part-of-speech) variants. The database was developed for English using a combination of resources and algorithms, including the LCS Verb and Preposition Databases (Dorr 2001), the Brown Corpus section of the Penn Treebank (Marcus et al. 1994), an English morphological analysis lexicon developed for PC-Kimmo (ENGLEX) (Antworth 1990), WordNet1.6 (Fellbaum 1998), an English Verb-Noun list extracted from Nomlex (Macleod et al. 1998), a similar list extracted from LDOCE (Procter 1978) and the Porter stemmer (Porter 1980).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 1<br /> | [[Categorial-Variation Database - RTE Users|Users]]<br /> <br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.acronym-guide.com/ Acronym Guide]<br /> | Word List<br /> | Acronym-Guide.com <br /> | Acronym and Abbreviation Lists for English, branched in thematic directories <br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|3<br /> |style="text-align: center;"| 0<br /> | [[Acronym Guide - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.ldc.upenn.edu/Catalog/CatalogEntry.jsp?catalogId=LDC2006T13 Web1T 5-grams]<br /> | Word list<br /> | Linguistic Data Consortium, University of Pennsylvania; Google Inc.<br /> | Data set containing English word n-grams and their observed frequency counts. The n-gram counts were generated from approximately 1 trillion word tokens of text from publicly accessible Web pages<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Web1T - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.coli.uni-saarland.de/%7Erwang/resources/RTE3_RTE4_NGD.zip Normalized Google Distance (RTE3&RTE4)]<br /> | Word Pair Co-occurrence<br /> | Saarland University<br /> | Co-occurrence of the word pairs in RTE3 and RTE4 using Normalized Google Distance (Cilibrasi and Vitanyi, 2004). The word pairs are all the possible combinations of content words in T and H. In practice, we used [http://m.www.yahoo.com/ Yahoo!] as the search engine.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Normalized Google Distance (RTE3&RTE4)- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.coli.uni-saarland.de/%7Erwang/resources/RTE5_NGD.zip Normalized Google Distance (RTE5)]<br /> | Word Pair Co-occurrence<br /> | Saarland University<br /> | Co-occurrence of the word pairs in RTE3 and RTE4 using Normalized Google Distance (Cilibrasi and Vitanyi, 2004). The word pairs are all the possible combinations of content words in T and H. In practice, we used [http://m.www.yahoo.com/ Yahoo!] as the search engine.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Normalized Google Distance (RTE5)- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://geonames.usgs.gov/index.html GNIS - Geographic Names Information System]<br /> | Gazetteer<br /> | USGS (United States Geological Survey)<br /> | Database containing the Federal and national standard toponyms for USA, associated areas and Antarctica<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[GNIS - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.geonames.org/ Geonames]<br /> | Gazetteer<br /> | <br /> | Database containing eight million geographical names. It is integrating geographical data such as names of places in various languages, elevation, population and others from various sources.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Geonames - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/paraphrase/ Sekine's Paraphrase Database]<br /> | Collection of paraphrases<br /> | Department of Computer Science, New York University <br /> | Data-base created using Sekine's method, NOT cleaned up by human. It includes 19,975 sets of paraphrases with 191,572 phrases.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Sekine's Paraphrase Database - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://research.microsoft.com/research/downloads/Details/607D14D9-20CD-47E3-85BC-A2F65CD28042/Details.aspx Microsoft Research Paraphrase Corpus]<br /> | Collection of paraphrases<br /> | Microsoft Research<br /> | Text file containing 5800 pairs of sentences which have been extracted from news sources on the web, along with human annotations indicating whether each pair captures a paraphrase/semantic equivalence relationship.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Microsoft Research Paraphrase Corpus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.cs.cornell.edu/~cristian/Without_a_doubt_-_Data.html Downward entailing operators] <br /> | Collection of entailing operators<br /> | Department of Computer Science, Cornell University, Ithaca NY<br /> | System output of an unsupervised algorithm recovering many Downward Entailing operators, like 'doubt'.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Downward entailing operators - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/WikiRules.html WikiRules!]<br /> | Lexical Reference rule-base<br /> | Bar-Ilan University<br /> | Extraction of about 8 million lexical reference rules from the text body (first sentence) and from metadata (links, redirects, parentheses) of Wikipedia. Provides better performance than other automatically constructed resources and comparable performance to WordNet. Offers complementary knowledge to WordNet. <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[WikiRules! - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://www.cs.utexas.edu/users/pclark/dart/ DART]<br /> | Collection of "world knowledge" propositions<br /> | Boeing Research and Technology<br /> | 23 million tuples such as "airplanes can fly to airports", "rivers can flood" collected from abstracted parse trees.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[DART - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://cs.biu.ac.il/~nlp/download/FRED/latest/FRED.zip FRED]<br /> | FrameNet-derived entailment rule-base<br /> | Bar-Ilan University<br /> | This package contains the outputs of the FRED algorithm, an algorithm which extracts entailment rules from FrameNet.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[FRED - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/DIRECT.html DIRECT]<br /> | Directional Distributional Term-Similarity Resource<br /> | Bar-Ilan University<br /> | This is a resource of directional distributional term-similarity rules (mostly lexical entailment rules) automatically extracted using the inclusion relation as described in (Kotlerman et.al., ACL-09).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[DIRECT - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/binaryDirt.html binaryDIRT]<br /> | Entailment rules between binary templates using DIRT algorithm<br /> | Bar-Ilan University<br /> | This resource contains entailment rules over binary templates learned over the Reuters corpus using<br /> the DIRT algorithm of Lin and Pantel.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[BinaryDIRT- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/unaryBInc.html unaryBInc]<br /> | Entailment rules between unary templates using BInc algorithm<br /> | Bar-Ilan University<br /> | This resource contains entailment rules over unary templates learned over the Reuters corpus using<br /> the BInc algorithm of Szpektor and Dagan (2008).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[unaryBInc- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''New resource''<br /> | <br /> | <br /> | ''Participants are encouraged to contribute''<br /> | style="text-align: center;"| <br /> | style="text-align: center;"| <br /> | style="text-align: center;"| <br /> |style="text-align: center;"| <br /> | [[New Resource2 - RTE Users|Users]]<br /> <br /> |}<br /> <br><br /> <br><br /> <br /> === Not available Resources ===<br /> The following table lists the unpublished resources used by RTE participants. Some of them have been developed by Users themselves specifically for RTE. Interested people may turn to authors to obtain further information.<br /> <br><br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! width="80"|Resource<br /> ! width="50"|Type<br /> ! width="180"|Author<br /> ! class="unsortable" width="500"|Brief description<br /> ! width="30"|<small>PAST Users</small><br /> ! width="30"|<small>RTE4 Users</small><br /> ! width="30"|<small>RTE5 Users</small><br /> ! width="30"|<small>RTE6 Users</small><br /> ! class="unsortable" width="30"|<small>Usage info</small><br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.parc.com/ PARC] Polarity Lexicon<br /> | Lexical DB<br /> | PARC - Palo Alto Research Center<br /> | Verbs classification with respect to semantic polarity<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Parc Polarity Lexicon - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Gazetteer from [http://trec.nist.gov/ TREC]<br /> | Gazetteer<br /> | NIST - National Institute of Standards and Technology<br /> | Cities and other geographical names<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Gazetteer from TREC - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DFKI Geographic Ontology<br>''(to be released)''<br /> | Ontology<br /> | DFKI - German Research Center for Artificial Intelligence<br /> | Ontology containing geographic terms and two kinds of relations: the directional ''part-of'' relation, and the ''equal'' relation for synonyms and abbreviations of the same geographic area (e.g ''the United Kingdom'', ''the UK'', ''Great Britain'', etc.) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Geographic Ontology - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Geo''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | Meronymy entailment rules, based on TREC’s TIPSTER gazetteer.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Geo - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Regex''<br /> | Collection of Entailment rules <br /> | Bar-Ilan University; Tel-Aviv University<br /> | Small set of entailment rules based on regular expressions, intended to address lexical variability involving temporal phrases<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Regex - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Syntactic rule base <br>''(to be released)''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | A manually-composed collection of entailment rules which define parse tree transformations. The rules cover generic syntactic phenomena such as appositions, conjunctions, passive, relative clause, etc. (Bar-Haim et al., AAAI-07) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Syntactic rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Polarity rule base <br>''(to be released)''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | A manually-composed collection of entailment rules which detect predicates whose polarity is negative (e.g. didn't dance) or unknown (e.g. plans to dance). The rules capture diverse phenomena that affect polarity, e.g. verbal negation, modal verbs, conditionals, and certain verbs that induce negative or "unknown" polarity context. The latter were taken mainly from VerbNet. Extends a resource described in (Bar-Haim et al., AAAI-07) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0 <br /> | style="text-align: center;"|<br /> | [[Polarity rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Lexical-Syntactic rule base<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | Extract lexical-syntactic entailment rules for predicates (verbal and nominal), including argument mapping. The resource is based on WordNet, Nomlex-Plus and Unary DIRT (Szpektor and Dagan, Coling 08)<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Lexical-Syntactic rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | OPENU Collection<br /> | Collection of Entailment Rules and Patterns<br /> | Open University<br /> | Collections of rules, patterns etc. for RTE purpose, extracted from Reuter corpus parsed using Minipar.<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"| <br /> | [[OPENU Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Abbr''<br /> | Collection of rules for abbreviation<br /> | Bar-Ilan University; Tel-Aviv University<br /> | 2000 Abbreviation rules, extracted from [http://badc.nerc.ac.uk/help/abbrevs.html BADC] and [http://www.acronym-guide.com/ Acronym Guide]<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Abbr - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Negation_list<br /> | Negation rules<br /> | „Al. I. Cuza“ University, Iasi, Romania <br /> | List of negative terms and words (verbs, adjectives, nouns) affecting modality or factuality of a infinitive verb preceded by the particle "to" (e.g. "believe","necessary", "attempt")<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1 <br /> | style="text-align: center;"|<br /> | [[UAIC Negation_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DLSIUAES Negation_list<br /> | List of negative terms<br /> | University of Alicante<br /> | Basic list of negative terms.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"| <br /> | [[DLSIUAES Negation_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Quantifier_list<br /> | List of quantifiers<br /> | „Al. I. Cuza“ University, Iasi, Romania <br /> | List of quantifiers affecting entailment judgment. The quantifiers are taken from a list which contains expressions like “more than”, “less than”, or words such as “over”, “under”, etc.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"| <br /> | [[UAIC Quantifier_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | FBKirst StopWord list<br /> | List of frequent words<br /> | FBK-Irst;<br/>University of Trento - Italy <br /> | A list of the 572 most frequent English words.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"| <br /> | [[FBKirst StopWord list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | IKOMA Dictionary of Named Entities Acronyms and Synonyms<br /> | Dictionary of Named Entities Acronyms and Synonyms<br /> | IKOMA; NEC Corporation, Takayama, Ikoma, Nara, Japan<br /> | Acronym dictionary constructed automatically from the corpus and a synonym dictionary that contains geographical terms.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | [[IKOMA2 - RTE Users|Users]]<br /> <br /> <br /> <br /> |}<br /> <br><br /> <br /> ==Footnotes==<br /> <references/></div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE_Knowledge_Resources&diff=8671 RTE Knowledge Resources 2011-02-04T13:33:27Z

<p>Amarchetti: /* Not available Resources */</p> <hr /> <div>Knowledge resources have shown their relevance for applied semantic inference, and are extensively used by applied inference systems, such as those developed within the Textual Entailment framework.<br><br /> <br><br /> This page presents a list of the knowledge resources used by systems that have participated in the last RTE challenges. The first table lists the publicly available resources, the second one lists unpublished resources. Both tables are sortable by Resource name, type, author and number of users.<br><br /> <br><br /> RTE Participants are encouraged to add information about all kind of knowledge resources used, from standard existing resources (e.g. WordNet) to knowledge collections created for specific purposes, which can be made available to the community.<br><br /> <br><br /> __TOC__<br /> <br><br /> === Call for Resources ===<br /> <br /> In order to help the research, all the participants are invited to contribute, sharing their own resources with the RTE community. <br /> <br><br /> Making the resources available to be used by other systems has several advantages. On the one hand, it helps improve the TE technology; on the other hand, it offers an opportunity to further test and evaluate the resource.<br /> <br><br /> <br><br /> [[RTE6 - Call for Resources]]<br /> <br><br /> === Ablation Tests ===<br /> An ablation test consists of removing one module at a time from a system, and rerunning the system on the test set with the other modules, except the one tested. <br /> <br><br /> Ablation test are meant to help better understand the relevance of the knowledge resources used by RTE systems, and evaluate the contribution of each of them to the systems' performances. In fact, comparing the results achieved in the ablation tests to those obtained by the systems as a whole allows assessing the contribution given by each single resource.<br /> <br><br /> <br><br /> *[[RTE5 - Ablation Tests]]<br /> *[[RTE6 - Ablation Tests]]<br /> <br><br /> <br /> === Publicly available Resources ===<br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! width="80"|Resource<br /> ! width="50"|Type<br /> ! width="150"|Author<br /> ! class="unsortable" width="300"|Brief description<br /> ! width="45"|<small>PAST Users <ref name:"rtethree">RTE-3 data have been provided by participants by means of a questionnaire.</ref></small><br /> ! width="45"|<small>RTE4 Users<ref name:"rtefour">RTE-4 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! width="45"|<small>RTE5 Users<ref name:"rtefive">RTE-5 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! width="45"|<small>RTE6 Users<ref name:"rtesix">RTE-6 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! class="unsortable" width="30"|<small>Usage info</small><br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[WordNet]]<br /> | Lexical DB<br /> | Princeton University<br /> | Lexical database of English nouns, verbs, adjectives and adverbs<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|21<br /> | style="text-align: center;"|18<br /> |style="text-align: center;"| 22<br /> | [[WordNet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://xwn.hlt.utdallas.edu/index.html eXtended Wordnet]<br /> | Lexical DB<br /> | Human Language Technology Research Institute, University of Texas at Dallas<br /> | Extension of WordNet based on the exploitation of the information contained in WordNet definitional glosses: the glosses are syntactically parsed, transformed into logic forms and content words are semantically disambiguated. The Extended Wordnet is an ongoing project. <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|2<br /> |style="text-align: center;"| 0<br /> | [[eXtended WordNet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://ai.stanford.edu/~rion/swn/ Augmented Wordnet]<br /> | Lexical DB<br /> | Stanford University<br /> | The resource is the result of the application of a learning algorithm for inducing semantic taxonomies from parsed text. The algorithm automatically acquires items of world knowledge, and uses these to produce significantly enhanced versions of WordNet (up to 40,000 synsets more).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Augmented Wordnet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://verbs.colorado.edu/~mpalmer/projects/verbnet.html Verbnet]<br /> | Lexical DB<br /> | University of Colorado Boulder<br /> | Lexicon for English verbs organized into classes extending Levin (1993) classes through refinement and addition of subclasses to achieve syntactic and semantic coherence among members of a class<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Verbnet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[VerbOcean]]<br /> | Lexical DB<br /> | Information Sciences Institute, University of Southern California<br /> | Broad-coverage semantic network of verbs<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|6<br /> |style="text-align: center;"| 7<br /> | [[VerbOcean - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://framenet.icsi.berkeley.edu/ FrameNet]<br /> | Lexical DB<br /> | ICSI (International Computer Science Institute) - Berkley University<br /> | Lexical resource for English words, based on frame semantics (valences) and supported by corpus evidence<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|2<br /> |style="text-align: center;"| 3<br /> | [[Framenet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/meyers/NomBank.html NomBank]<br /> | Lexical DB<br /> | New York University<br /> | Lexical resource containing syntactic frames for nouns, extracted from annotated corpora <br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[NomBank Resource - RTE Users|Users]] <br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://verbs.colorado.edu/~mpalmer/projects/ace.html PropBank]<br /> | Lexical DB<br /> | University of Colorado Boulder <br /> | Lexical resource containing syntactic frames for verbs, extracted from annotated corpora <br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[PropBank Resource - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/nomlex/index.html Nomlex] Plus<br /> | Lexical DB<br /> | New York University<br /> | Dictionary of English nominalizations: it describes the allowed complements for a nominalization and relates the nominal complements to the arguments of the corresponding verb<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Nomlex Plus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.cs.ualberta.ca/~lindek/downloads.htm Dekang Lin’s Thesaurus]<br /> | Thesaurus<br /> | University of Alberta<br /> | Thesaurus automatically constructed using a parsed corpus, based on distributional similarity scores<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 2<br /> | [[Dekang Lin’s Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://icon.shef.ac.uk/Moby/mthes.html Grady Ward's Moby Thesaurus]<br /> | Thesaurus<br /> | University of Sheffield<br /> | Thesaurus containing 30,260 root words, with 2,520,264 synonyms and related terms. Grady Ward placed this thesaurus in the public domain in 1996.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Grady Ward's Moby Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://en.wikipedia.org/wiki/Roget%27s_Thesaurus Roget's Thesaurus]<br /> | Thesaurus<br /> | Peter Mark Roget (Electronic version distributed by University of Chicago)<br /> | Roget's Thesaurus is a widely-used English thesaurus, created by Dr. Peter Mark Roget in 1805. The original edition had 15,000 words, and each new edition has been larger. The electronic edition ([http://machaut.uchicago.edu/rogets version 1.02]) is made available by University of Chicago.<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Roget's Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.wikipedia.org/ Wikipedia]<br /> | Encyclopedia<br /> | <br /> | Free encyclopedia. Used for extraction of lexical-semantic rules (from its more structured parts), named entity recognition, geographical information etc.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|6<br /> |style="text-align: center;"| 6<br /> | [[Wikipedia - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.umbel.org/ Umbel]<br /> | Ontology<br /> | Structured Dynamics LLC, Coralville, IA<br /> | UMBEL stands for Upper Mapping and Binding Exchange Layer and is a lightweight ontology structure for relating Web content and data to a standard set of subject concepts<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Umbel - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.mpi-inf.mpg.de/yago-naga/yago/ YAGO]<br /> | Ontology<br /> | Max-Planck Institute for Informatics, Saarbrücken, Germany<br /> | Light-weight and extensible ontology. It contains more than 2 million entities and 20 million facts about these entities. The facts have been automatically extracted from Wikipedia and unified with WordNet.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[YAGO - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://dbpedia.org/About DBpedia]<br /> | Ontology<br /> | Open community project<br /> | DBpedia is a community effort to extract structured information from Wikipedia and to make this information available on the Web. The DBpedia knowledge base currently describes more than 2.9 million things in 91 different languages and consists of 479 million pieces of information.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[DBpedia - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[DIRT Paraphrase Collection]]<br /> | Collection of paraphrases<br /> | University of Alberta<br /> | DIRT (Discovery of Inference Rules from Text) is both an algorithm and a resulting knowledge collection. The DIRT knowledge collection is the output of the DIRT algorithm over a 1GB set of newspaper text.<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|4<br /> | style="text-align: center;"|3<br /> |style="text-align: center;"| 4<br /> | [[DIRT Paraphrase Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[TEASE]] Collection<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University<br /> | Output of the TEASE algorithm <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Tease Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://badc.nerc.ac.uk/help/abbrevs.html BADC Acronym and Abbreviation List]<br /> | Word List<br /> | BADC (British Atmospheric Data Centre)<br /> | Acronym and Abbreviation List<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[BADC Acronym and Abbreviation List - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://clipdemos.umiacs.umd.edu/catvar/ Catvar The Categorial Variation Database (English)]<br /> | Word List<br /> | University of Maryland<br /> | A Categorial-Variation Database (or Catvar) is a database of clusters of uninflected words (lexemes) and their categorial (i.e. part-of-speech) variants. The database was developed for English using a combination of resources and algorithms, including the LCS Verb and Preposition Databases (Dorr 2001), the Brown Corpus section of the Penn Treebank (Marcus et al. 1994), an English morphological analysis lexicon developed for PC-Kimmo (ENGLEX) (Antworth 1990), WordNet1.6 (Fellbaum 1998), an English Verb-Noun list extracted from Nomlex (Macleod et al. 1998), a similar list extracted from LDOCE (Procter 1978) and the Porter stemmer (Porter 1980).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 1<br /> | [[Categorial-Variation Database - RTE Users|Users]]<br /> <br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.acronym-guide.com/ Acronym Guide]<br /> | Word List<br /> | Acronym-Guide.com <br /> | Acronym and Abbreviation Lists for English, branched in thematic directories <br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|3<br /> |style="text-align: center;"| 0<br /> | [[Acronym Guide - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.ldc.upenn.edu/Catalog/CatalogEntry.jsp?catalogId=LDC2006T13 Web1T 5-grams]<br /> | Word list<br /> | Linguistic Data Consortium, University of Pennsylvania; Google Inc.<br /> | Data set containing English word n-grams and their observed frequency counts. The n-gram counts were generated from approximately 1 trillion word tokens of text from publicly accessible Web pages<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Web1T - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.coli.uni-saarland.de/%7Erwang/resources/RTE3_RTE4_NGD.zip Normalized Google Distance (RTE3&RTE4)]<br /> | Word Pair Co-occurrence<br /> | Saarland University<br /> | Co-occurrence of the word pairs in RTE3 and RTE4 using Normalized Google Distance (Cilibrasi and Vitanyi, 2004). The word pairs are all the possible combinations of content words in T and H. In practice, we used [http://m.www.yahoo.com/ Yahoo!] as the search engine.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Normalized Google Distance (RTE3&RTE4)- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.coli.uni-saarland.de/%7Erwang/resources/RTE5_NGD.zip Normalized Google Distance (RTE5)]<br /> | Word Pair Co-occurrence<br /> | Saarland University<br /> | Co-occurrence of the word pairs in RTE3 and RTE4 using Normalized Google Distance (Cilibrasi and Vitanyi, 2004). The word pairs are all the possible combinations of content words in T and H. In practice, we used [http://m.www.yahoo.com/ Yahoo!] as the search engine.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Normalized Google Distance (RTE5)- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://geonames.usgs.gov/index.html GNIS - Geographic Names Information System]<br /> | Gazetteer<br /> | USGS (United States Geological Survey)<br /> | Database containing the Federal and national standard toponyms for USA, associated areas and Antarctica<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[GNIS - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.geonames.org/ Geonames]<br /> | Gazetteer<br /> | <br /> | Database containing eight million geographical names. It is integrating geographical data such as names of places in various languages, elevation, population and others from various sources.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Geonames - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/paraphrase/ Sekine's Paraphrase Database]<br /> | Collection of paraphrases<br /> | Department of Computer Science, New York University <br /> | Data-base created using Sekine's method, NOT cleaned up by human. It includes 19,975 sets of paraphrases with 191,572 phrases.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Sekine's Paraphrase Database - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://research.microsoft.com/research/downloads/Details/607D14D9-20CD-47E3-85BC-A2F65CD28042/Details.aspx Microsoft Research Paraphrase Corpus]<br /> | Collection of paraphrases<br /> | Microsoft Research<br /> | Text file containing 5800 pairs of sentences which have been extracted from news sources on the web, along with human annotations indicating whether each pair captures a paraphrase/semantic equivalence relationship.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Microsoft Research Paraphrase Corpus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.cs.cornell.edu/~cristian/Without_a_doubt_-_Data.html Downward entailing operators] <br /> | Collection of entailing operators<br /> | Department of Computer Science, Cornell University, Ithaca NY<br /> | System output of an unsupervised algorithm recovering many Downward Entailing operators, like 'doubt'.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Downward entailing operators - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/WikiRules.html WikiRules!]<br /> | Lexical Reference rule-base<br /> | Bar-Ilan University<br /> | Extraction of about 8 million lexical reference rules from the text body (first sentence) and from metadata (links, redirects, parentheses) of Wikipedia. Provides better performance than other automatically constructed resources and comparable performance to WordNet. Offers complementary knowledge to WordNet. <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[WikiRules! - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://www.cs.utexas.edu/users/pclark/dart/ DART]<br /> | Collection of "world knowledge" propositions<br /> | Boeing Research and Technology<br /> | 23 million tuples such as "airplanes can fly to airports", "rivers can flood" collected from abstracted parse trees.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[DART - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://cs.biu.ac.il/~nlp/download/FRED/latest/FRED.zip FRED]<br /> | FrameNet-derived entailment rule-base<br /> | Bar-Ilan University<br /> | This package contains the outputs of the FRED algorithm, an algorithm which extracts entailment rules from FrameNet.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[FRED - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/DIRECT.html DIRECT]<br /> | Directional Distributional Term-Similarity Resource<br /> | Bar-Ilan University<br /> | This is a resource of directional distributional term-similarity rules (mostly lexical entailment rules) automatically extracted using the inclusion relation as described in (Kotlerman et.al., ACL-09).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[DIRECT - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/binaryDirt.html binaryDIRT]<br /> | Entailment rules between binary templates using DIRT algorithm<br /> | Bar-Ilan University<br /> | This resource contains entailment rules over binary templates learned over the Reuters corpus using<br /> the DIRT algorithm of Lin and Pantel.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[BinaryDIRT- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/unaryBInc.html unaryBInc]<br /> | Entailment rules between unary templates using BInc algorithm<br /> | Bar-Ilan University<br /> | This resource contains entailment rules over unary templates learned over the Reuters corpus using<br /> the BInc algorithm of Szpektor and Dagan (2008).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[unaryBInc- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''New resource''<br /> | <br /> | <br /> | ''Participants are encouraged to contribute''<br /> | style="text-align: center;"| <br /> | style="text-align: center;"| <br /> | style="text-align: center;"| <br /> |style="text-align: center;"| <br /> | [[New Resource2 - RTE Users|Users]]<br /> <br /> |}<br /> <br><br /> <br><br /> <br /> === Not available Resources ===<br /> The following table lists the unpublished resources used by RTE participants. Some of them have been developed by Users themselves specifically for RTE. Interested people may turn to authors to obtain further information.<br /> <br><br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! width="80"|Resource<br /> ! width="50"|Type<br /> ! width="180"|Author<br /> ! class="unsortable" width="500"|Brief description<br /> ! width="30"|<small>PAST Users</small><br /> ! width="30"|<small>RTE4 Users</small><br /> ! width="30"|<small>RTE5 Users</small><br /> ! width="30"|<small>RTE6 Users</small><br /> ! class="unsortable" width="30"|<small>Usage info</small><br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.parc.com/ PARC] Polarity Lexicon<br /> | Lexical DB<br /> | PARC - Palo Alto Research Center<br /> | Verbs classification with respect to semantic polarity<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Parc Polarity Lexicon - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Gazetteer from [http://trec.nist.gov/ TREC]<br /> | Gazetteer<br /> | NIST - National Institute of Standards and Technology<br /> | Cities and other geographical names<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Gazetteer from TREC - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DFKI Geographic Ontology<br>''(to be released)''<br /> | Ontology<br /> | DFKI - German Research Center for Artificial Intelligence<br /> | Ontology containing geographic terms and two kinds of relations: the directional ''part-of'' relation, and the ''equal'' relation for synonyms and abbreviations of the same geographic area (e.g ''the United Kingdom'', ''the UK'', ''Great Britain'', etc.) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Geographic Ontology - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Geo''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | Meronymy entailment rules, based on TREC’s TIPSTER gazetteer.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Geo - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Regex''<br /> | Collection of Entailment rules <br /> | Bar-Ilan University; Tel-Aviv University<br /> | Small set of entailment rules based on regular expressions, intended to address lexical variability involving temporal phrases<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Regex - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Syntactic rule base <br>''(to be released)''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | A manually-composed collection of entailment rules which define parse tree transformations. The rules cover generic syntactic phenomena such as appositions, conjunctions, passive, relative clause, etc. (Bar-Haim et al., AAAI-07) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Syntactic rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Polarity rule base <br>''(to be released)''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | A manually-composed collection of entailment rules which detect predicates whose polarity is negative (e.g. didn't dance) or unknown (e.g. plans to dance). The rules capture diverse phenomena that affect polarity, e.g. verbal negation, modal verbs, conditionals, and certain verbs that induce negative or "unknown" polarity context. The latter were taken mainly from VerbNet. Extends a resource described in (Bar-Haim et al., AAAI-07) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0 <br /> | style="text-align: center;"|<br /> | [[Polarity rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Lexical-Syntactic rule base<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | Extract lexical-syntactic entailment rules for predicates (verbal and nominal), including argument mapping. The resource is based on WordNet, Nomlex-Plus and Unary DIRT (Szpektor and Dagan, Coling 08)<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Lexical-Syntactic rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | OPENU Collection<br /> | Collection of Entailment Rules and Patterns<br /> | Open University<br /> | Collections of rules, patterns etc. for RTE purpose, extracted from Reuter corpus parsed using Minipar.<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"| <br /> | [[OPENU Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Abbr''<br /> | Collection of rules for abbreviation<br /> | Bar-Ilan University; Tel-Aviv University<br /> | 2000 Abbreviation rules, extracted from [http://badc.nerc.ac.uk/help/abbrevs.html BADC] and [http://www.acronym-guide.com/ Acronym Guide]<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Abbr - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Negation_list<br /> | Negation rules<br /> | „Al. I. Cuza“ University, Iasi, Romania <br /> | List of negative terms and words (verbs, adjectives, nouns) affecting modality or factuality of a infinitive verb preceded by the particle "to" (e.g. "believe","necessary", "attempt")<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1 <br /> | style="text-align: center;"|<br /> | [[UAIC Negation_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DLSIUAES Negation_list<br /> | List of negative terms<br /> | University of Alicante<br /> | Basic list of negative terms.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"| <br /> | [[DLSIUAES Negation_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Quantifier_list<br /> | List of quantifiers<br /> | „Al. I. Cuza“ University, Iasi, Romania <br /> | List of quantifiers affecting entailment judgment. The quantifiers are taken from a list which contains expressions like “more than”, “less than”, or words such as “over”, “under”, etc.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"| <br /> | [[UAIC Quantifier_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | FBKirst StopWord list<br /> | List of frequent words<br /> | FBK-Irst;<br/>University of Trento - Italy <br /> | A list of the 572 most frequent English words.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"| <br /> | [[FBKirst StopWord list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | IKOMA Dictionary of Named Entities Acronyms and Synonyms<br /> | Dictionary of Named Entities Acronyms and Synonyms<br /> | IKOMA; NEC Corporation, Takayama, Ikoma, Nara, Japan<br /> | Acronym dictionary constructed automatically from the corpus and a synonym dictionary that contains geographical terms.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"| <br /> | [[IKOMA2 - RTE Users|Users]]<br /> <br /> <br /> <br /> |}<br /> <br><br /> <br /> ==Footnotes==<br /> <references/></div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE_Knowledge_Resources&diff=8670 RTE Knowledge Resources 2011-02-04T10:40:33Z

<p>Amarchetti: /* Not available Resources */</p> <hr /> <div>Knowledge resources have shown their relevance for applied semantic inference, and are extensively used by applied inference systems, such as those developed within the Textual Entailment framework.<br><br /> <br><br /> This page presents a list of the knowledge resources used by systems that have participated in the last RTE challenges. The first table lists the publicly available resources, the second one lists unpublished resources. Both tables are sortable by Resource name, type, author and number of users.<br><br /> <br><br /> RTE Participants are encouraged to add information about all kind of knowledge resources used, from standard existing resources (e.g. WordNet) to knowledge collections created for specific purposes, which can be made available to the community.<br><br /> <br><br /> __TOC__<br /> <br><br /> === Call for Resources ===<br /> <br /> In order to help the research, all the participants are invited to contribute, sharing their own resources with the RTE community. <br /> <br><br /> Making the resources available to be used by other systems has several advantages. On the one hand, it helps improve the TE technology; on the other hand, it offers an opportunity to further test and evaluate the resource.<br /> <br><br /> <br><br /> [[RTE6 - Call for Resources]]<br /> <br><br /> === Ablation Tests ===<br /> An ablation test consists of removing one module at a time from a system, and rerunning the system on the test set with the other modules, except the one tested. <br /> <br><br /> Ablation test are meant to help better understand the relevance of the knowledge resources used by RTE systems, and evaluate the contribution of each of them to the systems' performances. In fact, comparing the results achieved in the ablation tests to those obtained by the systems as a whole allows assessing the contribution given by each single resource.<br /> <br><br /> <br><br /> *[[RTE5 - Ablation Tests]]<br /> *[[RTE6 - Ablation Tests]]<br /> <br><br /> <br /> === Publicly available Resources ===<br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! width="80"|Resource<br /> ! width="50"|Type<br /> ! width="150"|Author<br /> ! class="unsortable" width="300"|Brief description<br /> ! width="45"|<small>PAST Users <ref name:"rtethree">RTE-3 data have been provided by participants by means of a questionnaire.</ref></small><br /> ! width="45"|<small>RTE4 Users<ref name:"rtefour">RTE-4 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! width="45"|<small>RTE5 Users<ref name:"rtefive">RTE-5 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! width="45"|<small>RTE6 Users<ref name:"rtesix">RTE-6 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! class="unsortable" width="30"|<small>Usage info</small><br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[WordNet]]<br /> | Lexical DB<br /> | Princeton University<br /> | Lexical database of English nouns, verbs, adjectives and adverbs<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|21<br /> | style="text-align: center;"|18<br /> |style="text-align: center;"| 22<br /> | [[WordNet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://xwn.hlt.utdallas.edu/index.html eXtended Wordnet]<br /> | Lexical DB<br /> | Human Language Technology Research Institute, University of Texas at Dallas<br /> | Extension of WordNet based on the exploitation of the information contained in WordNet definitional glosses: the glosses are syntactically parsed, transformed into logic forms and content words are semantically disambiguated. The Extended Wordnet is an ongoing project. <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|2<br /> |style="text-align: center;"| 0<br /> | [[eXtended WordNet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://ai.stanford.edu/~rion/swn/ Augmented Wordnet]<br /> | Lexical DB<br /> | Stanford University<br /> | The resource is the result of the application of a learning algorithm for inducing semantic taxonomies from parsed text. The algorithm automatically acquires items of world knowledge, and uses these to produce significantly enhanced versions of WordNet (up to 40,000 synsets more).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Augmented Wordnet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://verbs.colorado.edu/~mpalmer/projects/verbnet.html Verbnet]<br /> | Lexical DB<br /> | University of Colorado Boulder<br /> | Lexicon for English verbs organized into classes extending Levin (1993) classes through refinement and addition of subclasses to achieve syntactic and semantic coherence among members of a class<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Verbnet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[VerbOcean]]<br /> | Lexical DB<br /> | Information Sciences Institute, University of Southern California<br /> | Broad-coverage semantic network of verbs<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|6<br /> |style="text-align: center;"| 7<br /> | [[VerbOcean - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://framenet.icsi.berkeley.edu/ FrameNet]<br /> | Lexical DB<br /> | ICSI (International Computer Science Institute) - Berkley University<br /> | Lexical resource for English words, based on frame semantics (valences) and supported by corpus evidence<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|2<br /> |style="text-align: center;"| 3<br /> | [[Framenet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/meyers/NomBank.html NomBank]<br /> | Lexical DB<br /> | New York University<br /> | Lexical resource containing syntactic frames for nouns, extracted from annotated corpora <br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[NomBank Resource - RTE Users|Users]] <br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://verbs.colorado.edu/~mpalmer/projects/ace.html PropBank]<br /> | Lexical DB<br /> | University of Colorado Boulder <br /> | Lexical resource containing syntactic frames for verbs, extracted from annotated corpora <br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[PropBank Resource - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/nomlex/index.html Nomlex] Plus<br /> | Lexical DB<br /> | New York University<br /> | Dictionary of English nominalizations: it describes the allowed complements for a nominalization and relates the nominal complements to the arguments of the corresponding verb<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Nomlex Plus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.cs.ualberta.ca/~lindek/downloads.htm Dekang Lin’s Thesaurus]<br /> | Thesaurus<br /> | University of Alberta<br /> | Thesaurus automatically constructed using a parsed corpus, based on distributional similarity scores<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 2<br /> | [[Dekang Lin’s Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://icon.shef.ac.uk/Moby/mthes.html Grady Ward's Moby Thesaurus]<br /> | Thesaurus<br /> | University of Sheffield<br /> | Thesaurus containing 30,260 root words, with 2,520,264 synonyms and related terms. Grady Ward placed this thesaurus in the public domain in 1996.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Grady Ward's Moby Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://en.wikipedia.org/wiki/Roget%27s_Thesaurus Roget's Thesaurus]<br /> | Thesaurus<br /> | Peter Mark Roget (Electronic version distributed by University of Chicago)<br /> | Roget's Thesaurus is a widely-used English thesaurus, created by Dr. Peter Mark Roget in 1805. The original edition had 15,000 words, and each new edition has been larger. The electronic edition ([http://machaut.uchicago.edu/rogets version 1.02]) is made available by University of Chicago.<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Roget's Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.wikipedia.org/ Wikipedia]<br /> | Encyclopedia<br /> | <br /> | Free encyclopedia. Used for extraction of lexical-semantic rules (from its more structured parts), named entity recognition, geographical information etc.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|6<br /> |style="text-align: center;"| 6<br /> | [[Wikipedia - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.umbel.org/ Umbel]<br /> | Ontology<br /> | Structured Dynamics LLC, Coralville, IA<br /> | UMBEL stands for Upper Mapping and Binding Exchange Layer and is a lightweight ontology structure for relating Web content and data to a standard set of subject concepts<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Umbel - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.mpi-inf.mpg.de/yago-naga/yago/ YAGO]<br /> | Ontology<br /> | Max-Planck Institute for Informatics, Saarbrücken, Germany<br /> | Light-weight and extensible ontology. It contains more than 2 million entities and 20 million facts about these entities. The facts have been automatically extracted from Wikipedia and unified with WordNet.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[YAGO - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://dbpedia.org/About DBpedia]<br /> | Ontology<br /> | Open community project<br /> | DBpedia is a community effort to extract structured information from Wikipedia and to make this information available on the Web. The DBpedia knowledge base currently describes more than 2.9 million things in 91 different languages and consists of 479 million pieces of information.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[DBpedia - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[DIRT Paraphrase Collection]]<br /> | Collection of paraphrases<br /> | University of Alberta<br /> | DIRT (Discovery of Inference Rules from Text) is both an algorithm and a resulting knowledge collection. The DIRT knowledge collection is the output of the DIRT algorithm over a 1GB set of newspaper text.<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|4<br /> | style="text-align: center;"|3<br /> |style="text-align: center;"| 4<br /> | [[DIRT Paraphrase Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[TEASE]] Collection<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University<br /> | Output of the TEASE algorithm <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Tease Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://badc.nerc.ac.uk/help/abbrevs.html BADC Acronym and Abbreviation List]<br /> | Word List<br /> | BADC (British Atmospheric Data Centre)<br /> | Acronym and Abbreviation List<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[BADC Acronym and Abbreviation List - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://clipdemos.umiacs.umd.edu/catvar/ Catvar The Categorial Variation Database (English)]<br /> | Word List<br /> | University of Maryland<br /> | A Categorial-Variation Database (or Catvar) is a database of clusters of uninflected words (lexemes) and their categorial (i.e. part-of-speech) variants. The database was developed for English using a combination of resources and algorithms, including the LCS Verb and Preposition Databases (Dorr 2001), the Brown Corpus section of the Penn Treebank (Marcus et al. 1994), an English morphological analysis lexicon developed for PC-Kimmo (ENGLEX) (Antworth 1990), WordNet1.6 (Fellbaum 1998), an English Verb-Noun list extracted from Nomlex (Macleod et al. 1998), a similar list extracted from LDOCE (Procter 1978) and the Porter stemmer (Porter 1980).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 1<br /> | [[Categorial-Variation Database - RTE Users|Users]]<br /> <br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.acronym-guide.com/ Acronym Guide]<br /> | Word List<br /> | Acronym-Guide.com <br /> | Acronym and Abbreviation Lists for English, branched in thematic directories <br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|3<br /> |style="text-align: center;"| 0<br /> | [[Acronym Guide - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.ldc.upenn.edu/Catalog/CatalogEntry.jsp?catalogId=LDC2006T13 Web1T 5-grams]<br /> | Word list<br /> | Linguistic Data Consortium, University of Pennsylvania; Google Inc.<br /> | Data set containing English word n-grams and their observed frequency counts. The n-gram counts were generated from approximately 1 trillion word tokens of text from publicly accessible Web pages<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Web1T - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.coli.uni-saarland.de/%7Erwang/resources/RTE3_RTE4_NGD.zip Normalized Google Distance (RTE3&RTE4)]<br /> | Word Pair Co-occurrence<br /> | Saarland University<br /> | Co-occurrence of the word pairs in RTE3 and RTE4 using Normalized Google Distance (Cilibrasi and Vitanyi, 2004). The word pairs are all the possible combinations of content words in T and H. In practice, we used [http://m.www.yahoo.com/ Yahoo!] as the search engine.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Normalized Google Distance (RTE3&RTE4)- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.coli.uni-saarland.de/%7Erwang/resources/RTE5_NGD.zip Normalized Google Distance (RTE5)]<br /> | Word Pair Co-occurrence<br /> | Saarland University<br /> | Co-occurrence of the word pairs in RTE3 and RTE4 using Normalized Google Distance (Cilibrasi and Vitanyi, 2004). The word pairs are all the possible combinations of content words in T and H. In practice, we used [http://m.www.yahoo.com/ Yahoo!] as the search engine.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Normalized Google Distance (RTE5)- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://geonames.usgs.gov/index.html GNIS - Geographic Names Information System]<br /> | Gazetteer<br /> | USGS (United States Geological Survey)<br /> | Database containing the Federal and national standard toponyms for USA, associated areas and Antarctica<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[GNIS - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.geonames.org/ Geonames]<br /> | Gazetteer<br /> | <br /> | Database containing eight million geographical names. It is integrating geographical data such as names of places in various languages, elevation, population and others from various sources.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Geonames - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/paraphrase/ Sekine's Paraphrase Database]<br /> | Collection of paraphrases<br /> | Department of Computer Science, New York University <br /> | Data-base created using Sekine's method, NOT cleaned up by human. It includes 19,975 sets of paraphrases with 191,572 phrases.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Sekine's Paraphrase Database - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://research.microsoft.com/research/downloads/Details/607D14D9-20CD-47E3-85BC-A2F65CD28042/Details.aspx Microsoft Research Paraphrase Corpus]<br /> | Collection of paraphrases<br /> | Microsoft Research<br /> | Text file containing 5800 pairs of sentences which have been extracted from news sources on the web, along with human annotations indicating whether each pair captures a paraphrase/semantic equivalence relationship.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Microsoft Research Paraphrase Corpus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.cs.cornell.edu/~cristian/Without_a_doubt_-_Data.html Downward entailing operators] <br /> | Collection of entailing operators<br /> | Department of Computer Science, Cornell University, Ithaca NY<br /> | System output of an unsupervised algorithm recovering many Downward Entailing operators, like 'doubt'.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Downward entailing operators - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/WikiRules.html WikiRules!]<br /> | Lexical Reference rule-base<br /> | Bar-Ilan University<br /> | Extraction of about 8 million lexical reference rules from the text body (first sentence) and from metadata (links, redirects, parentheses) of Wikipedia. Provides better performance than other automatically constructed resources and comparable performance to WordNet. Offers complementary knowledge to WordNet. <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[WikiRules! - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://www.cs.utexas.edu/users/pclark/dart/ DART]<br /> | Collection of "world knowledge" propositions<br /> | Boeing Research and Technology<br /> | 23 million tuples such as "airplanes can fly to airports", "rivers can flood" collected from abstracted parse trees.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[DART - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://cs.biu.ac.il/~nlp/download/FRED/latest/FRED.zip FRED]<br /> | FrameNet-derived entailment rule-base<br /> | Bar-Ilan University<br /> | This package contains the outputs of the FRED algorithm, an algorithm which extracts entailment rules from FrameNet.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[FRED - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/DIRECT.html DIRECT]<br /> | Directional Distributional Term-Similarity Resource<br /> | Bar-Ilan University<br /> | This is a resource of directional distributional term-similarity rules (mostly lexical entailment rules) automatically extracted using the inclusion relation as described in (Kotlerman et.al., ACL-09).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[DIRECT - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/binaryDirt.html binaryDIRT]<br /> | Entailment rules between binary templates using DIRT algorithm<br /> | Bar-Ilan University<br /> | This resource contains entailment rules over binary templates learned over the Reuters corpus using<br /> the DIRT algorithm of Lin and Pantel.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[BinaryDIRT- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/unaryBInc.html unaryBInc]<br /> | Entailment rules between unary templates using BInc algorithm<br /> | Bar-Ilan University<br /> | This resource contains entailment rules over unary templates learned over the Reuters corpus using<br /> the BInc algorithm of Szpektor and Dagan (2008).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[unaryBInc- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''New resource''<br /> | <br /> | <br /> | ''Participants are encouraged to contribute''<br /> | style="text-align: center;"| <br /> | style="text-align: center;"| <br /> | style="text-align: center;"| <br /> |style="text-align: center;"| <br /> | [[New Resource2 - RTE Users|Users]]<br /> <br /> |}<br /> <br><br /> <br><br /> <br /> === Not available Resources ===<br /> The following table lists the unpublished resources used by RTE participants. Some of them have been developed by Users themselves specifically for RTE. Interested people may turn to authors to obtain further information.<br /> <br><br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! width="80"|Resource<br /> ! width="50"|Type<br /> ! width="180"|Author<br /> ! class="unsortable" width="500"|Brief description<br /> ! width="30"|<small>PAST Users</small><br /> ! width="30"|<small>RTE4 Users</small><br /> ! width="30"|<small>RTE5 Users</small><br /> ! width="30"|<small>RTE6 Users</small><br /> ! class="unsortable" width="30"|<small>Usage info</small><br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.parc.com/ PARC] Polarity Lexicon<br /> | Lexical DB<br /> | PARC - Palo Alto Research Center<br /> | Verbs classification with respect to semantic polarity<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Parc Polarity Lexicon - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Gazetteer from [http://trec.nist.gov/ TREC]<br /> | Gazetteer<br /> | NIST - National Institute of Standards and Technology<br /> | Cities and other geographical names<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Gazetteer from TREC - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DFKI Geographic Ontology<br>''(to be released)''<br /> | Ontology<br /> | DFKI - German Research Center for Artificial Intelligence<br /> | Ontology containing geographic terms and two kinds of relations: the directional ''part-of'' relation, and the ''equal'' relation for synonyms and abbreviations of the same geographic area (e.g ''the United Kingdom'', ''the UK'', ''Great Britain'', etc.) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Geographic Ontology - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Geo''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | Meronymy entailment rules, based on TREC’s TIPSTER gazetteer.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Geo - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Regex''<br /> | Collection of Entailment rules <br /> | Bar-Ilan University; Tel-Aviv University<br /> | Small set of entailment rules based on regular expressions, intended to address lexical variability involving temporal phrases<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Regex - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Syntactic rule base <br>''(to be released)''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | A manually-composed collection of entailment rules which define parse tree transformations. The rules cover generic syntactic phenomena such as appositions, conjunctions, passive, relative clause, etc. (Bar-Haim et al., AAAI-07) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Syntactic rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Polarity rule base <br>''(to be released)''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | A manually-composed collection of entailment rules which detect predicates whose polarity is negative (e.g. didn't dance) or unknown (e.g. plans to dance). The rules capture diverse phenomena that affect polarity, e.g. verbal negation, modal verbs, conditionals, and certain verbs that induce negative or "unknown" polarity context. The latter were taken mainly from VerbNet. Extends a resource described in (Bar-Haim et al., AAAI-07) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0 <br /> | style="text-align: center;"|<br /> | [[Polarity rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Lexical-Syntactic rule base<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | Extract lexical-syntactic entailment rules for predicates (verbal and nominal), including argument mapping. The resource is based on WordNet, Nomlex-Plus and Unary DIRT (Szpektor and Dagan, Coling 08)<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Lexical-Syntactic rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | OPENU Collection<br /> | Collection of Entailment Rules and Patterns<br /> | Open University<br /> | Collections of rules, patterns etc. for RTE purpose, extracted from Reuter corpus parsed using Minipar.<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"| <br /> | [[OPENU Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Abbr''<br /> | Collection of rules for abbreviation<br /> | Bar-Ilan University; Tel-Aviv University<br /> | 2000 Abbreviation rules, extracted from [http://badc.nerc.ac.uk/help/abbrevs.html BADC] and [http://www.acronym-guide.com/ Acronym Guide]<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Abbr - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Negation_list<br /> | Negation rules<br /> | „Al. I. Cuza“ University, Iasi, Romania <br /> | List of negative terms and words (verbs, adjectives, nouns) affecting modality or factuality of a infinitive verb preceded by the particle "to" (e.g. "believe","necessary", "attempt")<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1 <br /> | style="text-align: center;"|<br /> | [[UAIC Negation_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DLSIUAES Negation_list<br /> | List of negative terms<br /> | University of Alicante<br /> | Basic list of negative terms.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"| <br /> | [[DLSIUAES Negation_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Quantifier_list<br /> | List of quantifiers<br /> | „Al. I. Cuza“ University, Iasi, Romania <br /> | List of quantifiers affecting entailment judgment. The quantifiers are taken from a list which contains expressions like “more than”, “less than”, or words such as “over”, “under”, etc.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"| <br /> | [[UAIC Quantifier_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | FBKirst StopWord list<br /> | List of frequent words<br /> | FBK-Irst;<br/>University of Trento - Italy <br /> | A list of the 572 most frequent English words.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"| <br /> | [[FBKirst StopWord list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | IKOMA2 Dictionary of Named Entities Acronyms and Synonyms<br /> | Dictionary of Named Entities Acronyms and Synonyms<br /> | IKOMA; NEC Corporation, Takayama, Ikoma, Nara, Japan<br /> | Acronym dictionary constructed automatically from the corpus and a synonym dictionary that contains geographical terms.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"| <br /> | [[IKOMA2 - RTE Users|Users]]<br /> <br /> <br /> <br /> |}<br /> <br><br /> <br /> ==Footnotes==<br /> <references/></div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE_Knowledge_Resources&diff=8669 RTE Knowledge Resources 2011-02-04T10:34:08Z

<p>Amarchetti: /* Publicly available Resources */</p> <hr /> <div>Knowledge resources have shown their relevance for applied semantic inference, and are extensively used by applied inference systems, such as those developed within the Textual Entailment framework.<br><br /> <br><br /> This page presents a list of the knowledge resources used by systems that have participated in the last RTE challenges. The first table lists the publicly available resources, the second one lists unpublished resources. Both tables are sortable by Resource name, type, author and number of users.<br><br /> <br><br /> RTE Participants are encouraged to add information about all kind of knowledge resources used, from standard existing resources (e.g. WordNet) to knowledge collections created for specific purposes, which can be made available to the community.<br><br /> <br><br /> __TOC__<br /> <br><br /> === Call for Resources ===<br /> <br /> In order to help the research, all the participants are invited to contribute, sharing their own resources with the RTE community. <br /> <br><br /> Making the resources available to be used by other systems has several advantages. On the one hand, it helps improve the TE technology; on the other hand, it offers an opportunity to further test and evaluate the resource.<br /> <br><br /> <br><br /> [[RTE6 - Call for Resources]]<br /> <br><br /> === Ablation Tests ===<br /> An ablation test consists of removing one module at a time from a system, and rerunning the system on the test set with the other modules, except the one tested. <br /> <br><br /> Ablation test are meant to help better understand the relevance of the knowledge resources used by RTE systems, and evaluate the contribution of each of them to the systems' performances. In fact, comparing the results achieved in the ablation tests to those obtained by the systems as a whole allows assessing the contribution given by each single resource.<br /> <br><br /> <br><br /> *[[RTE5 - Ablation Tests]]<br /> *[[RTE6 - Ablation Tests]]<br /> <br><br /> <br /> === Publicly available Resources ===<br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! width="80"|Resource<br /> ! width="50"|Type<br /> ! width="150"|Author<br /> ! class="unsortable" width="300"|Brief description<br /> ! width="45"|<small>PAST Users <ref name:"rtethree">RTE-3 data have been provided by participants by means of a questionnaire.</ref></small><br /> ! width="45"|<small>RTE4 Users<ref name:"rtefour">RTE-4 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! width="45"|<small>RTE5 Users<ref name:"rtefive">RTE-5 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! width="45"|<small>RTE6 Users<ref name:"rtesix">RTE-6 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! class="unsortable" width="30"|<small>Usage info</small><br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[WordNet]]<br /> | Lexical DB<br /> | Princeton University<br /> | Lexical database of English nouns, verbs, adjectives and adverbs<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|21<br /> | style="text-align: center;"|18<br /> |style="text-align: center;"| 22<br /> | [[WordNet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://xwn.hlt.utdallas.edu/index.html eXtended Wordnet]<br /> | Lexical DB<br /> | Human Language Technology Research Institute, University of Texas at Dallas<br /> | Extension of WordNet based on the exploitation of the information contained in WordNet definitional glosses: the glosses are syntactically parsed, transformed into logic forms and content words are semantically disambiguated. The Extended Wordnet is an ongoing project. <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|2<br /> |style="text-align: center;"| 0<br /> | [[eXtended WordNet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://ai.stanford.edu/~rion/swn/ Augmented Wordnet]<br /> | Lexical DB<br /> | Stanford University<br /> | The resource is the result of the application of a learning algorithm for inducing semantic taxonomies from parsed text. The algorithm automatically acquires items of world knowledge, and uses these to produce significantly enhanced versions of WordNet (up to 40,000 synsets more).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Augmented Wordnet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://verbs.colorado.edu/~mpalmer/projects/verbnet.html Verbnet]<br /> | Lexical DB<br /> | University of Colorado Boulder<br /> | Lexicon for English verbs organized into classes extending Levin (1993) classes through refinement and addition of subclasses to achieve syntactic and semantic coherence among members of a class<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Verbnet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[VerbOcean]]<br /> | Lexical DB<br /> | Information Sciences Institute, University of Southern California<br /> | Broad-coverage semantic network of verbs<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|6<br /> |style="text-align: center;"| 7<br /> | [[VerbOcean - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://framenet.icsi.berkeley.edu/ FrameNet]<br /> | Lexical DB<br /> | ICSI (International Computer Science Institute) - Berkley University<br /> | Lexical resource for English words, based on frame semantics (valences) and supported by corpus evidence<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|2<br /> |style="text-align: center;"| 3<br /> | [[Framenet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/meyers/NomBank.html NomBank]<br /> | Lexical DB<br /> | New York University<br /> | Lexical resource containing syntactic frames for nouns, extracted from annotated corpora <br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[NomBank Resource - RTE Users|Users]] <br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://verbs.colorado.edu/~mpalmer/projects/ace.html PropBank]<br /> | Lexical DB<br /> | University of Colorado Boulder <br /> | Lexical resource containing syntactic frames for verbs, extracted from annotated corpora <br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[PropBank Resource - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/nomlex/index.html Nomlex] Plus<br /> | Lexical DB<br /> | New York University<br /> | Dictionary of English nominalizations: it describes the allowed complements for a nominalization and relates the nominal complements to the arguments of the corresponding verb<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Nomlex Plus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.cs.ualberta.ca/~lindek/downloads.htm Dekang Lin’s Thesaurus]<br /> | Thesaurus<br /> | University of Alberta<br /> | Thesaurus automatically constructed using a parsed corpus, based on distributional similarity scores<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 2<br /> | [[Dekang Lin’s Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://icon.shef.ac.uk/Moby/mthes.html Grady Ward's Moby Thesaurus]<br /> | Thesaurus<br /> | University of Sheffield<br /> | Thesaurus containing 30,260 root words, with 2,520,264 synonyms and related terms. Grady Ward placed this thesaurus in the public domain in 1996.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Grady Ward's Moby Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://en.wikipedia.org/wiki/Roget%27s_Thesaurus Roget's Thesaurus]<br /> | Thesaurus<br /> | Peter Mark Roget (Electronic version distributed by University of Chicago)<br /> | Roget's Thesaurus is a widely-used English thesaurus, created by Dr. Peter Mark Roget in 1805. The original edition had 15,000 words, and each new edition has been larger. The electronic edition ([http://machaut.uchicago.edu/rogets version 1.02]) is made available by University of Chicago.<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Roget's Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.wikipedia.org/ Wikipedia]<br /> | Encyclopedia<br /> | <br /> | Free encyclopedia. Used for extraction of lexical-semantic rules (from its more structured parts), named entity recognition, geographical information etc.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|6<br /> |style="text-align: center;"| 6<br /> | [[Wikipedia - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.umbel.org/ Umbel]<br /> | Ontology<br /> | Structured Dynamics LLC, Coralville, IA<br /> | UMBEL stands for Upper Mapping and Binding Exchange Layer and is a lightweight ontology structure for relating Web content and data to a standard set of subject concepts<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Umbel - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.mpi-inf.mpg.de/yago-naga/yago/ YAGO]<br /> | Ontology<br /> | Max-Planck Institute for Informatics, Saarbrücken, Germany<br /> | Light-weight and extensible ontology. It contains more than 2 million entities and 20 million facts about these entities. The facts have been automatically extracted from Wikipedia and unified with WordNet.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[YAGO - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://dbpedia.org/About DBpedia]<br /> | Ontology<br /> | Open community project<br /> | DBpedia is a community effort to extract structured information from Wikipedia and to make this information available on the Web. The DBpedia knowledge base currently describes more than 2.9 million things in 91 different languages and consists of 479 million pieces of information.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[DBpedia - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[DIRT Paraphrase Collection]]<br /> | Collection of paraphrases<br /> | University of Alberta<br /> | DIRT (Discovery of Inference Rules from Text) is both an algorithm and a resulting knowledge collection. The DIRT knowledge collection is the output of the DIRT algorithm over a 1GB set of newspaper text.<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|4<br /> | style="text-align: center;"|3<br /> |style="text-align: center;"| 4<br /> | [[DIRT Paraphrase Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[TEASE]] Collection<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University<br /> | Output of the TEASE algorithm <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Tease Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://badc.nerc.ac.uk/help/abbrevs.html BADC Acronym and Abbreviation List]<br /> | Word List<br /> | BADC (British Atmospheric Data Centre)<br /> | Acronym and Abbreviation List<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[BADC Acronym and Abbreviation List - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://clipdemos.umiacs.umd.edu/catvar/ Catvar The Categorial Variation Database (English)]<br /> | Word List<br /> | University of Maryland<br /> | A Categorial-Variation Database (or Catvar) is a database of clusters of uninflected words (lexemes) and their categorial (i.e. part-of-speech) variants. The database was developed for English using a combination of resources and algorithms, including the LCS Verb and Preposition Databases (Dorr 2001), the Brown Corpus section of the Penn Treebank (Marcus et al. 1994), an English morphological analysis lexicon developed for PC-Kimmo (ENGLEX) (Antworth 1990), WordNet1.6 (Fellbaum 1998), an English Verb-Noun list extracted from Nomlex (Macleod et al. 1998), a similar list extracted from LDOCE (Procter 1978) and the Porter stemmer (Porter 1980).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 1<br /> | [[Categorial-Variation Database - RTE Users|Users]]<br /> <br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.acronym-guide.com/ Acronym Guide]<br /> | Word List<br /> | Acronym-Guide.com <br /> | Acronym and Abbreviation Lists for English, branched in thematic directories <br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|3<br /> |style="text-align: center;"| 0<br /> | [[Acronym Guide - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.ldc.upenn.edu/Catalog/CatalogEntry.jsp?catalogId=LDC2006T13 Web1T 5-grams]<br /> | Word list<br /> | Linguistic Data Consortium, University of Pennsylvania; Google Inc.<br /> | Data set containing English word n-grams and their observed frequency counts. The n-gram counts were generated from approximately 1 trillion word tokens of text from publicly accessible Web pages<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Web1T - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.coli.uni-saarland.de/%7Erwang/resources/RTE3_RTE4_NGD.zip Normalized Google Distance (RTE3&RTE4)]<br /> | Word Pair Co-occurrence<br /> | Saarland University<br /> | Co-occurrence of the word pairs in RTE3 and RTE4 using Normalized Google Distance (Cilibrasi and Vitanyi, 2004). The word pairs are all the possible combinations of content words in T and H. In practice, we used [http://m.www.yahoo.com/ Yahoo!] as the search engine.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Normalized Google Distance (RTE3&RTE4)- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.coli.uni-saarland.de/%7Erwang/resources/RTE5_NGD.zip Normalized Google Distance (RTE5)]<br /> | Word Pair Co-occurrence<br /> | Saarland University<br /> | Co-occurrence of the word pairs in RTE3 and RTE4 using Normalized Google Distance (Cilibrasi and Vitanyi, 2004). The word pairs are all the possible combinations of content words in T and H. In practice, we used [http://m.www.yahoo.com/ Yahoo!] as the search engine.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Normalized Google Distance (RTE5)- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://geonames.usgs.gov/index.html GNIS - Geographic Names Information System]<br /> | Gazetteer<br /> | USGS (United States Geological Survey)<br /> | Database containing the Federal and national standard toponyms for USA, associated areas and Antarctica<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[GNIS - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.geonames.org/ Geonames]<br /> | Gazetteer<br /> | <br /> | Database containing eight million geographical names. It is integrating geographical data such as names of places in various languages, elevation, population and others from various sources.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Geonames - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/paraphrase/ Sekine's Paraphrase Database]<br /> | Collection of paraphrases<br /> | Department of Computer Science, New York University <br /> | Data-base created using Sekine's method, NOT cleaned up by human. It includes 19,975 sets of paraphrases with 191,572 phrases.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Sekine's Paraphrase Database - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://research.microsoft.com/research/downloads/Details/607D14D9-20CD-47E3-85BC-A2F65CD28042/Details.aspx Microsoft Research Paraphrase Corpus]<br /> | Collection of paraphrases<br /> | Microsoft Research<br /> | Text file containing 5800 pairs of sentences which have been extracted from news sources on the web, along with human annotations indicating whether each pair captures a paraphrase/semantic equivalence relationship.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Microsoft Research Paraphrase Corpus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.cs.cornell.edu/~cristian/Without_a_doubt_-_Data.html Downward entailing operators] <br /> | Collection of entailing operators<br /> | Department of Computer Science, Cornell University, Ithaca NY<br /> | System output of an unsupervised algorithm recovering many Downward Entailing operators, like 'doubt'.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Downward entailing operators - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/WikiRules.html WikiRules!]<br /> | Lexical Reference rule-base<br /> | Bar-Ilan University<br /> | Extraction of about 8 million lexical reference rules from the text body (first sentence) and from metadata (links, redirects, parentheses) of Wikipedia. Provides better performance than other automatically constructed resources and comparable performance to WordNet. Offers complementary knowledge to WordNet. <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[WikiRules! - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://www.cs.utexas.edu/users/pclark/dart/ DART]<br /> | Collection of "world knowledge" propositions<br /> | Boeing Research and Technology<br /> | 23 million tuples such as "airplanes can fly to airports", "rivers can flood" collected from abstracted parse trees.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[DART - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://cs.biu.ac.il/~nlp/download/FRED/latest/FRED.zip FRED]<br /> | FrameNet-derived entailment rule-base<br /> | Bar-Ilan University<br /> | This package contains the outputs of the FRED algorithm, an algorithm which extracts entailment rules from FrameNet.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[FRED - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/DIRECT.html DIRECT]<br /> | Directional Distributional Term-Similarity Resource<br /> | Bar-Ilan University<br /> | This is a resource of directional distributional term-similarity rules (mostly lexical entailment rules) automatically extracted using the inclusion relation as described in (Kotlerman et.al., ACL-09).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[DIRECT - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/binaryDirt.html binaryDIRT]<br /> | Entailment rules between binary templates using DIRT algorithm<br /> | Bar-Ilan University<br /> | This resource contains entailment rules over binary templates learned over the Reuters corpus using<br /> the DIRT algorithm of Lin and Pantel.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[BinaryDIRT- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/unaryBInc.html unaryBInc]<br /> | Entailment rules between unary templates using BInc algorithm<br /> | Bar-Ilan University<br /> | This resource contains entailment rules over unary templates learned over the Reuters corpus using<br /> the BInc algorithm of Szpektor and Dagan (2008).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[unaryBInc- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''New resource''<br /> | <br /> | <br /> | ''Participants are encouraged to contribute''<br /> | style="text-align: center;"| <br /> | style="text-align: center;"| <br /> | style="text-align: center;"| <br /> |style="text-align: center;"| <br /> | [[New Resource2 - RTE Users|Users]]<br /> <br /> |}<br /> <br><br /> <br><br /> <br /> === Not available Resources ===<br /> The following table lists the unpublished resources used by RTE participants. Some of them have been developed by Users themselves specifically for RTE. Interested people may turn to authors to obtain further information.<br /> <br><br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! width="80"|Resource<br /> ! width="50"|Type<br /> ! width="180"|Author<br /> ! class="unsortable" width="500"|Brief description<br /> ! width="30"|<small>PAST Users</small><br /> ! width="30"|<small>RTE4 Users</small><br /> ! width="30"|<small>RTE5 Users</small><br /> ! width="30"|<small>RTE6 Users</small><br /> ! class="unsortable" width="30"|<small>Usage info</small><br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.parc.com/ PARC] Polarity Lexicon<br /> | Lexical DB<br /> | PARC - Palo Alto Research Center<br /> | Verbs classification with respect to semantic polarity<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Parc Polarity Lexicon - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Gazetteer from [http://trec.nist.gov/ TREC]<br /> | Gazetteer<br /> | NIST - National Institute of Standards and Technology<br /> | Cities and other geographical names<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Gazetteer from TREC - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DFKI Geographic Ontology<br>''(to be released)''<br /> | Ontology<br /> | DFKI - German Research Center for Artificial Intelligence<br /> | Ontology containing geographic terms and two kinds of relations: the directional ''part-of'' relation, and the ''equal'' relation for synonyms and abbreviations of the same geographic area (e.g ''the United Kingdom'', ''the UK'', ''Great Britain'', etc.) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Geographic Ontology - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Geo''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | Meronymy entailment rules, based on TREC’s TIPSTER gazetteer.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Geo - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Regex''<br /> | Collection of Entailment rules <br /> | Bar-Ilan University; Tel-Aviv University<br /> | Small set of entailment rules based on regular expressions, intended to address lexical variability involving temporal phrases<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Regex - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Syntactic rule base <br>''(to be released)''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | A manually-composed collection of entailment rules which define parse tree transformations. The rules cover generic syntactic phenomena such as appositions, conjunctions, passive, relative clause, etc. (Bar-Haim et al., AAAI-07) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Syntactic rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Polarity rule base <br>''(to be released)''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | A manually-composed collection of entailment rules which detect predicates whose polarity is negative (e.g. didn't dance) or unknown (e.g. plans to dance). The rules capture diverse phenomena that affect polarity, e.g. verbal negation, modal verbs, conditionals, and certain verbs that induce negative or "unknown" polarity context. The latter were taken mainly from VerbNet. Extends a resource described in (Bar-Haim et al., AAAI-07) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0 <br /> | style="text-align: center;"|<br /> | [[Polarity rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Lexical-Syntactic rule base<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | Extract lexical-syntactic entailment rules for predicates (verbal and nominal), including argument mapping. The resource is based on WordNet, Nomlex-Plus and Unary DIRT (Szpektor and Dagan, Coling 08)<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Lexical-Syntactic rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | OPENU Collection<br /> | Collection of Entailment Rules and Patterns<br /> | Open University<br /> | Collections of rules, patterns etc. for RTE purpose, extracted from Reuter corpus parsed using Minipar.<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"| <br /> | [[OPENU Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Abbr''<br /> | Collection of rules for abbreviation<br /> | Bar-Ilan University; Tel-Aviv University<br /> | 2000 Abbreviation rules, extracted from [http://badc.nerc.ac.uk/help/abbrevs.html BADC] and [http://www.acronym-guide.com/ Acronym Guide]<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Abbr - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Negation_list<br /> | Negation rules<br /> | „Al. I. Cuza“ University, Iasi, Romania <br /> | List of negative terms and words (verbs, adjectives, nouns) affecting modality or factuality of a infinitive verb preceded by the particle "to" (e.g. "believe","necessary", "attempt")<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1 <br /> | style="text-align: center;"|<br /> | [[UAIC Negation_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DLSIUAES Negation_list<br /> | List of negative terms<br /> | University of Alicante<br /> | Basic list of negative terms.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"| <br /> | [[DLSIUAES Negation_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Quantifier_list<br /> | List of quantifiers<br /> | „Al. I. Cuza“ University, Iasi, Romania <br /> | List of quantifiers affecting entailment judgment. The quantifiers are taken from a list which contains expressions like “more than”, “less than”, or words such as “over”, “under”, etc.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"| <br /> | [[UAIC Quantifier_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | FBKirst StopWord list<br /> | List of frequent words<br /> | FBK-Irst;<br/>University of Trento - Italy <br /> | A list of the 572 most frequent English words.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"| <br /> | [[FBKirst StopWord list - RTE Users|Users]]<br /> <br /> |}<br /> <br><br /> <br /> ==Footnotes==<br /> <references/></div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE_Knowledge_Resources&diff=8668 RTE Knowledge Resources 2011-02-04T10:33:41Z

<p>Amarchetti: /* Publicly available Resources */</p> <hr /> <div>Knowledge resources have shown their relevance for applied semantic inference, and are extensively used by applied inference systems, such as those developed within the Textual Entailment framework.<br><br /> <br><br /> This page presents a list of the knowledge resources used by systems that have participated in the last RTE challenges. The first table lists the publicly available resources, the second one lists unpublished resources. Both tables are sortable by Resource name, type, author and number of users.<br><br /> <br><br /> RTE Participants are encouraged to add information about all kind of knowledge resources used, from standard existing resources (e.g. WordNet) to knowledge collections created for specific purposes, which can be made available to the community.<br><br /> <br><br /> __TOC__<br /> <br><br /> === Call for Resources ===<br /> <br /> In order to help the research, all the participants are invited to contribute, sharing their own resources with the RTE community. <br /> <br><br /> Making the resources available to be used by other systems has several advantages. On the one hand, it helps improve the TE technology; on the other hand, it offers an opportunity to further test and evaluate the resource.<br /> <br><br /> <br><br /> [[RTE6 - Call for Resources]]<br /> <br><br /> === Ablation Tests ===<br /> An ablation test consists of removing one module at a time from a system, and rerunning the system on the test set with the other modules, except the one tested. <br /> <br><br /> Ablation test are meant to help better understand the relevance of the knowledge resources used by RTE systems, and evaluate the contribution of each of them to the systems' performances. In fact, comparing the results achieved in the ablation tests to those obtained by the systems as a whole allows assessing the contribution given by each single resource.<br /> <br><br /> <br><br /> *[[RTE5 - Ablation Tests]]<br /> *[[RTE6 - Ablation Tests]]<br /> <br><br /> <br /> === Publicly available Resources ===<br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! width="80"|Resource<br /> ! width="50"|Type<br /> ! width="150"|Author<br /> ! class="unsortable" width="300"|Brief description<br /> ! width="45"|<small>PAST Users <ref name:"rtethree">RTE-3 data have been provided by participants by means of a questionnaire.</ref></small><br /> ! width="45"|<small>RTE4 Users<ref name:"rtefour">RTE-4 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! width="45"|<small>RTE5 Users<ref name:"rtefive">RTE-5 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! width="45"|<small>RTE6 Users<ref name:"rtesix">RTE-6 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! class="unsortable" width="30"|<small>Usage info</small><br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[WordNet]]<br /> | Lexical DB<br /> | Princeton University<br /> | Lexical database of English nouns, verbs, adjectives and adverbs<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|21<br /> | style="text-align: center;"|18<br /> |style="text-align: center;"| 22<br /> | [[WordNet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://xwn.hlt.utdallas.edu/index.html eXtended Wordnet]<br /> | Lexical DB<br /> | Human Language Technology Research Institute, University of Texas at Dallas<br /> | Extension of WordNet based on the exploitation of the information contained in WordNet definitional glosses: the glosses are syntactically parsed, transformed into logic forms and content words are semantically disambiguated. The Extended Wordnet is an ongoing project. <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|2<br /> |style="text-align: center;"| 0<br /> | [[eXtended WordNet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://ai.stanford.edu/~rion/swn/ Augmented Wordnet]<br /> | Lexical DB<br /> | Stanford University<br /> | The resource is the result of the application of a learning algorithm for inducing semantic taxonomies from parsed text. The algorithm automatically acquires items of world knowledge, and uses these to produce significantly enhanced versions of WordNet (up to 40,000 synsets more).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Augmented Wordnet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://verbs.colorado.edu/~mpalmer/projects/verbnet.html Verbnet]<br /> | Lexical DB<br /> | University of Colorado Boulder<br /> | Lexicon for English verbs organized into classes extending Levin (1993) classes through refinement and addition of subclasses to achieve syntactic and semantic coherence among members of a class<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Verbnet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[VerbOcean]]<br /> | Lexical DB<br /> | Information Sciences Institute, University of Southern California<br /> | Broad-coverage semantic network of verbs<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|6<br /> |style="text-align: center;"| 7<br /> | [[VerbOcean - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://framenet.icsi.berkeley.edu/ FrameNet]<br /> | Lexical DB<br /> | ICSI (International Computer Science Institute) - Berkley University<br /> | Lexical resource for English words, based on frame semantics (valences) and supported by corpus evidence<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|2<br /> |style="text-align: center;"| 3<br /> | [[Framenet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/meyers/NomBank.html NomBank]<br /> | Lexical DB<br /> | New York University<br /> | Lexical resource containing syntactic frames for nouns, extracted from annotated corpora <br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[NomBank Resource - RTE Users|Users]] <br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://verbs.colorado.edu/~mpalmer/projects/ace.html PropBank]<br /> | Lexical DB<br /> | University of Colorado Boulder <br /> | Lexical resource containing syntactic frames for verbs, extracted from annotated corpora <br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[PropBank Resource - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/nomlex/index.html Nomlex] Plus<br /> | Lexical DB<br /> | New York University<br /> | Dictionary of English nominalizations: it describes the allowed complements for a nominalization and relates the nominal complements to the arguments of the corresponding verb<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Nomlex Plus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.cs.ualberta.ca/~lindek/downloads.htm Dekang Lin’s Thesaurus]<br /> | Thesaurus<br /> | University of Alberta<br /> | Thesaurus automatically constructed using a parsed corpus, based on distributional similarity scores<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 2<br /> | [[Dekang Lin’s Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://icon.shef.ac.uk/Moby/mthes.html Grady Ward's Moby Thesaurus]<br /> | Thesaurus<br /> | University of Sheffield<br /> | Thesaurus containing 30,260 root words, with 2,520,264 synonyms and related terms. Grady Ward placed this thesaurus in the public domain in 1996.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Grady Ward's Moby Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://en.wikipedia.org/wiki/Roget%27s_Thesaurus Roget's Thesaurus]<br /> | Thesaurus<br /> | Peter Mark Roget (Electronic version distributed by University of Chicago)<br /> | Roget's Thesaurus is a widely-used English thesaurus, created by Dr. Peter Mark Roget in 1805. The original edition had 15,000 words, and each new edition has been larger. The electronic edition ([http://machaut.uchicago.edu/rogets version 1.02]) is made available by University of Chicago.<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Roget's Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.wikipedia.org/ Wikipedia]<br /> | Encyclopedia<br /> | <br /> | Free encyclopedia. Used for extraction of lexical-semantic rules (from its more structured parts), named entity recognition, geographical information etc.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|6<br /> |style="text-align: center;"| 6<br /> | [[Wikipedia - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.umbel.org/ Umbel]<br /> | Ontology<br /> | Structured Dynamics LLC, Coralville, IA<br /> | UMBEL stands for Upper Mapping and Binding Exchange Layer and is a lightweight ontology structure for relating Web content and data to a standard set of subject concepts<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Umbel - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.mpi-inf.mpg.de/yago-naga/yago/ YAGO]<br /> | Ontology<br /> | Max-Planck Institute for Informatics, Saarbrücken, Germany<br /> | Light-weight and extensible ontology. It contains more than 2 million entities and 20 million facts about these entities. The facts have been automatically extracted from Wikipedia and unified with WordNet.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[YAGO - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://dbpedia.org/About DBpedia]<br /> | Ontology<br /> | Open community project<br /> | DBpedia is a community effort to extract structured information from Wikipedia and to make this information available on the Web. The DBpedia knowledge base currently describes more than 2.9 million things in 91 different languages and consists of 479 million pieces of information.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[DBpedia - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[DIRT Paraphrase Collection]]<br /> | Collection of paraphrases<br /> | University of Alberta<br /> | DIRT (Discovery of Inference Rules from Text) is both an algorithm and a resulting knowledge collection. The DIRT knowledge collection is the output of the DIRT algorithm over a 1GB set of newspaper text.<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|4<br /> | style="text-align: center;"|3<br /> |style="text-align: center;"| 4<br /> | [[DIRT Paraphrase Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[TEASE]] Collection<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University<br /> | Output of the TEASE algorithm <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Tease Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://badc.nerc.ac.uk/help/abbrevs.html BADC Acronym and Abbreviation List]<br /> | Word List<br /> | BADC (British Atmospheric Data Centre)<br /> | Acronym and Abbreviation List<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[BADC Acronym and Abbreviation List - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://clipdemos.umiacs.umd.edu/catvar/ Catvar The Categorial Variation Database (English)]<br /> | Word List<br /> | University of Maryland<br /> | A Categorial-Variation Database (or Catvar) is a database of clusters of uninflected words (lexemes) and their categorial (i.e. part-of-speech) variants. The database was developed for English using a combination of resources and algorithms, including the LCS Verb and Preposition Databases (Dorr 2001), the Brown Corpus section of the Penn Treebank (Marcus et al. 1994), an English morphological analysis lexicon developed for PC-Kimmo (ENGLEX) (Antworth 1990), WordNet1.6 (Fellbaum 1998), an English Verb-Noun list extracted from Nomlex (Macleod et al. 1998), a similar list extracted from LDOCE (Procter 1978) and the Porter stemmer (Porter 1980).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 1<br /> | [[Categorial-Variation Database - RTE Users|Users]]<br /> <br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.acronym-guide.com/ Acronym Guide]<br /> | Word List<br /> | Acronym-Guide.com <br /> | Acronym and Abbreviation Lists for English, branched in thematic directories <br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|3<br /> |style="text-align: center;"| 0<br /> | [[Acronym Guide - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.ldc.upenn.edu/Catalog/CatalogEntry.jsp?catalogId=LDC2006T13 Web1T 5-grams]<br /> | Word list<br /> | Linguistic Data Consortium, University of Pennsylvania; Google Inc.<br /> | Data set containing English word n-grams and their observed frequency counts. The n-gram counts were generated from approximately 1 trillion word tokens of text from publicly accessible Web pages<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Web1T - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.coli.uni-saarland.de/%7Erwang/resources/RTE3_RTE4_NGD.zip Normalized Google Distance (RTE3&RTE4)]<br /> | Word Pair Co-occurrence<br /> | Saarland University<br /> | Co-occurrence of the word pairs in RTE3 and RTE4 using Normalized Google Distance (Cilibrasi and Vitanyi, 2004). The word pairs are all the possible combinations of content words in T and H. In practice, we used [http://m.www.yahoo.com/ Yahoo!] as the search engine.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Normalized Google Distance (RTE3&RTE4)- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.coli.uni-saarland.de/%7Erwang/resources/RTE5_NGD.zip Normalized Google Distance (RTE5)]<br /> | Word Pair Co-occurrence<br /> | Saarland University<br /> | Co-occurrence of the word pairs in RTE3 and RTE4 using Normalized Google Distance (Cilibrasi and Vitanyi, 2004). The word pairs are all the possible combinations of content words in T and H. In practice, we used [http://m.www.yahoo.com/ Yahoo!] as the search engine.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Normalized Google Distance (RTE5)- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://geonames.usgs.gov/index.html GNIS - Geographic Names Information System]<br /> | Gazetteer<br /> | USGS (United States Geological Survey)<br /> | Database containing the Federal and national standard toponyms for USA, associated areas and Antarctica<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[GNIS - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.geonames.org/ Geonames]<br /> | Gazetteer<br /> | <br /> | Database containing eight million geographical names. It is integrating geographical data such as names of places in various languages, elevation, population and others from various sources.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Geonames - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/paraphrase/ Sekine's Paraphrase Database]<br /> | Collection of paraphrases<br /> | Department of Computer Science, New York University <br /> | Data-base created using Sekine's method, NOT cleaned up by human. It includes 19,975 sets of paraphrases with 191,572 phrases.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Sekine's Paraphrase Database - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://research.microsoft.com/research/downloads/Details/607D14D9-20CD-47E3-85BC-A2F65CD28042/Details.aspx Microsoft Research Paraphrase Corpus]<br /> | Collection of paraphrases<br /> | Microsoft Research<br /> | Text file containing 5800 pairs of sentences which have been extracted from news sources on the web, along with human annotations indicating whether each pair captures a paraphrase/semantic equivalence relationship.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Microsoft Research Paraphrase Corpus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.cs.cornell.edu/~cristian/Without_a_doubt_-_Data.html Downward entailing operators] <br /> | Collection of entailing operators<br /> | Department of Computer Science, Cornell University, Ithaca NY<br /> | System output of an unsupervised algorithm recovering many Downward Entailing operators, like 'doubt'.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Downward entailing operators - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/WikiRules.html WikiRules!]<br /> | Lexical Reference rule-base<br /> | Bar-Ilan University<br /> | Extraction of about 8 million lexical reference rules from the text body (first sentence) and from metadata (links, redirects, parentheses) of Wikipedia. Provides better performance than other automatically constructed resources and comparable performance to WordNet. Offers complementary knowledge to WordNet. <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[WikiRules! - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://www.cs.utexas.edu/users/pclark/dart/ DART]<br /> | Collection of "world knowledge" propositions<br /> | Boeing Research and Technology<br /> | 23 million tuples such as "airplanes can fly to airports", "rivers can flood" collected from abstracted parse trees.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[DART - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://cs.biu.ac.il/~nlp/download/FRED/latest/FRED.zip FRED]<br /> | FrameNet-derived entailment rule-base<br /> | Bar-Ilan University<br /> | This package contains the outputs of the FRED algorithm, an algorithm which extracts entailment rules from FrameNet.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[FRED - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/DIRECT.html DIRECT]<br /> | Directional Distributional Term-Similarity Resource<br /> | Bar-Ilan University<br /> | This is a resource of directional distributional term-similarity rules (mostly lexical entailment rules) automatically extracted using the inclusion relation as described in (Kotlerman et.al., ACL-09).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[DIRECT - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/binaryDirt.html binaryDIRT]<br /> | Entailment rules between binary templates using DIRT algorithm<br /> | Bar-Ilan University<br /> | This resource contains entailment rules over binary templates learned over the Reuters corpus using<br /> the DIRT algorithm of Lin and Pantel.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[BinaryDIRT- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/unaryBInc.html unaryBInc]<br /> | Entailment rules between unary templates using BInc algorithm<br /> | Bar-Ilan University<br /> | This resource contains entailment rules over unary templates learned over the Reuters corpus using<br /> the BInc algorithm of Szpektor and Dagan (2008).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[unaryBInc- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''New resource''<br /> | <br /> | <br /> | ''Participants are encouraged to contribute''<br /> | style="text-align: center;"| <br /> | style="text-align: center;"| <br /> | style="text-align: center;"| <br /> |style="text-align: center;"| 0<br /> | [[New Resource2 - RTE Users|Users]]<br /> <br /> |}<br /> <br><br /> <br><br /> <br /> === Not available Resources ===<br /> The following table lists the unpublished resources used by RTE participants. Some of them have been developed by Users themselves specifically for RTE. Interested people may turn to authors to obtain further information.<br /> <br><br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! width="80"|Resource<br /> ! width="50"|Type<br /> ! width="180"|Author<br /> ! class="unsortable" width="500"|Brief description<br /> ! width="30"|<small>PAST Users</small><br /> ! width="30"|<small>RTE4 Users</small><br /> ! width="30"|<small>RTE5 Users</small><br /> ! width="30"|<small>RTE6 Users</small><br /> ! class="unsortable" width="30"|<small>Usage info</small><br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.parc.com/ PARC] Polarity Lexicon<br /> | Lexical DB<br /> | PARC - Palo Alto Research Center<br /> | Verbs classification with respect to semantic polarity<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Parc Polarity Lexicon - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Gazetteer from [http://trec.nist.gov/ TREC]<br /> | Gazetteer<br /> | NIST - National Institute of Standards and Technology<br /> | Cities and other geographical names<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Gazetteer from TREC - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DFKI Geographic Ontology<br>''(to be released)''<br /> | Ontology<br /> | DFKI - German Research Center for Artificial Intelligence<br /> | Ontology containing geographic terms and two kinds of relations: the directional ''part-of'' relation, and the ''equal'' relation for synonyms and abbreviations of the same geographic area (e.g ''the United Kingdom'', ''the UK'', ''Great Britain'', etc.) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Geographic Ontology - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Geo''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | Meronymy entailment rules, based on TREC’s TIPSTER gazetteer.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Geo - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Regex''<br /> | Collection of Entailment rules <br /> | Bar-Ilan University; Tel-Aviv University<br /> | Small set of entailment rules based on regular expressions, intended to address lexical variability involving temporal phrases<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Regex - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Syntactic rule base <br>''(to be released)''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | A manually-composed collection of entailment rules which define parse tree transformations. The rules cover generic syntactic phenomena such as appositions, conjunctions, passive, relative clause, etc. (Bar-Haim et al., AAAI-07) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Syntactic rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Polarity rule base <br>''(to be released)''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | A manually-composed collection of entailment rules which detect predicates whose polarity is negative (e.g. didn't dance) or unknown (e.g. plans to dance). The rules capture diverse phenomena that affect polarity, e.g. verbal negation, modal verbs, conditionals, and certain verbs that induce negative or "unknown" polarity context. The latter were taken mainly from VerbNet. Extends a resource described in (Bar-Haim et al., AAAI-07) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0 <br /> | style="text-align: center;"|<br /> | [[Polarity rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Lexical-Syntactic rule base<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | Extract lexical-syntactic entailment rules for predicates (verbal and nominal), including argument mapping. The resource is based on WordNet, Nomlex-Plus and Unary DIRT (Szpektor and Dagan, Coling 08)<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Lexical-Syntactic rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | OPENU Collection<br /> | Collection of Entailment Rules and Patterns<br /> | Open University<br /> | Collections of rules, patterns etc. for RTE purpose, extracted from Reuter corpus parsed using Minipar.<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"| <br /> | [[OPENU Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Abbr''<br /> | Collection of rules for abbreviation<br /> | Bar-Ilan University; Tel-Aviv University<br /> | 2000 Abbreviation rules, extracted from [http://badc.nerc.ac.uk/help/abbrevs.html BADC] and [http://www.acronym-guide.com/ Acronym Guide]<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Abbr - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Negation_list<br /> | Negation rules<br /> | „Al. I. Cuza“ University, Iasi, Romania <br /> | List of negative terms and words (verbs, adjectives, nouns) affecting modality or factuality of a infinitive verb preceded by the particle "to" (e.g. "believe","necessary", "attempt")<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1 <br /> | style="text-align: center;"|<br /> | [[UAIC Negation_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DLSIUAES Negation_list<br /> | List of negative terms<br /> | University of Alicante<br /> | Basic list of negative terms.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"| <br /> | [[DLSIUAES Negation_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Quantifier_list<br /> | List of quantifiers<br /> | „Al. I. Cuza“ University, Iasi, Romania <br /> | List of quantifiers affecting entailment judgment. The quantifiers are taken from a list which contains expressions like “more than”, “less than”, or words such as “over”, “under”, etc.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"| <br /> | [[UAIC Quantifier_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | FBKirst StopWord list<br /> | List of frequent words<br /> | FBK-Irst;<br/>University of Trento - Italy <br /> | A list of the 572 most frequent English words.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"| <br /> | [[FBKirst StopWord list - RTE Users|Users]]<br /> <br /> |}<br /> <br><br /> <br /> ==Footnotes==<br /> <references/></div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE_Knowledge_Resources&diff=8667 RTE Knowledge Resources 2011-02-04T10:26:13Z

<p>Amarchetti: /* Publicly available Resources */</p> <hr /> <div>Knowledge resources have shown their relevance for applied semantic inference, and are extensively used by applied inference systems, such as those developed within the Textual Entailment framework.<br><br /> <br><br /> This page presents a list of the knowledge resources used by systems that have participated in the last RTE challenges. The first table lists the publicly available resources, the second one lists unpublished resources. Both tables are sortable by Resource name, type, author and number of users.<br><br /> <br><br /> RTE Participants are encouraged to add information about all kind of knowledge resources used, from standard existing resources (e.g. WordNet) to knowledge collections created for specific purposes, which can be made available to the community.<br><br /> <br><br /> __TOC__<br /> <br><br /> === Call for Resources ===<br /> <br /> In order to help the research, all the participants are invited to contribute, sharing their own resources with the RTE community. <br /> <br><br /> Making the resources available to be used by other systems has several advantages. On the one hand, it helps improve the TE technology; on the other hand, it offers an opportunity to further test and evaluate the resource.<br /> <br><br /> <br><br /> [[RTE6 - Call for Resources]]<br /> <br><br /> === Ablation Tests ===<br /> An ablation test consists of removing one module at a time from a system, and rerunning the system on the test set with the other modules, except the one tested. <br /> <br><br /> Ablation test are meant to help better understand the relevance of the knowledge resources used by RTE systems, and evaluate the contribution of each of them to the systems' performances. In fact, comparing the results achieved in the ablation tests to those obtained by the systems as a whole allows assessing the contribution given by each single resource.<br /> <br><br /> <br><br /> *[[RTE5 - Ablation Tests]]<br /> *[[RTE6 - Ablation Tests]]<br /> <br><br /> <br /> === Publicly available Resources ===<br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! width="80"|Resource<br /> ! width="50"|Type<br /> ! width="150"|Author<br /> ! class="unsortable" width="300"|Brief description<br /> ! width="45"|<small>PAST Users <ref name:"rtethree">RTE-3 data have been provided by participants by means of a questionnaire.</ref></small><br /> ! width="45"|<small>RTE4 Users<ref name:"rtefour">RTE-4 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! width="45"|<small>RTE5 Users<ref name:"rtefive">RTE-5 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! width="45"|<small>RTE6 Users<ref name:"rtesix">RTE-6 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! class="unsortable" width="30"|<small>Usage info</small><br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[WordNet]]<br /> | Lexical DB<br /> | Princeton University<br /> | Lexical database of English nouns, verbs, adjectives and adverbs<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|21<br /> | style="text-align: center;"|18<br /> |style="text-align: center;"| 22<br /> | [[WordNet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://xwn.hlt.utdallas.edu/index.html eXtended Wordnet]<br /> | Lexical DB<br /> | Human Language Technology Research Institute, University of Texas at Dallas<br /> | Extension of WordNet based on the exploitation of the information contained in WordNet definitional glosses: the glosses are syntactically parsed, transformed into logic forms and content words are semantically disambiguated. The Extended Wordnet is an ongoing project. <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|2<br /> |style="text-align: center;"| 0<br /> | [[eXtended WordNet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://ai.stanford.edu/~rion/swn/ Augmented Wordnet]<br /> | Lexical DB<br /> | Stanford University<br /> | The resource is the result of the application of a learning algorithm for inducing semantic taxonomies from parsed text. The algorithm automatically acquires items of world knowledge, and uses these to produce significantly enhanced versions of WordNet (up to 40,000 synsets more).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Augmented Wordnet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://verbs.colorado.edu/~mpalmer/projects/verbnet.html Verbnet]<br /> | Lexical DB<br /> | University of Colorado Boulder<br /> | Lexicon for English verbs organized into classes extending Levin (1993) classes through refinement and addition of subclasses to achieve syntactic and semantic coherence among members of a class<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Verbnet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[VerbOcean]]<br /> | Lexical DB<br /> | Information Sciences Institute, University of Southern California<br /> | Broad-coverage semantic network of verbs<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|6<br /> |style="text-align: center;"| 7<br /> | [[VerbOcean - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://framenet.icsi.berkeley.edu/ FrameNet]<br /> | Lexical DB<br /> | ICSI (International Computer Science Institute) - Berkley University<br /> | Lexical resource for English words, based on frame semantics (valences) and supported by corpus evidence<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|2<br /> |style="text-align: center;"| 3<br /> | [[Framenet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/meyers/NomBank.html NomBank]<br /> | Lexical DB<br /> | New York University<br /> | Lexical resource containing syntactic frames for nouns, extracted from annotated corpora <br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[NomBank Resource - RTE Users|Users]] <br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://verbs.colorado.edu/~mpalmer/projects/ace.html PropBank]<br /> | Lexical DB<br /> | University of Colorado Boulder <br /> | Lexical resource containing syntactic frames for verbs, extracted from annotated corpora <br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[PropBank Resource - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/nomlex/index.html Nomlex] Plus<br /> | Lexical DB<br /> | New York University<br /> | Dictionary of English nominalizations: it describes the allowed complements for a nominalization and relates the nominal complements to the arguments of the corresponding verb<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Nomlex Plus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.cs.ualberta.ca/~lindek/downloads.htm Dekang Lin’s Thesaurus]<br /> | Thesaurus<br /> | University of Alberta<br /> | Thesaurus automatically constructed using a parsed corpus, based on distributional similarity scores<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 2<br /> | [[Dekang Lin’s Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://icon.shef.ac.uk/Moby/mthes.html Grady Ward's Moby Thesaurus]<br /> | Thesaurus<br /> | University of Sheffield<br /> | Thesaurus containing 30,260 root words, with 2,520,264 synonyms and related terms. Grady Ward placed this thesaurus in the public domain in 1996.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Grady Ward's Moby Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://en.wikipedia.org/wiki/Roget%27s_Thesaurus Roget's Thesaurus]<br /> | Thesaurus<br /> | Peter Mark Roget (Electronic version distributed by University of Chicago)<br /> | Roget's Thesaurus is a widely-used English thesaurus, created by Dr. Peter Mark Roget in 1805. The original edition had 15,000 words, and each new edition has been larger. The electronic edition ([http://machaut.uchicago.edu/rogets version 1.02]) is made available by University of Chicago.<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Roget's Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.wikipedia.org/ Wikipedia]<br /> | Encyclopedia<br /> | <br /> | Free encyclopedia. Used for extraction of lexical-semantic rules (from its more structured parts), named entity recognition, geographical information etc.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|6<br /> |style="text-align: center;"| 6<br /> | [[Wikipedia - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.umbel.org/ Umbel]<br /> | Ontology<br /> | Structured Dynamics LLC, Coralville, IA<br /> | UMBEL stands for Upper Mapping and Binding Exchange Layer and is a lightweight ontology structure for relating Web content and data to a standard set of subject concepts<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Umbel - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.mpi-inf.mpg.de/yago-naga/yago/ YAGO]<br /> | Ontology<br /> | Max-Planck Institute for Informatics, Saarbrücken, Germany<br /> | Light-weight and extensible ontology. It contains more than 2 million entities and 20 million facts about these entities. The facts have been automatically extracted from Wikipedia and unified with WordNet.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[YAGO - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://dbpedia.org/About DBpedia]<br /> | Ontology<br /> | Open community project<br /> | DBpedia is a community effort to extract structured information from Wikipedia and to make this information available on the Web. The DBpedia knowledge base currently describes more than 2.9 million things in 91 different languages and consists of 479 million pieces of information.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[DBpedia - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[DIRT Paraphrase Collection]]<br /> | Collection of paraphrases<br /> | University of Alberta<br /> | DIRT (Discovery of Inference Rules from Text) is both an algorithm and a resulting knowledge collection. The DIRT knowledge collection is the output of the DIRT algorithm over a 1GB set of newspaper text.<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|4<br /> | style="text-align: center;"|3<br /> |style="text-align: center;"| 4<br /> | [[DIRT Paraphrase Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[TEASE]] Collection<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University<br /> | Output of the TEASE algorithm <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Tease Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://badc.nerc.ac.uk/help/abbrevs.html BADC Acronym and Abbreviation List]<br /> | Word List<br /> | BADC (British Atmospheric Data Centre)<br /> | Acronym and Abbreviation List<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[BADC Acronym and Abbreviation List - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://clipdemos.umiacs.umd.edu/catvar/ Catvar The Categorial Variation Database (English)]<br /> | Word List<br /> | University of Maryland<br /> | A Categorial-Variation Database (or Catvar) is a database of clusters of uninflected words (lexemes) and their categorial (i.e. part-of-speech) variants. The database was developed for English using a combination of resources and algorithms, including the LCS Verb and Preposition Databases (Dorr 2001), the Brown Corpus section of the Penn Treebank (Marcus et al. 1994), an English morphological analysis lexicon developed for PC-Kimmo (ENGLEX) (Antworth 1990), WordNet1.6 (Fellbaum 1998), an English Verb-Noun list extracted from Nomlex (Macleod et al. 1998), a similar list extracted from LDOCE (Procter 1978) and the Porter stemmer (Porter 1980).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 1<br /> | [[Categorial-Variation Database - RTE Users|Users]]<br /> <br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.acronym-guide.com/ Acronym Guide]<br /> | Word List<br /> | Acronym-Guide.com <br /> | Acronym and Abbreviation Lists for English, branched in thematic directories <br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|3<br /> |style="text-align: center;"| <br /> | [[Acronym Guide - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.ldc.upenn.edu/Catalog/CatalogEntry.jsp?catalogId=LDC2006T13 Web1T 5-grams]<br /> | Word list<br /> | Linguistic Data Consortium, University of Pennsylvania; Google Inc.<br /> | Data set containing English word n-grams and their observed frequency counts. The n-gram counts were generated from approximately 1 trillion word tokens of text from publicly accessible Web pages<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[Web1T - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.coli.uni-saarland.de/%7Erwang/resources/RTE3_RTE4_NGD.zip Normalized Google Distance (RTE3&RTE4)]<br /> | Word Pair Co-occurrence<br /> | Saarland University<br /> | Co-occurrence of the word pairs in RTE3 and RTE4 using Normalized Google Distance (Cilibrasi and Vitanyi, 2004). The word pairs are all the possible combinations of content words in T and H. In practice, we used [http://m.www.yahoo.com/ Yahoo!] as the search engine.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Normalized Google Distance (RTE3&RTE4)- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.coli.uni-saarland.de/%7Erwang/resources/RTE5_NGD.zip Normalized Google Distance (RTE5)]<br /> | Word Pair Co-occurrence<br /> | Saarland University<br /> | Co-occurrence of the word pairs in RTE3 and RTE4 using Normalized Google Distance (Cilibrasi and Vitanyi, 2004). The word pairs are all the possible combinations of content words in T and H. In practice, we used [http://m.www.yahoo.com/ Yahoo!] as the search engine.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Normalized Google Distance (RTE5)- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://geonames.usgs.gov/index.html GNIS - Geographic Names Information System]<br /> | Gazetteer<br /> | USGS (United States Geological Survey)<br /> | Database containing the Federal and national standard toponyms for USA, associated areas and Antarctica<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[GNIS - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.geonames.org/ Geonames]<br /> | Gazetteer<br /> | <br /> | Database containing eight million geographical names. It is integrating geographical data such as names of places in various languages, elevation, population and others from various sources.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[Geonames - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/paraphrase/ Sekine's Paraphrase Database]<br /> | Collection of paraphrases<br /> | Department of Computer Science, New York University <br /> | Data-base created using Sekine's method, NOT cleaned up by human. It includes 19,975 sets of paraphrases with 191,572 phrases.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[Sekine's Paraphrase Database - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://research.microsoft.com/research/downloads/Details/607D14D9-20CD-47E3-85BC-A2F65CD28042/Details.aspx Microsoft Research Paraphrase Corpus]<br /> | Collection of paraphrases<br /> | Microsoft Research<br /> | Text file containing 5800 pairs of sentences which have been extracted from news sources on the web, along with human annotations indicating whether each pair captures a paraphrase/semantic equivalence relationship.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[Microsoft Research Paraphrase Corpus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.cs.cornell.edu/~cristian/Without_a_doubt_-_Data.html Downward entailing operators] <br /> | Collection of entailing operators<br /> | Department of Computer Science, Cornell University, Ithaca NY<br /> | System output of an unsupervised algorithm recovering many Downward Entailing operators, like 'doubt'.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Downward entailing operators - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/WikiRules.html WikiRules!]<br /> | Lexical Reference rule-base<br /> | Bar-Ilan University<br /> | Extraction of about 8 million lexical reference rules from the text body (first sentence) and from metadata (links, redirects, parentheses) of Wikipedia. Provides better performance than other automatically constructed resources and comparable performance to WordNet. Offers complementary knowledge to WordNet. <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[WikiRules! - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://www.cs.utexas.edu/users/pclark/dart/ DART]<br /> | Collection of "world knowledge" propositions<br /> | Boeing Research and Technology<br /> | 23 million tuples such as "airplanes can fly to airports", "rivers can flood" collected from abstracted parse trees.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[DART - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://cs.biu.ac.il/~nlp/download/FRED/latest/FRED.zip FRED]<br /> | FrameNet-derived entailment rule-base<br /> | Bar-Ilan University<br /> | This package contains the outputs of the FRED algorithm, an algorithm which extracts entailment rules from FrameNet.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[FRED - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/DIRECT.html DIRECT]<br /> | Directional Distributional Term-Similarity Resource<br /> | Bar-Ilan University<br /> | This is a resource of directional distributional term-similarity rules (mostly lexical entailment rules) automatically extracted using the inclusion relation as described in (Kotlerman et.al., ACL-09).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[DIRECT - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/binaryDirt.html binaryDIRT]<br /> | Entailment rules between binary templates using DIRT algorithm<br /> | Bar-Ilan University<br /> | This resource contains entailment rules over binary templates learned over the Reuters corpus using<br /> the DIRT algorithm of Lin and Pantel.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[BinaryDIRT- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/unaryBInc.html unaryBInc]<br /> | Entailment rules between unary templates using BInc algorithm<br /> | Bar-Ilan University<br /> | This resource contains entailment rules over unary templates learned over the Reuters corpus using<br /> the BInc algorithm of Szpektor and Dagan (2008).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[unaryBInc- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''New resource''<br /> | <br /> | <br /> | ''Participants are encouraged to contribute''<br /> | style="text-align: center;"| <br /> | style="text-align: center;"| <br /> | style="text-align: center;"| <br /> |style="text-align: center;"|<br /> | [[New Resource2 - RTE Users|Users]]<br /> <br /> |}<br /> <br><br /> <br><br /> <br /> === Not available Resources ===<br /> The following table lists the unpublished resources used by RTE participants. Some of them have been developed by Users themselves specifically for RTE. Interested people may turn to authors to obtain further information.<br /> <br><br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! width="80"|Resource<br /> ! width="50"|Type<br /> ! width="180"|Author<br /> ! class="unsortable" width="500"|Brief description<br /> ! width="30"|<small>PAST Users</small><br /> ! width="30"|<small>RTE4 Users</small><br /> ! width="30"|<small>RTE5 Users</small><br /> ! width="30"|<small>RTE6 Users</small><br /> ! class="unsortable" width="30"|<small>Usage info</small><br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.parc.com/ PARC] Polarity Lexicon<br /> | Lexical DB<br /> | PARC - Palo Alto Research Center<br /> | Verbs classification with respect to semantic polarity<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Parc Polarity Lexicon - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Gazetteer from [http://trec.nist.gov/ TREC]<br /> | Gazetteer<br /> | NIST - National Institute of Standards and Technology<br /> | Cities and other geographical names<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Gazetteer from TREC - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DFKI Geographic Ontology<br>''(to be released)''<br /> | Ontology<br /> | DFKI - German Research Center for Artificial Intelligence<br /> | Ontology containing geographic terms and two kinds of relations: the directional ''part-of'' relation, and the ''equal'' relation for synonyms and abbreviations of the same geographic area (e.g ''the United Kingdom'', ''the UK'', ''Great Britain'', etc.) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Geographic Ontology - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Geo''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | Meronymy entailment rules, based on TREC’s TIPSTER gazetteer.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Geo - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Regex''<br /> | Collection of Entailment rules <br /> | Bar-Ilan University; Tel-Aviv University<br /> | Small set of entailment rules based on regular expressions, intended to address lexical variability involving temporal phrases<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Regex - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Syntactic rule base <br>''(to be released)''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | A manually-composed collection of entailment rules which define parse tree transformations. The rules cover generic syntactic phenomena such as appositions, conjunctions, passive, relative clause, etc. (Bar-Haim et al., AAAI-07) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Syntactic rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Polarity rule base <br>''(to be released)''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | A manually-composed collection of entailment rules which detect predicates whose polarity is negative (e.g. didn't dance) or unknown (e.g. plans to dance). The rules capture diverse phenomena that affect polarity, e.g. verbal negation, modal verbs, conditionals, and certain verbs that induce negative or "unknown" polarity context. The latter were taken mainly from VerbNet. Extends a resource described in (Bar-Haim et al., AAAI-07) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0 <br /> | style="text-align: center;"|<br /> | [[Polarity rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Lexical-Syntactic rule base<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | Extract lexical-syntactic entailment rules for predicates (verbal and nominal), including argument mapping. The resource is based on WordNet, Nomlex-Plus and Unary DIRT (Szpektor and Dagan, Coling 08)<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Lexical-Syntactic rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | OPENU Collection<br /> | Collection of Entailment Rules and Patterns<br /> | Open University<br /> | Collections of rules, patterns etc. for RTE purpose, extracted from Reuter corpus parsed using Minipar.<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"| <br /> | [[OPENU Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Abbr''<br /> | Collection of rules for abbreviation<br /> | Bar-Ilan University; Tel-Aviv University<br /> | 2000 Abbreviation rules, extracted from [http://badc.nerc.ac.uk/help/abbrevs.html BADC] and [http://www.acronym-guide.com/ Acronym Guide]<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Abbr - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Negation_list<br /> | Negation rules<br /> | „Al. I. Cuza“ University, Iasi, Romania <br /> | List of negative terms and words (verbs, adjectives, nouns) affecting modality or factuality of a infinitive verb preceded by the particle "to" (e.g. "believe","necessary", "attempt")<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1 <br /> | style="text-align: center;"|<br /> | [[UAIC Negation_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DLSIUAES Negation_list<br /> | List of negative terms<br /> | University of Alicante<br /> | Basic list of negative terms.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"| <br /> | [[DLSIUAES Negation_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Quantifier_list<br /> | List of quantifiers<br /> | „Al. I. Cuza“ University, Iasi, Romania <br /> | List of quantifiers affecting entailment judgment. The quantifiers are taken from a list which contains expressions like “more than”, “less than”, or words such as “over”, “under”, etc.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"| <br /> | [[UAIC Quantifier_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | FBKirst StopWord list<br /> | List of frequent words<br /> | FBK-Irst;<br/>University of Trento - Italy <br /> | A list of the 572 most frequent English words.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"| <br /> | [[FBKirst StopWord list - RTE Users|Users]]<br /> <br /> |}<br /> <br><br /> <br /> ==Footnotes==<br /> <references/></div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE_Knowledge_Resources&diff=8666 RTE Knowledge Resources 2011-02-04T10:17:46Z

<p>Amarchetti: /* Publicly available Resources */</p> <hr /> <div>Knowledge resources have shown their relevance for applied semantic inference, and are extensively used by applied inference systems, such as those developed within the Textual Entailment framework.<br><br /> <br><br /> This page presents a list of the knowledge resources used by systems that have participated in the last RTE challenges. The first table lists the publicly available resources, the second one lists unpublished resources. Both tables are sortable by Resource name, type, author and number of users.<br><br /> <br><br /> RTE Participants are encouraged to add information about all kind of knowledge resources used, from standard existing resources (e.g. WordNet) to knowledge collections created for specific purposes, which can be made available to the community.<br><br /> <br><br /> __TOC__<br /> <br><br /> === Call for Resources ===<br /> <br /> In order to help the research, all the participants are invited to contribute, sharing their own resources with the RTE community. <br /> <br><br /> Making the resources available to be used by other systems has several advantages. On the one hand, it helps improve the TE technology; on the other hand, it offers an opportunity to further test and evaluate the resource.<br /> <br><br /> <br><br /> [[RTE6 - Call for Resources]]<br /> <br><br /> === Ablation Tests ===<br /> An ablation test consists of removing one module at a time from a system, and rerunning the system on the test set with the other modules, except the one tested. <br /> <br><br /> Ablation test are meant to help better understand the relevance of the knowledge resources used by RTE systems, and evaluate the contribution of each of them to the systems' performances. In fact, comparing the results achieved in the ablation tests to those obtained by the systems as a whole allows assessing the contribution given by each single resource.<br /> <br><br /> <br><br /> *[[RTE5 - Ablation Tests]]<br /> *[[RTE6 - Ablation Tests]]<br /> <br><br /> <br /> === Publicly available Resources ===<br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! width="80"|Resource<br /> ! width="50"|Type<br /> ! width="150"|Author<br /> ! class="unsortable" width="300"|Brief description<br /> ! width="45"|<small>PAST Users <ref name:"rtethree">RTE-3 data have been provided by participants by means of a questionnaire.</ref></small><br /> ! width="45"|<small>RTE4 Users<ref name:"rtefour">RTE-4 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! width="45"|<small>RTE5 Users<ref name:"rtefive">RTE-5 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! width="45"|<small>RTE6 Users<ref name:"rtesix">RTE-6 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! class="unsortable" width="30"|<small>Usage info</small><br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[WordNet]]<br /> | Lexical DB<br /> | Princeton University<br /> | Lexical database of English nouns, verbs, adjectives and adverbs<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|21<br /> | style="text-align: center;"|18<br /> |style="text-align: center;"| 22<br /> | [[WordNet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://xwn.hlt.utdallas.edu/index.html eXtended Wordnet]<br /> | Lexical DB<br /> | Human Language Technology Research Institute, University of Texas at Dallas<br /> | Extension of WordNet based on the exploitation of the information contained in WordNet definitional glosses: the glosses are syntactically parsed, transformed into logic forms and content words are semantically disambiguated. The Extended Wordnet is an ongoing project. <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|2<br /> |style="text-align: center;"| 0<br /> | [[eXtended WordNet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://ai.stanford.edu/~rion/swn/ Augmented Wordnet]<br /> | Lexical DB<br /> | Stanford University<br /> | The resource is the result of the application of a learning algorithm for inducing semantic taxonomies from parsed text. The algorithm automatically acquires items of world knowledge, and uses these to produce significantly enhanced versions of WordNet (up to 40,000 synsets more).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Augmented Wordnet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://verbs.colorado.edu/~mpalmer/projects/verbnet.html Verbnet]<br /> | Lexical DB<br /> | University of Colorado Boulder<br /> | Lexicon for English verbs organized into classes extending Levin (1993) classes through refinement and addition of subclasses to achieve syntactic and semantic coherence among members of a class<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Verbnet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[VerbOcean]]<br /> | Lexical DB<br /> | Information Sciences Institute, University of Southern California<br /> | Broad-coverage semantic network of verbs<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|6<br /> |style="text-align: center;"| 7<br /> | [[VerbOcean - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://framenet.icsi.berkeley.edu/ FrameNet]<br /> | Lexical DB<br /> | ICSI (International Computer Science Institute) - Berkley University<br /> | Lexical resource for English words, based on frame semantics (valences) and supported by corpus evidence<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|2<br /> |style="text-align: center;"| 3<br /> | [[Framenet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/meyers/NomBank.html NomBank]<br /> | Lexical DB<br /> | New York University<br /> | Lexical resource containing syntactic frames for nouns, extracted from annotated corpora <br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[NomBank Resource - RTE Users|Users]] <br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://verbs.colorado.edu/~mpalmer/projects/ace.html PropBank]<br /> | Lexical DB<br /> | University of Colorado Boulder <br /> | Lexical resource containing syntactic frames for verbs, extracted from annotated corpora <br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[PropBank Resource - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/nomlex/index.html Nomlex] Plus<br /> | Lexical DB<br /> | New York University<br /> | Dictionary of English nominalizations: it describes the allowed complements for a nominalization and relates the nominal complements to the arguments of the corresponding verb<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Nomlex Plus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.cs.ualberta.ca/~lindek/downloads.htm Dekang Lin’s Thesaurus]<br /> | Thesaurus<br /> | University of Alberta<br /> | Thesaurus automatically constructed using a parsed corpus, based on distributional similarity scores<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 2<br /> | [[Dekang Lin’s Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://icon.shef.ac.uk/Moby/mthes.html Grady Ward's Moby Thesaurus]<br /> | Thesaurus<br /> | University of Sheffield<br /> | Thesaurus containing 30,260 root words, with 2,520,264 synonyms and related terms. Grady Ward placed this thesaurus in the public domain in 1996.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Grady Ward's Moby Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://en.wikipedia.org/wiki/Roget%27s_Thesaurus Roget's Thesaurus]<br /> | Thesaurus<br /> | Peter Mark Roget (Electronic version distributed by University of Chicago)<br /> | Roget's Thesaurus is a widely-used English thesaurus, created by Dr. Peter Mark Roget in 1805. The original edition had 15,000 words, and each new edition has been larger. The electronic edition ([http://machaut.uchicago.edu/rogets version 1.02]) is made available by University of Chicago.<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Roget's Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.wikipedia.org/ Wikipedia]<br /> | Encyclopedia<br /> | <br /> | Free encyclopedia. Used for extraction of lexical-semantic rules (from its more structured parts), named entity recognition, geographical information etc.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|6<br /> |style="text-align: center;"| 6<br /> | [[Wikipedia - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.umbel.org/ Umbel]<br /> | Ontology<br /> | Structured Dynamics LLC, Coralville, IA<br /> | UMBEL stands for Upper Mapping and Binding Exchange Layer and is a lightweight ontology structure for relating Web content and data to a standard set of subject concepts<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[Umbel - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.mpi-inf.mpg.de/yago-naga/yago/ YAGO]<br /> | Ontology<br /> | Max-Planck Institute for Informatics, Saarbrücken, Germany<br /> | Light-weight and extensible ontology. It contains more than 2 million entities and 20 million facts about these entities. The facts have been automatically extracted from Wikipedia and unified with WordNet.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[YAGO - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://dbpedia.org/About DBpedia]<br /> | Ontology<br /> | Open community project<br /> | DBpedia is a community effort to extract structured information from Wikipedia and to make this information available on the Web. The DBpedia knowledge base currently describes more than 2.9 million things in 91 different languages and consists of 479 million pieces of information.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[DBpedia - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[DIRT Paraphrase Collection]]<br /> | Collection of paraphrases<br /> | University of Alberta<br /> | DIRT (Discovery of Inference Rules from Text) is both an algorithm and a resulting knowledge collection. The DIRT knowledge collection is the output of the DIRT algorithm over a 1GB set of newspaper text.<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|4<br /> | style="text-align: center;"|3<br /> |style="text-align: center;"| 4<br /> | [[DIRT Paraphrase Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[TEASE]] Collection<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University<br /> | Output of the TEASE algorithm <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"| 0<br /> | [[Tease Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://badc.nerc.ac.uk/help/abbrevs.html BADC Acronym and Abbreviation List]<br /> | Word List<br /> | BADC (British Atmospheric Data Centre)<br /> | Acronym and Abbreviation List<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"| 0<br /> | [[BADC Acronym and Abbreviation List - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.acronym-guide.com/ Acronym Guide]<br /> | Word List<br /> | Acronym-Guide.com <br /> | Acronym and Abbreviation Lists for English, branched in thematic directories <br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|3<br /> |style="text-align: center;"| <br /> | [[Acronym Guide - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.ldc.upenn.edu/Catalog/CatalogEntry.jsp?catalogId=LDC2006T13 Web1T 5-grams]<br /> | Word list<br /> | Linguistic Data Consortium, University of Pennsylvania; Google Inc.<br /> | Data set containing English word n-grams and their observed frequency counts. The n-gram counts were generated from approximately 1 trillion word tokens of text from publicly accessible Web pages<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[Web1T - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.coli.uni-saarland.de/%7Erwang/resources/RTE3_RTE4_NGD.zip Normalized Google Distance (RTE3&RTE4)]<br /> | Word Pair Co-occurrence<br /> | Saarland University<br /> | Co-occurrence of the word pairs in RTE3 and RTE4 using Normalized Google Distance (Cilibrasi and Vitanyi, 2004). The word pairs are all the possible combinations of content words in T and H. In practice, we used [http://m.www.yahoo.com/ Yahoo!] as the search engine.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Normalized Google Distance (RTE3&RTE4)- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.coli.uni-saarland.de/%7Erwang/resources/RTE5_NGD.zip Normalized Google Distance (RTE5)]<br /> | Word Pair Co-occurrence<br /> | Saarland University<br /> | Co-occurrence of the word pairs in RTE3 and RTE4 using Normalized Google Distance (Cilibrasi and Vitanyi, 2004). The word pairs are all the possible combinations of content words in T and H. In practice, we used [http://m.www.yahoo.com/ Yahoo!] as the search engine.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Normalized Google Distance (RTE5)- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://geonames.usgs.gov/index.html GNIS - Geographic Names Information System]<br /> | Gazetteer<br /> | USGS (United States Geological Survey)<br /> | Database containing the Federal and national standard toponyms for USA, associated areas and Antarctica<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[GNIS - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.geonames.org/ Geonames]<br /> | Gazetteer<br /> | <br /> | Database containing eight million geographical names. It is integrating geographical data such as names of places in various languages, elevation, population and others from various sources.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[Geonames - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/paraphrase/ Sekine's Paraphrase Database]<br /> | Collection of paraphrases<br /> | Department of Computer Science, New York University <br /> | Data-base created using Sekine's method, NOT cleaned up by human. It includes 19,975 sets of paraphrases with 191,572 phrases.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[Sekine's Paraphrase Database - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://research.microsoft.com/research/downloads/Details/607D14D9-20CD-47E3-85BC-A2F65CD28042/Details.aspx Microsoft Research Paraphrase Corpus]<br /> | Collection of paraphrases<br /> | Microsoft Research<br /> | Text file containing 5800 pairs of sentences which have been extracted from news sources on the web, along with human annotations indicating whether each pair captures a paraphrase/semantic equivalence relationship.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[Microsoft Research Paraphrase Corpus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.cs.cornell.edu/~cristian/Without_a_doubt_-_Data.html Downward entailing operators] <br /> | Collection of entailing operators<br /> | Department of Computer Science, Cornell University, Ithaca NY<br /> | System output of an unsupervised algorithm recovering many Downward Entailing operators, like 'doubt'.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Downward entailing operators - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/WikiRules.html WikiRules!]<br /> | Lexical Reference rule-base<br /> | Bar-Ilan University<br /> | Extraction of about 8 million lexical reference rules from the text body (first sentence) and from metadata (links, redirects, parentheses) of Wikipedia. Provides better performance than other automatically constructed resources and comparable performance to WordNet. Offers complementary knowledge to WordNet. <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[WikiRules! - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://www.cs.utexas.edu/users/pclark/dart/ DART]<br /> | Collection of "world knowledge" propositions<br /> | Boeing Research and Technology<br /> | 23 million tuples such as "airplanes can fly to airports", "rivers can flood" collected from abstracted parse trees.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[DART - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://cs.biu.ac.il/~nlp/download/FRED/latest/FRED.zip FRED]<br /> | FrameNet-derived entailment rule-base<br /> | Bar-Ilan University<br /> | This package contains the outputs of the FRED algorithm, an algorithm which extracts entailment rules from FrameNet.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[FRED - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/DIRECT.html DIRECT]<br /> | Directional Distributional Term-Similarity Resource<br /> | Bar-Ilan University<br /> | This is a resource of directional distributional term-similarity rules (mostly lexical entailment rules) automatically extracted using the inclusion relation as described in (Kotlerman et.al., ACL-09).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[DIRECT - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/binaryDirt.html binaryDIRT]<br /> | Entailment rules between binary templates using DIRT algorithm<br /> | Bar-Ilan University<br /> | This resource contains entailment rules over binary templates learned over the Reuters corpus using<br /> the DIRT algorithm of Lin and Pantel.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[BinaryDIRT- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/unaryBInc.html unaryBInc]<br /> | Entailment rules between unary templates using BInc algorithm<br /> | Bar-Ilan University<br /> | This resource contains entailment rules over unary templates learned over the Reuters corpus using<br /> the BInc algorithm of Szpektor and Dagan (2008).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[unaryBInc- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''New resource''<br /> | <br /> | <br /> | ''Participants are encouraged to contribute''<br /> | style="text-align: center;"| <br /> | style="text-align: center;"| <br /> | style="text-align: center;"| <br /> |style="text-align: center;"|<br /> | [[New Resource2 - RTE Users|Users]]<br /> <br /> |}<br /> <br><br /> <br><br /> <br /> === Not available Resources ===<br /> The following table lists the unpublished resources used by RTE participants. Some of them have been developed by Users themselves specifically for RTE. Interested people may turn to authors to obtain further information.<br /> <br><br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! width="80"|Resource<br /> ! width="50"|Type<br /> ! width="180"|Author<br /> ! class="unsortable" width="500"|Brief description<br /> ! width="30"|<small>PAST Users</small><br /> ! width="30"|<small>RTE4 Users</small><br /> ! width="30"|<small>RTE5 Users</small><br /> ! width="30"|<small>RTE6 Users</small><br /> ! class="unsortable" width="30"|<small>Usage info</small><br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.parc.com/ PARC] Polarity Lexicon<br /> | Lexical DB<br /> | PARC - Palo Alto Research Center<br /> | Verbs classification with respect to semantic polarity<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Parc Polarity Lexicon - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Gazetteer from [http://trec.nist.gov/ TREC]<br /> | Gazetteer<br /> | NIST - National Institute of Standards and Technology<br /> | Cities and other geographical names<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Gazetteer from TREC - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DFKI Geographic Ontology<br>''(to be released)''<br /> | Ontology<br /> | DFKI - German Research Center for Artificial Intelligence<br /> | Ontology containing geographic terms and two kinds of relations: the directional ''part-of'' relation, and the ''equal'' relation for synonyms and abbreviations of the same geographic area (e.g ''the United Kingdom'', ''the UK'', ''Great Britain'', etc.) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Geographic Ontology - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Geo''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | Meronymy entailment rules, based on TREC’s TIPSTER gazetteer.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Geo - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Regex''<br /> | Collection of Entailment rules <br /> | Bar-Ilan University; Tel-Aviv University<br /> | Small set of entailment rules based on regular expressions, intended to address lexical variability involving temporal phrases<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Regex - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Syntactic rule base <br>''(to be released)''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | A manually-composed collection of entailment rules which define parse tree transformations. The rules cover generic syntactic phenomena such as appositions, conjunctions, passive, relative clause, etc. (Bar-Haim et al., AAAI-07) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Syntactic rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Polarity rule base <br>''(to be released)''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | A manually-composed collection of entailment rules which detect predicates whose polarity is negative (e.g. didn't dance) or unknown (e.g. plans to dance). The rules capture diverse phenomena that affect polarity, e.g. verbal negation, modal verbs, conditionals, and certain verbs that induce negative or "unknown" polarity context. The latter were taken mainly from VerbNet. Extends a resource described in (Bar-Haim et al., AAAI-07) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0 <br /> | style="text-align: center;"|<br /> | [[Polarity rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Lexical-Syntactic rule base<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | Extract lexical-syntactic entailment rules for predicates (verbal and nominal), including argument mapping. The resource is based on WordNet, Nomlex-Plus and Unary DIRT (Szpektor and Dagan, Coling 08)<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Lexical-Syntactic rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | OPENU Collection<br /> | Collection of Entailment Rules and Patterns<br /> | Open University<br /> | Collections of rules, patterns etc. for RTE purpose, extracted from Reuter corpus parsed using Minipar.<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"| <br /> | [[OPENU Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Abbr''<br /> | Collection of rules for abbreviation<br /> | Bar-Ilan University; Tel-Aviv University<br /> | 2000 Abbreviation rules, extracted from [http://badc.nerc.ac.uk/help/abbrevs.html BADC] and [http://www.acronym-guide.com/ Acronym Guide]<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Abbr - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Negation_list<br /> | Negation rules<br /> | „Al. I. Cuza“ University, Iasi, Romania <br /> | List of negative terms and words (verbs, adjectives, nouns) affecting modality or factuality of a infinitive verb preceded by the particle "to" (e.g. "believe","necessary", "attempt")<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1 <br /> | style="text-align: center;"|<br /> | [[UAIC Negation_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DLSIUAES Negation_list<br /> | List of negative terms<br /> | University of Alicante<br /> | Basic list of negative terms.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"| <br /> | [[DLSIUAES Negation_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Quantifier_list<br /> | List of quantifiers<br /> | „Al. I. Cuza“ University, Iasi, Romania <br /> | List of quantifiers affecting entailment judgment. The quantifiers are taken from a list which contains expressions like “more than”, “less than”, or words such as “over”, “under”, etc.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"| <br /> | [[UAIC Quantifier_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | FBKirst StopWord list<br /> | List of frequent words<br /> | FBK-Irst;<br/>University of Trento - Italy <br /> | A list of the 572 most frequent English words.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"| <br /> | [[FBKirst StopWord list - RTE Users|Users]]<br /> <br /> |}<br /> <br><br /> <br /> ==Footnotes==<br /> <references/></div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE6_-_Ablation_Tests&diff=8665 RTE6 - Ablation Tests 2011-02-03T16:22:03Z

<p>Amarchetti: </p> <hr /> <div>The following table lists the results of the ablation tests submitted by participants to RTE6 .<br><br /> The exploratory effort started in RTE5 as a mandatory track, was not only reiterated in RTE6, but also extended to tools.<br /> The first column contains the specific resources which have been ablated.<br>The second column lists the Team Run in the form ''[name_of_the_Team][number_of_the_submitted_run].[submission_task]'' (e.g. BIU1.2way, Boeing3.3way).<br>The third column presents the normalized difference between the accuracy of the complete system run and the accuracy of the ablation run (i.e. the output of the complete system without the ablated resource), showing the impact of the resource on the performance of the system.<br><br /> The fourth column contains a brief description of the specific usage of the resource. It is based on the information provided both in the "readme" files submitted together with the ablation tests and in the system reports published in the RTE6 proceedings.<br><br /> If the ablated resource is highlighted in yellow, it is a tool, otherwise is a knowledge resource.<br><br /> <br /> Participants are kindly invited to check if all the inserted information is correct and complete.<br /> <br /> <br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! Ablated Component<br /> ! Ablation Run<ref>For further information about participants, click here: [[RTE Challenges - Data about participants]]</ref><br /> ! <small>Resource impact - F1 </small><br /> ! Resource Usage Description<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | BIU1_abl-1<br /> | style="text-align: center;"| 0.9<br /> | style="text-align: center;"| No Word-Net. On Dev set: 39.18% (compared to 40.73% when WN is used)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | CatVar<br /> | BIU1_abl-2<br /> | style="text-align: center;"| 0.63<br /> | style="text-align: center;"| No CatVar. On Dev set achieved about 40.20% (compared to 40.73% when CatVar is used)<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Coreference resolver<br /> | BIU1_abl-3<br /> | style="text-align: center;"| -0.88<br /> | style="text-align: center;"| No coreference resolver<br /> On Dev set 41.62% (Compared to 40.73% when Coreference resolver is used).<br /> This ablation test is an unusual ablation test, since it shows that the co-reference resolution component has a negative impact.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DIRT<br /> | Boeing1_abl-1<br /> | style="text-align: center;"| 3.97<br /> | style="text-align: center;"| DIRT removed<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | Boeing1_abl-2<br /> | style="text-align: center;"| 4.42<br /> | style="text-align: center;"| No WordNet<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Name Normalization<br /> | budapestcad2_abl-2<br /> | style="text-align: center;"| 0.65<br /> | style="text-align: center;"| no name normalization was performed (e.g. George W. Bush -> Bush).<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entities Recognition<br /> | budapestcad2_abl-3<br /> | style="text-align: center;"| -1.23<br /> | style="text-align: center;"| no NER<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | budapestcad2_abl-4<br /> | style="text-align: center;"| -1.11<br /> | style="text-align: center;"| No WordNet. (In the original run, WordNet was used to find the synonyms of words in the triplets, and additional triplets were generated from all possible combinations.)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb1_abl-1<br /> | style="text-align: center;"| 8.68<br /> | style="text-align: center;"| Wordnet is albated in this test.No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | deb_iitb1_abl-2<br /> | style="text-align: center;"| 1.87<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb2_abl-1<br /> | style="text-align: center;"| 7.9<br /> | style="text-align: center;"| Wordnet is albated in this test.No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | deb_iitb2_abl-2<br /> | style="text-align: center;"| 0.94<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb3_abl-1<br /> | style="text-align: center;"| 11.43<br /> | style="text-align: center;"| Wordnet is albated in this test. No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb3_abl-2<br /> | style="text-align: center;"| 2.54<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | POS-Tagger<br /> | DFKI1_abl-4<br /> | style="text-align: center;"| 4.99<br /> | style="text-align: center;"| No wordform/POS-tags included for the comparison.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | POS-Tagger<br /> | DFKI1_abl-6<br /> | style="text-align: center;"| 2.22<br /> | style="text-align: center;"| No named entity recognition for the comparison.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | DFKI1_abl-7<br /> | style="text-align: center;"| -0.23<br /> | style="text-align: center;"| No WordNet similarity for the comparison.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Coreference resolver<br /> | DFKI1_Main<br /> | style="text-align: center;"| -1.54<br /> | style="text-align: center;"| Coreference resolution used for the comparison.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | DirRelCond23_abl-1<br /> | style="text-align: center;"| 8.43<br /> | style="text-align: center;"| WordNet removed. Only basic word comparison used instead of word relations.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -23.91<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from Wikipedia<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -3.58<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from Wikipedia with probability above 0.7<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Proximity similarity dictionary of Dekang Lin<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -7.79<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from proximity similarity dictionary of Dekang Lin<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -3.21<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from WordNet<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -2.08<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from WordNet with probability above 0.7<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -4<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from Verbocean<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Dependency similarity dictionary of Dekang Lin<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -13.56<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from dependency similarity dictionary of Dekang Lin<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Dictionary of Named Entities Acronyms and Synonyms <br /> | IKOMA2_abl-3<br /> | style="text-align: center;"| -0.76<br /> | style="text-align: center;"| Remove synonym dictionaries: as acronym dictionary constructed automatically from the corpus and a synonym dictionary that contains geographical terms.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | JU_CSE_TAC1_abl-1<br /> | style="text-align: center;"| 13.29<br /> | style="text-align: center;"| The Run-1 is based on the composition of lexical based RTE methods and Syntactic RTE Method. The lexical based RTE methods are: WordNet based unigram match, bigram match, longest common sub-sequence, skip-gram and stemming. Here we ablated the WordNet based unigram match only.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | JU_CSE_TAC2_abl-1<br /> | style="text-align: center;"| 10.19<br /> | style="text-align: center;"| The Run-2 is based on the composition of lexical based RTE methods, Syntactic RTE Method, Chunk and Named Entity Methods. The lexical based RTE methods are: WordNet based unigram match, bigram match, longest common sub-sequence, skip-gram and stemming. Here we ablated the WordNet based unigram match only.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | JU_CSE_TAC3_abl-1<br /> | style="text-align: center;"| 3.86<br /> | style="text-align: center;"| The Run-3 is based on the Support Vector Machine that uses twenty five features for lexical similarity, the output tag from a rule based syntactic two-way TE system as feature, and output from a rule based Chunk Module and Named Entity Module. The important lexical features that are used in the present system are: WordNet based unigram match, bigram match, longest common sub-sequence, skip-gram, stemming and lexical distance (17 features). Here we ablated the WordNet based unigram match only.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | LingPipe co-reference<br /> | PKUTM2_abl-1<br /> | style="text-align: center;"| 0.17<br /> | style="text-align: center;"| Lingpipe co-reference are removed, the experiment was based on named-entity, wordnet, verbocean<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | PKUTM2_abl-2<br /> | style="text-align: center;"| 1.02<br /> | style="text-align: center;"| Verbocean are removed, the experiment was based on named-entity, wordnet, co-reference<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | LingPipe Named Entities<br /> | PKUTM2_abl-3<br /> | style="text-align: center;"| 13.84<br /> | style="text-align: center;"| Lingpipe named-entity are removed, the experiment was based on wordnet, co-reference, verbocean<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | saicnlp1_abl-1<br /> | style="text-align: center;"| -0.02<br /> | style="text-align: center;"| Ablation run, with WordNet stubbed.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Shalmaneser Parser<br /> | Sangyan1_abl-1<br /> | style="text-align: center;"| -1.76<br /> | style="text-align: center;"| This ablation run was executed with the Shalmaneser parser ablated from our system. Our system uses the Shalmaneser Parser for Framenet frame assignment.<br /> Our system required minor tweaking of the source code in order ablate Shalmaneser Parser.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | Sangyan1_abl-2<br /> | style="text-align: center;"| -0.39<br /> | style="text-align: center;"| This ablation run was executed with the WordNet ablated from our system. Our system uses the Wordnet for Word mathcing.<br /> Our system required minor tweaking of the source code in order ablate Wordnet.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | Sangyan1_abl-3<br /> | style="text-align: center;"| 0.07<br /> | style="text-align: center;"| This ablation run was executed with the Verb Ocean ablated from our system. Our system uses the Verb Ocean resource for detection of antonyms. Our system required minor tweaking of the source code in order ablate Verb Ocean resource.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entities Recognition<br /> | SINAI1_abl-1<br /> | style="text-align: center;"| 7.62<br /> | style="text-align: center;"| Only PPVs on WN 1.7<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wordnet<br /> | SINAI1_abl-2<br /> | style="text-align: center;"| 15.87<br /> | style="text-align: center;"| Only NEs on WN 1.7<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entities Recognition<br /> | SINAI2_abl-1<br /> | style="text-align: center;"| 20.25<br /> | style="text-align: center;"| Only PPVs on WN 3.0<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wordnet<br /> | SINAI2_abl-2<br /> | style="text-align: center;"| 18.28<br /> | style="text-align: center;"| Only NEs on WN 3.0<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wordnet<br /> | SJTU_CIT3_abl-1<br /> | style="text-align: center;"| 0.03<br /> | style="text-align: center;"| The resource that has been ablated is WordNet.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | SJTU_CIT3_abl-2<br /> | style="text-align: center;"| 4.7<br /> | style="text-align: center;"| The resource that has been ablated is Wikipedia.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | SJTU_CIT3_abl-3<br /> | style="text-align: center;"| -1.15<br /> | style="text-align: center;"| The resource that has been ablated is VerbOcean.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | UAIC20101_abl-1<br /> | style="text-align: center;"| 1.93<br /> | style="text-align: center;"| Run 1 with no BK<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DIRT<br /> | UAIC20101_abl-2<br /> | style="text-align: center;"| -1.56<br /> | style="text-align: center;"| Run 1 without DIRT.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | UAIC20102_abl-1<br /> | style="text-align: center;"| 1.08<br /> | style="text-align: center;"| Run 2 with no BK<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DIRT<br /> | UAIC20102_abl-2<br /> | style="text-align: center;"| -0.72<br /> | style="text-align: center;"| Run 2 without DIRT.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | UAIC20103_abl-1<br /> | style="text-align: center;"| 1.3<br /> | style="text-align: center;"| Run 3 with no BK<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DIRT<br /> | UAIC20103_abl-2<br /> | style="text-align: center;"| -0.98<br /> | style="text-align: center;"| Run 3 without DIRT.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | UB.dmirg1_abl-1<br /> | style="text-align: center;"| 1.7<br /> | style="text-align: center;"| WordNet has been removed. No system changes required.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Framenet<br /> | UB.dmirg1_abl-2<br /> | style="text-align: center;"| -0.1<br /> | style="text-align: center;"| FrameNet has been removed. No system changes required.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | UB.dmirg2_abl-1<br /> | style="text-align: center;"| 0.6<br /> | style="text-align: center;"| WordNet has been removed. No system changes required.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Framenet<br /> | UB.dmirg2_abl-2<br /> | style="text-align: center;"| -1.84<br /> | style="text-align: center;"| FrameNet has been removed. No system changes required.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | UB.dmirg3_abl-1<br /> | style="text-align: center;"| -0.78<br /> | style="text-align: center;"| WordNet has been removed. No system changes required.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Framenet<br /> | UB.dmirg3_abl-2<br /> | style="text-align: center;"| -1.81<br /> | style="text-align: center;"| FrameNet has been removed. No system changes required.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | UIUC1_abl-1<br /> | style="text-align: center;"| -3.07<br /> | style="text-align: center;"| Abalation 1 - Removed wordnet (No locations and no synset expansion for non-entity terms). Threshold tuned on development collection to achieve similar precision and recall as Run 1.<br /> <br /> |}<br /> <br><br /> ==Footnotes==<br /> <references /><br /> <br /> <br /> Return to [[RTE Knowledge Resources]]</div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE6_-_Ablation_Tests&diff=8664 RTE6 - Ablation Tests 2011-02-03T16:17:45Z

<p>Amarchetti: </p> <hr /> <div>The following table lists the results of the ablation tests submitted by participants to RTE6 .<br><br /> The exploratory effort started in RTE5 as a mandatory track, was not only reiterated in RTE6, but also extended to tools.<br /> The first column contains the specific resources which have been ablated.<br>The second column lists the Team Run in the form ''[name_of_the_Team][number_of_the_submitted_run].[submission_task]'' (e.g. BIU1.2way, Boeing3.3way).<br>The third column presents the normalized difference between the accuracy of the complete system run and the accuracy of the ablation run (i.e. the output of the complete system without the ablated resource), showing the impact of the resource on the performance of the system.<br><br /> The fourth column contains a brief description of the specific usage of the resource. It is based on the information provided both in the "readme" files submitted together with the ablation tests and in the system reports published in the RTE6 proceedings.<br><br /> If the ablated resource is highlighted in yellow, it is a tool, otherwise is a knowledge resource.<br><br /> <br /> Participants are kindly invited to check if all the inserted information is correct and complete.<br /> <br /> <br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! Ablated Component<br /> ! Ablation Run<ref>For further information about participants, click here: [[RTE Challenges - Data about participants]]</ref><br /> ! <small>Resource impact - F1 </small><br /> ! Resource Usage Description<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | BIU1_abl-1<br /> | style="text-align: center;"| 0.9<br /> | style="text-align: center;"| No Word-Net. On Dev set: 39.18% (compared to 40.73% when WN is used)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | CatVar<br /> | BIU1_abl-2<br /> | style="text-align: center;"| 0.63<br /> | style="text-align: center;"| No CatVar. On Dev set achieved about 40.20% (compared to 40.73% when CatVar is used)<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Coreference resolver<br /> | BIU1_abl-3<br /> | style="text-align: center;"| -0.88<br /> | style="text-align: center;"| No coreference resolver<br /> On Dev set 41.62% (Compared to 40.73% when Coreference resolver is used).<br /> This ablation test is an unusual ablation test, since it shows that the co-reference resolution component has a negative impact.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DIRT<br /> | Boeing1_abl-1<br /> | style="text-align: center;"| 3.97<br /> | style="text-align: center;"| DIRT removed<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | Boeing1_abl-2<br /> | style="text-align: center;"| 4.42<br /> | style="text-align: center;"| No WordNet<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Name Normalization<br /> | budapestcad2_abl-2<br /> | style="text-align: center;"| 0.65<br /> | style="text-align: center;"| no name normalization was performed (e.g. George W. Bush -> Bush).<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entities Recognition<br /> | budapestcad2_abl-3<br /> | style="text-align: center;"| -1.23<br /> | style="text-align: center;"| no NER<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | budapestcad2_abl-4<br /> | style="text-align: center;"| -1.11<br /> | style="text-align: center;"| No WordNet. (In the original run, WordNet was used to find the synonyms of words in the triplets, and additional triplets were generated from all possible combinations.)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb1_abl-1<br /> | style="text-align: center;"| 8.68<br /> | style="text-align: center;"| Wordnet is albated in this test.No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | deb_iitb1_abl-2<br /> | style="text-align: center;"| 1.87<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb2_abl-1<br /> | style="text-align: center;"| 7.9<br /> | style="text-align: center;"| Wordnet is albated in this test.No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | deb_iitb2_abl-2<br /> | style="text-align: center;"| 0.94<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb3_abl-1<br /> | style="text-align: center;"| 11.43<br /> | style="text-align: center;"| Wordnet is albated in this test. No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb3_abl-2<br /> | style="text-align: center;"| 2.54<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | POS-Tagger<br /> | DFKI1_abl-4<br /> | style="text-align: center;"| 4.99<br /> | style="text-align: center;"| No wordform/POS-tags included for the comparison.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | POS-Tagger<br /> | DFKI1_abl-6<br /> | style="text-align: center;"| 2.22<br /> | style="text-align: center;"| No named entity recognition for the comparison.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | DFKI1_abl-7<br /> | style="text-align: center;"| -0.23<br /> | style="text-align: center;"| No WordNet similarity for the comparison.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Coreference resolver<br /> | DFKI1_Main<br /> | style="text-align: center;"| -1.54<br /> | style="text-align: center;"| Coreference resolution used for the comparison.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | DirRelCond23_abl-1<br /> | style="text-align: center;"| 8.43<br /> | style="text-align: center;"| WordNet removed. Only basic word comparison used instead of word relations.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -23.91<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from Wikipedia<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -3.58<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from Wikipedia with probability above 0.7<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Proximity similarity dictionary of Dekang Lin<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -7.79<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from proximity similarity dictionary of Dekang Lin<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -3.21<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from WordNet<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -2.08<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from WordNet with probability above 0.7<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -4<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from Verbocean<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Dependency similarity dictionary of Dekang Lin<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -13.56<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from dependency similarity dictionary of Dekang Lin<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Dictionary of Named Entities Acronyms and Synonyms <br /> | IKOMA2_abl-3<br /> | style="text-align: center;"| -0.76<br /> | style="text-align: center;"| Remove synonym dictionaries: as acronym dictionary constructed automatically from the corpus and a synonym dictionary that contains geographical terms.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | JU_CSE_TAC1_abl-1<br /> | style="text-align: center;"| 13.29<br /> | style="text-align: center;"| The Run-1 is based on the composition of lexical based RTE methods and Syntactic RTE Method. The lexical based RTE methods are: WordNet based unigram match, bigram match, longest common sub-sequence, skip-gram and stemming. Here we ablated the WordNet based unigram match only.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | JU_CSE_TAC2_abl-1<br /> | style="text-align: center;"| 10.19<br /> | style="text-align: center;"| The Run-2 is based on the composition of lexical based RTE methods, Syntactic RTE Method, Chunk and Named Entity Methods. The lexical based RTE methods are: WordNet based unigram match, bigram match, longest common sub-sequence, skip-gram and stemming. Here we ablated the WordNet based unigram match only.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | JU_CSE_TAC3_abl-1<br /> | style="text-align: center;"| 3.86<br /> | style="text-align: center;"| The Run-3 is based on the Support Vector Machine that uses twenty five features for lexical similarity, the output tag from a rule based syntactic two-way TE system as feature, and output from a rule based Chunk Module and Named Entity Module. The important lexical features that are used in the present system are: WordNet based unigram match, bigram match, longest common sub-sequence, skip-gram, stemming and lexical distance (17 features). Here we ablated the WordNet based unigram match only.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | LingPipe co-reference<br /> | PKUTM2_abl-1<br /> | style="text-align: center;"| 0.17<br /> | style="text-align: center;"| Lingpipe co-reference are removed, the experiment was based on named-entity, wordnet, verbocean<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | PKUTM2_abl-2<br /> | style="text-align: center;"| 1.02<br /> | style="text-align: center;"| Verbocean are removed, the experiment was based on named-entity, wordnet, co-reference<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | LingPipe Named Entities<br /> | PKUTM2_abl-3<br /> | style="text-align: center;"| 13.84<br /> | style="text-align: center;"| Lingpipe named-entity are removed, the experiment was based on wordnet, co-reference, verbocean<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | saicnlp1_abl-1<br /> | style="text-align: center;"| -0.02<br /> | style="text-align: center;"| Ablation run, with WordNet stubbed.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Shalmanesar Parser<br /> | Sangyan1_abl-1<br /> | style="text-align: center;"| -1.76<br /> | style="text-align: center;"| This ablation run was executed with the Shalmanesar parser ablated from our system. Our system uses the Shalmanesar Parser for Framenet frame assignment.<br /> Our system required minor tweaking of the source code in order ablate Shalmanesar Parser.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | Sangyan1_abl-2<br /> | style="text-align: center;"| -0.39<br /> | style="text-align: center;"| This ablation run was executed with the WordNet ablated from our system. Our system uses the Wordnet for Word mathcing.<br /> Our system required minor tweaking of the source code in order ablate Wordnet.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | Sangyan1_abl-3<br /> | style="text-align: center;"| 0.07<br /> | style="text-align: center;"| This ablation run was executed with the Verb Ocean ablated from our system. Our system uses the Verb Ocean resource for detection of antonyms. Our system required minor tweaking of the source code in order ablate Verb Ocean resource.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entities Recognition<br /> | SINAI1_abl-1<br /> | style="text-align: center;"| 7.62<br /> | style="text-align: center;"| Only PPVs on WN 1.7<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wordnet<br /> | SINAI1_abl-2<br /> | style="text-align: center;"| 15.87<br /> | style="text-align: center;"| Only NEs on WN 1.7<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entities Recognition<br /> | SINAI2_abl-1<br /> | style="text-align: center;"| 20.25<br /> | style="text-align: center;"| Only PPVs on WN 3.0<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wordnet<br /> | SINAI2_abl-2<br /> | style="text-align: center;"| 18.28<br /> | style="text-align: center;"| Only NEs on WN 3.0<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wordnet<br /> | SJTU_CIT3_abl-1<br /> | style="text-align: center;"| 0.03<br /> | style="text-align: center;"| The resource that has been ablated is WordNet.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | SJTU_CIT3_abl-2<br /> | style="text-align: center;"| 4.7<br /> | style="text-align: center;"| The resource that has been ablated is Wikipedia.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | SJTU_CIT3_abl-3<br /> | style="text-align: center;"| -1.15<br /> | style="text-align: center;"| The resource that has been ablated is VerbOcean.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | UAIC20101_abl-1<br /> | style="text-align: center;"| 1.93<br /> | style="text-align: center;"| Run 1 with no BK<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DIRT<br /> | UAIC20101_abl-2<br /> | style="text-align: center;"| -1.56<br /> | style="text-align: center;"| Run 1 without DIRT.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | UAIC20102_abl-1<br /> | style="text-align: center;"| 1.08<br /> | style="text-align: center;"| Run 2 with no BK<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DIRT<br /> | UAIC20102_abl-2<br /> | style="text-align: center;"| -0.72<br /> | style="text-align: center;"| Run 2 without DIRT.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | UAIC20103_abl-1<br /> | style="text-align: center;"| 1.3<br /> | style="text-align: center;"| Run 3 with no BK<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DIRT<br /> | UAIC20103_abl-2<br /> | style="text-align: center;"| -0.98<br /> | style="text-align: center;"| Run 3 without DIRT.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | UB.dmirg1_abl-1<br /> | style="text-align: center;"| 1.7<br /> | style="text-align: center;"| WordNet has been removed. No system changes required.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Framenet<br /> | UB.dmirg1_abl-2<br /> | style="text-align: center;"| -0.1<br /> | style="text-align: center;"| FrameNet has been removed. No system changes required.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | UB.dmirg2_abl-1<br /> | style="text-align: center;"| 0.6<br /> | style="text-align: center;"| WordNet has been removed. No system changes required.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Framenet<br /> | UB.dmirg2_abl-2<br /> | style="text-align: center;"| -1.84<br /> | style="text-align: center;"| FrameNet has been removed. No system changes required.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | UB.dmirg3_abl-1<br /> | style="text-align: center;"| -0.78<br /> | style="text-align: center;"| WordNet has been removed. No system changes required.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Framenet<br /> | UB.dmirg3_abl-2<br /> | style="text-align: center;"| -1.81<br /> | style="text-align: center;"| FrameNet has been removed. No system changes required.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | UIUC1_abl-1<br /> | style="text-align: center;"| -3.07<br /> | style="text-align: center;"| Abalation 1 - Removed wordnet (No locations and no synset expansion for non-entity terms). Threshold tuned on development collection to achieve similar precision and recall as Run 1.<br /> <br /> |}<br /> <br><br /> ==Footnotes==<br /> <references /><br /> <br /> <br /> Return to [[RTE Knowledge Resources]]</div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE6_-_Ablation_Tests&diff=8663 RTE6 - Ablation Tests 2011-02-03T15:41:27Z

<p>Amarchetti: </p> <hr /> <div>The following table lists the results of the ablation tests (a mandatory track since the RTE5 campaign), submitted by participants to RTE6 .<br><br /> The first column contains the specific resources which have been ablated.<br>The second column lists the Team Run in the form ''[name_of_the_Team][number_of_the_submitted_run].[submission_task]'' (e.g. BIU1.2way, Boeing3.3way).<br>The third column presents the normalized difference between the accuracy of the complete system run and the accuracy of the ablation run (i.e. the output of the complete system without the ablated resource), showing the impact of the resource on the performance of the system.<br><br /> The fourth column contains a brief description of the specific usage of the resource. It is based on the information provided both in the "readme" files submitted together with the ablation tests and in the system reports published in the RTE6 proceedings.<br><br><br /> <br /> <br /> <br /> Participants are kindly invited to check if all the inserted information is correct and complete.<br /> <br /> <br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! Ablated Component<br /> ! Ablation Run<ref>For further information about participants, click here: [[RTE Challenges - Data about participants]]</ref><br /> ! <small>Resource impact - F1 </small><br /> ! Resource Usage Description<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | BIU1_abl-1<br /> | style="text-align: center;"| 0.9<br /> | style="text-align: center;"| No Word-Net. On Dev set: 39.18% (compared to 40.73% when WN is used)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | CatVar<br /> | BIU1_abl-2<br /> | style="text-align: center;"| 0.63<br /> | style="text-align: center;"| No CatVar. On Dev set achieved about 40.20% (compared to 40.73% when CatVar is used)<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Coreference resolver<br /> | BIU1_abl-3<br /> | style="text-align: center;"| -0.88<br /> | style="text-align: center;"| No coreference resolver<br /> On Dev set 41.62% (Compared to 40.73% when Coreference resolver is used).<br /> This ablation test is an unusual ablation test, since it shows that the co-reference resolution component has a negative impact.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DIRT<br /> | Boeing1_abl-1<br /> | style="text-align: center;"| 3.97<br /> | style="text-align: center;"| DIRT removed<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | Boeing1_abl-2<br /> | style="text-align: center;"| 4.42<br /> | style="text-align: center;"| No WordNet<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Name Normalization<br /> | budapestcad2_abl-2<br /> | style="text-align: center;"| 0.65<br /> | style="text-align: center;"| no name normalization was performed (e.g. George W. Bush -> Bush).<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entities Recognition<br /> | budapestcad2_abl-3<br /> | style="text-align: center;"| -1.23<br /> | style="text-align: center;"| no NER<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | budapestcad2_abl-4<br /> | style="text-align: center;"| -1.11<br /> | style="text-align: center;"| No WordNet. (In the original run, WordNet was used to find the synonyms of words in the triplets, and additional triplets were generated from all possible combinations.)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb1_abl-1<br /> | style="text-align: center;"| 8.68<br /> | style="text-align: center;"| Wordnet is albated in this test.No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | deb_iitb1_abl-2<br /> | style="text-align: center;"| 1.87<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb2_abl-1<br /> | style="text-align: center;"| 7.9<br /> | style="text-align: center;"| Wordnet is albated in this test.No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | deb_iitb2_abl-2<br /> | style="text-align: center;"| 0.94<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb3_abl-1<br /> | style="text-align: center;"| 11.43<br /> | style="text-align: center;"| Wordnet is albated in this test. No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb3_abl-2<br /> | style="text-align: center;"| 2.54<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | POS-Tagger<br /> | DFKI1_abl-4<br /> | style="text-align: center;"| 4.99<br /> | style="text-align: center;"| No wordform/POS-tags included for the comparison.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | POS-Tagger<br /> | DFKI1_abl-6<br /> | style="text-align: center;"| 2.22<br /> | style="text-align: center;"| No named entity recognition for the comparison.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | DFKI1_abl-7<br /> | style="text-align: center;"| -0.23<br /> | style="text-align: center;"| No WordNet similarity for the comparison.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Coreference resolver<br /> | DFKI1_Main<br /> | style="text-align: center;"| -1.54<br /> | style="text-align: center;"| Coreference resolution used for the comparison.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | DirRelCond23_abl-1<br /> | style="text-align: center;"| 8.43<br /> | style="text-align: center;"| WordNet removed. Only basic word comparison used instead of word relations.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -23.91<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from Wikipedia<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -3.58<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from Wikipedia with probability above 0.7<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Proximity similarity dictionary of Dekang Lin<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -7.79<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from proximity similarity dictionary of Dekang Lin<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -3.21<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from WordNet<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -2.08<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from WordNet with probability above 0.7<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -4<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from Verbocean<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Dependency similarity dictionary of Dekang Lin<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -13.56<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from dependency similarity dictionary of Dekang Lin<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Dictionary of Named Entities Acronyms and Synonyms <br /> | IKOMA2_abl-3<br /> | style="text-align: center;"| -0.76<br /> | style="text-align: center;"| Remove synonym dictionaries: as acronym dictionary constructed automatically from the corpus and a synonym dictionary that contains geographical terms.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | JU_CSE_TAC1_abl-1<br /> | style="text-align: center;"| 13.29<br /> | style="text-align: center;"| The Run-1 is based on the composition of lexical based RTE methods and Syntactic RTE Method. The lexical based RTE methods are: WordNet based unigram match, bigram match, longest common sub-sequence, skip-gram and stemming. Here we ablated the WordNet based unigram match only.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | JU_CSE_TAC2_abl-1<br /> | style="text-align: center;"| 10.19<br /> | style="text-align: center;"| The Run-2 is based on the composition of lexical based RTE methods, Syntactic RTE Method, Chunk and Named Entity Methods. The lexical based RTE methods are: WordNet based unigram match, bigram match, longest common sub-sequence, skip-gram and stemming. Here we ablated the WordNet based unigram match only.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | JU_CSE_TAC3_abl-1<br /> | style="text-align: center;"| 3.86<br /> | style="text-align: center;"| The Run-3 is based on the Support Vector Machine that uses twenty five features for lexical similarity, the output tag from a rule based syntactic two-way TE system as feature, and output from a rule based Chunk Module and Named Entity Module. The important lexical features that are used in the present system are: WordNet based unigram match, bigram match, longest common sub-sequence, skip-gram, stemming and lexical distance (17 features). Here we ablated the WordNet based unigram match only.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | LingPipe co-reference<br /> | PKUTM2_abl-1<br /> | style="text-align: center;"| 0.17<br /> | style="text-align: center;"| Lingpipe co-reference are removed, the experiment was based on named-entity, wordnet, verbocean<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | PKUTM2_abl-2<br /> | style="text-align: center;"| 1.02<br /> | style="text-align: center;"| Verbocean are removed, the experiment was based on named-entity, wordnet, co-reference<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | LingPipe Named Entities<br /> | PKUTM2_abl-3<br /> | style="text-align: center;"| 13.84<br /> | style="text-align: center;"| Lingpipe named-entity are removed, the experiment was based on wordnet, co-reference, verbocean<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | saicnlp1_abl-1<br /> | style="text-align: center;"| -0.02<br /> | style="text-align: center;"| Ablation run, with WordNet stubbed.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Shalmanesar Parser<br /> | Sangyan1_abl-1<br /> | style="text-align: center;"| -1.76<br /> | style="text-align: center;"| This ablation run was executed with the Shalmanesar parser ablated from our system. Our system uses the Shalmanesar Parser for Framenet frame assignment.<br /> Our system required minor tweaking of the source code in order ablate Shalmanesar Parser.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | Sangyan1_abl-2<br /> | style="text-align: center;"| -0.39<br /> | style="text-align: center;"| This ablation run was executed with the WordNet ablated from our system. Our system uses the Wordnet for Word mathcing.<br /> Our system required minor tweaking of the source code in order ablate Wordnet.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | Sangyan1_abl-3<br /> | style="text-align: center;"| 0.07<br /> | style="text-align: center;"| This ablation run was executed with the Verb Ocean ablated from our system. Our system uses the Verb Ocean resource for detection of antonyms. Our system required minor tweaking of the source code in order ablate Verb Ocean resource.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entities Recognition<br /> | SINAI1_abl-1<br /> | style="text-align: center;"| 7.62<br /> | style="text-align: center;"| Only PPVs on WN 1.7<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wordnet<br /> | SINAI1_abl-2<br /> | style="text-align: center;"| 15.87<br /> | style="text-align: center;"| Only NEs on WN 1.7<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entities Recognition<br /> | SINAI2_abl-1<br /> | style="text-align: center;"| 20.25<br /> | style="text-align: center;"| Only PPVs on WN 3.0<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wordnet<br /> | SINAI2_abl-2<br /> | style="text-align: center;"| 18.28<br /> | style="text-align: center;"| Only NEs on WN 3.0<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wordnet<br /> | SJTU_CIT3_abl-1<br /> | style="text-align: center;"| 0.03<br /> | style="text-align: center;"| The resource that has been ablated is WordNet.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | SJTU_CIT3_abl-2<br /> | style="text-align: center;"| 4.7<br /> | style="text-align: center;"| The resource that has been ablated is Wikipedia.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | SJTU_CIT3_abl-3<br /> | style="text-align: center;"| -1.15<br /> | style="text-align: center;"| The resource that has been ablated is VerbOcean.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | UAIC20101_abl-1<br /> | style="text-align: center;"| 1.93<br /> | style="text-align: center;"| Run 1 with no BK<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DIRT<br /> | UAIC20101_abl-2<br /> | style="text-align: center;"| -1.56<br /> | style="text-align: center;"| Run 1 without DIRT.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | UAIC20102_abl-1<br /> | style="text-align: center;"| 1.08<br /> | style="text-align: center;"| Run 2 with no BK<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DIRT<br /> | UAIC20102_abl-2<br /> | style="text-align: center;"| -0.72<br /> | style="text-align: center;"| Run 2 without DIRT.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | UAIC20103_abl-1<br /> | style="text-align: center;"| 1.3<br /> | style="text-align: center;"| Run 3 with no BK<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DIRT<br /> | UAIC20103_abl-2<br /> | style="text-align: center;"| -0.98<br /> | style="text-align: center;"| Run 3 without DIRT.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | UB.dmirg1_abl-1<br /> | style="text-align: center;"| 1.7<br /> | style="text-align: center;"| WordNet has been removed. No system changes required.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Framenet<br /> | UB.dmirg1_abl-2<br /> | style="text-align: center;"| -0.1<br /> | style="text-align: center;"| FrameNet has been removed. No system changes required.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | UB.dmirg2_abl-1<br /> | style="text-align: center;"| 0.6<br /> | style="text-align: center;"| WordNet has been removed. No system changes required.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Framenet<br /> | UB.dmirg2_abl-2<br /> | style="text-align: center;"| -1.84<br /> | style="text-align: center;"| FrameNet has been removed. No system changes required.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | UB.dmirg3_abl-1<br /> | style="text-align: center;"| -0.78<br /> | style="text-align: center;"| WordNet has been removed. No system changes required.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Framenet<br /> | UB.dmirg3_abl-2<br /> | style="text-align: center;"| -1.81<br /> | style="text-align: center;"| FrameNet has been removed. No system changes required.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | UIUC1_abl-1<br /> | style="text-align: center;"| -3.07<br /> | style="text-align: center;"| Abalation 1 - Removed wordnet (No locations and no synset expansion for non-entity terms). Threshold tuned on development collection to achieve similar precision and recall as Run 1.<br /> <br /> |}<br /> <br><br /> ==Footnotes==<br /> <references /><br /> <br /> <br /> Return to [[RTE Knowledge Resources]]</div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE6_-_Ablation_Tests&diff=8662 RTE6 - Ablation Tests 2011-02-03T15:40:31Z

<p>Amarchetti: </p> <hr /> <div>The following table lists the results of the ablation tests (a mandatory track since the RTE5 campaign), submitted by participants to RTE6 .<br><br /> The first column contains the specific resources which have been ablated.<br>The second column lists the Team Run in the form ''[name_of_the_Team][number_of_the_submitted_run].[submission_task]'' (e.g. BIU1.2way, Boeing3.3way).<br>The third column present the normalized difference between the accuracy of the complete system run and the accuracy of the ablation run (i.e. the output of the complete system without the ablated resource), showing the impact of the resource on the performance of the system. The third refers to the score obtained in the 2-way task, the fourth to that obtained in the 3-way task. For all the runs submitted as 3-way, also the 2-way derived accuracy has been calculated.<br><br /> The fourth column contains a brief description of the specific usage of the resource. It is based on the information provided both in the "readme" files submitted together with the ablation tests and in the system reports published in the RTE6 proceedings.<br><br><br /> <br /> <br /> <br /> Participants are kindly invited to check if all the inserted information is correct and complete.<br /> <br /> <br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! Ablated Component<br /> ! Ablation Run<ref>For further information about participants, click here: [[RTE Challenges - Data about participants]]</ref><br /> ! <small>Resource impact - F1 </small><br /> ! Resource Usage Description<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | BIU1_abl-1<br /> | style="text-align: center;"| 0.9<br /> | style="text-align: center;"| No Word-Net. On Dev set: 39.18% (compared to 40.73% when WN is used)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | CatVar<br /> | BIU1_abl-2<br /> | style="text-align: center;"| 0.63<br /> | style="text-align: center;"| No CatVar. On Dev set achieved about 40.20% (compared to 40.73% when CatVar is used)<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Coreference resolver<br /> | BIU1_abl-3<br /> | style="text-align: center;"| -0.88<br /> | style="text-align: center;"| No coreference resolver<br /> On Dev set 41.62% (Compared to 40.73% when Coreference resolver is used).<br /> This ablation test is an unusual ablation test, since it shows that the co-reference resolution component has a negative impact.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DIRT<br /> | Boeing1_abl-1<br /> | style="text-align: center;"| 3.97<br /> | style="text-align: center;"| DIRT removed<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | Boeing1_abl-2<br /> | style="text-align: center;"| 4.42<br /> | style="text-align: center;"| No WordNet<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Name Normalization<br /> | budapestcad2_abl-2<br /> | style="text-align: center;"| 0.65<br /> | style="text-align: center;"| no name normalization was performed (e.g. George W. Bush -> Bush).<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entities Recognition<br /> | budapestcad2_abl-3<br /> | style="text-align: center;"| -1.23<br /> | style="text-align: center;"| no NER<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | budapestcad2_abl-4<br /> | style="text-align: center;"| -1.11<br /> | style="text-align: center;"| No WordNet. (In the original run, WordNet was used to find the synonyms of words in the triplets, and additional triplets were generated from all possible combinations.)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb1_abl-1<br /> | style="text-align: center;"| 8.68<br /> | style="text-align: center;"| Wordnet is albated in this test.No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | deb_iitb1_abl-2<br /> | style="text-align: center;"| 1.87<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb2_abl-1<br /> | style="text-align: center;"| 7.9<br /> | style="text-align: center;"| Wordnet is albated in this test.No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | deb_iitb2_abl-2<br /> | style="text-align: center;"| 0.94<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb3_abl-1<br /> | style="text-align: center;"| 11.43<br /> | style="text-align: center;"| Wordnet is albated in this test. No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb3_abl-2<br /> | style="text-align: center;"| 2.54<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | POS-Tagger<br /> | DFKI1_abl-4<br /> | style="text-align: center;"| 4.99<br /> | style="text-align: center;"| No wordform/POS-tags included for the comparison.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | POS-Tagger<br /> | DFKI1_abl-6<br /> | style="text-align: center;"| 2.22<br /> | style="text-align: center;"| No named entity recognition for the comparison.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | DFKI1_abl-7<br /> | style="text-align: center;"| -0.23<br /> | style="text-align: center;"| No WordNet similarity for the comparison.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Coreference resolver<br /> | DFKI1_Main<br /> | style="text-align: center;"| -1.54<br /> | style="text-align: center;"| Coreference resolution used for the comparison.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | DirRelCond23_abl-1<br /> | style="text-align: center;"| 8.43<br /> | style="text-align: center;"| WordNet removed. Only basic word comparison used instead of word relations.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -23.91<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from Wikipedia<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -3.58<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from Wikipedia with probability above 0.7<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Proximity similarity dictionary of Dekang Lin<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -7.79<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from proximity similarity dictionary of Dekang Lin<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -3.21<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from WordNet<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -2.08<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from WordNet with probability above 0.7<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -4<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from Verbocean<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Dependency similarity dictionary of Dekang Lin<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -13.56<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from dependency similarity dictionary of Dekang Lin<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Dictionary of Named Entities Acronyms and Synonyms <br /> | IKOMA2_abl-3<br /> | style="text-align: center;"| -0.76<br /> | style="text-align: center;"| Remove synonym dictionaries: as acronym dictionary constructed automatically from the corpus and a synonym dictionary that contains geographical terms.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | JU_CSE_TAC1_abl-1<br /> | style="text-align: center;"| 13.29<br /> | style="text-align: center;"| The Run-1 is based on the composition of lexical based RTE methods and Syntactic RTE Method. The lexical based RTE methods are: WordNet based unigram match, bigram match, longest common sub-sequence, skip-gram and stemming. Here we ablated the WordNet based unigram match only.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | JU_CSE_TAC2_abl-1<br /> | style="text-align: center;"| 10.19<br /> | style="text-align: center;"| The Run-2 is based on the composition of lexical based RTE methods, Syntactic RTE Method, Chunk and Named Entity Methods. The lexical based RTE methods are: WordNet based unigram match, bigram match, longest common sub-sequence, skip-gram and stemming. Here we ablated the WordNet based unigram match only.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | JU_CSE_TAC3_abl-1<br /> | style="text-align: center;"| 3.86<br /> | style="text-align: center;"| The Run-3 is based on the Support Vector Machine that uses twenty five features for lexical similarity, the output tag from a rule based syntactic two-way TE system as feature, and output from a rule based Chunk Module and Named Entity Module. The important lexical features that are used in the present system are: WordNet based unigram match, bigram match, longest common sub-sequence, skip-gram, stemming and lexical distance (17 features). Here we ablated the WordNet based unigram match only.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | LingPipe co-reference<br /> | PKUTM2_abl-1<br /> | style="text-align: center;"| 0.17<br /> | style="text-align: center;"| Lingpipe co-reference are removed, the experiment was based on named-entity, wordnet, verbocean<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | PKUTM2_abl-2<br /> | style="text-align: center;"| 1.02<br /> | style="text-align: center;"| Verbocean are removed, the experiment was based on named-entity, wordnet, co-reference<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | LingPipe Named Entities<br /> | PKUTM2_abl-3<br /> | style="text-align: center;"| 13.84<br /> | style="text-align: center;"| Lingpipe named-entity are removed, the experiment was based on wordnet, co-reference, verbocean<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | saicnlp1_abl-1<br /> | style="text-align: center;"| -0.02<br /> | style="text-align: center;"| Ablation run, with WordNet stubbed.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Shalmanesar Parser<br /> | Sangyan1_abl-1<br /> | style="text-align: center;"| -1.76<br /> | style="text-align: center;"| This ablation run was executed with the Shalmanesar parser ablated from our system. Our system uses the Shalmanesar Parser for Framenet frame assignment.<br /> Our system required minor tweaking of the source code in order ablate Shalmanesar Parser.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | Sangyan1_abl-2<br /> | style="text-align: center;"| -0.39<br /> | style="text-align: center;"| This ablation run was executed with the WordNet ablated from our system. Our system uses the Wordnet for Word mathcing.<br /> Our system required minor tweaking of the source code in order ablate Wordnet.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | Sangyan1_abl-3<br /> | style="text-align: center;"| 0.07<br /> | style="text-align: center;"| This ablation run was executed with the Verb Ocean ablated from our system. Our system uses the Verb Ocean resource for detection of antonyms. Our system required minor tweaking of the source code in order ablate Verb Ocean resource.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entities Recognition<br /> | SINAI1_abl-1<br /> | style="text-align: center;"| 7.62<br /> | style="text-align: center;"| Only PPVs on WN 1.7<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wordnet<br /> | SINAI1_abl-2<br /> | style="text-align: center;"| 15.87<br /> | style="text-align: center;"| Only NEs on WN 1.7<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entities Recognition<br /> | SINAI2_abl-1<br /> | style="text-align: center;"| 20.25<br /> | style="text-align: center;"| Only PPVs on WN 3.0<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wordnet<br /> | SINAI2_abl-2<br /> | style="text-align: center;"| 18.28<br /> | style="text-align: center;"| Only NEs on WN 3.0<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wordnet<br /> | SJTU_CIT3_abl-1<br /> | style="text-align: center;"| 0.03<br /> | style="text-align: center;"| The resource that has been ablated is WordNet.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | SJTU_CIT3_abl-2<br /> | style="text-align: center;"| 4.7<br /> | style="text-align: center;"| The resource that has been ablated is Wikipedia.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | SJTU_CIT3_abl-3<br /> | style="text-align: center;"| -1.15<br /> | style="text-align: center;"| The resource that has been ablated is VerbOcean.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | UAIC20101_abl-1<br /> | style="text-align: center;"| 1.93<br /> | style="text-align: center;"| Run 1 with no BK<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DIRT<br /> | UAIC20101_abl-2<br /> | style="text-align: center;"| -1.56<br /> | style="text-align: center;"| Run 1 without DIRT.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | UAIC20102_abl-1<br /> | style="text-align: center;"| 1.08<br /> | style="text-align: center;"| Run 2 with no BK<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DIRT<br /> | UAIC20102_abl-2<br /> | style="text-align: center;"| -0.72<br /> | style="text-align: center;"| Run 2 without DIRT.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | UAIC20103_abl-1<br /> | style="text-align: center;"| 1.3<br /> | style="text-align: center;"| Run 3 with no BK<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DIRT<br /> | UAIC20103_abl-2<br /> | style="text-align: center;"| -0.98<br /> | style="text-align: center;"| Run 3 without DIRT.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | UB.dmirg1_abl-1<br /> | style="text-align: center;"| 1.7<br /> | style="text-align: center;"| WordNet has been removed. No system changes required.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Framenet<br /> | UB.dmirg1_abl-2<br /> | style="text-align: center;"| -0.1<br /> | style="text-align: center;"| FrameNet has been removed. No system changes required.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | UB.dmirg2_abl-1<br /> | style="text-align: center;"| 0.6<br /> | style="text-align: center;"| WordNet has been removed. No system changes required.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Framenet<br /> | UB.dmirg2_abl-2<br /> | style="text-align: center;"| -1.84<br /> | style="text-align: center;"| FrameNet has been removed. No system changes required.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | UB.dmirg3_abl-1<br /> | style="text-align: center;"| -0.78<br /> | style="text-align: center;"| WordNet has been removed. No system changes required.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Framenet<br /> | UB.dmirg3_abl-2<br /> | style="text-align: center;"| -1.81<br /> | style="text-align: center;"| FrameNet has been removed. No system changes required.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | UIUC1_abl-1<br /> | style="text-align: center;"| -3.07<br /> | style="text-align: center;"| Abalation 1 - Removed wordnet (No locations and no synset expansion for non-entity terms). Threshold tuned on development collection to achieve similar precision and recall as Run 1.<br /> <br /> |}<br /> <br><br /> ==Footnotes==<br /> <references /><br /> <br /> <br /> Return to [[RTE Knowledge Resources]]</div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE_Challenges_-_Data_about_participants&diff=8661 RTE Challenges - Data about participants 2011-02-03T15:35:27Z

<p>Amarchetti: </p> <hr /> <div>The table below lists information about Users in the RTE challenges.<br><br /> The first column shows the ID names of systems as reported in the "RTE Users" pages.<br /> <br><br /> <br><br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" " border="1"<br /> |- bgcolor="#CDCDCD"<br /> ! System<br /> ! Affiliation<br /> |- bgcolor="#ECECEC" "align="left"<br /> | AUEBNLP<br /> | Department of Informatics, <br> Athens University of Economics and Business<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | BIU<br /> | Computer Science Department, Bar-Ilan University, Israel;<br>School of Computer Science, Tel-Aviv University, Israel;<br>Department of Linguistics, Tel-Aviv University, Israel<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Boeing<br /> | Boeing Phantom Works, The Boeing Company, Seattle<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Budapestcad<br /> | Computer and Automation Research Institute of the Hungarian Academy of Sciences, Budapest, Hungary<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Cambridge<br /> | Austrian Academy of Sciences, University of Cambridge Computer Laboratory, Cambridge, UK<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | CERES<br /> | Science Applications International Corporation, Orlando, Florida, US<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | CLEAR<br /> | Boulder Language Technologies, Boulder, CO;<br> Center for Computational Language and Education Research, University of Colorado, Boulder<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | CLR<br /> | CL Research, Damascus, Maryland, US<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Cswhu<br /> | School of Computer Science, Wuhan University, Wuhan, Hubei, PRC<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | deb_iitb<br /> | IITBombay, Mumbai, India<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DFKI<br /> | Saarland University, Saarbruecken, Germany; <br>DFKI GmbH, Saarbruecken, Germany<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DirRelCond<br /> | Babes-Bolyai University of Cluj-Napoca, Romania<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DLSIUAES<br /> | Universidad de Alicante, Departamento de Lenguajes y Sistemas Informáticos, Alicante<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Emory<br /> | Mathematics & Computer Science Department, Emory University<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | FbkIrst<br /> | FBK-Irst;<br>University of Trento, Povo, Trento, Italy<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | FSC<br /> | Textual Entailment – Fitchburg State College<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | IIITSum08<br /> | Search and Information Extraction Lab., International Institute of Information Technology, Hyderabad, India (RTE4)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | IKOMA<br /> | NEC Corporation, Takayama, Ikoma, Nara, Japan<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | IPD<br /> | L3S Research Center, Universität Hannover, Germany<br>Department of Computer Science and Software Engineering, Concordia University, Montréal, Canada<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | JU_CSE_TAC<br /> | Jadavpur University, Kolkata, India;<br/>National Polytechnic Institute, Mexico City, Mexico<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | KUNLP<br /> | Koç University<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Lcc<br /> | Language Computer Corporation, Richardson, Texas USA<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | OAQA<br /> | CLaC Laboratory<br/>Department of Computer Science and Software Engineering, Concordia University, Montreal, Quebec, Canada<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | PeMoZa<br /> | DISP, University of Rome “Tor Vergata”, Roma, Italy;<br>Computerlinguistik, Universität des Saarlandes, Saarbrücken, Germany;<br>DISI, University of Trento, Povo di Trento, Italy<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | PKUTM<br /> | Institute of Computer Science and Technology, Peking University, Beijing, PRC<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | QUANTA<br /> | State Key Laboratory on Intelligent Technology and Systems,<br>Tsinghua National Laboratory of Intelligent Technology and Systems (LITS),<br>Department of Computer Science and Technology,<br>Tsinghua University, Beijing, PRC<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Rhodes<br /> | Rhodes College, Memphis, TN <br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Sagan<br /> | Faculty of Mathematics Astronomy and Physics, Department of Computer Science, National University of Cordoba, Argentina<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | saicnlp<br /> | Science Applications International Corporation (SAIC), McLean, VA, U.S.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Sangyan<br /> | NEC HCL System Technologies Ltd. New Delhi, India<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Siel_09<br /> | Search and Information Extraction Lab., International Institute of Information Technology, Hyderabad, India<br/>(RTE5; previuos: IIITSum08)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | SINAI<br /> | University of Jaén, Jaén, Spain<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | SJTU_CIT<br /> | Shanghai JiaoTong University, Shanghai, PRC<br /> |- bgcolor="#ECECEC" "align="left"<br /> | Stanford<br /> | Stanford University, Stanford, CA, U.S.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC<br /> | „Al. I. Cuza“ University, Faculty of Computer Science, Iasi, Romania<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UB.dmirg<br /> | Centre for Informatics and Applied Optimization, University of Ballarat, Victoria, Australia<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UIUC<br /> | University of Illinois at Urbana-Champaign, Urbana, IL, U.S. (RTE4)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UI_ccg<br /> | University of Illinois at Urbana-Champaign, Urbana, IL, U.S. <br/>(RTE5; previous: UIUC)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UMD<br /> | Institute for Advanced Computer Studies, University of Maryland, MD, U.S.<br /> |- bgcolor="#ECECEC" "align="left"<br /> | UNED<br /> | Departmento de Lenguajes y Sistemas Informáticos, Universidad Nacional de Educación a Distancia, Madrid, Spain<br /> |- bgcolor="#ECECEC" "align="left"<br /> | Uoeltg<br /> | University of Edinburgh<br /> |- bgcolor="#ECECEC" "align="left"<br /> | UPC<br /> | TALP Research Center, Universitat Politècnica de Catalunya, Spain;<br>Universidad de Sevilla, Spain<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Unimelb<br /> | University of Melbourne<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VENSES<br /> | Venice Semantic Evaluation System - Department of Language Sciences, Università Ca’ Foscari – Ca’ Bembo, Venezia, Italy<br /> |- bgcolor="#ECECEC" "align="left"<br /> | OPENU<br /> | Centre for Research in Computing, The Open University, Walton Hall, Milton Keynes, MK7 6AA, UK<br /> <br /> |}<br /> <br></div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE_Challenges_-_Data_about_participants&diff=8660 RTE Challenges - Data about participants 2011-02-03T15:24:56Z

<p>Amarchetti: </p> <hr /> <div>The table below lists information about Users in the RTE challenges.<br><br /> The first column shows the ID names of systems as reported in the "RTE Users" pages.<br /> <br><br /> <br><br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" " border="1"<br /> |- bgcolor="#CDCDCD"<br /> ! System<br /> ! Affiliation<br /> |- bgcolor="#ECECEC" "align="left"<br /> | AUEBNLP<br /> | Department of Informatics, <br> Athens University of Economics and Business<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | BIU<br /> | Computer Science Department, Bar-Ilan University, Israel;<br>School of Computer Science, Tel-Aviv University, Israel;<br>Department of Linguistics, Tel-Aviv University, Israel<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Boeing<br /> | Boeing Phantom Works, The Boeing Company, Seattle<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Budapestcad<br /> | Computer and Automation Research Institute of the Hungarian Academy of Sciences, Budapest, Hungary<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Cambridge<br /> | Austrian Academy of Sciences, University of Cambridge Computer Laboratory, Cambridge, UK<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | CERES<br /> | Science Applications International Corporation, Orlando, Florida, US<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | CLEAR<br /> | Boulder Language Technologies, Boulder, CO;<br> Center for Computational Language and Education Research, University of Colorado, Boulder<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | CLR<br /> | CL Research, Damascus, Maryland, US<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Cswhu<br /> | School of Computer Science, Wuhan University, Wuhan, Hubei, PRC<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | deb_iitb<br /> | IITBombay, Mumbai, India<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DFKI<br /> | Saarland University, Saarbruecken, Germany; <br>DFKI GmbH, Saarbruecken, Germany<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DirRelCond<br /> | Babes-Bolyai University of Cluj-Napoca, Romania<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DLSIUAES<br /> | Universidad de Alicante, Departamento de Lenguajes y Sistemas Informáticos, Alicante<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Emory<br /> | Mathematics & Computer Science Department, Emory University<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | FbkIrst<br /> | FBK-Irst;<br>University of Trento, Povo, Trento, Italy<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | FSC<br /> | Textual Entailment – Fitchburg State College<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | IIITSum08<br /> | Search and Information Extraction Lab., International Institute of Information Technology, Hyderabad, India (RTE4)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | IKOMA<br /> | NEC Corporation, Takayama, Ikoma, Nara, Japan<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | IPD<br /> | L3S Research Center, Universität Hannover, Germany<br>Department of Computer Science and Software Engineering, Concordia University, Montréal, Canada<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | JU_CSE_TAC<br /> | Jadavpur University, Kolkata, India;<br/>National Polytechnic Institute, Mexico City, Mexico<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | KUNLP<br /> | Koç University<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Lcc<br /> | Language Computer Corporation, Richardson, Texas USA<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | OAQA<br /> | CLaC Laboratory<br/>Department of Computer Science and Software Engineering, Concordia University, Montreal, Quebec, Canada<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | PeMoZa<br /> | DISP, University of Rome “Tor Vergata”, Roma, Italy;<br>Computerlinguistik, Universität des Saarlandes, Saarbrücken, Germany;<br>DISI, University of Trento, Povo di Trento, Italy<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | QUANTA<br /> | State Key Laboratory on Intelligent Technology and Systems,<br>Tsinghua National Laboratory of Intelligent Technology and Systems (LITS),<br>Department of Computer Science and Technology,<br>Tsinghua University, Beijing, China<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Rhodes<br /> | Rhodes College, Memphis, TN <br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Sagan<br /> | Faculty of Mathematics Astronomy and Physics, Department of Computer Science, National University of Cordoba, Argentina<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Siel_09<br /> | Search and Information Extraction Lab., International Institute of Information Technology, Hyderabad, India<br/>(RTE5; previuos: IIITSum08)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Stanford<br /> | Stanford University<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC<br /> | „Al. I. Cuza“ University, Faculty of Computer Science, Iasi, Romania<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UB.dmirg<br /> | Centre for Informatics and Applied Optimization, University of Ballarat, Victoria, Australia<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UIUC<br /> | University of Illinois at Urbana-Champaign, Urbana, IL (RTE4)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UI_ccg<br /> | University of Illinois at Urbana-Champaign, Urbana, IL <br/>(RTE5; previous: UIUC)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UMD<br /> | Institute for Advanced Computer Studies, University of Maryland, MD<br /> |- bgcolor="#ECECEC" "align="left"<br /> | UNED<br /> | Departmento de Lenguajes y Sistemas Informáticos, Universidad Nacional de Educación a Distancia, Madrid, Spain<br /> |- bgcolor="#ECECEC" "align="left"<br /> | Uoeltg<br /> | University of Edinburgh<br /> |- bgcolor="#ECECEC" "align="left"<br /> | UPC<br /> | TALP Research Center, Universitat Politècnica de Catalunya, Spain;<br>Universidad de Sevilla, Spain<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Unimelb<br /> | University of Melbourne<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VENSES<br /> | Venice Semantic Evaluation System - Department of Language Sciences, Università Ca’ Foscari – Ca’ Bembo, Venezia, Italy<br /> |- bgcolor="#ECECEC" "align="left"<br /> | OPENU<br /> | Centre for Research in Computing, The Open University, Walton Hall, Milton Keynes, MK7 6AA, UK<br /> <br /> |}<br /> <br></div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE6_-_Ablation_Tests&diff=8659 RTE6 - Ablation Tests 2011-02-03T13:31:20Z

<p>Amarchetti: </p> <hr /> <div>The following table lists the results of the ablation tests (a mandatory track since the RTE5 campaign), submitted by participants to RTE6 .<br><br><br /> <br /> <br /> Participants are kindly invited to check if all the inserted information is correct and complete.<br /> <br /> <br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! Ablated Component<br /> ! Ablation Run<ref>For further information about participants, click here: [[RTE Challenges - Data about participants]]</ref><br /> ! <small>Resource impact - F1 </small><br /> ! Resource Usage Description<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | BIU1_abl-1<br /> | style="text-align: center;"| 0.9<br /> | style="text-align: center;"| No Word-Net. On Dev set: 39.18% (compared to 40.73% when WN is used)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | CatVar<br /> | BIU1_abl-2<br /> | style="text-align: center;"| 0.63<br /> | style="text-align: center;"| No CatVar. On Dev set achieved about 40.20% (compared to 40.73% when CatVar is used)<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Coreference resolver<br /> | BIU1_abl-3<br /> | style="text-align: center;"| -0.88<br /> | style="text-align: center;"| No coreference resolver<br /> On Dev set 41.62% (Compared to 40.73% when Coreference resolver is used).<br /> This ablation test is an unusual ablation test, since it shows that the co-reference resolution component has a negative impact.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DIRT<br /> | Boeing1_abl-1<br /> | style="text-align: center;"| 3.97<br /> | style="text-align: center;"| DIRT removed<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | Boeing1_abl-2<br /> | style="text-align: center;"| 4.42<br /> | style="text-align: center;"| No WordNet<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Name Normalization<br /> | budapestcad2_abl-2<br /> | style="text-align: center;"| 0.65<br /> | style="text-align: center;"| no name normalization was performed (e.g. George W. Bush -> Bush).<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entities Recognition<br /> | budapestcad2_abl-3<br /> | style="text-align: center;"| -1.23<br /> | style="text-align: center;"| no NER<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | budapestcad2_abl-4<br /> | style="text-align: center;"| -1.11<br /> | style="text-align: center;"| No WordNet. (In the original run, WordNet was used to find the synonyms of words in the triplets, and additional triplets were generated from all possible combinations.)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb1_abl-1<br /> | style="text-align: center;"| 8.68<br /> | style="text-align: center;"| Wordnet is albated in this test.No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | deb_iitb1_abl-2<br /> | style="text-align: center;"| 1.87<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb2_abl-1<br /> | style="text-align: center;"| 7.9<br /> | style="text-align: center;"| Wordnet is albated in this test.No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | deb_iitb2_abl-2<br /> | style="text-align: center;"| 0.94<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb3_abl-1<br /> | style="text-align: center;"| 11.43<br /> | style="text-align: center;"| Wordnet is albated in this test. No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb3_abl-2<br /> | style="text-align: center;"| 2.54<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | POS-Tagger<br /> | DFKI1_abl-4<br /> | style="text-align: center;"| 4.99<br /> | style="text-align: center;"| No wordform/POS-tags included for the comparison.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | POS-Tagger<br /> | DFKI1_abl-6<br /> | style="text-align: center;"| 2.22<br /> | style="text-align: center;"| No named entity recognition for the comparison.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | DFKI1_abl-7<br /> | style="text-align: center;"| -0.23<br /> | style="text-align: center;"| No WordNet similarity for the comparison.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Coreference resolver<br /> | DFKI1_Main<br /> | style="text-align: center;"| -1.54<br /> | style="text-align: center;"| Coreference resolution used for the comparison.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | DirRelCond23_abl-1<br /> | style="text-align: center;"| 8.43<br /> | style="text-align: center;"| WordNet removed. Only basic word comparison used instead of word relations.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -23.91<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from Wikipedia<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -3.58<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from Wikipedia with probability above 0.7<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Proximity similarity dictionary of Dekang Lin<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -7.79<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from proximity similarity dictionary of Dekang Lin<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -3.21<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from WordNet<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -2.08<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from WordNet with probability above 0.7<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -4<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from Verbocean<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Dependency similarity dictionary of Dekang Lin<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -13.56<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from dependency similarity dictionary of Dekang Lin<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Dictionary of Named Entities Acronyms and Synonyms <br /> | IKOMA2_abl-3<br /> | style="text-align: center;"| -0.76<br /> | style="text-align: center;"| Remove synonym dictionaries: as acronym dictionary constructed automatically from the corpus and a synonym dictionary that contains geographical terms.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | JU_CSE_TAC1_abl-1<br /> | style="text-align: center;"| 13.29<br /> | style="text-align: center;"| The Run-1 is based on the composition of lexical based RTE methods and Syntactic RTE Method. The lexical based RTE methods are: WordNet based unigram match, bigram match, longest common sub-sequence, skip-gram and stemming. Here we ablated the WordNet based unigram match only.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | JU_CSE_TAC2_abl-1<br /> | style="text-align: center;"| 10.19<br /> | style="text-align: center;"| The Run-2 is based on the composition of lexical based RTE methods, Syntactic RTE Method, Chunk and Named Entity Methods. The lexical based RTE methods are: WordNet based unigram match, bigram match, longest common sub-sequence, skip-gram and stemming. Here we ablated the WordNet based unigram match only.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | JU_CSE_TAC3_abl-1<br /> | style="text-align: center;"| 3.86<br /> | style="text-align: center;"| The Run-3 is based on the Support Vector Machine that uses twenty five features for lexical similarity, the output tag from a rule based syntactic two-way TE system as feature, and output from a rule based Chunk Module and Named Entity Module. The important lexical features that are used in the present system are: WordNet based unigram match, bigram match, longest common sub-sequence, skip-gram, stemming and lexical distance (17 features). Here we ablated the WordNet based unigram match only.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | LingPipe co-reference<br /> | PKUTM2_abl-1<br /> | style="text-align: center;"| 0.17<br /> | style="text-align: center;"| Lingpipe co-reference are removed, the experiment was based on named-entity, wordnet, verbocean<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | PKUTM2_abl-2<br /> | style="text-align: center;"| 1.02<br /> | style="text-align: center;"| Verbocean are removed, the experiment was based on named-entity, wordnet, co-reference<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | LingPipe Named Entities<br /> | PKUTM2_abl-3<br /> | style="text-align: center;"| 13.84<br /> | style="text-align: center;"| Lingpipe named-entity are removed, the experiment was based on wordnet, co-reference, verbocean<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | saicnlp1_abl-1<br /> | style="text-align: center;"| -0.02<br /> | style="text-align: center;"| Ablation run, with WordNet stubbed.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Shalmanesar Parser<br /> | Sangyan1_abl-1<br /> | style="text-align: center;"| -1.76<br /> | style="text-align: center;"| This ablation run was executed with the Shalmanesar parser ablated from our system. Our system uses the Shalmanesar Parser for Framenet frame assignment.<br /> Our system required minor tweaking of the source code in order ablate Shalmanesar Parser.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | Sangyan1_abl-2<br /> | style="text-align: center;"| -0.39<br /> | style="text-align: center;"| This ablation run was executed with the WordNet ablated from our system. Our system uses the Wordnet for Word mathcing.<br /> Our system required minor tweaking of the source code in order ablate Wordnet.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | Sangyan1_abl-3<br /> | style="text-align: center;"| 0.07<br /> | style="text-align: center;"| This ablation run was executed with the Verb Ocean ablated from our system. Our system uses the Verb Ocean resource for detection of antonyms. Our system required minor tweaking of the source code in order ablate Verb Ocean resource.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entities Recognition<br /> | SINAI1_abl-1<br /> | style="text-align: center;"| 7.62<br /> | style="text-align: center;"| Only PPVs on WN 1.7<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wordnet<br /> | SINAI1_abl-2<br /> | style="text-align: center;"| 15.87<br /> | style="text-align: center;"| Only NEs on WN 1.7<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entities Recognition<br /> | SINAI2_abl-1<br /> | style="text-align: center;"| 20.25<br /> | style="text-align: center;"| Only PPVs on WN 3.0<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wordnet<br /> | SINAI2_abl-2<br /> | style="text-align: center;"| 18.28<br /> | style="text-align: center;"| Only NEs on WN 3.0<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wordnet<br /> | SJTU_CIT3_abl-1<br /> | style="text-align: center;"| 0.03<br /> | style="text-align: center;"| The resource that has been ablated is WordNet.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | SJTU_CIT3_abl-2<br /> | style="text-align: center;"| 4.7<br /> | style="text-align: center;"| The resource that has been ablated is Wikipedia.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | SJTU_CIT3_abl-3<br /> | style="text-align: center;"| -1.15<br /> | style="text-align: center;"| The resource that has been ablated is VerbOcean.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | UAIC20101_abl-1<br /> | style="text-align: center;"| 1.93<br /> | style="text-align: center;"| Run 1 with no BK<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DIRT<br /> | UAIC20101_abl-2<br /> | style="text-align: center;"| -1.56<br /> | style="text-align: center;"| Run 1 without DIRT.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | UAIC20102_abl-1<br /> | style="text-align: center;"| 1.08<br /> | style="text-align: center;"| Run 2 with no BK<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DIRT<br /> | UAIC20102_abl-2<br /> | style="text-align: center;"| -0.72<br /> | style="text-align: center;"| Run 2 without DIRT.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | UAIC20103_abl-1<br /> | style="text-align: center;"| 1.3<br /> | style="text-align: center;"| Run 3 with no BK<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DIRT<br /> | UAIC20103_abl-2<br /> | style="text-align: center;"| -0.98<br /> | style="text-align: center;"| Run 3 without DIRT.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | UB.dmirg1_abl-1<br /> | style="text-align: center;"| 1.7<br /> | style="text-align: center;"| WordNet has been removed. No system changes required.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Framenet<br /> | UB.dmirg1_abl-2<br /> | style="text-align: center;"| -0.1<br /> | style="text-align: center;"| FrameNet has been removed. No system changes required.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | UB.dmirg2_abl-1<br /> | style="text-align: center;"| 0.6<br /> | style="text-align: center;"| WordNet has been removed. No system changes required.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Framenet<br /> | UB.dmirg2_abl-2<br /> | style="text-align: center;"| -1.84<br /> | style="text-align: center;"| FrameNet has been removed. No system changes required.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | UB.dmirg3_abl-1<br /> | style="text-align: center;"| -0.78<br /> | style="text-align: center;"| WordNet has been removed. No system changes required.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Framenet<br /> | UB.dmirg3_abl-2<br /> | style="text-align: center;"| -1.81<br /> | style="text-align: center;"| FrameNet has been removed. No system changes required.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | UIUC1_abl-1<br /> | style="text-align: center;"| -3.07<br /> | style="text-align: center;"| Abalation 1 - Removed wordnet (No locations and no synset expansion for non-entity terms). Threshold tuned on development collection to achieve similar precision and recall as Run 1.<br /> <br /> |}<br /> <br><br /> ==Footnotes==<br /> <references /><br /> <br /> <br /> Return to [[RTE Knowledge Resources]]</div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE6_-_Ablation_Tests&diff=8658 RTE6 - Ablation Tests 2011-02-03T13:25:18Z

<p>Amarchetti: </p> <hr /> <div>The following table lists the results of the ablation tests (a mandatory track since the RTE5 campaign), submitted by participants to RTE6 .<br><br><br /> <br /> <br /> Participants are kindly invited to check if all the inserted information is correct and complete.<br /> <br /> <br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! Ablated Component<br /> ! Ablation Run<ref>For further information about participants, click here: [[RTE Challenges - Data about participants]]</ref><br /> ! <small>Resource impact - F1 </small><br /> ! Resource Usage Description<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | BIU1_abl-1<br /> | style="text-align: center;"| 0.9<br /> | style="text-align: center;"| No Word-Net. On Dev set: 39.18% (compared to 40.73% when WN is used)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | CatVar<br /> | BIU1_abl-2<br /> | style="text-align: center;"| 0.63<br /> | style="text-align: center;"| No CatVar. On Dev set achieved about 40.20% (compared to 40.73% when CatVar is used)<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Coreference resolver<br /> | BIU1_abl-3<br /> | style="text-align: center;"| -0.88<br /> | style="text-align: center;"| No coreference resolver<br /> On Dev set 41.62% (Compared to 40.73% when Coreference resolver is used).<br /> This ablation test is an unusual ablation test, since it shows that the co-reference resolution component has a negative impact.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DIRT<br /> | Boeing1_abl-1<br /> | style="text-align: center;"| 3.97<br /> | style="text-align: center;"| DIRT removed<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | Boeing1_abl-2<br /> | style="text-align: center;"| 4.42<br /> | style="text-align: center;"| No WordNet<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Name Normalization<br /> | budapestcad2_abl-2<br /> | style="text-align: center;"| 0.65<br /> | style="text-align: center;"| no name normalization was performed (e.g. George W. Bush -> Bush).<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entities Recognition<br /> | budapestcad2_abl-3<br /> | style="text-align: center;"| -1.23<br /> | style="text-align: center;"| no NER<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | budapestcad2_abl-4<br /> | style="text-align: center;"| -1.11<br /> | style="text-align: center;"| No WordNet. (In the original run, WordNet was used to find the synonyms of words in the triplets, and additional triplets were generated from all possible combinations.)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb1_abl-1<br /> | style="text-align: center;"| 8.68<br /> | style="text-align: center;"| Wordnet is albated in this test.No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | deb_iitb1_abl-2<br /> | style="text-align: center;"| 1.87<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb2_abl-1<br /> | style="text-align: center;"| 7.9<br /> | style="text-align: center;"| Wordnet is albated in this test.No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | deb_iitb2_abl-2<br /> | style="text-align: center;"| 0.94<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb3_abl-1<br /> | style="text-align: center;"| 11.43<br /> | style="text-align: center;"| Wordnet is albated in this test. No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb3_abl-2<br /> | style="text-align: center;"| 2.54<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | POS-Tagger<br /> | DFKI1_abl-4<br /> | style="text-align: center;"| 4.99<br /> | style="text-align: center;"| No wordform/POS-tags included for the comparison.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | POS-Tagger<br /> | DFKI1_abl-6<br /> | style="text-align: center;"| 2.22<br /> | style="text-align: center;"| No named entity recognition for the comparison.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | DFKI1_abl-7<br /> | style="text-align: center;"| -0.23<br /> | style="text-align: center;"| No WordNet similarity for the comparison.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Coreference resolver<br /> | DFKI1_Main<br /> | style="text-align: center;"| -1.54<br /> | style="text-align: center;"| Coreference resolution used for the comparison.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | DirRelCond23_abl-1<br /> | style="text-align: center;"| 8.43<br /> | style="text-align: center;"| WordNet removed. Only basic word comparison used instead of word relations.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -23.91<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from Wikipedia<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -3.58<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from Wikipedia with probability above 0.7<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Proximity similarity dictionary of Dekang Lin<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -7.79<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from proximity similarity dictionary of Dekang Lin<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -3.21<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from WordNet<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -2.08<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from WordNet with probability above 0.7<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -4<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from Verbocean<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Dependency similarity dictionary of Dekang Lin<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -13.56<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from dependency similarity dictionary of Dekang Lin<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Dictionary of Named Entities Acronyms and Synonyms <br /> | IKOMA2_abl-3<br /> | style="text-align: center;"| -0.76<br /> | style="text-align: center;"| Remove synonym dictionaries: as acronym dictionary constructed automatically from the corpus and a synonym dictionary that contains geographical terms.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | JU_CSE_TAC1_abl-1<br /> | style="text-align: center;"| 13.29<br /> | style="text-align: center;"| The Run-1 is based on the composition of lexical based RTE methods and Syntactic RTE Method. The lexical based RTE methods are: WordNet based unigram match, bigram match, longest common sub-sequence, skip-gram and stemming. Here we ablated the WordNet based unigram match only.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | JU_CSE_TAC2_abl-1<br /> | style="text-align: center;"| 10.19<br /> | style="text-align: center;"| The Run-2 is based on the composition of lexical based RTE methods, Syntactic RTE Method, Chunk and Named Entity Methods. The lexical based RTE methods are: WordNet based unigram match, bigram match, longest common sub-sequence, skip-gram and stemming. Here we ablated the WordNet based unigram match only.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | JU_CSE_TAC3_abl-1<br /> | style="text-align: center;"| 3.86<br /> | style="text-align: center;"| The Run-3 is based on the Support Vector Machine that uses twenty five features for lexical similarity, the output tag from a rule based syntactic two-way TE system as feature, and output from a rule based Chunk Module and Named Entity Module. The important lexical features that are used in the present system are: WordNet based unigram match, bigram match, longest common sub-sequence, skip-gram, stemming and lexical distance (17 features). Here we ablated the WordNet based unigram match only.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | LingPipe co-reference<br /> | PKUTM2_abl-1<br /> | style="text-align: center;"| 0.17<br /> | style="text-align: center;"| Lingpipe co-reference are removed, the experiment was based on named-entity, wordnet, verbocean<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | PKUTM2_abl-2<br /> | style="text-align: center;"| 1.02<br /> | style="text-align: center;"| Verbocean are removed, the experiment was based on named-entity, wordnet, co-reference<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | LingPipe Named Entities<br /> | PKUTM2_abl-3<br /> | style="text-align: center;"| 13.84<br /> | style="text-align: center;"| Lingpipe named-entity are removed, the experiment was based on wordnet, co-reference, verbocean<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | saicnlp1_abl-1<br /> | style="text-align: center;"| -0.02<br /> | style="text-align: center;"| Ablation run, with WordNet stubbed.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Shalmanesar Parser<br /> | Sangyan1_abl-1<br /> | style="text-align: center;"| -1.76<br /> | style="text-align: center;"| This ablation run was executed with the Shalmanesar parser ablated from our system. Our system uses the Shalmanesar Parser for Framenet frame assignment.<br /> Our system required minor tweaking of the source code in order ablate Shalmanesar Parser.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | Sangyan1_abl-2<br /> | style="text-align: center;"| -0.39<br /> | style="text-align: center;"| This ablation run was executed with the WordNet ablated from our system. Our system uses the Wordnet for Word mathcing.<br /> Our system required minor tweaking of the source code in order ablate Wordnet.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | Sangyan1_abl-3<br /> | style="text-align: center;"| 0.07<br /> | style="text-align: center;"| This ablation run was executed with the Verb Ocean ablated from our system. Our system uses the Verb Ocean resource for detection of antonyms. Our system required minor tweaking of the source code in order ablate Verb Ocean resource.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entities Recognition<br /> | SINAI1_abl-1<br /> | style="text-align: center;"| 7.62<br /> | style="text-align: center;"| Only PPVs on WN 1.7<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wordnet<br /> | SINAI1_abl-2<br /> | style="text-align: center;"| 15.87<br /> | style="text-align: center;"| Only NEs on WN 1.7<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entities Recognition<br /> | SINAI2_abl-1<br /> | style="text-align: center;"| 20.25<br /> | style="text-align: center;"| Only PPVs on WN 3.0<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wordnet<br /> | SINAI2_abl-2<br /> | style="text-align: center;"| 18.28<br /> | style="text-align: center;"| Only NEs on WN 3.0<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wordnet<br /> | SJTU_CIT3_abl-1<br /> | style="text-align: center;"| 0.03<br /> | style="text-align: center;"| The resource that has been ablated is WordNet.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | SJTU_CIT3_abl-2<br /> | style="text-align: center;"| 4.7<br /> | style="text-align: center;"| The resource that has been ablated is Wikipedia.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | SJTU_CIT3_abl-3<br /> | style="text-align: center;"| -1.15<br /> | style="text-align: center;"| The resource that has been ablated is VerbOcean.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | UAIC20101_abl-1<br /> | style="text-align: center;"| 1.93<br /> | style="text-align: center;"| Run 1 with no BK<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DIRT<br /> | UAIC20101_abl-2<br /> | style="text-align: center;"| -1.56<br /> | style="text-align: center;"| Run 1 without DIRT.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | UAIC20102_abl-1<br /> | style="text-align: center;"| 1.08<br /> | style="text-align: center;"| Run 2 with no BK<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DIRT<br /> | UAIC20102_abl-2<br /> | style="text-align: center;"| -0.72<br /> | style="text-align: center;"| Run 2 without DIRT.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | UAIC20103_abl-1<br /> | style="text-align: center;"| 1.3<br /> | style="text-align: center;"| Run 3 with no BK<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DIRT<br /> | UAIC20103_abl-2<br /> | style="text-align: center;"| -0.98<br /> | style="text-align: center;"| Run 3 without DIRT.<br /> |}<br /> <br><br /> ==Footnotes==<br /> <references /><br /> <br /> <br /> Return to [[RTE Knowledge Resources]]</div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE6_-_Ablation_Tests&diff=8657 RTE6 - Ablation Tests 2011-02-03T13:18:39Z

<p>Amarchetti: </p> <hr /> <div>The following table lists the results of the ablation tests (a mandatory track since the RTE5 campaign), submitted by participants to RTE6 .<br><br><br /> <br /> <br /> Participants are kindly invited to check if all the inserted information is correct and complete.<br /> <br /> <br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! Ablated Component<br /> ! Ablation Run<ref>For further information about participants, click here: [[RTE Challenges - Data about participants]]</ref><br /> ! <small>Resource impact - F1 </small><br /> ! Resource Usage Description<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | BIU1_abl-1<br /> | style="text-align: center;"| 0.9<br /> | style="text-align: center;"| No Word-Net. On Dev set: 39.18% (compared to 40.73% when WN is used)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | CatVar<br /> | BIU1_abl-2<br /> | style="text-align: center;"| 0.63<br /> | style="text-align: center;"| No CatVar. On Dev set achieved about 40.20% (compared to 40.73% when CatVar is used)<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Coreference resolver<br /> | BIU1_abl-3<br /> | style="text-align: center;"| -0.88<br /> | style="text-align: center;"| No coreference resolver<br /> On Dev set 41.62% (Compared to 40.73% when Coreference resolver is used).<br /> This ablation test is an unusual ablation test, since it shows that the co-reference resolution component has a negative impact.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DIRT<br /> | Boeing1_abl-1<br /> | style="text-align: center;"| 3.97<br /> | style="text-align: center;"| DIRT removed<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | Boeing1_abl-2<br /> | style="text-align: center;"| 4.42<br /> | style="text-align: center;"| No WordNet<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Name Normalization<br /> | budapestcad2_abl-2<br /> | style="text-align: center;"| 0.65<br /> | style="text-align: center;"| no name normalization was performed (e.g. George W. Bush -> Bush).<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entities Recognition<br /> | budapestcad2_abl-3<br /> | style="text-align: center;"| -1.23<br /> | style="text-align: center;"| no NER<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | budapestcad2_abl-4<br /> | style="text-align: center;"| -1.11<br /> | style="text-align: center;"| No WordNet. (In the original run, WordNet was used to find the synonyms of words in the triplets, and additional triplets were generated from all possible combinations.)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb1_abl-1<br /> | style="text-align: center;"| 8.68<br /> | style="text-align: center;"| Wordnet is albated in this test.No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | deb_iitb1_abl-2<br /> | style="text-align: center;"| 1.87<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb2_abl-1<br /> | style="text-align: center;"| 7.9<br /> | style="text-align: center;"| Wordnet is albated in this test.No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | deb_iitb2_abl-2<br /> | style="text-align: center;"| 0.94<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb3_abl-1<br /> | style="text-align: center;"| 11.43<br /> | style="text-align: center;"| Wordnet is albated in this test. No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb3_abl-2<br /> | style="text-align: center;"| 2.54<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | POS-Tagger<br /> | DFKI1_abl-4<br /> | style="text-align: center;"| 4.99<br /> | style="text-align: center;"| No wordform/POS-tags included for the comparison.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | POS-Tagger<br /> | DFKI1_abl-6<br /> | style="text-align: center;"| 2.22<br /> | style="text-align: center;"| No named entity recognition for the comparison.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | DFKI1_abl-7<br /> | style="text-align: center;"| -0.23<br /> | style="text-align: center;"| No WordNet similarity for the comparison.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Coreference resolver<br /> | DFKI1_Main<br /> | style="text-align: center;"| -1.54<br /> | style="text-align: center;"| Coreference resolution used for the comparison.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | DirRelCond23_abl-1<br /> | style="text-align: center;"| 8.43<br /> | style="text-align: center;"| WordNet removed. Only basic word comparison used instead of word relations.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -23.91<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from Wikipedia<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -3.58<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from Wikipedia with probability above 0.7<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Proximity similarity dictionary of Dekang Lin<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -7.79<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from proximity similarity dictionary of Dekang Lin<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -3.21<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from WordNet<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -2.08<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from WordNet with probability above 0.7<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -4<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from Verbocean<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Dependency similarity dictionary of Dekang Lin<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -13.56<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from dependency similarity dictionary of Dekang Lin<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Dictionary of Named Entities Acronyms and Synonyms <br /> | IKOMA2_abl-3<br /> | style="text-align: center;"| -0.76<br /> | style="text-align: center;"| Remove synonym dictionaries: as acronym dictionary constructed automatically from the corpus and a synonym dictionary that contains geographical terms.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | JU_CSE_TAC1_abl-1<br /> | style="text-align: center;"| 13.29<br /> | style="text-align: center;"| The Run-1 is based on the composition of lexical based RTE methods and Syntactic RTE Method. The lexical based RTE methods are: WordNet based unigram match, bigram match, longest common sub-sequence, skip-gram and stemming. Here we ablated the WordNet based unigram match only.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | JU_CSE_TAC2_abl-1<br /> | style="text-align: center;"| 10.19<br /> | style="text-align: center;"| The Run-2 is based on the composition of lexical based RTE methods, Syntactic RTE Method, Chunk and Named Entity Methods. The lexical based RTE methods are: WordNet based unigram match, bigram match, longest common sub-sequence, skip-gram and stemming. Here we ablated the WordNet based unigram match only.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | JU_CSE_TAC3_abl-1<br /> | style="text-align: center;"| 3.86<br /> | style="text-align: center;"| The Run-3 is based on the Support Vector Machine that uses twenty five features for lexical similarity, the output tag from a rule based syntactic two-way TE system as feature, and output from a rule based Chunk Module and Named Entity Module. The important lexical features that are used in the present system are: WordNet based unigram match, bigram match, longest common sub-sequence, skip-gram, stemming and lexical distance (17 features). Here we ablated the WordNet based unigram match only.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | LingPipe co-reference<br /> | PKUTM2_abl-1<br /> | style="text-align: center;"| 0.17<br /> | style="text-align: center;"| Lingpipe co-reference are removed, the experiment was based on named-entity, wordnet, verbocean<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | PKUTM2_abl-2<br /> | style="text-align: center;"| 1.02<br /> | style="text-align: center;"| Verbocean are removed, the experiment was based on named-entity, wordnet, co-reference<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | LingPipe Named Entities<br /> | PKUTM2_abl-3<br /> | style="text-align: center;"| 13.84<br /> | style="text-align: center;"| Lingpipe named-entity are removed, the experiment was based on wordnet, co-reference, verbocean<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | saicnlp1_abl-1<br /> | style="text-align: center;"| -0.02<br /> | style="text-align: center;"| Ablation run, with WordNet stubbed.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Shalmanesar Parser<br /> | Sangyan1_abl-1<br /> | style="text-align: center;"| -1.76<br /> | style="text-align: center;"| This ablation run was executed with the Shalmanesar parser ablated from our system. Our system uses the Shalmanesar Parser for Framenet frame assignment.<br /> Our system required minor tweaking of the source code in order ablate Shalmanesar Parser.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | Sangyan1_abl-2<br /> | style="text-align: center;"| -0.39<br /> | style="text-align: center;"| This ablation run was executed with the WordNet ablated from our system. Our system uses the Wordnet for Word mathcing.<br /> Our system required minor tweaking of the source code in order ablate Wordnet.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | Sangyan1_abl-3<br /> | style="text-align: center;"| 0.07<br /> | style="text-align: center;"| This ablation run was executed with the Verb Ocean ablated from our system. Our system uses the Verb Ocean resource for detection of antonyms. Our system required minor tweaking of the source code in order ablate Verb Ocean resource.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entities Recognition<br /> | SINAI1_abl-1<br /> | style="text-align: center;"| 7.62<br /> | style="text-align: center;"| Only PPVs on WN 1.7<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wordnet<br /> | SINAI1_abl-2<br /> | style="text-align: center;"| 15.87<br /> | style="text-align: center;"| Only NEs on WN 1.7<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entities Recognition<br /> | SINAI2_abl-1<br /> | style="text-align: center;"| 20.25<br /> | style="text-align: center;"| Only PPVs on WN 3.0<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wordnet<br /> | SINAI2_abl-2<br /> | style="text-align: center;"| 18.28<br /> | style="text-align: center;"| Only NEs on WN 3.0<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wordnet<br /> | SJTU_CIT3_abl-1<br /> | style="text-align: center;"| 0.03<br /> | style="text-align: center;"| The resource that has been ablated is WordNet.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | SJTU_CIT3_abl-2<br /> | style="text-align: center;"| 4.7<br /> | style="text-align: center;"| The resource that has been ablated is Wikipedia.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | SJTU_CIT3_abl-3<br /> | style="text-align: center;"| -1.15<br /> | style="text-align: center;"| The resource that has been ablated is VerbOcean.<br /> <br /> <br /> <br /> |}<br /> <br><br /> ==Footnotes==<br /> <references /><br /> <br /> <br /> Return to [[RTE Knowledge Resources]]</div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE6_-_Ablation_Tests&diff=8656 RTE6 - Ablation Tests 2011-02-03T13:10:27Z

<p>Amarchetti: </p> <hr /> <div>The following table lists the results of the ablation tests (a mandatory track since the RTE5 campaign), submitted by participants to RTE6 .<br><br><br /> <br /> <br /> Participants are kindly invited to check if all the inserted information is correct and complete.<br /> <br /> <br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! Ablated Component<br /> ! Ablation Run<ref>For further information about participants, click here: [[RTE Challenges - Data about participants]]</ref><br /> ! <small>Resource impact - F1 </small><br /> ! Resource Usage Description<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | BIU1_abl-1<br /> | style="text-align: center;"| 0.9<br /> | style="text-align: center;"| No Word-Net. On Dev set: 39.18% (compared to 40.73% when WN is used)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | CatVar<br /> | BIU1_abl-2<br /> | style="text-align: center;"| 0.63<br /> | style="text-align: center;"| No CatVar. On Dev set achieved about 40.20% (compared to 40.73% when CatVar is used)<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Coreference resolver<br /> | BIU1_abl-3<br /> | style="text-align: center;"| -0.88<br /> | style="text-align: center;"| No coreference resolver<br /> On Dev set 41.62% (Compared to 40.73% when Coreference resolver is used).<br /> This ablation test is an unusual ablation test, since it shows that the co-reference resolution component has a negative impact.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DIRT<br /> | Boeing1_abl-1<br /> | style="text-align: center;"| 3.97<br /> | style="text-align: center;"| DIRT removed<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | Boeing1_abl-2<br /> | style="text-align: center;"| 4.42<br /> | style="text-align: center;"| No WordNet<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Name Normalization<br /> | budapestcad2_abl-2<br /> | style="text-align: center;"| 0.65<br /> | style="text-align: center;"| no name normalization was performed (e.g. George W. Bush -> Bush).<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entities Recognition<br /> | budapestcad2_abl-3<br /> | style="text-align: center;"| -1.23<br /> | style="text-align: center;"| no NER<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | budapestcad2_abl-4<br /> | style="text-align: center;"| -1.11<br /> | style="text-align: center;"| No WordNet. (In the original run, WordNet was used to find the synonyms of words in the triplets, and additional triplets were generated from all possible combinations.)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb1_abl-1<br /> | style="text-align: center;"| 8.68<br /> | style="text-align: center;"| Wordnet is albated in this test.No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | deb_iitb1_abl-2<br /> | style="text-align: center;"| 1.87<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb2_abl-1<br /> | style="text-align: center;"| 7.9<br /> | style="text-align: center;"| Wordnet is albated in this test.No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | deb_iitb2_abl-2<br /> | style="text-align: center;"| 0.94<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb3_abl-1<br /> | style="text-align: center;"| 11.43<br /> | style="text-align: center;"| Wordnet is albated in this test. No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb3_abl-2<br /> | style="text-align: center;"| 2.54<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | POS-Tagger<br /> | DFKI1_abl-4<br /> | style="text-align: center;"| 4.99<br /> | style="text-align: center;"| No wordform/POS-tags included for the comparison.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | POS-Tagger<br /> | DFKI1_abl-6<br /> | style="text-align: center;"| 2.22<br /> | style="text-align: center;"| No named entity recognition for the comparison.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | DFKI1_abl-7<br /> | style="text-align: center;"| -0.23<br /> | style="text-align: center;"| No WordNet similarity for the comparison.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Coreference resolver<br /> | DFKI1_Main<br /> | style="text-align: center;"| -1.54<br /> | style="text-align: center;"| Coreference resolution used for the comparison.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | DirRelCond23_abl-1<br /> | style="text-align: center;"| 8.43<br /> | style="text-align: center;"| WordNet removed. Only basic word comparison used instead of word relations.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -23.91<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from Wikipedia<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -3.58<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from Wikipedia with probability above 0.7<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Proximity similarity dictionary of Dekang Lin<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -7.79<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from proximity similarity dictionary of Dekang Lin<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -3.21<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from WordNet<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -2.08<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from WordNet with probability above 0.7<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -4<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from Verbocean<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Dependency similarity dictionary of Dekang Lin<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -13.56<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from dependency similarity dictionary of Dekang Lin<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Dictionary of Named Entities Acronyms and Synonyms <br /> | IKOMA2_abl-3<br /> | style="text-align: center;"| -0.76<br /> | style="text-align: center;"| Remove synonym dictionaries: as acronym dictionary constructed automatically from the corpus and a synonym dictionary that contains geographical terms.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | JU_CSE_TAC1_abl-1<br /> | style="text-align: center;"| 13.29<br /> | style="text-align: center;"| The Run-1 is based on the composition of lexical based RTE methods and Syntactic RTE Method. The lexical based RTE methods are: WordNet based unigram match, bigram match, longest common sub-sequence, skip-gram and stemming. Here we ablated the WordNet based unigram match only.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | JU_CSE_TAC2_abl-1<br /> | style="text-align: center;"| 10.19<br /> | style="text-align: center;"| The Run-2 is based on the composition of lexical based RTE methods, Syntactic RTE Method, Chunk and Named Entity Methods. The lexical based RTE methods are: WordNet based unigram match, bigram match, longest common sub-sequence, skip-gram and stemming. Here we ablated the WordNet based unigram match only.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | JU_CSE_TAC3_abl-1<br /> | style="text-align: center;"| 3.86<br /> | style="text-align: center;"| The Run-3 is based on the Support Vector Machine that uses twenty five features for lexical similarity, the output tag from a rule based syntactic two-way TE system as feature, and output from a rule based Chunk Module and Named Entity Module. The important lexical features that are used in the present system are: WordNet based unigram match, bigram match, longest common sub-sequence, skip-gram, stemming and lexical distance (17 features). Here we ablated the WordNet based unigram match only.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | LingPipe co-reference<br /> | PKUTM2_abl-1<br /> | style="text-align: center;"| 0.17<br /> | style="text-align: center;"| Lingpipe co-reference are removed, the experiment was based on named-entity, wordnet, verbocean<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | PKUTM2_abl-2<br /> | style="text-align: center;"| 1.02<br /> | style="text-align: center;"| Verbocean are removed, the experiment was based on named-entity, wordnet, co-reference<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | LingPipe Named Entities<br /> | PKUTM2_abl-3<br /> | style="text-align: center;"| 13.84<br /> | style="text-align: center;"| Lingpipe named-entity are removed, the experiment was based on wordnet, co-reference, verbocean<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | saicnlp1_abl-1<br /> | style="text-align: center;"| -0.02<br /> | style="text-align: center;"| Ablation run, with WordNet stubbed.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Shalmanesar Parser<br /> | Sangyan1_abl-1<br /> | style="text-align: center;"| -1.76<br /> | style="text-align: center;"| This ablation run was executed with the Shalmanesar parser ablated from our system. Our system uses the Shalmanesar Parser for Framenet frame assignment.<br /> Our system required minor tweaking of the source code in order ablate Shalmanesar Parser.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | Sangyan1_abl-2<br /> | style="text-align: center;"| -0.39<br /> | style="text-align: center;"| This ablation run was executed with the WordNet ablated from our system. Our system uses the Wordnet for Word mathcing.<br /> Our system required minor tweaking of the source code in order ablate Wordnet.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | Sangyan1_abl-3<br /> | style="text-align: center;"| 0.07<br /> | style="text-align: center;"| This ablation run was executed with the Verb Ocean ablated from our system. Our system uses the Verb Ocean resource for detection of antonyms. Our system required minor tweaking of the source code in order ablate Verb Ocean resource.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entities Recognition<br /> | SINAI1_abl-1<br /> | style="text-align: center;"| 7.62<br /> | style="text-align: center;"| Only PPVs on WN 1.7<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wordnet<br /> | SINAI1_abl-2<br /> | style="text-align: center;"| 15.87<br /> | style="text-align: center;"| Only NEs on WN 1.7<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entities Recognition<br /> | SINAI2_abl-1<br /> | style="text-align: center;"| 20.25<br /> | style="text-align: center;"| Only PPVs on WN 3.0<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wordnet<br /> | SINAI2_abl-2<br /> | style="text-align: center;"| 18.28<br /> | style="text-align: center;"| Only NEs on WN 3.0<br /> <br /> |}<br /> <br><br /> ==Footnotes==<br /> <references /><br /> <br /> <br /> Return to [[RTE Knowledge Resources]]</div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE6_-_Ablation_Tests&diff=8655 RTE6 - Ablation Tests 2011-02-03T12:17:29Z

<p>Amarchetti: </p> <hr /> <div>The following table lists the results of the ablation tests (a mandatory track since the RTE5 campaign), submitted by participants to RTE6 .<br><br><br /> <br /> <br /> Participants are kindly invited to check if all the inserted information is correct and complete.<br /> <br /> <br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! Ablated Component<br /> ! Ablation Run<ref>For further information about participants, click here: [[RTE Challenges - Data about participants]]</ref><br /> ! <small>Resource impact - F1 </small><br /> ! Resource Usage Description<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | BIU1_abl-1<br /> | style="text-align: center;"| 0.9<br /> | style="text-align: center;"| No Word-Net. On Dev set: 39.18% (compared to 40.73% when WN is used)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | CatVar<br /> | BIU1_abl-2<br /> | style="text-align: center;"| 0.63<br /> | style="text-align: center;"| No CatVar. On Dev set achieved about 40.20% (compared to 40.73% when CatVar is used)<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Coreference resolver<br /> | BIU1_abl-3<br /> | style="text-align: center;"| -0.88<br /> | style="text-align: center;"| No coreference resolver<br /> On Dev set 41.62% (Compared to 40.73% when Coreference resolver is used).<br /> This ablation test is an unusual ablation test, since it shows that the co-reference resolution component has a negative impact.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DIRT<br /> | Boeing1_abl-1<br /> | style="text-align: center;"| 3.97<br /> | style="text-align: center;"| DIRT removed<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | Boeing1_abl-2<br /> | style="text-align: center;"| 4.42<br /> | style="text-align: center;"| No WordNet<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Name Normalization<br /> | budapestcad2_abl-2<br /> | style="text-align: center;"| 0.65<br /> | style="text-align: center;"| no name normalization was performed (e.g. George W. Bush -> Bush).<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entities Recognition<br /> | budapestcad2_abl-3<br /> | style="text-align: center;"| -1.23<br /> | style="text-align: center;"| no NER<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | budapestcad2_abl-4<br /> | style="text-align: center;"| -1.11<br /> | style="text-align: center;"| No WordNet. (In the original run, WordNet was used to find the synonyms of words in the triplets, and additional triplets were generated from all possible combinations.)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb1_abl-1<br /> | style="text-align: center;"| 8.68<br /> | style="text-align: center;"| Wordnet is albated in this test.No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | deb_iitb1_abl-2<br /> | style="text-align: center;"| 1.87<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb2_abl-1<br /> | style="text-align: center;"| 7.9<br /> | style="text-align: center;"| Wordnet is albated in this test.No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | deb_iitb2_abl-2<br /> | style="text-align: center;"| 0.94<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb3_abl-1<br /> | style="text-align: center;"| 11.43<br /> | style="text-align: center;"| Wordnet is albated in this test. No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb3_abl-2<br /> | style="text-align: center;"| 2.54<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | POS-Tagger<br /> | DFKI1_abl-4<br /> | style="text-align: center;"| 4.99<br /> | style="text-align: center;"| No wordform/POS-tags included for the comparison.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | POS-Tagger<br /> | DFKI1_abl-6<br /> | style="text-align: center;"| 2.22<br /> | style="text-align: center;"| No named entity recognition for the comparison.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | DFKI1_abl-7<br /> | style="text-align: center;"| -0.23<br /> | style="text-align: center;"| No WordNet similarity for the comparison.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Coreference resolver<br /> | DFKI1_Main<br /> | style="text-align: center;"| -1.54<br /> | style="text-align: center;"| Coreference resolution used for the comparison.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | DirRelCond23_abl-1<br /> | style="text-align: center;"| 8.43<br /> | style="text-align: center;"| WordNet removed. Only basic word comparison used instead of word relations.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -23.91<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from Wikipedia<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -3.58<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from Wikipedia with probability above 0.7<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Proximity similarity dictionary of Dekang Lin<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -7.79<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from proximity similarity dictionary of Dekang Lin<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -3.21<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from WordNet<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -2.08<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from WordNet with probability above 0.7<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -4<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from Verbocean<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Dependency similarity dictionary of Dekang Lin<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -13.56<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from dependency similarity dictionary of Dekang Lin<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Dictionary of Named Entities Acronyms and Synonyms <br /> | IKOMA2_abl-3<br /> | style="text-align: center;"| -0.76<br /> | style="text-align: center;"| Remove synonym dictionaries: as acronym dictionary constructed automatically from the corpus and a synonym dictionary that contains geographical terms.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | JU_CSE_TAC1_abl-1<br /> | style="text-align: center;"| 13.29<br /> | style="text-align: center;"| The Run-1 is based on the composition of lexical based RTE methods and Syntactic RTE Method. The lexical based RTE methods are: WordNet based unigram match, bigram match, longest common sub-sequence, skip-gram and stemming. Here we ablated the WordNet based unigram match only.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | JU_CSE_TAC2_abl-1<br /> | style="text-align: center;"| 10.19<br /> | style="text-align: center;"| The Run-2 is based on the composition of lexical based RTE methods, Syntactic RTE Method, Chunk and Named Entity Methods. The lexical based RTE methods are: WordNet based unigram match, bigram match, longest common sub-sequence, skip-gram and stemming. Here we ablated the WordNet based unigram match only.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | JU_CSE_TAC3_abl-1<br /> | style="text-align: center;"| 3.86<br /> | style="text-align: center;"| The Run-3 is based on the Support Vector Machine that uses twenty five features for lexical similarity, the output tag from a rule based syntactic two-way TE system as feature, and output from a rule based Chunk Module and Named Entity Module. The important lexical features that are used in the present system are: WordNet based unigram match, bigram match, longest common sub-sequence, skip-gram, stemming and lexical distance (17 features). Here we ablated the WordNet based unigram match only.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | LingPipe co-reference<br /> | PKUTM2_abl-1<br /> | style="text-align: center;"| 0.17<br /> | style="text-align: center;"| Lingpipe co-reference are removed, the experiment was based on named-entity, wordnet, verbocean<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | PKUTM2_abl-2<br /> | style="text-align: center;"| 1.02<br /> | style="text-align: center;"| Verbocean are removed, the experiment was based on named-entity, wordnet, co-reference<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | LingPipe Named Entities<br /> | PKUTM2_abl-3<br /> | style="text-align: center;"| 13.84<br /> | style="text-align: center;"| Lingpipe named-entity are removed, the experiment was based on wordnet, co-reference, verbocean<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | saicnlp1_abl-1<br /> | style="text-align: center;"| -0.02<br /> | style="text-align: center;"| Ablation run, with WordNet stubbed.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Shalmanesar Parser<br /> | Sangyan1_abl-1<br /> | style="text-align: center;"| -1.76<br /> | style="text-align: center;"| This ablation run was executed with the Shalmanesar parser ablated from our system. Our system uses the Shalmanesar Parser for Framenet frame assignment.<br /> Our system required minor tweaking of the source code in order ablate Shalmanesar Parser.<br /> <br /> |}<br /> <br><br /> ==Footnotes==<br /> <references /><br /> <br /> <br /> Return to [[RTE Knowledge Resources]]</div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE6_-_Ablation_Tests&diff=8654 RTE6 - Ablation Tests 2011-02-03T12:15:49Z

<p>Amarchetti: </p> <hr /> <div>The following table lists the results of the ablation tests (a mandatory track since the RTE5 campaign), submitted by participants to RTE6 .<br><br><br /> <br /> <br /> Participants are kindly invited to check if all the inserted information is correct and complete.<br /> <br /> <br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! Ablated Component<br /> ! Ablation Run<ref>For further information about participants, click here: [[RTE Challenges - Data about participants]]</ref><br /> ! <small>Resource impact - F1 </small><br /> ! Resource Usage Description<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | BIU1_abl-1<br /> | style="text-align: center;"| 0.9<br /> | style="text-align: center;"| No Word-Net. On Dev set: 39.18% (compared to 40.73% when WN is used)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | CatVar<br /> | BIU1_abl-2<br /> | style="text-align: center;"| 0.63<br /> | style="text-align: center;"| No CatVar. On Dev set achieved about 40.20% (compared to 40.73% when CatVar is used)<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Coreference resolver<br /> | BIU1_abl-3<br /> | style="text-align: center;"| -0.88<br /> | style="text-align: center;"| No coreference resolver<br /> On Dev set 41.62% (Compared to 40.73% when Coreference resolver is used).<br /> This ablation test is an unusual ablation test, since it shows that the co-reference resolution component has a negative impact.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DIRT<br /> | Boeing1_abl-1<br /> | style="text-align: center;"| 3.97<br /> | style="text-align: center;"| DIRT removed<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | Boeing1_abl-2<br /> | style="text-align: center;"| 4.42<br /> | style="text-align: center;"| No WordNet<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Name Normalization<br /> | budapestcad2_abl-2<br /> | style="text-align: center;"| 0.65<br /> | style="text-align: center;"| no name normalization was performed (e.g. George W. Bush -> Bush).<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entities Recognition<br /> | budapestcad2_abl-3<br /> | style="text-align: center;"| -1.23<br /> | style="text-align: center;"| no NER<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | budapestcad2_abl-4<br /> | style="text-align: center;"| -1.11<br /> | style="text-align: center;"| No WordNet. (In the original run, WordNet was used to find the synonyms of words in the triplets, and additional triplets were generated from all possible combinations.)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb1_abl-1<br /> | style="text-align: center;"| 8.68<br /> | style="text-align: center;"| Wordnet is albated in this test.No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | deb_iitb1_abl-2<br /> | style="text-align: center;"| 1.87<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb2_abl-1<br /> | style="text-align: center;"| 7.9<br /> | style="text-align: center;"| Wordnet is albated in this test.No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | deb_iitb2_abl-2<br /> | style="text-align: center;"| 0.94<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb3_abl-1<br /> | style="text-align: center;"| 11.43<br /> | style="text-align: center;"| Wordnet is albated in this test. No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb3_abl-2<br /> | style="text-align: center;"| 2.54<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | POS-Tagger<br /> | DFKI1_abl-4<br /> | style="text-align: center;"| 4.99<br /> | style="text-align: center;"| No wordform/POS-tags included for the comparison.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | POS-Tagger<br /> | DFKI1_abl-6<br /> | style="text-align: center;"| 2.22<br /> | style="text-align: center;"| No named entity recognition for the comparison.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | DFKI1_abl-7<br /> | style="text-align: center;"| -0.23<br /> | style="text-align: center;"| No WordNet similarity for the comparison.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Coreference resolver<br /> | DFKI1_Main<br /> | style="text-align: center;"| -1.54<br /> | style="text-align: center;"| Coreference resolution used for the comparison.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | DirRelCond23_abl-1<br /> | style="text-align: center;"| 8.43<br /> | style="text-align: center;"| WordNet removed. Only basic word comparison used instead of word relations.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -23.91<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from Wikipedia<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -3.58<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from Wikipedia with probability above 0.7<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Proximity similarity dictionary of Dekang Lin<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -7.79<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from proximity similarity dictionary of Dekang Lin<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -3.21<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from WordNet<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -2.08<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from WordNet with probability above 0.7<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -4<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from Verbocean<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Dependency similarity dictionary of Dekang Lin<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -13.56<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from dependency similarity dictionary of Dekang Lin<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Dictionary of Named Entities Acronyms and Synonyms <br /> | IKOMA2_abl-3<br /> | style="text-align: center;"| -0.76<br /> | style="text-align: center;"| Remove synonym dictionaries: as acronym dictionary constructed automatically from the corpus and a synonym dictionary that contains geographical terms.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | JU_CSE_TAC1_abl-1<br /> | style="text-align: center;"| 13.29<br /> | style="text-align: center;"| The Run-1 is based on the composition of lexical based RTE methods and Syntactic RTE Method. The lexical based RTE methods are: WordNet based unigram match, bigram match, longest common sub-sequence, skip-gram and stemming. Here we ablated the WordNet based unigram match only.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | JU_CSE_TAC2_abl-1<br /> | style="text-align: center;"| 10.19<br /> | style="text-align: center;"| The Run-2 is based on the composition of lexical based RTE methods, Syntactic RTE Method, Chunk and Named Entity Methods. The lexical based RTE methods are: WordNet based unigram match, bigram match, longest common sub-sequence, skip-gram and stemming. Here we ablated the WordNet based unigram match only.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | JU_CSE_TAC3_abl-1<br /> | style="text-align: center;"| 3.86<br /> | style="text-align: center;"| The Run-3 is based on the Support Vector Machine that uses twenty five features for lexical similarity, the output tag from a rule based syntactic two-way TE system as feature, and output from a rule based Chunk Module and Named Entity Module. The important lexical features that are used in the present system are: WordNet based unigram match, bigram match, longest common sub-sequence, skip-gram, stemming and lexical distance (17 features). Here we ablated the WordNet based unigram match only.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | LingPipe co-reference<br /> | PKUTM2_abl-1<br /> | style="text-align: center;"| 0.17<br /> | style="text-align: center;"| Lingpipe co-reference are removed, the experiment was based on named-entity, wordnet, verbocean<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | PKUTM2_abl-2<br /> | style="text-align: center;"| 1.02<br /> | style="text-align: center;"| Verbocean are removed, the experiment was based on named-entity, wordnet, co-reference<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | LingPipe Named Entities<br /> | PKUTM2_abl-3<br /> | style="text-align: center;"| 13.84<br /> | style="text-align: center;"| Lingpipe named-entity are removed, the experiment was based on wordnet, co-reference, verbocean<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | saicnlp1_abl-1<br /> | style="text-align: center;"| -0.02<br /> | style="text-align: center;"| Ablation run, with WordNet stubbed.<br /> <br /> |}<br /> <br><br /> ==Footnotes==<br /> <references /><br /> <br /> <br /> Return to [[RTE Knowledge Resources]]</div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE6_-_Ablation_Tests&diff=8653 RTE6 - Ablation Tests 2011-02-03T11:40:17Z

<p>Amarchetti: </p> <hr /> <div>The following table lists the results of the ablation tests (a mandatory track since the RTE5 campaign), submitted by participants to RTE6 .<br><br><br /> <br /> <br /> Participants are kindly invited to check if all the inserted information is correct and complete.<br /> <br /> <br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! Ablated Component<br /> ! Ablation Run<ref>For further information about participants, click here: [[RTE Challenges - Data about participants]]</ref><br /> ! <small>Resource impact - F1 </small><br /> ! Resource Usage Description<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | BIU1_abl-1<br /> | style="text-align: center;"| 0.9<br /> | style="text-align: center;"| No Word-Net. On Dev set: 39.18% (compared to 40.73% when WN is used)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | CatVar<br /> | BIU1_abl-2<br /> | style="text-align: center;"| 0.63<br /> | style="text-align: center;"| No CatVar. On Dev set achieved about 40.20% (compared to 40.73% when CatVar is used)<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Coreference resolver<br /> | BIU1_abl-3<br /> | style="text-align: center;"| -0.88<br /> | style="text-align: center;"| No coreference resolver<br /> On Dev set 41.62% (Compared to 40.73% when Coreference resolver is used).<br /> This ablation test is an unusual ablation test, since it shows that the co-reference resolution component has a negative impact.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DIRT<br /> | Boeing1_abl-1<br /> | style="text-align: center;"| 3.97<br /> | style="text-align: center;"| DIRT removed<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | Boeing1_abl-2<br /> | style="text-align: center;"| 4.42<br /> | style="text-align: center;"| No WordNet<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Name Normalization<br /> | budapestcad2_abl-2<br /> | style="text-align: center;"| 0.65<br /> | style="text-align: center;"| no name normalization was performed (e.g. George W. Bush -> Bush).<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entities Recognition<br /> | budapestcad2_abl-3<br /> | style="text-align: center;"| -1.23<br /> | style="text-align: center;"| no NER<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | budapestcad2_abl-4<br /> | style="text-align: center;"| -1.11<br /> | style="text-align: center;"| No WordNet. (In the original run, WordNet was used to find the synonyms of words in the triplets, and additional triplets were generated from all possible combinations.)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb1_abl-1<br /> | style="text-align: center;"| 8.68<br /> | style="text-align: center;"| Wordnet is albated in this test.No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | deb_iitb1_abl-2<br /> | style="text-align: center;"| 1.87<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb2_abl-1<br /> | style="text-align: center;"| 7.9<br /> | style="text-align: center;"| Wordnet is albated in this test.No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | deb_iitb2_abl-2<br /> | style="text-align: center;"| 0.94<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb3_abl-1<br /> | style="text-align: center;"| 11.43<br /> | style="text-align: center;"| Wordnet is albated in this test. No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb3_abl-2<br /> | style="text-align: center;"| 2.54<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | POS-Tagger<br /> | DFKI1_abl-4<br /> | style="text-align: center;"| 4.99<br /> | style="text-align: center;"| No wordform/POS-tags included for the comparison.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | POS-Tagger<br /> | DFKI1_abl-6<br /> | style="text-align: center;"| 2.22<br /> | style="text-align: center;"| No named entity recognition for the comparison.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | DFKI1_abl-7<br /> | style="text-align: center;"| -0.23<br /> | style="text-align: center;"| No WordNet similarity for the comparison.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Coreference resolver<br /> | DFKI1_Main<br /> | style="text-align: center;"| -1.54<br /> | style="text-align: center;"| Coreference resolution used for the comparison.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | DirRelCond23_abl-1<br /> | style="text-align: center;"| 8.43<br /> | style="text-align: center;"| WordNet removed. Only basic word comparison used instead of word relations.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -23.91<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from Wikipedia<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Wikipedia<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -3.58<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from Wikipedia with probability above 0.7<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Proximity similarity dictionary of Dekang Lin<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -7.79<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from proximity similarity dictionary of Dekang Lin<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -3.21<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from WordNet<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -2.08<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from WordNet with probability above 0.7<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -4<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from Verbocean<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Dependency similarity dictionary of Dekang Lin<br /> | FBK_irst3_Main<br /> | style="text-align: center;"| -13.56<br /> | style="text-align: center;"| This run is produced by the system configuration for run3 and uses rules extracted from dependency similarity dictionary of Dekang Lin<br /> <br /> <br /> <br /> |}<br /> <br><br /> ==Footnotes==<br /> <references /><br /> <br /> <br /> Return to [[RTE Knowledge Resources]]</div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE6_-_Ablation_Tests&diff=8651 RTE6 - Ablation Tests 2011-02-02T16:52:24Z

<p>Amarchetti: </p> <hr /> <div>The following table lists the results of the ablation tests (a mandatory track since the RTE5 campaign), submitted by participants to RTE6 .<br><br><br /> <br /> <br /> Participants are kindly invited to check if all the inserted information is correct and complete.<br /> <br /> <br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! Ablated Component<br /> ! Ablation Run<ref>For further information about participants, click here: [[RTE Challenges - Data about participants]]</ref><br /> ! <small>Resource impact - F1 </small><br /> ! Resource Usage Description<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | BIU1_abl-1<br /> | style="text-align: center;"| 0.9<br /> | style="text-align: center;"| No Word-Net. On Dev set: 39.18% (compared to 40.73% when WN is used)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | CatVar<br /> | BIU1_abl-2<br /> | style="text-align: center;"| 0.63<br /> | style="text-align: center;"| No CatVar. On Dev set achieved about 40.20% (compared to 40.73% when CatVar is used)<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Coreference resolver<br /> | BIU1_abl-3<br /> | style="text-align: center;"| -0.88<br /> | style="text-align: center;"| No coreference resolver<br /> On Dev set 41.62% (Compared to 40.73% when Coreference resolver is used).<br /> This ablation test is an unusual ablation test, since it shows that the co-reference resolution component has a negative impact.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DIRT<br /> | Boeing1_abl-1<br /> | style="text-align: center;"| 3.97<br /> | style="text-align: center;"| DIRT removed<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | Boeing1_abl-2<br /> | style="text-align: center;"| 4.42<br /> | style="text-align: center;"| No WordNet<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Name Normalization<br /> | budapestcad2_abl-2<br /> | style="text-align: center;"| 0.65<br /> | style="text-align: center;"| no name normalization was performed (e.g. George W. Bush -> Bush).<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entities Recognition<br /> | budapestcad2_abl-3<br /> | style="text-align: center;"| -1.23<br /> | style="text-align: center;"| no NER<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | budapestcad2_abl-4<br /> | style="text-align: center;"| -1.11<br /> | style="text-align: center;"| No WordNet. (In the original run, WordNet was used to find the synonyms of words in the triplets, and additional triplets were generated from all possible combinations.)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb1_abl-1<br /> | style="text-align: center;"| 8.68<br /> | style="text-align: center;"| Wordnet is albated in this test.No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | deb_iitb1_abl-2<br /> | style="text-align: center;"| 1.87<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb2_abl-1<br /> | style="text-align: center;"| 7.9<br /> | style="text-align: center;"| Wordnet is albated in this test.No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | deb_iitb2_abl-2<br /> | style="text-align: center;"| 0.94<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb3_abl-1<br /> | style="text-align: center;"| 11.43<br /> | style="text-align: center;"| Wordnet is albated in this test. No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb3_abl-2<br /> | style="text-align: center;"| 2.54<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | POS-Tagger<br /> | DFKI1_abl-4<br /> | style="text-align: center;"| 4.99<br /> | style="text-align: center;"| No wordform/POS-tags included for the comparison.<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | POS-Tagger<br /> | DFKI1_abl-6<br /> | style="text-align: center;"| 2.22<br /> | style="text-align: center;"| No named entity recognition for the comparison.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | DFKI1_abl-7<br /> | style="text-align: center;"| -0.23<br /> | style="text-align: center;"| No WordNet similarity for the comparison.<br /> |}<br /> <br><br /> ==Footnotes==<br /> <references /><br /> <br /> <br /> Return to [[RTE Knowledge Resources]]</div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE6_-_Ablation_Tests&diff=8650 RTE6 - Ablation Tests 2011-02-02T16:20:10Z

<p>Amarchetti: </p> <hr /> <div>The following table lists the results of the ablation tests (a mandatory track since the RTE5 campaign), submitted by participants to RTE6 .<br><br><br /> <br /> <br /> Participants are kindly invited to check if all the inserted information is correct and complete.<br /> <br /> <br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! Ablated Component<br /> ! Ablation Run<ref>For further information about participants, click here: [[RTE Challenges - Data about participants]]</ref><br /> ! <small>Resource impact - F1 </small><br /> ! Resource Usage Description<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | BIU1_abl-1<br /> | style="text-align: center;"| 0.9<br /> | style="text-align: center;"| No Word-Net. On Dev set: 39.18% (compared to 40.73% when WN is used)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | CatVar<br /> | BIU1_abl-2<br /> | style="text-align: center;"| 0.63<br /> | style="text-align: center;"| No CatVar. On Dev set achieved about 40.20% (compared to 40.73% when CatVar is used)<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Coreference resolver<br /> | BIU1_abl-3<br /> | style="text-align: center;"| -0.88<br /> | style="text-align: center;"| No coreference resolver<br /> On Dev set 41.62% (Compared to 40.73% when Coreference resolver is used).<br /> This ablation test is an unusual ablation test, since it shows that the co-reference resolution component has a negative impact.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DIRT<br /> | Boeing1_abl-1<br /> | style="text-align: center;"| 3.97<br /> | style="text-align: center;"| DIRT removed<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | Boeing1_abl-2<br /> | style="text-align: center;"| 4.42<br /> | style="text-align: center;"| No WordNet<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Name Normalization<br /> | budapestcad2_abl-2<br /> | style="text-align: center;"| 0.65<br /> | style="text-align: center;"| no name normalization was performed (e.g. George W. Bush -> Bush).<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entities Recognition<br /> | budapestcad2_abl-3<br /> | style="text-align: center;"| -1.23<br /> | style="text-align: center;"| no NER<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | budapestcad2_abl-4<br /> | style="text-align: center;"| -1.11<br /> | style="text-align: center;"| No WordNet. (In the original run, WordNet was used to find the synonyms of words in the triplets, and additional triplets were generated from all possible combinations.)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb1_abl-1<br /> | style="text-align: center;"| 8.68<br /> | style="text-align: center;"| Wordnet is albated in this test.No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | deb_iitb1_abl-2<br /> | style="text-align: center;"| 1.87<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb2_abl-1<br /> | style="text-align: center;"| 7.9<br /> | style="text-align: center;"| Wordnet is albated in this test.No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | VerbOcean<br /> | deb_iitb2_abl-2<br /> | style="text-align: center;"| 0.94<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb3_abl-1<br /> | style="text-align: center;"| 11.43<br /> | style="text-align: center;"| Wordnet is albated in this test. No change of code required only wordnet module is removed while matching.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | deb_iitb3_abl-2<br /> | style="text-align: center;"| 2.54<br /> | style="text-align: center;"| VerbOcean is albated in this test.No change of code required only VerbOcean module is removed while matching.<br /> |}<br /> <br><br /> ==Footnotes==<br /> <references /><br /> <br /> <br /> Return to [[RTE Knowledge Resources]]</div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE6_-_Ablation_Tests&diff=8649 RTE6 - Ablation Tests 2011-02-02T15:46:11Z

<p>Amarchetti: </p> <hr /> <div>The following table lists the results of the ablation tests (a mandatory track since the RTE5 campaign), submitted by participants to RTE6 .<br><br><br /> <br /> <br /> Participants are kindly invited to check if all the inserted information is correct and complete.<br /> <br /> <br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! Ablated Component<br /> ! Ablation Run<ref>For further information about participants, click here: [[RTE Challenges - Data about participants]]</ref><br /> ! <small>Resource impact - F1 </small><br /> ! Resource Usage Description<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | BIU1_abl-1<br /> | style="text-align: center;"| 0.9<br /> | style="text-align: center;"| No Word-Net. On Dev set: 39.18% (compared to 40.73% when WN is used)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | CatVar<br /> | BIU1_abl-2<br /> | style="text-align: center;"| 0.63<br /> | style="text-align: center;"| No CatVar. On Dev set achieved about 40.20% (compared to 40.73% when CatVar is used)<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Coreference resolver<br /> | BIU1_abl-3<br /> | style="text-align: center;"| -0.88<br /> | style="text-align: center;"| No coreference resolver<br /> On Dev set 41.62% (Compared to 40.73% when Coreference resolver is used).<br /> This ablation test is an unusual ablation test, since it shows that the co-reference resolution component has a negative impact.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DIRT<br /> | Boeing1_abl-1<br /> | style="text-align: center;"| 3.97<br /> | style="text-align: center;"| DIRT removed<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | Boeing1_abl-2<br /> | style="text-align: center;"| 4.42<br /> | style="text-align: center;"| No WordNet<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Name Normalization<br /> | budapestcad2_abl2<br /> | style="text-align: center;"| 0.65<br /> | style="text-align: center;"| no name normalization was performed (e.g. George W. Bush -> Bush).<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Named Entities Recognition<br /> | budapestcad2_abl3<br /> | style="text-align: center;"| -1.23<br /> | style="text-align: center;"| no NER<br /> <br /> |}<br /> <br><br /> ==Footnotes==<br /> <references /><br /> <br /> <br /> Return to [[RTE Knowledge Resources]]</div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE6_-_Ablation_Tests&diff=8648 RTE6 - Ablation Tests 2011-02-02T15:38:08Z

<p>Amarchetti: </p> <hr /> <div>The following table lists the results of the ablation tests (a mandatory track since the RTE5 campaign), submitted by participants to RTE6 .<br><br><br /> <br /> <br /> Participants are kindly invited to check if all the inserted information is correct and complete.<br /> <br /> <br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! Ablated Component<br /> ! Ablation Run<ref>For further information about participants, click here: [[RTE Challenges - Data about participants]]</ref><br /> ! <small>Resource impact - F1 </small><br /> ! Resource Usage Description<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | BIU1_abl-1<br /> | style="text-align: center;"| 0.9<br /> | style="text-align: center;"| No Word-Net. On Dev set: 39.18% (compared to 40.73% when WN is used)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | CatVar<br /> | BIU1_abl-2<br /> | style="text-align: center;"| 0.63<br /> | style="text-align: center;"| No CatVar. On Dev set achieved about 40.20% (compared to 40.73% when CatVar is used)<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Coreference resolver<br /> | BIU1_abl-3<br /> | style="text-align: center;"| -0.88<br /> | style="text-align: center;"| No coreference resolver<br /> On Dev set 41.62% (Compared to 40.73% when Coreference resolver is used).<br /> This ablation test is an unusual ablation test, since it shows that the co-reference resolution component has a negative impact.<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DIRT<br /> | Boeing1_abl-1<br /> | style="text-align: center;"| 3.97<br /> | style="text-align: center;"| DIRT removed<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | Boeing1_abl-2<br /> | style="text-align: center;"| 4.42<br /> | style="text-align: center;"| No WordNet<br /> <br /> <br /> |}<br /> <br><br /> ==Footnotes==<br /> <references /><br /> <br /> <br /> Return to [[RTE Knowledge Resources]]</div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE6_-_Ablation_Tests&diff=8647 RTE6 - Ablation Tests 2011-02-02T15:24:25Z

<p>Amarchetti: New page: The following table lists the results of the ablation tests (a mandatory track since the RTE5 campaign), submitted by participants to RTE6 .<br><br> Participants are kindly invited to ch...</p> <hr /> <div>The following table lists the results of the ablation tests (a mandatory track since the RTE5 campaign), submitted by participants to RTE6 .<br><br><br /> <br /> <br /> Participants are kindly invited to check if all the inserted information is correct and complete.<br /> <br /> <br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! Ablated Component<br /> ! Ablation Run<ref>For further information about participants, click here: [[RTE Challenges - Data about participants]]</ref><br /> ! <small>Resource impact - F1 </small><br /> ! Resource Usage Description<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | WordNet<br /> | BIU1_abl-1<br /> | style="text-align: center;"| 0.9<br /> | style="text-align: center;"| No Word-Net. On Dev set: 39.18% (compared to 40.73% when WN is used)<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | CatVar<br /> | BIU1_abl-2<br /> | style="text-align: center;"| 0.63<br /> | style="text-align: center;"| No CatVar. On Dev set achieved about 40.20% (compared to 40.73% when CatVar is used)<br /> <br /> |- bgcolor="#FFFFF" "align="left"<br /> | Coreference resolver<br /> | BIU1_abl-3<br /> | style="text-align: center;"| -0.88<br /> | style="text-align: center;"| No coreference resolver<br /> On Dev set 41.62% (Compared to 40.73% when Coreference resolver is used).<br /> This ablation test is an unusual ablation test, since it shows that the co-reference resolution component has a negative impact.<br /> <br /> <br /> <br /> <br /> <br /> <br /> <br /> <br /> <br /> <br /> <br /> <br /> |}<br /> <br><br /> ==Footnotes==<br /> <references /><br /> <br /> <br /> Return to [[RTE Knowledge Resources]]</div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE_Knowledge_Resources&diff=8646 RTE Knowledge Resources 2011-02-02T13:17:08Z

<p>Amarchetti: /* Ablation Tests */</p> <hr /> <div>Knowledge resources have shown their relevance for applied semantic inference, and are extensively used by applied inference systems, such as those developed within the Textual Entailment framework.<br><br /> <br><br /> This page presents a list of the knowledge resources used by systems that have participated in the last RTE challenges. The first table lists the publicly available resources, the second one lists unpublished resources. Both tables are sortable by Resource name, type, author and number of users.<br><br /> <br><br /> RTE Participants are encouraged to add information about all kind of knowledge resources used, from standard existing resources (e.g. WordNet) to knowledge collections created for specific purposes, which can be made available to the community.<br><br /> <br><br /> __TOC__<br /> <br><br /> === Call for Resources ===<br /> <br /> In order to help the research, all the participants are invited to contribute, sharing their own resources with the RTE community. <br /> <br><br /> Making the resources available to be used by other systems has several advantages. On the one hand, it helps improve the TE technology; on the other hand, it offers an opportunity to further test and evaluate the resource.<br /> <br><br /> <br><br /> [[RTE6 - Call for Resources]]<br /> <br><br /> === Ablation Tests ===<br /> An ablation test consists of removing one module at a time from a system, and rerunning the system on the test set with the other modules, except the one tested. <br /> <br><br /> Ablation test are meant to help better understand the relevance of the knowledge resources used by RTE systems, and evaluate the contribution of each of them to the systems' performances. In fact, comparing the results achieved in the ablation tests to those obtained by the systems as a whole allows assessing the contribution given by each single resource.<br /> <br><br /> <br><br /> *[[RTE5 - Ablation Tests]]<br /> *[[RTE6 - Ablation Tests]]<br /> <br><br /> <br /> === Publicly available Resources ===<br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! width="80"|Resource<br /> ! width="50"|Type<br /> ! width="150"|Author<br /> ! class="unsortable" width="300"|Brief description<br /> ! width="45"|<small>PAST Users <ref name:"rtethree">RTE-3 data have been provided by participants by means of a questionnaire.</ref></small><br /> ! width="45"|<small>RTE4 Users<ref name:"rtefour">RTE-4 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! width="45"|<small>RTE5 Users<ref name:"rtefive">RTE-5 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! width="45"|<small>RTE6 Users<ref name:"rtesix">RTE-6 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! class="unsortable" width="30"|<small>Usage info</small><br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[WordNet]]<br /> | Lexical DB<br /> | Princeton University<br /> | Lexical database of English nouns, verbs, adjectives and adverbs<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|21<br /> | style="text-align: center;"|18<br /> |style="text-align: center;"|<br /> | [[WordNet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://xwn.hlt.utdallas.edu/index.html eXtended Wordnet]<br /> | Lexical DB<br /> | Human Language Technology Research Institute, University of Texas at Dallas<br /> | Extension of WordNet based on the exploitation of the information contained in WordNet definitional glosses: the glosses are syntactically parsed, transformed into logic forms and content words are semantically disambiguated. The Extended Wordnet is an ongoing project. <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|2<br /> |style="text-align: center;"|<br /> | [[eXtended WordNet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://ai.stanford.edu/~rion/swn/ Augmented Wordnet]<br /> | Lexical DB<br /> | Stanford University<br /> | The resource is the result of the application of a learning algorithm for inducing semantic taxonomies from parsed text. The algorithm automatically acquires items of world knowledge, and uses these to produce significantly enhanced versions of WordNet (up to 40,000 synsets more).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Augmented Wordnet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://verbs.colorado.edu/~mpalmer/projects/verbnet.html Verbnet]<br /> | Lexical DB<br /> | University of Colorado Boulder<br /> | Lexicon for English verbs organized into classes extending Levin (1993) classes through refinement and addition of subclasses to achieve syntactic and semantic coherence among members of a class<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Verbnet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[VerbOcean]]<br /> | Lexical DB<br /> | Information Sciences Institute, University of Southern California<br /> | Broad-coverage semantic network of verbs<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|6<br /> |style="text-align: center;"|<br /> | [[VerbOcean - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://framenet.icsi.berkeley.edu/ FrameNet]<br /> | Lexical DB<br /> | ICSI (International Computer Science Institute) - Berkley University<br /> | Lexical resource for English words, based on frame semantics (valences) and supported by corpus evidence<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|2<br /> |style="text-align: center;"|<br /> | [[Framenet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/meyers/NomBank.html NomBank]<br /> | Lexical DB<br /> | New York University<br /> | Lexical resource containing syntactic frames for nouns, extracted from annotated corpora <br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[NomBank Resource - RTE Users|Users]] <br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://verbs.colorado.edu/~mpalmer/projects/ace.html PropBank]<br /> | Lexical DB<br /> | University of Colorado Boulder <br /> | Lexical resource containing syntactic frames for verbs, extracted from annotated corpora <br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[PropBank Resource - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/nomlex/index.html Nomlex] Plus<br /> | Lexical DB<br /> | New York University<br /> | Dictionary of English nominalizations: it describes the allowed complements for a nominalization and relates the nominal complements to the arguments of the corresponding verb<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[Nomlex Plus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.cs.ualberta.ca/~lindek/downloads.htm Dekang Lin’s Thesaurus]<br /> | Thesaurus<br /> | University of Alberta<br /> | Thesaurus automatically constructed using a parsed corpus, based on distributional similarity scores<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Dekang Lin’s Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://icon.shef.ac.uk/Moby/mthes.html Grady Ward's Moby Thesaurus]<br /> | Thesaurus<br /> | University of Sheffield<br /> | Thesaurus containing 30,260 root words, with 2,520,264 synonyms and related terms. Grady Ward placed this thesaurus in the public domain in 1996.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Grady Ward's Moby Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://en.wikipedia.org/wiki/Roget%27s_Thesaurus Roget's Thesaurus]<br /> | Thesaurus<br /> | Peter Mark Roget (Electronic version distributed by University of Chicago)<br /> | Roget's Thesaurus is a widely-used English thesaurus, created by Dr. Peter Mark Roget in 1805. The original edition had 15,000 words, and each new edition has been larger. The electronic edition ([http://machaut.uchicago.edu/rogets version 1.02]) is made available by University of Chicago.<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Roget's Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.wikipedia.org/ Wikipedia]<br /> | Encyclopedia<br /> | <br /> | Free encyclopedia. Used for extraction of lexical-semantic rules (from its more structured parts), named entity recognition, geographical information etc.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|6<br /> |style="text-align: center;"|<br /> | [[Wikipedia - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.umbel.org/ Umbel]<br /> | Ontology<br /> | Structured Dynamics LLC, Coralville, IA<br /> | UMBEL stands for Upper Mapping and Binding Exchange Layer and is a lightweight ontology structure for relating Web content and data to a standard set of subject concepts<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Umbel - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.mpi-inf.mpg.de/yago-naga/yago/ YAGO]<br /> | Ontology<br /> | Max-Planck Institute for Informatics, Saarbrücken, Germany<br /> | Light-weight and extensible ontology. It contains more than 2 million entities and 20 million facts about these entities. The facts have been automatically extracted from Wikipedia and unified with WordNet.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[YAGO - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://dbpedia.org/About DBpedia]<br /> | Ontology<br /> | Open community project<br /> | DBpedia is a community effort to extract structured information from Wikipedia and to make this information available on the Web. The DBpedia knowledge base currently describes more than 2.9 million things in 91 different languages and consists of 479 million pieces of information.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[DBpedia - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[DIRT Paraphrase Collection]]<br /> | Collection of paraphrases<br /> | University of Alberta<br /> | DIRT (Discovery of Inference Rules from Text) is both an algorithm and a resulting knowledge collection. The DIRT knowledge collection is the output of the DIRT algorithm over a 1GB set of newspaper text.<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|4<br /> | style="text-align: center;"|3<br /> |style="text-align: center;"|<br /> | [[DIRT Paraphrase Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[TEASE]] Collection<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University<br /> | Output of the TEASE algorithm <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[Tease Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://badc.nerc.ac.uk/help/abbrevs.html BADC Acronym and Abbreviation List]<br /> | Word List<br /> | BADC (British Atmospheric Data Centre)<br /> | Acronym and Abbreviation List<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[BADC Acronym and Abbreviation List - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.acronym-guide.com/ Acronym Guide]<br /> | Word List<br /> | Acronym-Guide.com <br /> | Acronym and Abbreviation Lists for English, branched in thematic directories <br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|3<br /> |style="text-align: center;"|<br /> | [[Acronym Guide - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.ldc.upenn.edu/Catalog/CatalogEntry.jsp?catalogId=LDC2006T13 Web1T 5-grams]<br /> | Word list<br /> | Linguistic Data Consortium, University of Pennsylvania; Google Inc.<br /> | Data set containing English word n-grams and their observed frequency counts. The n-gram counts were generated from approximately 1 trillion word tokens of text from publicly accessible Web pages<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[Web1T - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.coli.uni-saarland.de/%7Erwang/resources/RTE3_RTE4_NGD.zip Normalized Google Distance (RTE3&RTE4)]<br /> | Word Pair Co-occurrence<br /> | Saarland University<br /> | Co-occurrence of the word pairs in RTE3 and RTE4 using Normalized Google Distance (Cilibrasi and Vitanyi, 2004). The word pairs are all the possible combinations of content words in T and H. In practice, we used [http://m.www.yahoo.com/ Yahoo!] as the search engine.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Normalized Google Distance (RTE3&RTE4)- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.coli.uni-saarland.de/%7Erwang/resources/RTE5_NGD.zip Normalized Google Distance (RTE5)]<br /> | Word Pair Co-occurrence<br /> | Saarland University<br /> | Co-occurrence of the word pairs in RTE3 and RTE4 using Normalized Google Distance (Cilibrasi and Vitanyi, 2004). The word pairs are all the possible combinations of content words in T and H. In practice, we used [http://m.www.yahoo.com/ Yahoo!] as the search engine.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Normalized Google Distance (RTE5)- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://geonames.usgs.gov/index.html GNIS - Geographic Names Information System]<br /> | Gazetteer<br /> | USGS (United States Geological Survey)<br /> | Database containing the Federal and national standard toponyms for USA, associated areas and Antarctica<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[GNIS - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.geonames.org/ Geonames]<br /> | Gazetteer<br /> | <br /> | Database containing eight million geographical names. It is integrating geographical data such as names of places in various languages, elevation, population and others from various sources.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[Geonames - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/paraphrase/ Sekine's Paraphrase Database]<br /> | Collection of paraphrases<br /> | Department of Computer Science, New York University <br /> | Data-base created using Sekine's method, NOT cleaned up by human. It includes 19,975 sets of paraphrases with 191,572 phrases.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[Sekine's Paraphrase Database - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://research.microsoft.com/research/downloads/Details/607D14D9-20CD-47E3-85BC-A2F65CD28042/Details.aspx Microsoft Research Paraphrase Corpus]<br /> | Collection of paraphrases<br /> | Microsoft Research<br /> | Text file containing 5800 pairs of sentences which have been extracted from news sources on the web, along with human annotations indicating whether each pair captures a paraphrase/semantic equivalence relationship.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[Microsoft Research Paraphrase Corpus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.cs.cornell.edu/~cristian/Without_a_doubt_-_Data.html Downward entailing operators] <br /> | Collection of entailing operators<br /> | Department of Computer Science, Cornell University, Ithaca NY<br /> | System output of an unsupervised algorithm recovering many Downward Entailing operators, like 'doubt'.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Downward entailing operators - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/WikiRules.html WikiRules!]<br /> | Lexical Reference rule-base<br /> | Bar-Ilan University<br /> | Extraction of about 8 million lexical reference rules from the text body (first sentence) and from metadata (links, redirects, parentheses) of Wikipedia. Provides better performance than other automatically constructed resources and comparable performance to WordNet. Offers complementary knowledge to WordNet. <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[WikiRules! - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://www.cs.utexas.edu/users/pclark/dart/ DART]<br /> | Collection of "world knowledge" propositions<br /> | Boeing Research and Technology<br /> | 23 million tuples such as "airplanes can fly to airports", "rivers can flood" collected from abstracted parse trees.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[DART - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://cs.biu.ac.il/~nlp/download/FRED/latest/FRED.zip FRED]<br /> | FrameNet-derived entailment rule-base<br /> | Bar-Ilan University<br /> | This package contains the outputs of the FRED algorithm, an algorithm which extracts entailment rules from FrameNet.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[FRED - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/DIRECT.html DIRECT]<br /> | Directional Distributional Term-Similarity Resource<br /> | Bar-Ilan University<br /> | This is a resource of directional distributional term-similarity rules (mostly lexical entailment rules) automatically extracted using the inclusion relation as described in (Kotlerman et.al., ACL-09).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[DIRECT - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/binaryDirt.html binaryDIRT]<br /> | Entailment rules between binary templates using DIRT algorithm<br /> | Bar-Ilan University<br /> | This resource contains entailment rules over binary templates learned over the Reuters corpus using<br /> the DIRT algorithm of Lin and Pantel.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[BinaryDIRT- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/unaryBInc.html unaryBInc]<br /> | Entailment rules between unary templates using BInc algorithm<br /> | Bar-Ilan University<br /> | This resource contains entailment rules over unary templates learned over the Reuters corpus using<br /> the BInc algorithm of Szpektor and Dagan (2008).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[unaryBInc- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''New resource''<br /> | <br /> | <br /> | ''Participants are encouraged to contribute''<br /> | style="text-align: center;"| <br /> | style="text-align: center;"| <br /> | style="text-align: center;"| <br /> |style="text-align: center;"|<br /> | [[New Resource2 - RTE Users|Users]]<br /> <br /> |}<br /> <br><br /> <br><br /> <br /> === Not available Resources ===<br /> The following table lists the unpublished resources used by RTE participants. Some of them have been developed by Users themselves specifically for RTE. Interested people may turn to authors to obtain further information.<br /> <br><br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! width="80"|Resource<br /> ! width="50"|Type<br /> ! width="180"|Author<br /> ! class="unsortable" width="500"|Brief description<br /> ! width="30"|<small>PAST Users</small><br /> ! width="30"|<small>RTE4 Users</small><br /> ! width="30"|<small>RTE5 Users</small><br /> ! width="30"|<small>RTE6 Users</small><br /> ! class="unsortable" width="30"|<small>Usage info</small><br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.parc.com/ PARC] Polarity Lexicon<br /> | Lexical DB<br /> | PARC - Palo Alto Research Center<br /> | Verbs classification with respect to semantic polarity<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Parc Polarity Lexicon - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Gazetteer from [http://trec.nist.gov/ TREC]<br /> | Gazetteer<br /> | NIST - National Institute of Standards and Technology<br /> | Cities and other geographical names<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Gazetteer from TREC - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DFKI Geographic Ontology<br>''(to be released)''<br /> | Ontology<br /> | DFKI - German Research Center for Artificial Intelligence<br /> | Ontology containing geographic terms and two kinds of relations: the directional ''part-of'' relation, and the ''equal'' relation for synonyms and abbreviations of the same geographic area (e.g ''the United Kingdom'', ''the UK'', ''Great Britain'', etc.) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Geographic Ontology - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Geo''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | Meronymy entailment rules, based on TREC’s TIPSTER gazetteer.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Geo - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Regex''<br /> | Collection of Entailment rules <br /> | Bar-Ilan University; Tel-Aviv University<br /> | Small set of entailment rules based on regular expressions, intended to address lexical variability involving temporal phrases<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Regex - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Syntactic rule base <br>''(to be released)''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | A manually-composed collection of entailment rules which define parse tree transformations. The rules cover generic syntactic phenomena such as appositions, conjunctions, passive, relative clause, etc. (Bar-Haim et al., AAAI-07) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Syntactic rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Polarity rule base <br>''(to be released)''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | A manually-composed collection of entailment rules which detect predicates whose polarity is negative (e.g. didn't dance) or unknown (e.g. plans to dance). The rules capture diverse phenomena that affect polarity, e.g. verbal negation, modal verbs, conditionals, and certain verbs that induce negative or "unknown" polarity context. The latter were taken mainly from VerbNet. Extends a resource described in (Bar-Haim et al., AAAI-07) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0 <br /> | style="text-align: center;"|<br /> | [[Polarity rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Lexical-Syntactic rule base<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | Extract lexical-syntactic entailment rules for predicates (verbal and nominal), including argument mapping. The resource is based on WordNet, Nomlex-Plus and Unary DIRT (Szpektor and Dagan, Coling 08)<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Lexical-Syntactic rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | OPENU Collection<br /> | Collection of Entailment Rules and Patterns<br /> | Open University<br /> | Collections of rules, patterns etc. for RTE purpose, extracted from Reuter corpus parsed using Minipar.<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"| <br /> | [[OPENU Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Abbr''<br /> | Collection of rules for abbreviation<br /> | Bar-Ilan University; Tel-Aviv University<br /> | 2000 Abbreviation rules, extracted from [http://badc.nerc.ac.uk/help/abbrevs.html BADC] and [http://www.acronym-guide.com/ Acronym Guide]<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Abbr - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Negation_list<br /> | Negation rules<br /> | „Al. I. Cuza“ University, Iasi, Romania <br /> | List of negative terms and words (verbs, adjectives, nouns) affecting modality or factuality of a infinitive verb preceded by the particle "to" (e.g. "believe","necessary", "attempt")<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1 <br /> | style="text-align: center;"|<br /> | [[UAIC Negation_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DLSIUAES Negation_list<br /> | List of negative terms<br /> | University of Alicante<br /> | Basic list of negative terms.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"| <br /> | [[DLSIUAES Negation_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Quantifier_list<br /> | List of quantifiers<br /> | „Al. I. Cuza“ University, Iasi, Romania <br /> | List of quantifiers affecting entailment judgment. The quantifiers are taken from a list which contains expressions like “more than”, “less than”, or words such as “over”, “under”, etc.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"| <br /> | [[UAIC Quantifier_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | FBKirst StopWord list<br /> | List of frequent words<br /> | FBK-Irst;<br/>University of Trento - Italy <br /> | A list of the 572 most frequent English words.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"| <br /> | [[FBKirst StopWord list - RTE Users|Users]]<br /> <br /> |}<br /> <br><br /> <br /> ==Footnotes==<br /> <references/></div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE_Knowledge_Resources&diff=8645 RTE Knowledge Resources 2011-02-02T11:55:49Z

<p>Amarchetti: /* Not available Resources */</p> <hr /> <div>Knowledge resources have shown their relevance for applied semantic inference, and are extensively used by applied inference systems, such as those developed within the Textual Entailment framework.<br><br /> <br><br /> This page presents a list of the knowledge resources used by systems that have participated in the last RTE challenges. The first table lists the publicly available resources, the second one lists unpublished resources. Both tables are sortable by Resource name, type, author and number of users.<br><br /> <br><br /> RTE Participants are encouraged to add information about all kind of knowledge resources used, from standard existing resources (e.g. WordNet) to knowledge collections created for specific purposes, which can be made available to the community.<br><br /> <br><br /> __TOC__<br /> <br><br /> === Call for Resources ===<br /> <br /> In order to help the research, all the participants are invited to contribute, sharing their own resources with the RTE community. <br /> <br><br /> Making the resources available to be used by other systems has several advantages. On the one hand, it helps improve the TE technology; on the other hand, it offers an opportunity to further test and evaluate the resource.<br /> <br><br /> <br><br /> [[RTE6 - Call for Resources]]<br /> <br><br /> === Ablation Tests ===<br /> An ablation test consists of removing one module at a time from a system, and rerunning the system on the test set with the other modules, except the one tested. <br /> <br><br /> Ablation test are meant to help better understand the relevance of the knowledge resources used by RTE systems, and evaluate the contribution of each of them to the systems' performances. In fact, comparing the results achieved in the ablation tests to those obtained by the systems as a whole allows assessing the contribution given by each single resource.<br /> <br><br /> <br><br /> [[RTE5 - Ablation Tests]]<br /> <br><br /> <br /> === Publicly available Resources ===<br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! width="80"|Resource<br /> ! width="50"|Type<br /> ! width="150"|Author<br /> ! class="unsortable" width="300"|Brief description<br /> ! width="45"|<small>PAST Users <ref name:"rtethree">RTE-3 data have been provided by participants by means of a questionnaire.</ref></small><br /> ! width="45"|<small>RTE4 Users<ref name:"rtefour">RTE-4 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! width="45"|<small>RTE5 Users<ref name:"rtefive">RTE-5 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! width="45"|<small>RTE6 Users<ref name:"rtesix">RTE-6 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! class="unsortable" width="30"|<small>Usage info</small><br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[WordNet]]<br /> | Lexical DB<br /> | Princeton University<br /> | Lexical database of English nouns, verbs, adjectives and adverbs<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|21<br /> | style="text-align: center;"|18<br /> |style="text-align: center;"|<br /> | [[WordNet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://xwn.hlt.utdallas.edu/index.html eXtended Wordnet]<br /> | Lexical DB<br /> | Human Language Technology Research Institute, University of Texas at Dallas<br /> | Extension of WordNet based on the exploitation of the information contained in WordNet definitional glosses: the glosses are syntactically parsed, transformed into logic forms and content words are semantically disambiguated. The Extended Wordnet is an ongoing project. <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|2<br /> |style="text-align: center;"|<br /> | [[eXtended WordNet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://ai.stanford.edu/~rion/swn/ Augmented Wordnet]<br /> | Lexical DB<br /> | Stanford University<br /> | The resource is the result of the application of a learning algorithm for inducing semantic taxonomies from parsed text. The algorithm automatically acquires items of world knowledge, and uses these to produce significantly enhanced versions of WordNet (up to 40,000 synsets more).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Augmented Wordnet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://verbs.colorado.edu/~mpalmer/projects/verbnet.html Verbnet]<br /> | Lexical DB<br /> | University of Colorado Boulder<br /> | Lexicon for English verbs organized into classes extending Levin (1993) classes through refinement and addition of subclasses to achieve syntactic and semantic coherence among members of a class<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Verbnet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[VerbOcean]]<br /> | Lexical DB<br /> | Information Sciences Institute, University of Southern California<br /> | Broad-coverage semantic network of verbs<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|6<br /> |style="text-align: center;"|<br /> | [[VerbOcean - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://framenet.icsi.berkeley.edu/ FrameNet]<br /> | Lexical DB<br /> | ICSI (International Computer Science Institute) - Berkley University<br /> | Lexical resource for English words, based on frame semantics (valences) and supported by corpus evidence<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|2<br /> |style="text-align: center;"|<br /> | [[Framenet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/meyers/NomBank.html NomBank]<br /> | Lexical DB<br /> | New York University<br /> | Lexical resource containing syntactic frames for nouns, extracted from annotated corpora <br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[NomBank Resource - RTE Users|Users]] <br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://verbs.colorado.edu/~mpalmer/projects/ace.html PropBank]<br /> | Lexical DB<br /> | University of Colorado Boulder <br /> | Lexical resource containing syntactic frames for verbs, extracted from annotated corpora <br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[PropBank Resource - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/nomlex/index.html Nomlex] Plus<br /> | Lexical DB<br /> | New York University<br /> | Dictionary of English nominalizations: it describes the allowed complements for a nominalization and relates the nominal complements to the arguments of the corresponding verb<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[Nomlex Plus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.cs.ualberta.ca/~lindek/downloads.htm Dekang Lin’s Thesaurus]<br /> | Thesaurus<br /> | University of Alberta<br /> | Thesaurus automatically constructed using a parsed corpus, based on distributional similarity scores<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Dekang Lin’s Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://icon.shef.ac.uk/Moby/mthes.html Grady Ward's Moby Thesaurus]<br /> | Thesaurus<br /> | University of Sheffield<br /> | Thesaurus containing 30,260 root words, with 2,520,264 synonyms and related terms. Grady Ward placed this thesaurus in the public domain in 1996.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Grady Ward's Moby Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://en.wikipedia.org/wiki/Roget%27s_Thesaurus Roget's Thesaurus]<br /> | Thesaurus<br /> | Peter Mark Roget (Electronic version distributed by University of Chicago)<br /> | Roget's Thesaurus is a widely-used English thesaurus, created by Dr. Peter Mark Roget in 1805. The original edition had 15,000 words, and each new edition has been larger. The electronic edition ([http://machaut.uchicago.edu/rogets version 1.02]) is made available by University of Chicago.<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Roget's Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.wikipedia.org/ Wikipedia]<br /> | Encyclopedia<br /> | <br /> | Free encyclopedia. Used for extraction of lexical-semantic rules (from its more structured parts), named entity recognition, geographical information etc.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|6<br /> |style="text-align: center;"|<br /> | [[Wikipedia - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.umbel.org/ Umbel]<br /> | Ontology<br /> | Structured Dynamics LLC, Coralville, IA<br /> | UMBEL stands for Upper Mapping and Binding Exchange Layer and is a lightweight ontology structure for relating Web content and data to a standard set of subject concepts<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Umbel - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.mpi-inf.mpg.de/yago-naga/yago/ YAGO]<br /> | Ontology<br /> | Max-Planck Institute for Informatics, Saarbrücken, Germany<br /> | Light-weight and extensible ontology. It contains more than 2 million entities and 20 million facts about these entities. The facts have been automatically extracted from Wikipedia and unified with WordNet.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[YAGO - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://dbpedia.org/About DBpedia]<br /> | Ontology<br /> | Open community project<br /> | DBpedia is a community effort to extract structured information from Wikipedia and to make this information available on the Web. The DBpedia knowledge base currently describes more than 2.9 million things in 91 different languages and consists of 479 million pieces of information.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[DBpedia - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[DIRT Paraphrase Collection]]<br /> | Collection of paraphrases<br /> | University of Alberta<br /> | DIRT (Discovery of Inference Rules from Text) is both an algorithm and a resulting knowledge collection. The DIRT knowledge collection is the output of the DIRT algorithm over a 1GB set of newspaper text.<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|4<br /> | style="text-align: center;"|3<br /> |style="text-align: center;"|<br /> | [[DIRT Paraphrase Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[TEASE]] Collection<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University<br /> | Output of the TEASE algorithm <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[Tease Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://badc.nerc.ac.uk/help/abbrevs.html BADC Acronym and Abbreviation List]<br /> | Word List<br /> | BADC (British Atmospheric Data Centre)<br /> | Acronym and Abbreviation List<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[BADC Acronym and Abbreviation List - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.acronym-guide.com/ Acronym Guide]<br /> | Word List<br /> | Acronym-Guide.com <br /> | Acronym and Abbreviation Lists for English, branched in thematic directories <br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|3<br /> |style="text-align: center;"|<br /> | [[Acronym Guide - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.ldc.upenn.edu/Catalog/CatalogEntry.jsp?catalogId=LDC2006T13 Web1T 5-grams]<br /> | Word list<br /> | Linguistic Data Consortium, University of Pennsylvania; Google Inc.<br /> | Data set containing English word n-grams and their observed frequency counts. The n-gram counts were generated from approximately 1 trillion word tokens of text from publicly accessible Web pages<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[Web1T - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.coli.uni-saarland.de/%7Erwang/resources/RTE3_RTE4_NGD.zip Normalized Google Distance (RTE3&RTE4)]<br /> | Word Pair Co-occurrence<br /> | Saarland University<br /> | Co-occurrence of the word pairs in RTE3 and RTE4 using Normalized Google Distance (Cilibrasi and Vitanyi, 2004). The word pairs are all the possible combinations of content words in T and H. In practice, we used [http://m.www.yahoo.com/ Yahoo!] as the search engine.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Normalized Google Distance (RTE3&RTE4)- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.coli.uni-saarland.de/%7Erwang/resources/RTE5_NGD.zip Normalized Google Distance (RTE5)]<br /> | Word Pair Co-occurrence<br /> | Saarland University<br /> | Co-occurrence of the word pairs in RTE3 and RTE4 using Normalized Google Distance (Cilibrasi and Vitanyi, 2004). The word pairs are all the possible combinations of content words in T and H. In practice, we used [http://m.www.yahoo.com/ Yahoo!] as the search engine.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Normalized Google Distance (RTE5)- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://geonames.usgs.gov/index.html GNIS - Geographic Names Information System]<br /> | Gazetteer<br /> | USGS (United States Geological Survey)<br /> | Database containing the Federal and national standard toponyms for USA, associated areas and Antarctica<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[GNIS - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.geonames.org/ Geonames]<br /> | Gazetteer<br /> | <br /> | Database containing eight million geographical names. It is integrating geographical data such as names of places in various languages, elevation, population and others from various sources.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[Geonames - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/paraphrase/ Sekine's Paraphrase Database]<br /> | Collection of paraphrases<br /> | Department of Computer Science, New York University <br /> | Data-base created using Sekine's method, NOT cleaned up by human. It includes 19,975 sets of paraphrases with 191,572 phrases.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[Sekine's Paraphrase Database - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://research.microsoft.com/research/downloads/Details/607D14D9-20CD-47E3-85BC-A2F65CD28042/Details.aspx Microsoft Research Paraphrase Corpus]<br /> | Collection of paraphrases<br /> | Microsoft Research<br /> | Text file containing 5800 pairs of sentences which have been extracted from news sources on the web, along with human annotations indicating whether each pair captures a paraphrase/semantic equivalence relationship.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[Microsoft Research Paraphrase Corpus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.cs.cornell.edu/~cristian/Without_a_doubt_-_Data.html Downward entailing operators] <br /> | Collection of entailing operators<br /> | Department of Computer Science, Cornell University, Ithaca NY<br /> | System output of an unsupervised algorithm recovering many Downward Entailing operators, like 'doubt'.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Downward entailing operators - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/WikiRules.html WikiRules!]<br /> | Lexical Reference rule-base<br /> | Bar-Ilan University<br /> | Extraction of about 8 million lexical reference rules from the text body (first sentence) and from metadata (links, redirects, parentheses) of Wikipedia. Provides better performance than other automatically constructed resources and comparable performance to WordNet. Offers complementary knowledge to WordNet. <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[WikiRules! - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://www.cs.utexas.edu/users/pclark/dart/ DART]<br /> | Collection of "world knowledge" propositions<br /> | Boeing Research and Technology<br /> | 23 million tuples such as "airplanes can fly to airports", "rivers can flood" collected from abstracted parse trees.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[DART - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://cs.biu.ac.il/~nlp/download/FRED/latest/FRED.zip FRED]<br /> | FrameNet-derived entailment rule-base<br /> | Bar-Ilan University<br /> | This package contains the outputs of the FRED algorithm, an algorithm which extracts entailment rules from FrameNet.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[FRED - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/DIRECT.html DIRECT]<br /> | Directional Distributional Term-Similarity Resource<br /> | Bar-Ilan University<br /> | This is a resource of directional distributional term-similarity rules (mostly lexical entailment rules) automatically extracted using the inclusion relation as described in (Kotlerman et.al., ACL-09).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[DIRECT - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/binaryDirt.html binaryDIRT]<br /> | Entailment rules between binary templates using DIRT algorithm<br /> | Bar-Ilan University<br /> | This resource contains entailment rules over binary templates learned over the Reuters corpus using<br /> the DIRT algorithm of Lin and Pantel.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[BinaryDIRT- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/unaryBInc.html unaryBInc]<br /> | Entailment rules between unary templates using BInc algorithm<br /> | Bar-Ilan University<br /> | This resource contains entailment rules over unary templates learned over the Reuters corpus using<br /> the BInc algorithm of Szpektor and Dagan (2008).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[unaryBInc- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''New resource''<br /> | <br /> | <br /> | ''Participants are encouraged to contribute''<br /> | style="text-align: center;"| <br /> | style="text-align: center;"| <br /> | style="text-align: center;"| <br /> |style="text-align: center;"|<br /> | [[New Resource2 - RTE Users|Users]]<br /> <br /> |}<br /> <br><br /> <br><br /> <br /> === Not available Resources ===<br /> The following table lists the unpublished resources used by RTE participants. Some of them have been developed by Users themselves specifically for RTE. Interested people may turn to authors to obtain further information.<br /> <br><br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! width="80"|Resource<br /> ! width="50"|Type<br /> ! width="180"|Author<br /> ! class="unsortable" width="500"|Brief description<br /> ! width="30"|<small>PAST Users</small><br /> ! width="30"|<small>RTE4 Users</small><br /> ! width="30"|<small>RTE5 Users</small><br /> ! width="30"|<small>RTE6 Users</small><br /> ! class="unsortable" width="30"|<small>Usage info</small><br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.parc.com/ PARC] Polarity Lexicon<br /> | Lexical DB<br /> | PARC - Palo Alto Research Center<br /> | Verbs classification with respect to semantic polarity<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Parc Polarity Lexicon - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Gazetteer from [http://trec.nist.gov/ TREC]<br /> | Gazetteer<br /> | NIST - National Institute of Standards and Technology<br /> | Cities and other geographical names<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Gazetteer from TREC - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DFKI Geographic Ontology<br>''(to be released)''<br /> | Ontology<br /> | DFKI - German Research Center for Artificial Intelligence<br /> | Ontology containing geographic terms and two kinds of relations: the directional ''part-of'' relation, and the ''equal'' relation for synonyms and abbreviations of the same geographic area (e.g ''the United Kingdom'', ''the UK'', ''Great Britain'', etc.) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Geographic Ontology - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Geo''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | Meronymy entailment rules, based on TREC’s TIPSTER gazetteer.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Geo - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Regex''<br /> | Collection of Entailment rules <br /> | Bar-Ilan University; Tel-Aviv University<br /> | Small set of entailment rules based on regular expressions, intended to address lexical variability involving temporal phrases<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Regex - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Syntactic rule base <br>''(to be released)''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | A manually-composed collection of entailment rules which define parse tree transformations. The rules cover generic syntactic phenomena such as appositions, conjunctions, passive, relative clause, etc. (Bar-Haim et al., AAAI-07) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Syntactic rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Polarity rule base <br>''(to be released)''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | A manually-composed collection of entailment rules which detect predicates whose polarity is negative (e.g. didn't dance) or unknown (e.g. plans to dance). The rules capture diverse phenomena that affect polarity, e.g. verbal negation, modal verbs, conditionals, and certain verbs that induce negative or "unknown" polarity context. The latter were taken mainly from VerbNet. Extends a resource described in (Bar-Haim et al., AAAI-07) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0 <br /> | style="text-align: center;"|<br /> | [[Polarity rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Lexical-Syntactic rule base<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | Extract lexical-syntactic entailment rules for predicates (verbal and nominal), including argument mapping. The resource is based on WordNet, Nomlex-Plus and Unary DIRT (Szpektor and Dagan, Coling 08)<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|<br /> | [[Lexical-Syntactic rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | OPENU Collection<br /> | Collection of Entailment Rules and Patterns<br /> | Open University<br /> | Collections of rules, patterns etc. for RTE purpose, extracted from Reuter corpus parsed using Minipar.<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"| <br /> | [[OPENU Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Abbr''<br /> | Collection of rules for abbreviation<br /> | Bar-Ilan University; Tel-Aviv University<br /> | 2000 Abbreviation rules, extracted from [http://badc.nerc.ac.uk/help/abbrevs.html BADC] and [http://www.acronym-guide.com/ Acronym Guide]<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|<br /> | [[Abbr - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Negation_list<br /> | Negation rules<br /> | „Al. I. Cuza“ University, Iasi, Romania <br /> | List of negative terms and words (verbs, adjectives, nouns) affecting modality or factuality of a infinitive verb preceded by the particle "to" (e.g. "believe","necessary", "attempt")<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1 <br /> | style="text-align: center;"|<br /> | [[UAIC Negation_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DLSIUAES Negation_list<br /> | List of negative terms<br /> | University of Alicante<br /> | Basic list of negative terms.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"| <br /> | [[DLSIUAES Negation_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Quantifier_list<br /> | List of quantifiers<br /> | „Al. I. Cuza“ University, Iasi, Romania <br /> | List of quantifiers affecting entailment judgment. The quantifiers are taken from a list which contains expressions like “more than”, “less than”, or words such as “over”, “under”, etc.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"| <br /> | [[UAIC Quantifier_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | FBKirst StopWord list<br /> | List of frequent words<br /> | FBK-Irst;<br/>University of Trento - Italy <br /> | A list of the 572 most frequent English words.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"| <br /> | [[FBKirst StopWord list - RTE Users|Users]]<br /> <br /> |}<br /> <br><br /> <br /> ==Footnotes==<br /> <references/></div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE_Knowledge_Resources&diff=8644 RTE Knowledge Resources 2011-02-02T11:53:46Z

<p>Amarchetti: /* Publicly available Resources */</p> <hr /> <div>Knowledge resources have shown their relevance for applied semantic inference, and are extensively used by applied inference systems, such as those developed within the Textual Entailment framework.<br><br /> <br><br /> This page presents a list of the knowledge resources used by systems that have participated in the last RTE challenges. The first table lists the publicly available resources, the second one lists unpublished resources. Both tables are sortable by Resource name, type, author and number of users.<br><br /> <br><br /> RTE Participants are encouraged to add information about all kind of knowledge resources used, from standard existing resources (e.g. WordNet) to knowledge collections created for specific purposes, which can be made available to the community.<br><br /> <br><br /> __TOC__<br /> <br><br /> === Call for Resources ===<br /> <br /> In order to help the research, all the participants are invited to contribute, sharing their own resources with the RTE community. <br /> <br><br /> Making the resources available to be used by other systems has several advantages. On the one hand, it helps improve the TE technology; on the other hand, it offers an opportunity to further test and evaluate the resource.<br /> <br><br /> <br><br /> [[RTE6 - Call for Resources]]<br /> <br><br /> === Ablation Tests ===<br /> An ablation test consists of removing one module at a time from a system, and rerunning the system on the test set with the other modules, except the one tested. <br /> <br><br /> Ablation test are meant to help better understand the relevance of the knowledge resources used by RTE systems, and evaluate the contribution of each of them to the systems' performances. In fact, comparing the results achieved in the ablation tests to those obtained by the systems as a whole allows assessing the contribution given by each single resource.<br /> <br><br /> <br><br /> [[RTE5 - Ablation Tests]]<br /> <br><br /> <br /> === Publicly available Resources ===<br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! width="80"|Resource<br /> ! width="50"|Type<br /> ! width="150"|Author<br /> ! class="unsortable" width="300"|Brief description<br /> ! width="45"|<small>PAST Users <ref name:"rtethree">RTE-3 data have been provided by participants by means of a questionnaire.</ref></small><br /> ! width="45"|<small>RTE4 Users<ref name:"rtefour">RTE-4 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! width="45"|<small>RTE5 Users<ref name:"rtefive">RTE-5 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! width="45"|<small>RTE6 Users<ref name:"rtesix">RTE-6 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! class="unsortable" width="30"|<small>Usage info</small><br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[WordNet]]<br /> | Lexical DB<br /> | Princeton University<br /> | Lexical database of English nouns, verbs, adjectives and adverbs<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|21<br /> | style="text-align: center;"|18<br /> |style="text-align: center;"|<br /> | [[WordNet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://xwn.hlt.utdallas.edu/index.html eXtended Wordnet]<br /> | Lexical DB<br /> | Human Language Technology Research Institute, University of Texas at Dallas<br /> | Extension of WordNet based on the exploitation of the information contained in WordNet definitional glosses: the glosses are syntactically parsed, transformed into logic forms and content words are semantically disambiguated. The Extended Wordnet is an ongoing project. <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|2<br /> |style="text-align: center;"|<br /> | [[eXtended WordNet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://ai.stanford.edu/~rion/swn/ Augmented Wordnet]<br /> | Lexical DB<br /> | Stanford University<br /> | The resource is the result of the application of a learning algorithm for inducing semantic taxonomies from parsed text. The algorithm automatically acquires items of world knowledge, and uses these to produce significantly enhanced versions of WordNet (up to 40,000 synsets more).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Augmented Wordnet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://verbs.colorado.edu/~mpalmer/projects/verbnet.html Verbnet]<br /> | Lexical DB<br /> | University of Colorado Boulder<br /> | Lexicon for English verbs organized into classes extending Levin (1993) classes through refinement and addition of subclasses to achieve syntactic and semantic coherence among members of a class<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Verbnet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[VerbOcean]]<br /> | Lexical DB<br /> | Information Sciences Institute, University of Southern California<br /> | Broad-coverage semantic network of verbs<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|6<br /> |style="text-align: center;"|<br /> | [[VerbOcean - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://framenet.icsi.berkeley.edu/ FrameNet]<br /> | Lexical DB<br /> | ICSI (International Computer Science Institute) - Berkley University<br /> | Lexical resource for English words, based on frame semantics (valences) and supported by corpus evidence<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|2<br /> |style="text-align: center;"|<br /> | [[Framenet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/meyers/NomBank.html NomBank]<br /> | Lexical DB<br /> | New York University<br /> | Lexical resource containing syntactic frames for nouns, extracted from annotated corpora <br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[NomBank Resource - RTE Users|Users]] <br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://verbs.colorado.edu/~mpalmer/projects/ace.html PropBank]<br /> | Lexical DB<br /> | University of Colorado Boulder <br /> | Lexical resource containing syntactic frames for verbs, extracted from annotated corpora <br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[PropBank Resource - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/nomlex/index.html Nomlex] Plus<br /> | Lexical DB<br /> | New York University<br /> | Dictionary of English nominalizations: it describes the allowed complements for a nominalization and relates the nominal complements to the arguments of the corresponding verb<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[Nomlex Plus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.cs.ualberta.ca/~lindek/downloads.htm Dekang Lin’s Thesaurus]<br /> | Thesaurus<br /> | University of Alberta<br /> | Thesaurus automatically constructed using a parsed corpus, based on distributional similarity scores<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Dekang Lin’s Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://icon.shef.ac.uk/Moby/mthes.html Grady Ward's Moby Thesaurus]<br /> | Thesaurus<br /> | University of Sheffield<br /> | Thesaurus containing 30,260 root words, with 2,520,264 synonyms and related terms. Grady Ward placed this thesaurus in the public domain in 1996.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Grady Ward's Moby Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://en.wikipedia.org/wiki/Roget%27s_Thesaurus Roget's Thesaurus]<br /> | Thesaurus<br /> | Peter Mark Roget (Electronic version distributed by University of Chicago)<br /> | Roget's Thesaurus is a widely-used English thesaurus, created by Dr. Peter Mark Roget in 1805. The original edition had 15,000 words, and each new edition has been larger. The electronic edition ([http://machaut.uchicago.edu/rogets version 1.02]) is made available by University of Chicago.<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Roget's Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.wikipedia.org/ Wikipedia]<br /> | Encyclopedia<br /> | <br /> | Free encyclopedia. Used for extraction of lexical-semantic rules (from its more structured parts), named entity recognition, geographical information etc.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|6<br /> |style="text-align: center;"|<br /> | [[Wikipedia - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.umbel.org/ Umbel]<br /> | Ontology<br /> | Structured Dynamics LLC, Coralville, IA<br /> | UMBEL stands for Upper Mapping and Binding Exchange Layer and is a lightweight ontology structure for relating Web content and data to a standard set of subject concepts<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Umbel - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.mpi-inf.mpg.de/yago-naga/yago/ YAGO]<br /> | Ontology<br /> | Max-Planck Institute for Informatics, Saarbrücken, Germany<br /> | Light-weight and extensible ontology. It contains more than 2 million entities and 20 million facts about these entities. The facts have been automatically extracted from Wikipedia and unified with WordNet.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[YAGO - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://dbpedia.org/About DBpedia]<br /> | Ontology<br /> | Open community project<br /> | DBpedia is a community effort to extract structured information from Wikipedia and to make this information available on the Web. The DBpedia knowledge base currently describes more than 2.9 million things in 91 different languages and consists of 479 million pieces of information.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[DBpedia - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[DIRT Paraphrase Collection]]<br /> | Collection of paraphrases<br /> | University of Alberta<br /> | DIRT (Discovery of Inference Rules from Text) is both an algorithm and a resulting knowledge collection. The DIRT knowledge collection is the output of the DIRT algorithm over a 1GB set of newspaper text.<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|4<br /> | style="text-align: center;"|3<br /> |style="text-align: center;"|<br /> | [[DIRT Paraphrase Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[TEASE]] Collection<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University<br /> | Output of the TEASE algorithm <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[Tease Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://badc.nerc.ac.uk/help/abbrevs.html BADC Acronym and Abbreviation List]<br /> | Word List<br /> | BADC (British Atmospheric Data Centre)<br /> | Acronym and Abbreviation List<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[BADC Acronym and Abbreviation List - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.acronym-guide.com/ Acronym Guide]<br /> | Word List<br /> | Acronym-Guide.com <br /> | Acronym and Abbreviation Lists for English, branched in thematic directories <br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|3<br /> |style="text-align: center;"|<br /> | [[Acronym Guide - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.ldc.upenn.edu/Catalog/CatalogEntry.jsp?catalogId=LDC2006T13 Web1T 5-grams]<br /> | Word list<br /> | Linguistic Data Consortium, University of Pennsylvania; Google Inc.<br /> | Data set containing English word n-grams and their observed frequency counts. The n-gram counts were generated from approximately 1 trillion word tokens of text from publicly accessible Web pages<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[Web1T - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.coli.uni-saarland.de/%7Erwang/resources/RTE3_RTE4_NGD.zip Normalized Google Distance (RTE3&RTE4)]<br /> | Word Pair Co-occurrence<br /> | Saarland University<br /> | Co-occurrence of the word pairs in RTE3 and RTE4 using Normalized Google Distance (Cilibrasi and Vitanyi, 2004). The word pairs are all the possible combinations of content words in T and H. In practice, we used [http://m.www.yahoo.com/ Yahoo!] as the search engine.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Normalized Google Distance (RTE3&RTE4)- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.coli.uni-saarland.de/%7Erwang/resources/RTE5_NGD.zip Normalized Google Distance (RTE5)]<br /> | Word Pair Co-occurrence<br /> | Saarland University<br /> | Co-occurrence of the word pairs in RTE3 and RTE4 using Normalized Google Distance (Cilibrasi and Vitanyi, 2004). The word pairs are all the possible combinations of content words in T and H. In practice, we used [http://m.www.yahoo.com/ Yahoo!] as the search engine.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Normalized Google Distance (RTE5)- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://geonames.usgs.gov/index.html GNIS - Geographic Names Information System]<br /> | Gazetteer<br /> | USGS (United States Geological Survey)<br /> | Database containing the Federal and national standard toponyms for USA, associated areas and Antarctica<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[GNIS - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.geonames.org/ Geonames]<br /> | Gazetteer<br /> | <br /> | Database containing eight million geographical names. It is integrating geographical data such as names of places in various languages, elevation, population and others from various sources.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[Geonames - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/paraphrase/ Sekine's Paraphrase Database]<br /> | Collection of paraphrases<br /> | Department of Computer Science, New York University <br /> | Data-base created using Sekine's method, NOT cleaned up by human. It includes 19,975 sets of paraphrases with 191,572 phrases.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[Sekine's Paraphrase Database - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://research.microsoft.com/research/downloads/Details/607D14D9-20CD-47E3-85BC-A2F65CD28042/Details.aspx Microsoft Research Paraphrase Corpus]<br /> | Collection of paraphrases<br /> | Microsoft Research<br /> | Text file containing 5800 pairs of sentences which have been extracted from news sources on the web, along with human annotations indicating whether each pair captures a paraphrase/semantic equivalence relationship.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[Microsoft Research Paraphrase Corpus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.cs.cornell.edu/~cristian/Without_a_doubt_-_Data.html Downward entailing operators] <br /> | Collection of entailing operators<br /> | Department of Computer Science, Cornell University, Ithaca NY<br /> | System output of an unsupervised algorithm recovering many Downward Entailing operators, like 'doubt'.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Downward entailing operators - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/WikiRules.html WikiRules!]<br /> | Lexical Reference rule-base<br /> | Bar-Ilan University<br /> | Extraction of about 8 million lexical reference rules from the text body (first sentence) and from metadata (links, redirects, parentheses) of Wikipedia. Provides better performance than other automatically constructed resources and comparable performance to WordNet. Offers complementary knowledge to WordNet. <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[WikiRules! - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://www.cs.utexas.edu/users/pclark/dart/ DART]<br /> | Collection of "world knowledge" propositions<br /> | Boeing Research and Technology<br /> | 23 million tuples such as "airplanes can fly to airports", "rivers can flood" collected from abstracted parse trees.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[DART - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://cs.biu.ac.il/~nlp/download/FRED/latest/FRED.zip FRED]<br /> | FrameNet-derived entailment rule-base<br /> | Bar-Ilan University<br /> | This package contains the outputs of the FRED algorithm, an algorithm which extracts entailment rules from FrameNet.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[FRED - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/DIRECT.html DIRECT]<br /> | Directional Distributional Term-Similarity Resource<br /> | Bar-Ilan University<br /> | This is a resource of directional distributional term-similarity rules (mostly lexical entailment rules) automatically extracted using the inclusion relation as described in (Kotlerman et.al., ACL-09).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[DIRECT - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/binaryDirt.html binaryDIRT]<br /> | Entailment rules between binary templates using DIRT algorithm<br /> | Bar-Ilan University<br /> | This resource contains entailment rules over binary templates learned over the Reuters corpus using<br /> the DIRT algorithm of Lin and Pantel.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[BinaryDIRT- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/unaryBInc.html unaryBInc]<br /> | Entailment rules between unary templates using BInc algorithm<br /> | Bar-Ilan University<br /> | This resource contains entailment rules over unary templates learned over the Reuters corpus using<br /> the BInc algorithm of Szpektor and Dagan (2008).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[unaryBInc- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''New resource''<br /> | <br /> | <br /> | ''Participants are encouraged to contribute''<br /> | style="text-align: center;"| <br /> | style="text-align: center;"| <br /> | style="text-align: center;"| <br /> |style="text-align: center;"|<br /> | [[New Resource2 - RTE Users|Users]]<br /> <br /> |}<br /> <br><br /> <br><br /> <br /> === Not available Resources ===<br /> The following table lists the unpublished resources used by RTE participants. Some of them have been developed by Users themselves specifically for RTE. Interested people may turn to authors to obtain further information.<br /> <br><br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! width="80"|Resource<br /> ! width="50"|Type<br /> ! width="180"|Author<br /> ! class="unsortable" width="500"|Brief description<br /> ! width="30"|<small>PAST Users</small><br /> ! width="30"|<small>RTE4 Users</small><br /> ! width="30"|<small>RTE5 Users</small><br /> ! class="unsortable" width="30"|<small>Usage info</small><br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.parc.com/ PARC] Polarity Lexicon<br /> | Lexical DB<br /> | PARC - Palo Alto Research Center<br /> | Verbs classification with respect to semantic polarity<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | [[Parc Polarity Lexicon - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Gazetteer from [http://trec.nist.gov/ TREC]<br /> | Gazetteer<br /> | NIST - National Institute of Standards and Technology<br /> | Cities and other geographical names<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | [[Gazetteer from TREC - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DFKI Geographic Ontology<br>''(to be released)''<br /> | Ontology<br /> | DFKI - German Research Center for Artificial Intelligence<br /> | Ontology containing geographic terms and two kinds of relations: the directional ''part-of'' relation, and the ''equal'' relation for synonyms and abbreviations of the same geographic area (e.g ''the United Kingdom'', ''the UK'', ''Great Britain'', etc.) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | [[Geographic Ontology - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Geo''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | Meronymy entailment rules, based on TREC’s TIPSTER gazetteer.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | [[Geo - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Regex''<br /> | Collection of Entailment rules <br /> | Bar-Ilan University; Tel-Aviv University<br /> | Small set of entailment rules based on regular expressions, intended to address lexical variability involving temporal phrases<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | [[Regex - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Syntactic rule base <br>''(to be released)''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | A manually-composed collection of entailment rules which define parse tree transformations. The rules cover generic syntactic phenomena such as appositions, conjunctions, passive, relative clause, etc. (Bar-Haim et al., AAAI-07) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | [[Syntactic rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Polarity rule base <br>''(to be released)''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | A manually-composed collection of entailment rules which detect predicates whose polarity is negative (e.g. didn't dance) or unknown (e.g. plans to dance). The rules capture diverse phenomena that affect polarity, e.g. verbal negation, modal verbs, conditionals, and certain verbs that induce negative or "unknown" polarity context. The latter were taken mainly from VerbNet. Extends a resource described in (Bar-Haim et al., AAAI-07) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0 <br /> | [[Polarity rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Lexical-Syntactic rule base<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | Extract lexical-syntactic entailment rules for predicates (verbal and nominal), including argument mapping. The resource is based on WordNet, Nomlex-Plus and Unary DIRT (Szpektor and Dagan, Coling 08)<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | [[Lexical-Syntactic rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | OPENU Collection<br /> | Collection of Entailment Rules and Patterns<br /> | Open University<br /> | Collections of rules, patterns etc. for RTE purpose, extracted from Reuter corpus parsed using Minipar.<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0 <br /> | [[OPENU Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Abbr''<br /> | Collection of rules for abbreviation<br /> | Bar-Ilan University; Tel-Aviv University<br /> | 2000 Abbreviation rules, extracted from [http://badc.nerc.ac.uk/help/abbrevs.html BADC] and [http://www.acronym-guide.com/ Acronym Guide]<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | [[Abbr - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Negation_list<br /> | Negation rules<br /> | „Al. I. Cuza“ University, Iasi, Romania <br /> | List of negative terms and words (verbs, adjectives, nouns) affecting modality or factuality of a infinitive verb preceded by the particle "to" (e.g. "believe","necessary", "attempt")<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1 <br /> | [[UAIC Negation_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DLSIUAES Negation_list<br /> | List of negative terms<br /> | University of Alicante<br /> | Basic list of negative terms.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1 <br /> | [[DLSIUAES Negation_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Quantifier_list<br /> | List of quantifiers<br /> | „Al. I. Cuza“ University, Iasi, Romania <br /> | List of quantifiers affecting entailment judgment. The quantifiers are taken from a list which contains expressions like “more than”, “less than”, or words such as “over”, “under”, etc.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1 <br /> | [[UAIC Quantifier_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | FBKirst StopWord list<br /> | List of frequent words<br /> | FBK-Irst;<br/>University of Trento - Italy <br /> | A list of the 572 most frequent English words.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1 <br /> | [[FBKirst StopWord list - RTE Users|Users]]<br /> <br /> |}<br /> <br><br /> <br /> ==Footnotes==<br /> <references/></div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=RTE_Knowledge_Resources&diff=8643 RTE Knowledge Resources 2011-02-02T11:51:48Z

<p>Amarchetti: /* Publicly available Resources */</p> <hr /> <div>Knowledge resources have shown their relevance for applied semantic inference, and are extensively used by applied inference systems, such as those developed within the Textual Entailment framework.<br><br /> <br><br /> This page presents a list of the knowledge resources used by systems that have participated in the last RTE challenges. The first table lists the publicly available resources, the second one lists unpublished resources. Both tables are sortable by Resource name, type, author and number of users.<br><br /> <br><br /> RTE Participants are encouraged to add information about all kind of knowledge resources used, from standard existing resources (e.g. WordNet) to knowledge collections created for specific purposes, which can be made available to the community.<br><br /> <br><br /> __TOC__<br /> <br><br /> === Call for Resources ===<br /> <br /> In order to help the research, all the participants are invited to contribute, sharing their own resources with the RTE community. <br /> <br><br /> Making the resources available to be used by other systems has several advantages. On the one hand, it helps improve the TE technology; on the other hand, it offers an opportunity to further test and evaluate the resource.<br /> <br><br /> <br><br /> [[RTE6 - Call for Resources]]<br /> <br><br /> === Ablation Tests ===<br /> An ablation test consists of removing one module at a time from a system, and rerunning the system on the test set with the other modules, except the one tested. <br /> <br><br /> Ablation test are meant to help better understand the relevance of the knowledge resources used by RTE systems, and evaluate the contribution of each of them to the systems' performances. In fact, comparing the results achieved in the ablation tests to those obtained by the systems as a whole allows assessing the contribution given by each single resource.<br /> <br><br /> <br><br /> [[RTE5 - Ablation Tests]]<br /> <br><br /> <br /> === Publicly available Resources ===<br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! width="80"|Resource<br /> ! width="50"|Type<br /> ! width="150"|Author<br /> ! class="unsortable" width="300"|Brief description<br /> ! width="45"|<small>PAST Users <ref name:"rtethree">RTE-3 data have been provided by participants by means of a questionnaire.</ref></small><br /> ! width="45"|<small>RTE4 Users<ref name:"rtefour">RTE-4 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! width="45"|<small>RTE5 Users<ref name:"rtefive">RTE-5 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! width="45"|<small>RTE6 Users<ref name:"rtesix">RTE-5 data have been provided by participants and have been integrated with information extracted from the related proceedings.</ref></small><br /> ! class="unsortable" width="30"|<small>Usage info</small><br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[WordNet]]<br /> | Lexical DB<br /> | Princeton University<br /> | Lexical database of English nouns, verbs, adjectives and adverbs<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|21<br /> | style="text-align: center;"|18<br /> |style="text-align: center;"|<br /> | [[WordNet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://xwn.hlt.utdallas.edu/index.html eXtended Wordnet]<br /> | Lexical DB<br /> | Human Language Technology Research Institute, University of Texas at Dallas<br /> | Extension of WordNet based on the exploitation of the information contained in WordNet definitional glosses: the glosses are syntactically parsed, transformed into logic forms and content words are semantically disambiguated. The Extended Wordnet is an ongoing project. <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|2<br /> |style="text-align: center;"|<br /> | [[eXtended WordNet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://ai.stanford.edu/~rion/swn/ Augmented Wordnet]<br /> | Lexical DB<br /> | Stanford University<br /> | The resource is the result of the application of a learning algorithm for inducing semantic taxonomies from parsed text. The algorithm automatically acquires items of world knowledge, and uses these to produce significantly enhanced versions of WordNet (up to 40,000 synsets more).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Augmented Wordnet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://verbs.colorado.edu/~mpalmer/projects/verbnet.html Verbnet]<br /> | Lexical DB<br /> | University of Colorado Boulder<br /> | Lexicon for English verbs organized into classes extending Levin (1993) classes through refinement and addition of subclasses to achieve syntactic and semantic coherence among members of a class<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Verbnet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[VerbOcean]]<br /> | Lexical DB<br /> | Information Sciences Institute, University of Southern California<br /> | Broad-coverage semantic network of verbs<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|6<br /> |style="text-align: center;"|<br /> | [[VerbOcean - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://framenet.icsi.berkeley.edu/ FrameNet]<br /> | Lexical DB<br /> | ICSI (International Computer Science Institute) - Berkley University<br /> | Lexical resource for English words, based on frame semantics (valences) and supported by corpus evidence<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|2<br /> |style="text-align: center;"|<br /> | [[Framenet - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/meyers/NomBank.html NomBank]<br /> | Lexical DB<br /> | New York University<br /> | Lexical resource containing syntactic frames for nouns, extracted from annotated corpora <br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[NomBank Resource - RTE Users|Users]] <br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://verbs.colorado.edu/~mpalmer/projects/ace.html PropBank]<br /> | Lexical DB<br /> | University of Colorado Boulder <br /> | Lexical resource containing syntactic frames for verbs, extracted from annotated corpora <br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[PropBank Resource - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/nomlex/index.html Nomlex] Plus<br /> | Lexical DB<br /> | New York University<br /> | Dictionary of English nominalizations: it describes the allowed complements for a nominalization and relates the nominal complements to the arguments of the corresponding verb<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[Nomlex Plus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.cs.ualberta.ca/~lindek/downloads.htm Dekang Lin’s Thesaurus]<br /> | Thesaurus<br /> | University of Alberta<br /> | Thesaurus automatically constructed using a parsed corpus, based on distributional similarity scores<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Dekang Lin’s Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://icon.shef.ac.uk/Moby/mthes.html Grady Ward's Moby Thesaurus]<br /> | Thesaurus<br /> | University of Sheffield<br /> | Thesaurus containing 30,260 root words, with 2,520,264 synonyms and related terms. Grady Ward placed this thesaurus in the public domain in 1996.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Grady Ward's Moby Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://en.wikipedia.org/wiki/Roget%27s_Thesaurus Roget's Thesaurus]<br /> | Thesaurus<br /> | Peter Mark Roget (Electronic version distributed by University of Chicago)<br /> | Roget's Thesaurus is a widely-used English thesaurus, created by Dr. Peter Mark Roget in 1805. The original edition had 15,000 words, and each new edition has been larger. The electronic edition ([http://machaut.uchicago.edu/rogets version 1.02]) is made available by University of Chicago.<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Roget's Thesaurus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.wikipedia.org/ Wikipedia]<br /> | Encyclopedia<br /> | <br /> | Free encyclopedia. Used for extraction of lexical-semantic rules (from its more structured parts), named entity recognition, geographical information etc.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|3<br /> | style="text-align: center;"|6<br /> |style="text-align: center;"|<br /> | [[Wikipedia - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.umbel.org/ Umbel]<br /> | Ontology<br /> | Structured Dynamics LLC, Coralville, IA<br /> | UMBEL stands for Upper Mapping and Binding Exchange Layer and is a lightweight ontology structure for relating Web content and data to a standard set of subject concepts<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Umbel - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.mpi-inf.mpg.de/yago-naga/yago/ YAGO]<br /> | Ontology<br /> | Max-Planck Institute for Informatics, Saarbrücken, Germany<br /> | Light-weight and extensible ontology. It contains more than 2 million entities and 20 million facts about these entities. The facts have been automatically extracted from Wikipedia and unified with WordNet.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[YAGO - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://dbpedia.org/About DBpedia]<br /> | Ontology<br /> | Open community project<br /> | DBpedia is a community effort to extract structured information from Wikipedia and to make this information available on the Web. The DBpedia knowledge base currently describes more than 2.9 million things in 91 different languages and consists of 479 million pieces of information.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[DBpedia - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[DIRT Paraphrase Collection]]<br /> | Collection of paraphrases<br /> | University of Alberta<br /> | DIRT (Discovery of Inference Rules from Text) is both an algorithm and a resulting knowledge collection. The DIRT knowledge collection is the output of the DIRT algorithm over a 1GB set of newspaper text.<br /> | style="text-align: center;"|2<br /> | style="text-align: center;"|4<br /> | style="text-align: center;"|3<br /> |style="text-align: center;"|<br /> | [[DIRT Paraphrase Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [[TEASE]] Collection<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University<br /> | Output of the TEASE algorithm <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[Tease Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://badc.nerc.ac.uk/help/abbrevs.html BADC Acronym and Abbreviation List]<br /> | Word List<br /> | BADC (British Atmospheric Data Centre)<br /> | Acronym and Abbreviation List<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[BADC Acronym and Abbreviation List - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.acronym-guide.com/ Acronym Guide]<br /> | Word List<br /> | Acronym-Guide.com <br /> | Acronym and Abbreviation Lists for English, branched in thematic directories <br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|3<br /> |style="text-align: center;"|<br /> | [[Acronym Guide - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.ldc.upenn.edu/Catalog/CatalogEntry.jsp?catalogId=LDC2006T13 Web1T 5-grams]<br /> | Word list<br /> | Linguistic Data Consortium, University of Pennsylvania; Google Inc.<br /> | Data set containing English word n-grams and their observed frequency counts. The n-gram counts were generated from approximately 1 trillion word tokens of text from publicly accessible Web pages<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[Web1T - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.coli.uni-saarland.de/%7Erwang/resources/RTE3_RTE4_NGD.zip Normalized Google Distance (RTE3&RTE4)]<br /> | Word Pair Co-occurrence<br /> | Saarland University<br /> | Co-occurrence of the word pairs in RTE3 and RTE4 using Normalized Google Distance (Cilibrasi and Vitanyi, 2004). The word pairs are all the possible combinations of content words in T and H. In practice, we used [http://m.www.yahoo.com/ Yahoo!] as the search engine.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Normalized Google Distance (RTE3&RTE4)- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.coli.uni-saarland.de/%7Erwang/resources/RTE5_NGD.zip Normalized Google Distance (RTE5)]<br /> | Word Pair Co-occurrence<br /> | Saarland University<br /> | Co-occurrence of the word pairs in RTE3 and RTE4 using Normalized Google Distance (Cilibrasi and Vitanyi, 2004). The word pairs are all the possible combinations of content words in T and H. In practice, we used [http://m.www.yahoo.com/ Yahoo!] as the search engine.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Normalized Google Distance (RTE5)- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://geonames.usgs.gov/index.html GNIS - Geographic Names Information System]<br /> | Gazetteer<br /> | USGS (United States Geological Survey)<br /> | Database containing the Federal and national standard toponyms for USA, associated areas and Antarctica<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[GNIS - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.geonames.org/ Geonames]<br /> | Gazetteer<br /> | <br /> | Database containing eight million geographical names. It is integrating geographical data such as names of places in various languages, elevation, population and others from various sources.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[Geonames - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://nlp.cs.nyu.edu/paraphrase/ Sekine's Paraphrase Database]<br /> | Collection of paraphrases<br /> | Department of Computer Science, New York University <br /> | Data-base created using Sekine's method, NOT cleaned up by human. It includes 19,975 sets of paraphrases with 191,572 phrases.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[Sekine's Paraphrase Database - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://research.microsoft.com/research/downloads/Details/607D14D9-20CD-47E3-85BC-A2F65CD28042/Details.aspx Microsoft Research Paraphrase Corpus]<br /> | Collection of paraphrases<br /> | Microsoft Research<br /> | Text file containing 5800 pairs of sentences which have been extracted from news sources on the web, along with human annotations indicating whether each pair captures a paraphrase/semantic equivalence relationship.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[Microsoft Research Paraphrase Corpus - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.cs.cornell.edu/~cristian/Without_a_doubt_-_Data.html Downward entailing operators] <br /> | Collection of entailing operators<br /> | Department of Computer Science, Cornell University, Ithaca NY<br /> | System output of an unsupervised algorithm recovering many Downward Entailing operators, like 'doubt'.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[Downward entailing operators - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/WikiRules.html WikiRules!]<br /> | Lexical Reference rule-base<br /> | Bar-Ilan University<br /> | Extraction of about 8 million lexical reference rules from the text body (first sentence) and from metadata (links, redirects, parentheses) of Wikipedia. Provides better performance than other automatically constructed resources and comparable performance to WordNet. Offers complementary knowledge to WordNet. <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> |style="text-align: center;"|<br /> | [[WikiRules! - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://www.cs.utexas.edu/users/pclark/dart/ DART]<br /> | Collection of "world knowledge" propositions<br /> | Boeing Research and Technology<br /> | 23 million tuples such as "airplanes can fly to airports", "rivers can flood" collected from abstracted parse trees.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[DART - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://cs.biu.ac.il/~nlp/download/FRED/latest/FRED.zip FRED]<br /> | FrameNet-derived entailment rule-base<br /> | Bar-Ilan University<br /> | This package contains the outputs of the FRED algorithm, an algorithm which extracts entailment rules from FrameNet.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[FRED - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/DIRECT.html DIRECT]<br /> | Directional Distributional Term-Similarity Resource<br /> | Bar-Ilan University<br /> | This is a resource of directional distributional term-similarity rules (mostly lexical entailment rules) automatically extracted using the inclusion relation as described in (Kotlerman et.al., ACL-09).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[DIRECT - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/binaryDirt.html binaryDIRT]<br /> | Entailment rules between binary templates using DIRT algorithm<br /> | Bar-Ilan University<br /> | This resource contains entailment rules over binary templates learned over the Reuters corpus using<br /> the DIRT algorithm of Lin and Pantel.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[BinaryDIRT- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> |[http://u.cs.biu.ac.il/~nlp/downloads/unaryBInc.html unaryBInc]<br /> | Entailment rules between unary templates using BInc algorithm<br /> | Bar-Ilan University<br /> | This resource contains entailment rules over unary templates learned over the Reuters corpus using<br /> the BInc algorithm of Szpektor and Dagan (2008).<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> |style="text-align: center;"|<br /> | [[unaryBInc- RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''New resource''<br /> | <br /> | <br /> | ''Participants are encouraged to contribute''<br /> | style="text-align: center;"| <br /> | style="text-align: center;"| <br /> | style="text-align: center;"| <br /> |style="text-align: center;"|<br /> | [[New Resource2 - RTE Users|Users]]<br /> <br /> |}<br /> <br><br /> <br><br /> <br /> === Not available Resources ===<br /> The following table lists the unpublished resources used by RTE participants. Some of them have been developed by Users themselves specifically for RTE. Interested people may turn to authors to obtain further information.<br /> <br><br /> {|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"<br /> <br /> |- bgcolor="#CDCDCD"<br /> ! width="80"|Resource<br /> ! width="50"|Type<br /> ! width="180"|Author<br /> ! class="unsortable" width="500"|Brief description<br /> ! width="30"|<small>PAST Users</small><br /> ! width="30"|<small>RTE4 Users</small><br /> ! width="30"|<small>RTE5 Users</small><br /> ! class="unsortable" width="30"|<small>Usage info</small><br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | [http://www.parc.com/ PARC] Polarity Lexicon<br /> | Lexical DB<br /> | PARC - Palo Alto Research Center<br /> | Verbs classification with respect to semantic polarity<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | [[Parc Polarity Lexicon - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Gazetteer from [http://trec.nist.gov/ TREC]<br /> | Gazetteer<br /> | NIST - National Institute of Standards and Technology<br /> | Cities and other geographical names<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | [[Gazetteer from TREC - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DFKI Geographic Ontology<br>''(to be released)''<br /> | Ontology<br /> | DFKI - German Research Center for Artificial Intelligence<br /> | Ontology containing geographic terms and two kinds of relations: the directional ''part-of'' relation, and the ''equal'' relation for synonyms and abbreviations of the same geographic area (e.g ''the United Kingdom'', ''the UK'', ''Great Britain'', etc.) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | [[Geographic Ontology - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Geo''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | Meronymy entailment rules, based on TREC’s TIPSTER gazetteer.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | [[Geo - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Regex''<br /> | Collection of Entailment rules <br /> | Bar-Ilan University; Tel-Aviv University<br /> | Small set of entailment rules based on regular expressions, intended to address lexical variability involving temporal phrases<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | [[Regex - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Syntactic rule base <br>''(to be released)''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | A manually-composed collection of entailment rules which define parse tree transformations. The rules cover generic syntactic phenomena such as appositions, conjunctions, passive, relative clause, etc. (Bar-Haim et al., AAAI-07) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|1<br /> | [[Syntactic rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Polarity rule base <br>''(to be released)''<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | A manually-composed collection of entailment rules which detect predicates whose polarity is negative (e.g. didn't dance) or unknown (e.g. plans to dance). The rules capture diverse phenomena that affect polarity, e.g. verbal negation, modal verbs, conditionals, and certain verbs that induce negative or "unknown" polarity context. The latter were taken mainly from VerbNet. Extends a resource described in (Bar-Haim et al., AAAI-07) <br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0 <br /> | [[Polarity rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | Lexical-Syntactic rule base<br /> | Collection of Entailment Rules<br /> | Bar-Ilan University; Tel-Aviv University<br /> | Extract lexical-syntactic entailment rules for predicates (verbal and nominal), including argument mapping. The resource is based on WordNet, Nomlex-Plus and Unary DIRT (Szpektor and Dagan, Coling 08)<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | [[Lexical-Syntactic rule base - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | OPENU Collection<br /> | Collection of Entailment Rules and Patterns<br /> | Open University<br /> | Collections of rules, patterns etc. for RTE purpose, extracted from Reuter corpus parsed using Minipar.<br /> | style="text-align: center;"|1<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0 <br /> | [[OPENU Collection - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | ''Abbr''<br /> | Collection of rules for abbreviation<br /> | Bar-Ilan University; Tel-Aviv University<br /> | 2000 Abbreviation rules, extracted from [http://badc.nerc.ac.uk/help/abbrevs.html BADC] and [http://www.acronym-guide.com/ Acronym Guide]<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1<br /> | [[Abbr - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Negation_list<br /> | Negation rules<br /> | „Al. I. Cuza“ University, Iasi, Romania <br /> | List of negative terms and words (verbs, adjectives, nouns) affecting modality or factuality of a infinitive verb preceded by the particle "to" (e.g. "believe","necessary", "attempt")<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1 <br /> | [[UAIC Negation_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | DLSIUAES Negation_list<br /> | List of negative terms<br /> | University of Alicante<br /> | Basic list of negative terms.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1 <br /> | [[DLSIUAES Negation_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | UAIC Quantifier_list<br /> | List of quantifiers<br /> | „Al. I. Cuza“ University, Iasi, Romania <br /> | List of quantifiers affecting entailment judgment. The quantifiers are taken from a list which contains expressions like “more than”, “less than”, or words such as “over”, “under”, etc.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1 <br /> | [[UAIC Quantifier_list - RTE Users|Users]]<br /> <br /> |- bgcolor="#ECECEC" "align="left"<br /> | FBKirst StopWord list<br /> | List of frequent words<br /> | FBK-Irst;<br/>University of Trento - Italy <br /> | A list of the 572 most frequent English words.<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|0<br /> | style="text-align: center;"|1 <br /> | [[FBKirst StopWord list - RTE Users|Users]]<br /> <br /> |}<br /> <br><br /> <br /> ==Footnotes==<br /> <references/></div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=Textual_Entailment_Resource_Pool&diff=8637 Textual Entailment Resource Pool 2011-01-31T17:15:00Z

<p>Amarchetti: /* Knowledge Resources */</p> <hr /> <div>[[Textual Entailment]] &gt; '''Resources''':<br /> ----<br /> <br /> [[Textual Entailment|Textual entailment]] systems rely on many different types of [[Natural Language Processing|NLP]] resources, including term banks, paraphrase lists, parsers, named-entity recognizers, etc. With so many resources being continuously released and improved, it can be difficult to know which particular resource to use when developing a system.<br /> <br /> In response, the [[Recognizing Textual Entailment|Recognizing Textual Entailment (RTE)]] shared task community initiated a new activity for building this ''Textual Entailment Resource Pool''. RTE participants and any other member of the NLP community are encouraged to contribute to the pool.<br /> <br /> In an effort to determine the relative impact of the resources, RTE participants are strongly encouraged to report, whenever possible, the contribution to the overall performance of each utilized resource. Formal qualitative and quantitative results should be included in a separate section of the system report as well as posted on the talk pages of this Textual Entailment Resource Pool.<br /> <br /> '''Adding''' a new resource is very easy. See how to '''use existing templates''' to do this in [[Help:Using Templates]].<br /> <br /> == Complete RTE Systems ==<br /> <br /> * [http://project.cgm.unive.it/html/venses.html VENSES] (from Ca' Foscari University of Venice, Italy)<br /> * [http://svn.ask.it.usyd.edu.au/trac/candc/wiki/nutcracker Nutcracker] (available for download)<br /> * [http://l2r.cs.uiuc.edu/~cogcomp/kindleDemo.php Entailment Demo] (from the University of Illinois at Urbana-Champaign) - INACTIVE (as of 2010-12-22)<br /> * [http://edits.fbk.eu/ EDITS - Edit Distance Textual Entailment Suite] (open source software developed by [http://hlt.fbk.eu/ Human Language Technology (HLT) group at FBK-Irst])<br /> <br /> == RTE data sets ==<br /> === Past campaigns data sets ===<br /> * [http://pascallin.ecs.soton.ac.uk/Challenges/RTE/Datasets RTE1 dataset] - provided by [http://pascallin.ecs.soton.ac.uk PASCAL]<br /> * [http://pascallin.ecs.soton.ac.uk/Challenges/RTE2/Datasets RTE2 dataset] - provided by [http://pascallin.ecs.soton.ac.uk PASCAL]<br /> * [http://pascallin.ecs.soton.ac.uk/Challenges/RTE3/Datasets RTE3 dataset] - provided by [http://pascallin.ecs.soton.ac.uk PASCAL]<br /> * [http://www.nist.gov/tac/data/past/2008/RTE-4.html RTE4 dataset] - provided by [http://www.nist.gov/index.html NIST] - freely available upon request. For details see [http://www.nist.gov/tac/data/forms/index.html TAC User Agreements]<br /> * [http://www.nist.gov/tac/data/past/2009/RTE-5.html RTE5 dataset] - provided by [http://www.nist.gov/index.html NIST] - freely available upon request. For details see [http://www.nist.gov/tac/data/forms/index.html TAC User Agreements]<br /> * [http://www.nist.gov/tac/2010/RTE/index.html RTE6 dataset] - provided by [http://www.nist.gov/index.html NIST] - freely available upon request. For details see [http://www.nist.gov/tac/data/forms/index.html TAC User Agreements]<br /> <br /> === Other data sets ===<br /> * [http://www.coli.uni-saarland.de/projects/salsa/fate FrameNet manually annotated RTE 2006 Test Set.] Provided by [http://www.coli.uni-saarland.de/projects/salsa/ SALSA project, Saarland University.]<br /> * [http://www.cs.biu.ac.il/~nlp/files/RTE_2006_Aligned.zip Manually Word Aligned RTE 2006 Data Sets.] Provided by [http://research.microsoft.com/nlp/ the Natural Language Processing Group, Microsoft Research.]<br /> * [http://www-nlp.stanford.edu/projects/contradiction/ RTE data sets annotated for a 3-way decision: entails, contradicts, unknown.] Provided by Stanford NLP Group.<br /> * [http://www.cs.utexas.edu/~pclark/bpi-test-suite/ BPI RTE data set] - 250 pairs, focusing on world knowledge. Provided jointly by [http://www.boeing.com/phantom/math_ct/index.html Boeing], [http://wordnet.cs.princeton.edu/ Princeton], and [http://www.isi.edu ISI].<br /> * [http://hlt.fbk.eu/en/Technology/TE_Specialized_Data Textual Entailment Specialized Data Sets] - 90 RTE-5 Test Set pairs annotated with linguistic phenomena + 203 monothematic pairs (i.e. pairs where only one linguistic phenomenon is relevant to the entailment relation) created from the 90 annotated pairs. Provided jointly by [http://hlt.fbk.eu/en/home FBK-Irst], and [http://www.celct.it/ CELCT].<br /> * [http://www.nist.gov/tac/data/ RTE-5 Search Pilot Data Set annotated with anaphora and coreference information] - RTE-5 Search Data Set annotated with anaphora/coreference information + Augmented RTE-5 Search Data Set, where all the referring expressions which need to be resolved in the entailing sentences are substituted by explicit expressions on the basis of the anaphora/coreference annotation. Provided by [http://www.celct.it/ CELCT] and distributed by [http://www.nist.gov/index.html NIST] at the [http://www.nist.gov/tac/data/ Past TAC Data] web page (2009 Search Pilot, annotated test/dev data).<br /> * [http://www.investigacion.frc.utn.edu.ar/mslabs/~jcastillo/Sagan-test-suite/ RTE-3-Expanded, RTE-4-Expanded, RTE-5-Expanded.] RTE data set expanded in the two and three way task, at least 2000 pairs in each data set.<br /> * [https://agora.cs.illinois.edu/display/rtedata/Explanation+Based+Analysis+of+RTE+Data Explanation-Based Analysis annotation of RTE 5 Main Task subset] described in [http://l2r.cs.uiuc.edu/~danr/Papers/SammonsVyRo10.pdf this ACL 2010 paper]<br /> <br /> == Knowledge Resources ==<br /> The [[RTE Knowledge Resources]] page presents: <br /> <br /> * a [[RTE Knowledge Resources#Call for Resources|call for resources]], inviting system developers to share the resources used by their own TE engines, to both help improve the TE technology and further test and evaluate such resources;<br /> * [[RTE Knowledge Resources#Ablation tests|the ablation tests]] carried out in the RTE challenges in order to evaluate the impact of knowledge resources and tools on TE system performances;<br /> * [[RTE Knowledge Resources#Publicly available Resources|lists of knowledge resources]], both publically available and unpublished, used by systems participating in the last RTE challenges.<br /> <br /> <br /> == Tools ==<br /> <br /> === Parsers ===<br /> * [http://svn.ask.it.usyd.edu.au/trac/candc C&C parser for Combinatory Categorial Grammar]<br /> * [[Minipar]]<br /> * [http://l2r.cs.uiuc.edu/~cogcomp/asoftware.php?skey=SP Shallow Parser] - from the University of Illinois at Urbana-Champaign, see a [http://l2r.cs.uiuc.edu/~cogcomp/shallow_parse_demo.php web demo] of this tool<br /> <br /> === Role Labelling ===<br /> * [http://cemantix.org/assert.html ASSERT]<br /> * [http://www.coli.uni-saarland.de/projects/salsa/shal/ Shalmaneser]<br /> * [http://l2r.cs.uiuc.edu/~cogcomp/asoftware.php?skey=SRL Semantic Role Labeler] - from the University of Illinois at Urbana-Champaign, see a [http://l2r.cs.uiuc.edu/~cogcomp/srl-demo.php web demo] of this tool<br /> <br /> === Entity Recognition Tools ===<br /> * [http://l2r.cs.uiuc.edu/~cogcomp/asoftware.php?skey=NE Illinois Named Entity Tagger] - see a [http://l2r.cs.uiuc.edu/~cogcomp/ne_demo.php web demo] of this tool<br /> * [http://l2r.cs.uiuc.edu/~cogcomp/asoftware.php?skey=CORANKER Illinois Multi-lingual Named Entity Discovery Tool] - see a [http://l2r.cs.uiuc.edu/~cogcomp/ne_matcher_demo.php web demo] of this tool<br /> <br /> === Similarity / Relatedness Tools ===<br /> * [http://ixa2.si.ehu.es/ukb UKB]: Open source WordNet-based similarity/relatedness tool, includes also pre-computed semantic vectors for all words<br /> <br /> === Corpus Readers ===<br /> * [http://nltk.org NLTK] provides a corpus reader for the data from RTE Challenges 1, 2, and 3 - see the [http://nltk.org/doc/guides/corpus.html#rte Corpus Readers] Guide for more information.<br /> <br /> === Related Libraries ===<br /> <br /> * [http://www.semantilog.org/pypes.html PyPES] general purpose library containing evaluation environment for RTE and McPIET text inference engine based on the ERG (English Resource Grammar)<br /> <br /> == Links ==<br /> * [http://homepages.inf.ed.ac.uk/jbos/rte/ Textual Entailment site by Johan Bos]<br /> * [http://ai-nlp.info.uniroma2.it/research/te/ Textual Entailment at the University of Rome "Tor Vergata"]<br /> [[Category:Textual Entailment Portal]]<br /> * [http://cogcomp.cs.illinois.edu/page/demo_view/18 Illinois Textual Entailment System Component demos]</div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=Textual_Entailment_Resource_Pool&diff=8635 Textual Entailment Resource Pool 2011-01-31T17:12:04Z

<p>Amarchetti: /* Other data sets */</p> <hr /> <div>[[Textual Entailment]] &gt; '''Resources''':<br /> ----<br /> <br /> [[Textual Entailment|Textual entailment]] systems rely on many different types of [[Natural Language Processing|NLP]] resources, including term banks, paraphrase lists, parsers, named-entity recognizers, etc. With so many resources being continuously released and improved, it can be difficult to know which particular resource to use when developing a system.<br /> <br /> In response, the [[Recognizing Textual Entailment|Recognizing Textual Entailment (RTE)]] shared task community initiated a new activity for building this ''Textual Entailment Resource Pool''. RTE participants and any other member of the NLP community are encouraged to contribute to the pool.<br /> <br /> In an effort to determine the relative impact of the resources, RTE participants are strongly encouraged to report, whenever possible, the contribution to the overall performance of each utilized resource. Formal qualitative and quantitative results should be included in a separate section of the system report as well as posted on the talk pages of this Textual Entailment Resource Pool.<br /> <br /> '''Adding''' a new resource is very easy. See how to '''use existing templates''' to do this in [[Help:Using Templates]].<br /> <br /> == Complete RTE Systems ==<br /> <br /> * [http://project.cgm.unive.it/html/venses.html VENSES] (from Ca' Foscari University of Venice, Italy)<br /> * [http://svn.ask.it.usyd.edu.au/trac/candc/wiki/nutcracker Nutcracker] (available for download)<br /> * [http://l2r.cs.uiuc.edu/~cogcomp/kindleDemo.php Entailment Demo] (from the University of Illinois at Urbana-Champaign) - INACTIVE (as of 2010-12-22)<br /> * [http://edits.fbk.eu/ EDITS - Edit Distance Textual Entailment Suite] (open source software developed by [http://hlt.fbk.eu/ Human Language Technology (HLT) group at FBK-Irst])<br /> <br /> == RTE data sets ==<br /> === Past campaigns data sets ===<br /> * [http://pascallin.ecs.soton.ac.uk/Challenges/RTE/Datasets RTE1 dataset] - provided by [http://pascallin.ecs.soton.ac.uk PASCAL]<br /> * [http://pascallin.ecs.soton.ac.uk/Challenges/RTE2/Datasets RTE2 dataset] - provided by [http://pascallin.ecs.soton.ac.uk PASCAL]<br /> * [http://pascallin.ecs.soton.ac.uk/Challenges/RTE3/Datasets RTE3 dataset] - provided by [http://pascallin.ecs.soton.ac.uk PASCAL]<br /> * [http://www.nist.gov/tac/data/past/2008/RTE-4.html RTE4 dataset] - provided by [http://www.nist.gov/index.html NIST] - freely available upon request. For details see [http://www.nist.gov/tac/data/forms/index.html TAC User Agreements]<br /> * [http://www.nist.gov/tac/data/past/2009/RTE-5.html RTE5 dataset] - provided by [http://www.nist.gov/index.html NIST] - freely available upon request. For details see [http://www.nist.gov/tac/data/forms/index.html TAC User Agreements]<br /> * [http://www.nist.gov/tac/2010/RTE/index.html RTE6 dataset] - provided by [http://www.nist.gov/index.html NIST] - freely available upon request. For details see [http://www.nist.gov/tac/data/forms/index.html TAC User Agreements]<br /> <br /> === Other data sets ===<br /> * [http://www.coli.uni-saarland.de/projects/salsa/fate FrameNet manually annotated RTE 2006 Test Set.] Provided by [http://www.coli.uni-saarland.de/projects/salsa/ SALSA project, Saarland University.]<br /> * [http://www.cs.biu.ac.il/~nlp/files/RTE_2006_Aligned.zip Manually Word Aligned RTE 2006 Data Sets.] Provided by [http://research.microsoft.com/nlp/ the Natural Language Processing Group, Microsoft Research.]<br /> * [http://www-nlp.stanford.edu/projects/contradiction/ RTE data sets annotated for a 3-way decision: entails, contradicts, unknown.] Provided by Stanford NLP Group.<br /> * [http://www.cs.utexas.edu/~pclark/bpi-test-suite/ BPI RTE data set] - 250 pairs, focusing on world knowledge. Provided jointly by [http://www.boeing.com/phantom/math_ct/index.html Boeing], [http://wordnet.cs.princeton.edu/ Princeton], and [http://www.isi.edu ISI].<br /> * [http://hlt.fbk.eu/en/Technology/TE_Specialized_Data Textual Entailment Specialized Data Sets] - 90 RTE-5 Test Set pairs annotated with linguistic phenomena + 203 monothematic pairs (i.e. pairs where only one linguistic phenomenon is relevant to the entailment relation) created from the 90 annotated pairs. Provided jointly by [http://hlt.fbk.eu/en/home FBK-Irst], and [http://www.celct.it/ CELCT].<br /> * [http://www.nist.gov/tac/data/ RTE-5 Search Pilot Data Set annotated with anaphora and coreference information] - RTE-5 Search Data Set annotated with anaphora/coreference information + Augmented RTE-5 Search Data Set, where all the referring expressions which need to be resolved in the entailing sentences are substituted by explicit expressions on the basis of the anaphora/coreference annotation. Provided by [http://www.celct.it/ CELCT] and distributed by [http://www.nist.gov/index.html NIST] at the [http://www.nist.gov/tac/data/ Past TAC Data] web page (2009 Search Pilot, annotated test/dev data).<br /> * [http://www.investigacion.frc.utn.edu.ar/mslabs/~jcastillo/Sagan-test-suite/ RTE-3-Expanded, RTE-4-Expanded, RTE-5-Expanded.] RTE data set expanded in the two and three way task, at least 2000 pairs in each data set.<br /> * [https://agora.cs.illinois.edu/display/rtedata/Explanation+Based+Analysis+of+RTE+Data Explanation-Based Analysis annotation of RTE 5 Main Task subset] described in [http://l2r.cs.uiuc.edu/~danr/Papers/SammonsVyRo10.pdf this ACL 2010 paper]<br /> <br /> == Knowledge Resources ==<br /> The [[RTE Knowledge Resources]] page presents: <br /> <br /> * a [[RTE Knowledge Resources#Call for Resources|call for resources]], inviting system developers to share the resources used by their own TE engines, to both help improve the TE technology and further test and evaluate such resources;<br /> * [[RTE Knowledge Resources#Ablation tests|the ablation tests]] carried out in the RTE challenges in order to evaluate the impact of knowledge resources and tools on TE system performances;<br /> * [[RTE Knowledge Resources#Publicly available Resources|lists of knowledge resources]], both publically available and unpublished, used by systems participating in the last RTE challenges.<br /> * [https://agora.cs.illinois.edu/display/rtedata/Explanation+Based+Analysis+of+RTE+Data Explanation-Based Analysis annotation of RTE 5 Main Task subset] described in [http://l2r.cs.uiuc.edu/~danr/Papers/SammonsVyRo10.pdf this ACL 2010 paper]<br /> <br /> == Tools ==<br /> <br /> === Parsers ===<br /> * [http://svn.ask.it.usyd.edu.au/trac/candc C&C parser for Combinatory Categorial Grammar]<br /> * [[Minipar]]<br /> * [http://l2r.cs.uiuc.edu/~cogcomp/asoftware.php?skey=SP Shallow Parser] - from the University of Illinois at Urbana-Champaign, see a [http://l2r.cs.uiuc.edu/~cogcomp/shallow_parse_demo.php web demo] of this tool<br /> <br /> === Role Labelling ===<br /> * [http://cemantix.org/assert.html ASSERT]<br /> * [http://www.coli.uni-saarland.de/projects/salsa/shal/ Shalmaneser]<br /> * [http://l2r.cs.uiuc.edu/~cogcomp/asoftware.php?skey=SRL Semantic Role Labeler] - from the University of Illinois at Urbana-Champaign, see a [http://l2r.cs.uiuc.edu/~cogcomp/srl-demo.php web demo] of this tool<br /> <br /> === Entity Recognition Tools ===<br /> * [http://l2r.cs.uiuc.edu/~cogcomp/asoftware.php?skey=NE Illinois Named Entity Tagger] - see a [http://l2r.cs.uiuc.edu/~cogcomp/ne_demo.php web demo] of this tool<br /> * [http://l2r.cs.uiuc.edu/~cogcomp/asoftware.php?skey=CORANKER Illinois Multi-lingual Named Entity Discovery Tool] - see a [http://l2r.cs.uiuc.edu/~cogcomp/ne_matcher_demo.php web demo] of this tool<br /> <br /> === Similarity / Relatedness Tools ===<br /> * [http://ixa2.si.ehu.es/ukb UKB]: Open source WordNet-based similarity/relatedness tool, includes also pre-computed semantic vectors for all words<br /> <br /> === Corpus Readers ===<br /> * [http://nltk.org NLTK] provides a corpus reader for the data from RTE Challenges 1, 2, and 3 - see the [http://nltk.org/doc/guides/corpus.html#rte Corpus Readers] Guide for more information.<br /> <br /> === Related Libraries ===<br /> <br /> * [http://www.semantilog.org/pypes.html PyPES] general purpose library containing evaluation environment for RTE and McPIET text inference engine based on the ERG (English Resource Grammar)<br /> <br /> == Links ==<br /> * [http://homepages.inf.ed.ac.uk/jbos/rte/ Textual Entailment site by Johan Bos]<br /> * [http://ai-nlp.info.uniroma2.it/research/te/ Textual Entailment at the University of Rome "Tor Vergata"]<br /> [[Category:Textual Entailment Portal]]<br /> * [http://cogcomp.cs.illinois.edu/page/demo_view/18 Illinois Textual Entailment System Component demos]</div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=Textual_Entailment_Resource_Pool&diff=8634 Textual Entailment Resource Pool 2011-01-31T17:07:04Z

<p>Amarchetti: /* Past campaigns data sets */</p> <hr /> <div>[[Textual Entailment]] &gt; '''Resources''':<br /> ----<br /> <br /> [[Textual Entailment|Textual entailment]] systems rely on many different types of [[Natural Language Processing|NLP]] resources, including term banks, paraphrase lists, parsers, named-entity recognizers, etc. With so many resources being continuously released and improved, it can be difficult to know which particular resource to use when developing a system.<br /> <br /> In response, the [[Recognizing Textual Entailment|Recognizing Textual Entailment (RTE)]] shared task community initiated a new activity for building this ''Textual Entailment Resource Pool''. RTE participants and any other member of the NLP community are encouraged to contribute to the pool.<br /> <br /> In an effort to determine the relative impact of the resources, RTE participants are strongly encouraged to report, whenever possible, the contribution to the overall performance of each utilized resource. Formal qualitative and quantitative results should be included in a separate section of the system report as well as posted on the talk pages of this Textual Entailment Resource Pool.<br /> <br /> '''Adding''' a new resource is very easy. See how to '''use existing templates''' to do this in [[Help:Using Templates]].<br /> <br /> == Complete RTE Systems ==<br /> <br /> * [http://project.cgm.unive.it/html/venses.html VENSES] (from Ca' Foscari University of Venice, Italy)<br /> * [http://svn.ask.it.usyd.edu.au/trac/candc/wiki/nutcracker Nutcracker] (available for download)<br /> * [http://l2r.cs.uiuc.edu/~cogcomp/kindleDemo.php Entailment Demo] (from the University of Illinois at Urbana-Champaign) - INACTIVE (as of 2010-12-22)<br /> * [http://edits.fbk.eu/ EDITS - Edit Distance Textual Entailment Suite] (open source software developed by [http://hlt.fbk.eu/ Human Language Technology (HLT) group at FBK-Irst])<br /> <br /> == RTE data sets ==<br /> === Past campaigns data sets ===<br /> * [http://pascallin.ecs.soton.ac.uk/Challenges/RTE/Datasets RTE1 dataset] - provided by [http://pascallin.ecs.soton.ac.uk PASCAL]<br /> * [http://pascallin.ecs.soton.ac.uk/Challenges/RTE2/Datasets RTE2 dataset] - provided by [http://pascallin.ecs.soton.ac.uk PASCAL]<br /> * [http://pascallin.ecs.soton.ac.uk/Challenges/RTE3/Datasets RTE3 dataset] - provided by [http://pascallin.ecs.soton.ac.uk PASCAL]<br /> * [http://www.nist.gov/tac/data/past/2008/RTE-4.html RTE4 dataset] - provided by [http://www.nist.gov/index.html NIST] - freely available upon request. For details see [http://www.nist.gov/tac/data/forms/index.html TAC User Agreements]<br /> * [http://www.nist.gov/tac/data/past/2009/RTE-5.html RTE5 dataset] - provided by [http://www.nist.gov/index.html NIST] - freely available upon request. For details see [http://www.nist.gov/tac/data/forms/index.html TAC User Agreements]<br /> * [http://www.nist.gov/tac/2010/RTE/index.html RTE6 dataset] - provided by [http://www.nist.gov/index.html NIST] - freely available upon request. For details see [http://www.nist.gov/tac/data/forms/index.html TAC User Agreements]<br /> <br /> === Other data sets ===<br /> * [http://www.coli.uni-saarland.de/projects/salsa/fate FrameNet manually annotated RTE 2006 Test Set.] Provided by [http://www.coli.uni-saarland.de/projects/salsa/ SALSA project, Saarland University.]<br /> * [http://www.cs.biu.ac.il/~nlp/files/RTE_2006_Aligned.zip Manually Word Aligned RTE 2006 Data Sets.] Provided by [http://research.microsoft.com/nlp/ the Natural Language Processing Group, Microsoft Research.]<br /> * [http://www-nlp.stanford.edu/projects/contradiction/ RTE data sets annotated for a 3-way decision: entails, contradicts, unknown.] Provided by Stanford NLP Group.<br /> * [http://www.cs.utexas.edu/~pclark/bpi-test-suite/ BPI RTE data set] - 250 pairs, focusing on world knowledge. Provided jointly by [http://www.boeing.com/phantom/math_ct/index.html Boeing], [http://wordnet.cs.princeton.edu/ Princeton], and [http://www.isi.edu ISI].<br /> * [http://hlt.fbk.eu/en/Technology/TE_Specialized_Data Textual Entailment Specialized Data Sets] - 90 RTE-5 Test Set pairs annotated with linguistic phenomena + 203 monothematic pairs (i.e. pairs where only one linguistic phenomenon is relevant to the entailment relation) created from the 90 annotated pairs. Provided jointly by [http://hlt.fbk.eu/en/home FBK-Irst], and [http://www.celct.it/ CELCT].<br /> * [http://www.nist.gov/tac/data/ RTE-5 Search Pilot Data Set annotated with anaphora and coreference information] - RTE-5 Search Data Set annotated with anaphora/coreference information + Augmented RTE-5 Search Data Set, where all the referring expressions which need to be resolved in the entailing sentences are substituted by explicit expressions on the basis of the anaphora/coreference annotation. Provided by [http://www.celct.it/ CELCT] and distributed by [http://www.nist.gov/index.html NIST] at the [http://www.nist.gov/tac/data/ Past TAC Data] web page (2009 Search Pilot, annotated test/dev data).<br /> * [http://www.investigacion.frc.utn.edu.ar/mslabs/~jcastillo/Sagan-test-suite/ RTE-3-Expanded, RTE-4-Expanded, RTE-5-Expanded.] RTE data set expanded in the two and three way task, at least 2000 pairs in each data set.<br /> <br /> == Knowledge Resources ==<br /> The [[RTE Knowledge Resources]] page presents: <br /> <br /> * a [[RTE Knowledge Resources#Call for Resources|call for resources]], inviting system developers to share the resources used by their own TE engines, to both help improve the TE technology and further test and evaluate such resources;<br /> * [[RTE Knowledge Resources#Ablation tests|the ablation tests]] carried out in the RTE challenges in order to evaluate the impact of knowledge resources and tools on TE system performances;<br /> * [[RTE Knowledge Resources#Publicly available Resources|lists of knowledge resources]], both publically available and unpublished, used by systems participating in the last RTE challenges.<br /> * [https://agora.cs.illinois.edu/display/rtedata/Explanation+Based+Analysis+of+RTE+Data Explanation-Based Analysis annotation of RTE 5 Main Task subset] described in [http://l2r.cs.uiuc.edu/~danr/Papers/SammonsVyRo10.pdf this ACL 2010 paper]<br /> <br /> == Tools ==<br /> <br /> === Parsers ===<br /> * [http://svn.ask.it.usyd.edu.au/trac/candc C&C parser for Combinatory Categorial Grammar]<br /> * [[Minipar]]<br /> * [http://l2r.cs.uiuc.edu/~cogcomp/asoftware.php?skey=SP Shallow Parser] - from the University of Illinois at Urbana-Champaign, see a [http://l2r.cs.uiuc.edu/~cogcomp/shallow_parse_demo.php web demo] of this tool<br /> <br /> === Role Labelling ===<br /> * [http://cemantix.org/assert.html ASSERT]<br /> * [http://www.coli.uni-saarland.de/projects/salsa/shal/ Shalmaneser]<br /> * [http://l2r.cs.uiuc.edu/~cogcomp/asoftware.php?skey=SRL Semantic Role Labeler] - from the University of Illinois at Urbana-Champaign, see a [http://l2r.cs.uiuc.edu/~cogcomp/srl-demo.php web demo] of this tool<br /> <br /> === Entity Recognition Tools ===<br /> * [http://l2r.cs.uiuc.edu/~cogcomp/asoftware.php?skey=NE Illinois Named Entity Tagger] - see a [http://l2r.cs.uiuc.edu/~cogcomp/ne_demo.php web demo] of this tool<br /> * [http://l2r.cs.uiuc.edu/~cogcomp/asoftware.php?skey=CORANKER Illinois Multi-lingual Named Entity Discovery Tool] - see a [http://l2r.cs.uiuc.edu/~cogcomp/ne_matcher_demo.php web demo] of this tool<br /> <br /> === Similarity / Relatedness Tools ===<br /> * [http://ixa2.si.ehu.es/ukb UKB]: Open source WordNet-based similarity/relatedness tool, includes also pre-computed semantic vectors for all words<br /> <br /> === Corpus Readers ===<br /> * [http://nltk.org NLTK] provides a corpus reader for the data from RTE Challenges 1, 2, and 3 - see the [http://nltk.org/doc/guides/corpus.html#rte Corpus Readers] Guide for more information.<br /> <br /> === Related Libraries ===<br /> <br /> * [http://www.semantilog.org/pypes.html PyPES] general purpose library containing evaluation environment for RTE and McPIET text inference engine based on the ERG (English Resource Grammar)<br /> <br /> == Links ==<br /> * [http://homepages.inf.ed.ac.uk/jbos/rte/ Textual Entailment site by Johan Bos]<br /> * [http://ai-nlp.info.uniroma2.it/research/te/ Textual Entailment at the University of Rome "Tor Vergata"]<br /> [[Category:Textual Entailment Portal]]<br /> * [http://cogcomp.cs.illinois.edu/page/demo_view/18 Illinois Textual Entailment System Component demos]</div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=Textual_Entailment_Resource_Pool&diff=8633 Textual Entailment Resource Pool 2011-01-31T17:04:17Z

<p>Amarchetti: /* Past campaigns data sets */</p> <hr /> <div>[[Textual Entailment]] &gt; '''Resources''':<br /> ----<br /> <br /> [[Textual Entailment|Textual entailment]] systems rely on many different types of [[Natural Language Processing|NLP]] resources, including term banks, paraphrase lists, parsers, named-entity recognizers, etc. With so many resources being continuously released and improved, it can be difficult to know which particular resource to use when developing a system.<br /> <br /> In response, the [[Recognizing Textual Entailment|Recognizing Textual Entailment (RTE)]] shared task community initiated a new activity for building this ''Textual Entailment Resource Pool''. RTE participants and any other member of the NLP community are encouraged to contribute to the pool.<br /> <br /> In an effort to determine the relative impact of the resources, RTE participants are strongly encouraged to report, whenever possible, the contribution to the overall performance of each utilized resource. Formal qualitative and quantitative results should be included in a separate section of the system report as well as posted on the talk pages of this Textual Entailment Resource Pool.<br /> <br /> '''Adding''' a new resource is very easy. See how to '''use existing templates''' to do this in [[Help:Using Templates]].<br /> <br /> == Complete RTE Systems ==<br /> <br /> * [http://project.cgm.unive.it/html/venses.html VENSES] (from Ca' Foscari University of Venice, Italy)<br /> * [http://svn.ask.it.usyd.edu.au/trac/candc/wiki/nutcracker Nutcracker] (available for download)<br /> * [http://l2r.cs.uiuc.edu/~cogcomp/kindleDemo.php Entailment Demo] (from the University of Illinois at Urbana-Champaign) - INACTIVE (as of 2010-12-22)<br /> * [http://edits.fbk.eu/ EDITS - Edit Distance Textual Entailment Suite] (open source software developed by [http://hlt.fbk.eu/ Human Language Technology (HLT) group at FBK-Irst])<br /> <br /> == RTE data sets ==<br /> === Past campaigns data sets ===<br /> * [http://pascallin.ecs.soton.ac.uk/Challenges/RTE/Datasets RTE1 dataset] - provided by [http://pascallin.ecs.soton.ac.uk PASCAL]<br /> * [http://pascallin.ecs.soton.ac.uk/Challenges/RTE2/Datasets RTE2 dataset] - provided by [http://pascallin.ecs.soton.ac.uk PASCAL]<br /> * [http://pascallin.ecs.soton.ac.uk/Challenges/RTE3/Datasets RTE3 dataset] - provided by [http://pascallin.ecs.soton.ac.uk PASCAL]<br /> * [http://www.nist.gov/tac/data/past/2008/RTE-4.html RTE4 dataset] - provided by [http://www.nist.gov/index.html NIST] - freely available upon request. For details see [http://www.nist.gov/tac/data/forms/index.html TAC User Agreements]<br /> <br /> === Other data sets ===<br /> * [http://www.coli.uni-saarland.de/projects/salsa/fate FrameNet manually annotated RTE 2006 Test Set.] Provided by [http://www.coli.uni-saarland.de/projects/salsa/ SALSA project, Saarland University.]<br /> * [http://www.cs.biu.ac.il/~nlp/files/RTE_2006_Aligned.zip Manually Word Aligned RTE 2006 Data Sets.] Provided by [http://research.microsoft.com/nlp/ the Natural Language Processing Group, Microsoft Research.]<br /> * [http://www-nlp.stanford.edu/projects/contradiction/ RTE data sets annotated for a 3-way decision: entails, contradicts, unknown.] Provided by Stanford NLP Group.<br /> * [http://www.cs.utexas.edu/~pclark/bpi-test-suite/ BPI RTE data set] - 250 pairs, focusing on world knowledge. Provided jointly by [http://www.boeing.com/phantom/math_ct/index.html Boeing], [http://wordnet.cs.princeton.edu/ Princeton], and [http://www.isi.edu ISI].<br /> * [http://hlt.fbk.eu/en/Technology/TE_Specialized_Data Textual Entailment Specialized Data Sets] - 90 RTE-5 Test Set pairs annotated with linguistic phenomena + 203 monothematic pairs (i.e. pairs where only one linguistic phenomenon is relevant to the entailment relation) created from the 90 annotated pairs. Provided jointly by [http://hlt.fbk.eu/en/home FBK-Irst], and [http://www.celct.it/ CELCT].<br /> * [http://www.nist.gov/tac/data/ RTE-5 Search Pilot Data Set annotated with anaphora and coreference information] - RTE-5 Search Data Set annotated with anaphora/coreference information + Augmented RTE-5 Search Data Set, where all the referring expressions which need to be resolved in the entailing sentences are substituted by explicit expressions on the basis of the anaphora/coreference annotation. Provided by [http://www.celct.it/ CELCT] and distributed by [http://www.nist.gov/index.html NIST] at the [http://www.nist.gov/tac/data/ Past TAC Data] web page (2009 Search Pilot, annotated test/dev data).<br /> * [http://www.investigacion.frc.utn.edu.ar/mslabs/~jcastillo/Sagan-test-suite/ RTE-3-Expanded, RTE-4-Expanded, RTE-5-Expanded.] RTE data set expanded in the two and three way task, at least 2000 pairs in each data set.<br /> <br /> == Knowledge Resources ==<br /> The [[RTE Knowledge Resources]] page presents: <br /> <br /> * a [[RTE Knowledge Resources#Call for Resources|call for resources]], inviting system developers to share the resources used by their own TE engines, to both help improve the TE technology and further test and evaluate such resources;<br /> * [[RTE Knowledge Resources#Ablation tests|the ablation tests]] carried out in the RTE challenges in order to evaluate the impact of knowledge resources and tools on TE system performances;<br /> * [[RTE Knowledge Resources#Publicly available Resources|lists of knowledge resources]], both publically available and unpublished, used by systems participating in the last RTE challenges.<br /> * [https://agora.cs.illinois.edu/display/rtedata/Explanation+Based+Analysis+of+RTE+Data Explanation-Based Analysis annotation of RTE 5 Main Task subset] described in [http://l2r.cs.uiuc.edu/~danr/Papers/SammonsVyRo10.pdf this ACL 2010 paper]<br /> <br /> == Tools ==<br /> <br /> === Parsers ===<br /> * [http://svn.ask.it.usyd.edu.au/trac/candc C&C parser for Combinatory Categorial Grammar]<br /> * [[Minipar]]<br /> * [http://l2r.cs.uiuc.edu/~cogcomp/asoftware.php?skey=SP Shallow Parser] - from the University of Illinois at Urbana-Champaign, see a [http://l2r.cs.uiuc.edu/~cogcomp/shallow_parse_demo.php web demo] of this tool<br /> <br /> === Role Labelling ===<br /> * [http://cemantix.org/assert.html ASSERT]<br /> * [http://www.coli.uni-saarland.de/projects/salsa/shal/ Shalmaneser]<br /> * [http://l2r.cs.uiuc.edu/~cogcomp/asoftware.php?skey=SRL Semantic Role Labeler] - from the University of Illinois at Urbana-Champaign, see a [http://l2r.cs.uiuc.edu/~cogcomp/srl-demo.php web demo] of this tool<br /> <br /> === Entity Recognition Tools ===<br /> * [http://l2r.cs.uiuc.edu/~cogcomp/asoftware.php?skey=NE Illinois Named Entity Tagger] - see a [http://l2r.cs.uiuc.edu/~cogcomp/ne_demo.php web demo] of this tool<br /> * [http://l2r.cs.uiuc.edu/~cogcomp/asoftware.php?skey=CORANKER Illinois Multi-lingual Named Entity Discovery Tool] - see a [http://l2r.cs.uiuc.edu/~cogcomp/ne_matcher_demo.php web demo] of this tool<br /> <br /> === Similarity / Relatedness Tools ===<br /> * [http://ixa2.si.ehu.es/ukb UKB]: Open source WordNet-based similarity/relatedness tool, includes also pre-computed semantic vectors for all words<br /> <br /> === Corpus Readers ===<br /> * [http://nltk.org NLTK] provides a corpus reader for the data from RTE Challenges 1, 2, and 3 - see the [http://nltk.org/doc/guides/corpus.html#rte Corpus Readers] Guide for more information.<br /> <br /> === Related Libraries ===<br /> <br /> * [http://www.semantilog.org/pypes.html PyPES] general purpose library containing evaluation environment for RTE and McPIET text inference engine based on the ERG (English Resource Grammar)<br /> <br /> == Links ==<br /> * [http://homepages.inf.ed.ac.uk/jbos/rte/ Textual Entailment site by Johan Bos]<br /> * [http://ai-nlp.info.uniroma2.it/research/te/ Textual Entailment at the University of Rome "Tor Vergata"]<br /> [[Category:Textual Entailment Portal]]<br /> * [http://cogcomp.cs.illinois.edu/page/demo_view/18 Illinois Textual Entailment System Component demos]</div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=Textual_Entailment_Resource_Pool&diff=8632 Textual Entailment Resource Pool 2011-01-31T17:02:04Z

<p>Amarchetti: /* Past campaigns data sets */</p> <hr /> <div>[[Textual Entailment]] &gt; '''Resources''':<br /> ----<br /> <br /> [[Textual Entailment|Textual entailment]] systems rely on many different types of [[Natural Language Processing|NLP]] resources, including term banks, paraphrase lists, parsers, named-entity recognizers, etc. With so many resources being continuously released and improved, it can be difficult to know which particular resource to use when developing a system.<br /> <br /> In response, the [[Recognizing Textual Entailment|Recognizing Textual Entailment (RTE)]] shared task community initiated a new activity for building this ''Textual Entailment Resource Pool''. RTE participants and any other member of the NLP community are encouraged to contribute to the pool.<br /> <br /> In an effort to determine the relative impact of the resources, RTE participants are strongly encouraged to report, whenever possible, the contribution to the overall performance of each utilized resource. Formal qualitative and quantitative results should be included in a separate section of the system report as well as posted on the talk pages of this Textual Entailment Resource Pool.<br /> <br /> '''Adding''' a new resource is very easy. See how to '''use existing templates''' to do this in [[Help:Using Templates]].<br /> <br /> == Complete RTE Systems ==<br /> <br /> * [http://project.cgm.unive.it/html/venses.html VENSES] (from Ca' Foscari University of Venice, Italy)<br /> * [http://svn.ask.it.usyd.edu.au/trac/candc/wiki/nutcracker Nutcracker] (available for download)<br /> * [http://l2r.cs.uiuc.edu/~cogcomp/kindleDemo.php Entailment Demo] (from the University of Illinois at Urbana-Champaign) - INACTIVE (as of 2010-12-22)<br /> * [http://edits.fbk.eu/ EDITS - Edit Distance Textual Entailment Suite] (open source software developed by [http://hlt.fbk.eu/ Human Language Technology (HLT) group at FBK-Irst])<br /> <br /> == RTE data sets ==<br /> === Past campaigns data sets ===<br /> * [http://pascallin.ecs.soton.ac.uk/Challenges/RTE/Datasets RTE1 dataset] - provided by [http://pascallin.ecs.soton.ac.uk PASCAL]<br /> * [http://pascallin.ecs.soton.ac.uk/Challenges/RTE2/Datasets RTE2 dataset] - provided by [http://pascallin.ecs.soton.ac.uk PASCAL]<br /> * [http://pascallin.ecs.soton.ac.uk/Challenges/RTE3/Datasets RTE3 dataset] - provided by [http://pascallin.ecs.soton.ac.uk PASCAL]<br /> * [http://www.nist.gov/tac/2009/RTE/past_data/index.html RTE4 dataset] - provided by [http://www.nist.gov/index.html NIST] - freely available upon request. For details see [http://www.nist.gov/tac/data/forms/index.html TAC User Agreements]<br /> <br /> === Other data sets ===<br /> * [http://www.coli.uni-saarland.de/projects/salsa/fate FrameNet manually annotated RTE 2006 Test Set.] Provided by [http://www.coli.uni-saarland.de/projects/salsa/ SALSA project, Saarland University.]<br /> * [http://www.cs.biu.ac.il/~nlp/files/RTE_2006_Aligned.zip Manually Word Aligned RTE 2006 Data Sets.] Provided by [http://research.microsoft.com/nlp/ the Natural Language Processing Group, Microsoft Research.]<br /> * [http://www-nlp.stanford.edu/projects/contradiction/ RTE data sets annotated for a 3-way decision: entails, contradicts, unknown.] Provided by Stanford NLP Group.<br /> * [http://www.cs.utexas.edu/~pclark/bpi-test-suite/ BPI RTE data set] - 250 pairs, focusing on world knowledge. Provided jointly by [http://www.boeing.com/phantom/math_ct/index.html Boeing], [http://wordnet.cs.princeton.edu/ Princeton], and [http://www.isi.edu ISI].<br /> * [http://hlt.fbk.eu/en/Technology/TE_Specialized_Data Textual Entailment Specialized Data Sets] - 90 RTE-5 Test Set pairs annotated with linguistic phenomena + 203 monothematic pairs (i.e. pairs where only one linguistic phenomenon is relevant to the entailment relation) created from the 90 annotated pairs. Provided jointly by [http://hlt.fbk.eu/en/home FBK-Irst], and [http://www.celct.it/ CELCT].<br /> * [http://www.nist.gov/tac/data/ RTE-5 Search Pilot Data Set annotated with anaphora and coreference information] - RTE-5 Search Data Set annotated with anaphora/coreference information + Augmented RTE-5 Search Data Set, where all the referring expressions which need to be resolved in the entailing sentences are substituted by explicit expressions on the basis of the anaphora/coreference annotation. Provided by [http://www.celct.it/ CELCT] and distributed by [http://www.nist.gov/index.html NIST] at the [http://www.nist.gov/tac/data/ Past TAC Data] web page (2009 Search Pilot, annotated test/dev data).<br /> * [http://www.investigacion.frc.utn.edu.ar/mslabs/~jcastillo/Sagan-test-suite/ RTE-3-Expanded, RTE-4-Expanded, RTE-5-Expanded.] RTE data set expanded in the two and three way task, at least 2000 pairs in each data set.<br /> <br /> == Knowledge Resources ==<br /> The [[RTE Knowledge Resources]] page presents: <br /> <br /> * a [[RTE Knowledge Resources#Call for Resources|call for resources]], inviting system developers to share the resources used by their own TE engines, to both help improve the TE technology and further test and evaluate such resources;<br /> * [[RTE Knowledge Resources#Ablation tests|the ablation tests]] carried out in the RTE challenges in order to evaluate the impact of knowledge resources and tools on TE system performances;<br /> * [[RTE Knowledge Resources#Publicly available Resources|lists of knowledge resources]], both publically available and unpublished, used by systems participating in the last RTE challenges.<br /> * [https://agora.cs.illinois.edu/display/rtedata/Explanation+Based+Analysis+of+RTE+Data Explanation-Based Analysis annotation of RTE 5 Main Task subset] described in [http://l2r.cs.uiuc.edu/~danr/Papers/SammonsVyRo10.pdf this ACL 2010 paper]<br /> <br /> == Tools ==<br /> <br /> === Parsers ===<br /> * [http://svn.ask.it.usyd.edu.au/trac/candc C&C parser for Combinatory Categorial Grammar]<br /> * [[Minipar]]<br /> * [http://l2r.cs.uiuc.edu/~cogcomp/asoftware.php?skey=SP Shallow Parser] - from the University of Illinois at Urbana-Champaign, see a [http://l2r.cs.uiuc.edu/~cogcomp/shallow_parse_demo.php web demo] of this tool<br /> <br /> === Role Labelling ===<br /> * [http://cemantix.org/assert.html ASSERT]<br /> * [http://www.coli.uni-saarland.de/projects/salsa/shal/ Shalmaneser]<br /> * [http://l2r.cs.uiuc.edu/~cogcomp/asoftware.php?skey=SRL Semantic Role Labeler] - from the University of Illinois at Urbana-Champaign, see a [http://l2r.cs.uiuc.edu/~cogcomp/srl-demo.php web demo] of this tool<br /> <br /> === Entity Recognition Tools ===<br /> * [http://l2r.cs.uiuc.edu/~cogcomp/asoftware.php?skey=NE Illinois Named Entity Tagger] - see a [http://l2r.cs.uiuc.edu/~cogcomp/ne_demo.php web demo] of this tool<br /> * [http://l2r.cs.uiuc.edu/~cogcomp/asoftware.php?skey=CORANKER Illinois Multi-lingual Named Entity Discovery Tool] - see a [http://l2r.cs.uiuc.edu/~cogcomp/ne_matcher_demo.php web demo] of this tool<br /> <br /> === Similarity / Relatedness Tools ===<br /> * [http://ixa2.si.ehu.es/ukb UKB]: Open source WordNet-based similarity/relatedness tool, includes also pre-computed semantic vectors for all words<br /> <br /> === Corpus Readers ===<br /> * [http://nltk.org NLTK] provides a corpus reader for the data from RTE Challenges 1, 2, and 3 - see the [http://nltk.org/doc/guides/corpus.html#rte Corpus Readers] Guide for more information.<br /> <br /> === Related Libraries ===<br /> <br /> * [http://www.semantilog.org/pypes.html PyPES] general purpose library containing evaluation environment for RTE and McPIET text inference engine based on the ERG (English Resource Grammar)<br /> <br /> == Links ==<br /> * [http://homepages.inf.ed.ac.uk/jbos/rte/ Textual Entailment site by Johan Bos]<br /> * [http://ai-nlp.info.uniroma2.it/research/te/ Textual Entailment at the University of Rome "Tor Vergata"]<br /> [[Category:Textual Entailment Portal]]<br /> * [http://cogcomp.cs.illinois.edu/page/demo_view/18 Illinois Textual Entailment System Component demos]</div>

Amarchetti https://www.aclweb.org/aclwiki/index.php?title=Textual_Entailment_Resource_Pool&diff=8624 Textual Entailment Resource Pool 2011-01-27T14:30:10Z

<p>Amarchetti: /* RTE data sets */</p> <hr /> <div>[[Textual Entailment]] &gt; '''Resources''':<br /> ----<br /> <br /> [[Textual Entailment|Textual entailment]] systems rely on many different types of [[Natural Language Processing|NLP]] resources, including term banks, paraphrase lists, parsers, named-entity recognizers, etc. With so many resources being continuously released and improved, it can be difficult to know which particular resource to use when developing a system.<br /> <br /> In response, the [[Recognizing Textual Entailment|Recognizing Textual Entailment (RTE)]] shared task community initiated a new activity for building this ''Textual Entailment Resource Pool''. RTE participants and any other member of the NLP community are encouraged to contribute to the pool.<br /> <br /> In an effort to determine the relative impact of the resources, RTE participants are strongly encouraged to report, whenever possible, the contribution to the overall performance of each utilized resource. Formal qualitative and quantitative results should be included in a separate section of the system report as well as posted on the talk pages of this Textual Entailment Resource Pool.<br /> <br /> '''Adding''' a new resource is very easy. See how to '''use existing templates''' to do this in [[Help:Using Templates]].<br /> <br /> == Complete RTE Systems ==<br /> <br /> * [http://project.cgm.unive.it/html/venses.html VENSES] (from Ca' Foscari University of Venice, Italy)<br /> * [http://svn.ask.it.usyd.edu.au/trac/candc/wiki/nutcracker Nutcracker] (available for download)<br /> * [http://l2r.cs.uiuc.edu/~cogcomp/kindleDemo.php Entailment Demo] (from the University of Illinois at Urbana-Champaign) - INACTIVE (as of 2010-12-22)<br /> * [http://edits.fbk.eu/ EDITS - Edit Distance Textual Entailment Suite] (open source software developed by [http://hlt.fbk.eu/ Human Language Technology (HLT) group at FBK-Irst])<br /> <br /> == RTE data sets ==<br /> === Past campaigns data sets ===<br /> * [http://pascallin.ecs.soton.ac.uk/Challenges/RTE/Datasets RTE1 dataset] - provided by [http://pascallin.ecs.soton.ac.uk PASCAL]<br /> * [http://pascallin.ecs.soton.ac.uk/Challenges/RTE2/Datasets RTE2 dataset] - provided by [http://pascallin.ecs.soton.ac.uk PASCAL]<br /> * [http://pascallin.ecs.soton.ac.uk/Challenges/RTE3/Datasets RTE3 dataset] - provided by [http://pascallin.ecs.soton.ac.uk PASCAL]<br /> <br /> === Other data sets ===<br /> * [http://www.coli.uni-saarland.de/projects/salsa/fate FrameNet manually annotated RTE 2006 Test Set.] Provided by [http://www.coli.uni-saarland.de/projects/salsa/ SALSA project, Saarland University.]<br /> * [http://www.cs.biu.ac.il/~nlp/files/RTE_2006_Aligned.zip Manually Word Aligned RTE 2006 Data Sets.] Provided by [http://research.microsoft.com/nlp/ the Natural Language Processing Group, Microsoft Research.]<br /> * [http://www-nlp.stanford.edu/projects/contradiction/ RTE data sets annotated for a 3-way decision: entails, contradicts, unknown.] Provided by Stanford NLP Group.<br /> * [http://www.cs.utexas.edu/~pclark/bpi-test-suite/ BPI RTE data set] - 250 pairs, focusing on world knowledge. Provided jointly by [http://www.boeing.com/phantom/math_ct/index.html Boeing], [http://wordnet.cs.princeton.edu/ Princeton], and [http://www.isi.edu ISI].<br /> * [http://hlt.fbk.eu/en/Technology/TE_Specialized_Data Textual Entailment Specialized Data Sets] - 90 RTE-5 Test Set pairs annotated with linguistic phenomena + 203 monothematic pairs (i.e. pairs where only one linguistic phenomenon is relevant to the entailment relation) created from the 90 annotated pairs. Provided jointly by [http://hlt.fbk.eu/en/home FBK-Irst], and [http://www.celct.it/ CELCT].<br /> * [http://www.nist.gov/tac/data/ RTE-5 Search Pilot Data Set annotated with anaphora and coreference information] - RTE-5 Search Data Set annotated with anaphora/coreference information + Augmented RTE-5 Search Data Set, where all the referring expressions which need to be resolved in the entailing sentences are substituted by explicit expressions on the basis of the anaphora/coreference annotation. Provided by [http://www.celct.it/ CELCT] and distributed by [http://www.nist.gov/index.html NIST] at the [http://www.nist.gov/tac/data/ Past TAC Data] web page (2009 Search Pilot, annotated test/dev data).<br /> * [http://www.investigacion.frc.utn.edu.ar/mslabs/~jcastillo/Sagan-test-suite/ RTE-3-Expanded, RTE-4-Expanded, RTE-5-Expanded.] RTE data set expanded in the two and three way task, at least 2000 pairs in each data set.<br /> <br /> == Knowledge Resources ==<br /> The [[RTE Knowledge Resources]] page presents: <br /> <br /> * a [[RTE Knowledge Resources#Call for Resources|call for resources]], inviting system developers to share the resources used by their own TE engines, to both help improve the TE technology and further test and evaluate such resources;<br /> * [[RTE Knowledge Resources#Ablation tests|the ablation tests]] carried out in the RTE challenges in order to evaluate the impact of knowledge resources and tools on TE system performances;<br /> * [[RTE Knowledge Resources#Publicly available Resources|lists of knowledge resources]], both publically available and unpublished, used by systems participating in the last RTE challenges.<br /> * [https://agora.cs.illinois.edu/display/rtedata/Explanation+Based+Analysis+of+RTE+Data Explanation-Based Analysis annotation of RTE 5 Main Task subset] described in [http://l2r.cs.uiuc.edu/~danr/Papers/SammonsVyRo10.pdf this ACL 2010 paper]<br /> <br /> == Tools ==<br /> <br /> === Parsers ===<br /> * [http://svn.ask.it.usyd.edu.au/trac/candc C&C parser for Combinatory Categorial Grammar]<br /> * [[Minipar]]<br /> * [http://l2r.cs.uiuc.edu/~cogcomp/asoftware.php?skey=SP Shallow Parser] - from the University of Illinois at Urbana-Champaign, see a [http://l2r.cs.uiuc.edu/~cogcomp/shallow_parse_demo.php web demo] of this tool<br /> <br /> === Role Labelling ===<br /> * [http://cemantix.org/assert.html ASSERT]<br /> * [http://www.coli.uni-saarland.de/projects/salsa/shal/ Shalmaneser]<br /> * [http://l2r.cs.uiuc.edu/~cogcomp/asoftware.php?skey=SRL Semantic Role Labeler] - from the University of Illinois at Urbana-Champaign, see a [http://l2r.cs.uiuc.edu/~cogcomp/srl-demo.php web demo] of this tool<br /> <br /> === Entity Recognition Tools ===<br /> * [http://l2r.cs.uiuc.edu/~cogcomp/asoftware.php?skey=NE Illinois Named Entity Tagger] - see a [http://l2r.cs.uiuc.edu/~cogcomp/ne_demo.php web demo] of this tool<br /> * [http://l2r.cs.uiuc.edu/~cogcomp/asoftware.php?skey=CORANKER Illinois Multi-lingual Named Entity Discovery Tool] - see a [http://l2r.cs.uiuc.edu/~cogcomp/ne_matcher_demo.php web demo] of this tool<br /> <br /> === Similarity / Relatedness Tools ===<br /> * [http://ixa2.si.ehu.es/ukb UKB]: Open source WordNet-based similarity/relatedness tool, includes also pre-computed semantic vectors for all words<br /> <br /> === Corpus Readers ===<br /> * [http://nltk.org NLTK] provides a corpus reader for the data from RTE Challenges 1, 2, and 3 - see the [http://nltk.org/doc/guides/corpus.html#rte Corpus Readers] Guide for more information.<br /> <br /> === Related Libraries ===<br /> <br /> * [http://www.semantilog.org/pypes.html PyPES] general purpose library containing evaluation environment for RTE and McPIET text inference engine based on the ERG (English Resource Grammar)<br /> <br /> == Links ==<br /> * [http://homepages.inf.ed.ac.uk/jbos/rte/ Textual Entailment site by Johan Bos]<br /> * [http://ai-nlp.info.uniroma2.it/research/te/ Textual Entailment at the University of Rome "Tor Vergata"]<br /> [[Category:Textual Entailment Portal]]<br /> * [http://cogcomp.cs.illinois.edu/page/demo_view/18 Illinois Textual Entailment System Component demos]</div>

Amarchetti