Difference between revisions of "DIRT Paraphrase Collection - RTE Users"

From ACL Wiki
Jump to navigation Jump to search
m
 
(10 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
When not otherwise specified, data about the usage of resources have been gathered from the RTE Knowledge Resources Questionnaire submitted to participants. <br>
 
 
<br />
 
<br />
−
{|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1" style="margin-left: 20px;"
+
{|class="wikitable sortable" cellpadding="3" cellspacing="0" border="1"
 
|- bgcolor="#CDCDCD"
 
|- bgcolor="#CDCDCD"
 
! nowrap="nowrap"|Participants*
 
! nowrap="nowrap"|Participants*
Line 7: Line 6:
 
! nowrap="nowrap"|Version
 
! nowrap="nowrap"|Version
 
! nowrap="nowrap"|Specific usage description
 
! nowrap="nowrap"|Specific usage description
−
! nowrap="nowrap"|Evalutations / Comments
+
! nowrap="nowrap"|Evaluations / Comments
 +
 
 +
|- bgcolor="#ECECEC" align="left"
 +
| BIU
 +
| RTE5
 +
|
 +
| We used the canonical DIRT rulebase version of Szpektor and Dagan (RANLP 2007), and considered top 25 rules.
 +
| Ablation test performed. Positive impact of the resource: +1.33% accuracy on two-way task.
 +
 
 +
|- bgcolor="#ECECEC" align="left"
 +
| Boeing
 +
| RTE5
 +
|
 +
| Verb paraphrases
 +
| Ablation test performed. Negative impact of the resource on two-way task: -1.17% accuracy. Null impact of the resource on three-way task.
 +
 
 +
|- bgcolor="#ECECEC" align="left"
 +
| UAIC
 +
| RTE5
 +
|
 +
| Use of DIRT relations to map verbs in T with verbs in H
 +
| Ablation test performed. Positive impact of the resource: +0.17% accuracy on two-way, +0.33% on three-way task.
 +
 
 +
|- bgcolor="#ECECEC" align="left"
 +
| BIU
 +
| RTE4
 +
|
 +
| We used the canonical DIRT rulebase version of Szpektor and Dagan (RANLP 2007), and considered top 25 rules.
 +
| +0.9% on RTE-4 ablation tests
 +
 
 
|- bgcolor="#ECECEC" align="left"
 
|- bgcolor="#ECECEC" align="left"
 
| Boeing
 
| Boeing
Line 16: Line 44:
 
boeing run1: 67%/6%;<br>
 
boeing run1: 67%/6%;<br>
 
boeing run2: 54%/30%
 
boeing run2: 54%/30%
−
|- bgcolor="#ECECEC" align="left"
+
 
−
| BIU
 
−
| RTE4
 
−
|
 
−
| We used the canonical DIRT rulebase version of Szpektor and Dagan (RANLP 2007), and considered top 25 rules.
 
−
| +0.9% on RTE-4 ablation tests
 
 
|- bgcolor="#ECECEC" align="left"
 
|- bgcolor="#ECECEC" align="left"
 
| UAIC
 
| UAIC
 
| RTE4
 
| RTE4
 
|  
 
|  
−
|  
+
| Use of DIRT relations to map verbs in T with verbs in H
−
| ''Data taken from the RTE4 proceedings. Participants are recommended to add further information.''
+
| Ablation test performed: +0.7% precision on two-way task.
 +
 
 
|- bgcolor="#ECECEC" align="left"
 
|- bgcolor="#ECECEC" align="left"
 
| Uoeltg
 
| Uoeltg
Line 34: Line 58:
 
|  
 
|  
 
| ''Data taken from the RTE4 proceedings. Participants are recommended to add further information.''
 
| ''Data taken from the RTE4 proceedings. Participants are recommended to add further information.''
 +
 
|- bgcolor="#ECECEC" align="left"
 
|- bgcolor="#ECECEC" align="left"
−
| ''New user''
+
| UAIC
 +
| RTE3
 
|  
 
|  
 +
| Use of DIRT relations to map verbs in T with verbs in H
 +
| Ablation test performed: +0.37% precision on two-way task.
 +
 +
|- bgcolor="#ECECEC" align="left"
 +
| UIUC
 +
| RTE3
 
|  
 
|  
 +
| Paired verb/argument patterns
 
|  
 
|  
−
| ''Participants are encouraged to contribute.''
+
 
 
|}
 
|}
 
{|class="wikitable" cellpadding="3" cellspacing="0" border="0" style="margin-left: 20px;"
 
{|class="wikitable" cellpadding="3" cellspacing="0" border="0" style="margin-left: 20px;"
 
|-
 
|-
−
! align="left"|Total: 4
+
! align="left"|Total: 9
 
|}
 
|}
 
<br>
 
<br>

Latest revision as of 11:32, 22 December 2009


Participants* Campaign Version Specific usage description Evaluations / Comments
BIU RTE5 We used the canonical DIRT rulebase version of Szpektor and Dagan (RANLP 2007), and considered top 25 rules. Ablation test performed. Positive impact of the resource: +1.33% accuracy on two-way task.
Boeing RTE5 Verb paraphrases Ablation test performed. Negative impact of the resource on two-way task: -1.17% accuracy. Null impact of the resource on three-way task.
UAIC RTE5 Use of DIRT relations to map verbs in T with verbs in H Ablation test performed. Positive impact of the resource: +0.17% accuracy on two-way, +0.33% on three-way task.
BIU RTE4 We used the canonical DIRT rulebase version of Szpektor and Dagan (RANLP 2007), and considered top 25 rules. +0.9% on RTE-4 ablation tests
Boeing RTE4 Original DIRT db Elaborate T sentence with DIRT-implied entailments precision/recall in RTE4:

boeing run1: 67%/6%;
boeing run2: 54%/30%

UAIC RTE4 Use of DIRT relations to map verbs in T with verbs in H Ablation test performed: +0.7% precision on two-way task.
Uoeltg RTE4 Data taken from the RTE4 proceedings. Participants are recommended to add further information.
UAIC RTE3 Use of DIRT relations to map verbs in T with verbs in H Ablation test performed: +0.37% precision on two-way task.
UIUC RTE3 Paired verb/argument patterns
Total: 9


[*] For further information about participants, click here: RTE Challenges - Data about participants 

   Return to RTE Knowledge Resources
Retrieved from "https://aclweb.org/aclwiki/index.php?title=DIRT_Paraphrase_Collection_-_RTE_Users&oldid=7740"