DIRT Paraphrase Collection - RTE Users: Difference between revisions
Jump to navigation
Jump to search
No edit summary |
No edit summary |
||
| Line 8: | Line 8: | ||
! nowrap="nowrap"|Specific usage description | ! nowrap="nowrap"|Specific usage description | ||
! nowrap="nowrap"|Evaluations / Comments | ! nowrap="nowrap"|Evaluations / Comments | ||
|- bgcolor="#ECECEC" align="left" | |- bgcolor="#ECECEC" align="left" | ||
| | | BIU | ||
| | | RTE5 | ||
| | | | ||
| | | We used the canonical DIRT rulebase version of Szpektor and Dagan (RANLP 2007), and considered top 25 rules. | ||
| Ablation test performed. Positive impact of the resource: +1.33% accuracy on two-way task. | |||
|- bgcolor="#ECECEC" align="left" | |||
| BIU | |||
| RTE4 | |||
| | | | ||
| We used the canonical DIRT rulebase version of Szpektor and Dagan (RANLP 2007), and considered top 25 rules. | |||
| +0.9% on RTE-4 ablation tests | |||
|- bgcolor="#ECECEC" align="left" | |- bgcolor="#ECECEC" align="left" | ||
| Boeing | | Boeing | ||
| Line 22: | Line 31: | ||
boeing run1: 67%/6%;<br> | boeing run1: 67%/6%;<br> | ||
boeing run2: 54%/30% | boeing run2: 54%/30% | ||
|- bgcolor="#ECECEC" align="left" | |- bgcolor="#ECECEC" align="left" | ||
| | | UAIC | ||
| RTE4 | | RTE4 | ||
| | | | ||
| | | | ||
| ''Data taken from the RTE4 proceedings. Participants are recommended to add further information.'' | |||
|- bgcolor="#ECECEC" align="left" | |- bgcolor="#ECECEC" align="left" | ||
| | | UIUC | ||
| | | RTE3 | ||
| | | | ||
| Paired verb/argument patterns | |||
| | | | ||
|- bgcolor="#ECECEC" align="left" | |- bgcolor="#ECECEC" align="left" | ||
| Uoeltg | | Uoeltg | ||
| Line 40: | Line 52: | ||
| | | | ||
| ''Data taken from the RTE4 proceedings. Participants are recommended to add further information.'' | | ''Data taken from the RTE4 proceedings. Participants are recommended to add further information.'' | ||
|- bgcolor="#ECECEC" align="left" | |- bgcolor="#ECECEC" align="left" | ||
| ''New user'' | | ''New user'' | ||
| Line 46: | Line 59: | ||
| | | | ||
| ''Participants are encouraged to contribute.'' | | ''Participants are encouraged to contribute.'' | ||
|} | |} | ||
{|class="wikitable" cellpadding="3" cellspacing="0" border="0" style="margin-left: 20px;" | {|class="wikitable" cellpadding="3" cellspacing="0" border="0" style="margin-left: 20px;" | ||
Revision as of 15:00, 26 November 2009
When not otherwise specified, the data about version, usage and evaluation of the resource have been provided by participants themselves.
| Participants* | Campaign | Version | Specific usage description | Evaluations / Comments |
|---|---|---|---|---|
| BIU | RTE5 | We used the canonical DIRT rulebase version of Szpektor and Dagan (RANLP 2007), and considered top 25 rules. | Ablation test performed. Positive impact of the resource: +1.33% accuracy on two-way task. | |
| BIU | RTE4 | We used the canonical DIRT rulebase version of Szpektor and Dagan (RANLP 2007), and considered top 25 rules. | +0.9% on RTE-4 ablation tests | |
| Boeing | RTE4 | Original DIRT db | Elaborate T sentence with DIRT-implied entailments | precision/recall in RTE4: boeing run1: 67%/6%; |
| UAIC | RTE4 | Data taken from the RTE4 proceedings. Participants are recommended to add further information. | ||
| UIUC | RTE3 | Paired verb/argument patterns | ||
| Uoeltg | RTE4 | Data taken from the RTE4 proceedings. Participants are recommended to add further information. | ||
| New user | Participants are encouraged to contribute. |
| Total: 5 |
|---|
[*] For further information about participants, click here: RTE Challenges - Data about participants
Return to RTE Knowledge Resources