SemEval-2017 Task 6: #HashtagWars: Learning a Sense of Humor

Peter Potash, Alexey Romanov, Anna Rumshisky


Abstract
This paper describes a new shared task for humor understanding that attempts to eschew the ubiquitous binary approach to humor detection and focus on comparative humor ranking instead. The task is based on a new dataset of funny tweets posted in response to shared hashtags, collected from the ‘Hashtag Wars’ segment of the TV show @midnight. The results are evaluated in two subtasks that require the participants to generate either the correct pairwise comparisons of tweets (subtask A), or the correct ranking of the tweets (subtask B) in terms of how funny they are. 7 teams participated in subtask A, and 5 teams participated in subtask B. The best accuracy in subtask A was 0.675. The best (lowest) rank edit distance for subtask B was 0.872.
Anthology ID:
S17-2004
Volume:
Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017)
Month:
August
Year:
2017
Address:
Vancouver, Canada
Editors:
Steven Bethard, Marine Carpuat, Marianna Apidianaki, Saif M. Mohammad, Daniel Cer, David Jurgens
Venue:
SemEval
SIG:
SIGLEX
Publisher:
Association for Computational Linguistics
Note:
Pages:
49–57
Language:
URL:
https://aclanthology.org/S17-2004
DOI:
10.18653/v1/S17-2004
Bibkey:
Cite (ACL):
Peter Potash, Alexey Romanov, and Anna Rumshisky. 2017. SemEval-2017 Task 6: #HashtagWars: Learning a Sense of Humor. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 49–57, Vancouver, Canada. Association for Computational Linguistics.
Cite (Informal):
SemEval-2017 Task 6: #HashtagWars: Learning a Sense of Humor (Potash et al., SemEval 2017)
Copy Citation:
PDF:
https://aclanthology.org/S17-2004.pdf