About
The "Perspective Argument Retrieval" task addresses the often-overlooked challenge of incorporating socio-cultural factors (such as political views, occupation, age, gender) in argument retrieval. By focusing on these aspects, we acknowledge their potential latent influence on argumentation.
With this shared task, we invite the community to develop methods that concentrate on this crucial area and advance state-of-the-art retrieval models by considering the perspective of societal diversity.
This task introduces three scenarios with varying levels of difficulty related to socio-cultural alignment. The scenarios include: a baseline, where arguments are selected purely by topic relevance without considering perspectivism; explicit perspectivism, where queries and candidates are matched based on a specified attribute, ensuring retrieved arguments align with this attribute; and implicit perspectivism, where the attribute is specified in the query but absent in the candidates.
Beyond relevance, we evaluate the diversity of the ranking to assess to what extend the top-ranked arguments reflect a variety of socio-cultural profiles.
Important dates
- Release Training Data: 4th March
- Release Test Data: 17 March
- First Evaluation Circle: 24 April
- Second Evaluation Circle: 30 April
- Submission Final Run: 7 May
- Submission System Description Paper: 20 May
- Camera-ready papers due: 1st July
- Workshop / Shared Task: 15th of August 2024
All deadlines are 11.59 pm UTC -12h (“anywhere on Earth”).
Registration
Register for the task using this form.
Check our website for more details.
You can download the data and evaluation script from this repository
Join our slack channel for any information or to ask questions here
Or write us an e-mail perspectiveargretrieval [at] gmail.com
Sponsoring
Thanks to our sponsor Summetix we will be able to provide prizes in form of Amazon vouchers for the three best teams in
each track: relevance and diversity.
- 1st place: 130 $
- 2nd place: 80 $
- 3rd place: 40 $
Organizers
- Andreas Waldis, University of Lucerne
- Neele Falk, University of Stuttgart
- Gabriella Lapesa, GESIS and University of Düsseldorf
- Iryna Gurevych, University of Darmstadt