Dear colleagues,
You are invited to participate in the shared task “Context24: Evidence and Grounding Context Identification for Scientific Claims” collocated with the 4th Workshop on Scholarly Document Processing (SDP 2024) to be held at ACL 2024. Participants of the competition are also invited to submit papers describing their findings.
All papers must follow the ACL format and conform to the ACL 2024 Submission Guidelines. Papers must be submitted via openreview:
Website: https://sdproc.org/2024/sharedtasks.html#context24
Dataset: https://github.com/oasisresearchlab/context24
Submission site: http://tinyurl.com/sdpproc
Google group for participants: context24sharedtask [at] googlegroups.com
Paper submission deadline: June 17th (Monday), 2024
Shared Task Description
Interpreting scientific claims in the context of empirical findings is a valuable practice, yet extremely time-consuming for researchers. Such interpretation of scientific claims requires identifying key results that provide supporting evidence from research papers, and contextualizing these results with associated methodological details (e.g., measures, sample, etc.). In this shared task, we are interested in automating identification of key results (or evidence) as well as additional grounding context to make claim interpretation more efficient.
Context24 will have two tracks:
Track 1: Evidence Identification
Given a scientific claim and a relevant research paper, identify key figures or tables from the paper that provide supporting evidence for the claim.
Track 2: Grounding Context Identification
Given a scientific claim and a relevant research paper, identify all grounding context from the paper discussing methodological details of the experiment that resulted in this claim. This grounding context is typically dispersed throughout the full-text, often far from where the supporting evidence is presented and can include figures, tables or text snippets.
Important Dates
a. Training set release: April 5 (Friday), 2024
b. Test set release: May 24th (Friday), 2024
c. Result announcement: May 31st (Friday), 2024
d. Paper submission deadline: June 17th (Monday), 2024
e. Notification of acceptance: June 28th (Friday), 2024
f. Camera-ready paper due: July 8th (Monday), 2024
g. Workshop dates: August 15th–16th, 2024
Organizing Committee
Joel Chan, University of Maryland, College Park, USA
Matt Akamatsu, University of Washington, USA
Aakanksha Naik, Allen Institute for AI, USA