First Call for Participation
Clinical TempEval 2015
http://alt.qcri.org/semeval2015/task6/
as part of
SemEval-2015
International Workshop on Semantic Evaluation 2015
http://alt.qcri.org/semeval2015/
== Introduction ==
Clinical TempEval brings the temporal information extraction tasks of previous TempEvals to the clinical domain, using clinical notes and pathology reports for cancer patients from the Mayo Clinic. This follows recent interest in temporal information extraction for the clinical domain (e.g., the i2b2 2012 shared task) and broadens our understanding of the language of time beyond newswire-specific expressions. As in prior TempEvals, Clinical TempEval focuses on discrete, well-defined tasks which allow rapid, reliable and repeatable evaluation:
TS: identifying the spans of time expressions
ES: Identifying the spans of event expressions
TA: identifying the attributes of time expressions
* type=DATE, TIME, DURATION, QUANTIFIER, PREPOSTEXP or SET
* value=TIMEX3 value string as defined by TimeML
EA: identifying the attributes of event expressions
* type=N/A, ASPECTUAL or EVIDENTIAL
* polarity=POS or NEG
* degree=N/A, MOST or LITTLE
* modality=ACTUAL, HEDGED, HYPOTHETICAL or GENERIC
DR: identifying the relation between an event and the document creation time
* docTimeRel=BEFORE, OVERLAP, BEFORE-OVERLAP or AFTER
CR: identifying narrative container relations (CONTAINS a.k.a. INCLUDES)
For example, given the text:
"April 23, 2014: The patient did not have any postoperative bleeding so we will resume chemotherapy with a larger bolus on Friday even if there is slight nausea."
Systems should identify:
TS:
* April 23, 2014
* postoperative
* Friday
ES:
* bleeding
* resume
* chemotherapy
* bolus
* nausea
TA:
* April 23, 2014: type=DATE, value=2014-04-23
* postoperative: type=PREPOSTEXP
* Friday: type=DATE, value=2014-04-25
EA:
* bleeding: type=N/A, degree=N/A, polarity=NEG, modality=ACTUAL
* resume: type=ASPECTUAL, degree=N/A, polarity=POS, modality=ACTUAL
* chemotherapy: type=N/A, degree=N/A, polarity=POS, modality=ACTUAL
* bolus: type=N/A, degree=N/A, polarity=POS, modality=ACTUAL
* nausea: type=N/A, degree=LITTLE, polarity=POS, modality=HYPOTHETICAL
DR:
* bleeding BEFORE docTime
* resume AFTER docTime
* chemotherapy AFTER docTime
* bolus AFTER docTime
* nausea AFTER docTime
CR:
* postoperative CONTAINS bleeding
* Friday CONTAINS resume
* Friday CONTAINS bolus
== Data ==
Clinical TempEval will use [the THYME corpus](https://github.com/stylerw/thymedata), which has been annotated using an extension of ISO TimeML developed by [the THYME project](http://thyme.healthnlp.org/) for the annotation of times, events and temporal relations in clinical notes. Normalized time expression values are additionally being annotated by [the PHEME project](http://www.pheme.eu/).
All notes that are part of the THYME corpus have been de-identified, but to access the source texts for the THYME corpus participants must sign a data use agreement with the Mayo Clinic to ensure that the data is handled appropriately. To request a data use agreement, please follow [the instructions at the THYME data website](https://github.com/stylerw/thymedata#DUA).
**Please apply for a data use agreement as soon as possible! The process may take some time.**
== Task Organizers ==
Steven Bethard, The University of Alabama at Birmingham
Leon Derczynski, The University of Sheffield
James Pustejovsky, Brandeis University
Marc Verhagen, Brandeis University
== Important Dates ==
Trial data ready: April 30, 2014
Training data ready: July 30, 2014
Evaluation period starts: November 15, 2014
Evaluation period ends: November 30, 2014
Paper submission due: January 30, 2015
Paper reviews due: February 28, 2015
Camera ready due: March 30, 2015
SemEval workshop: Summer 2015
== Participants ==
Interested researchers should:
* Sign up on the Google Group at https://groups.google.com/d/forum/clinical-tempeval
* Post a message to the group stating which tasks you intend to participate in
* File for a data use agreement following the instructions at https://github.com/stylerw/thymedata#DUA
* Begin developing their systems with the trial data available from https://github.com/stylerw/thymedata/releases/tag/0.9.0%2BTACL-train
If you have any further questions about the task, please post them to the Google Group where they will be answered by the organizers.