Call for Participation
Text Analysis Conference (TAC 2017)
https://tac.nist.gov/2017/
Conducted by:
U.S. National Institute of Standards and Technology (NIST)
With support from:
U.S. Department of Defense
INTRODUCTION
The Text Analysis Conference (TAC) is a series of shared tasks,
evaluations and workshops organized to promote research in Natural
Language Processing and related applications, by providing a large
test collection, common evaluation procedures, and a forum for
organizations to share their results. TAC comprises multiple tracks,
each of which focuses on a particular subproblem of NLP. TAC tracks
aim to improve end-user tasks, but also include diagnostic and
component evaluations situated within the context of end-user tasks.
TAC 2017 has six tracks in two reseach areas:
1. Knowledge Base Population (KBP)
2. Adverse Drug Reaction Extraction from Drug Labels (ADR)
You are invited to participate in TAC 2017. Organizations may choose
to participate in any or all of the TAC tracks. NIST will provide test
data for each track, and track participants will run their NLP systems
on the data and return their results to NIST for evaluation. The
annual conference culminates in a November workshop at NIST in
Gaithersburg, Maryland, USA.
All results submitted to NIST are archived on the TAC web site, and
all evaluations of submitted results are included in the conference
proceedings. Dissemination of TAC work and results other than in the
conference proceedings is welcomed, but the conditions of
participation specifically preclude any advertising claims based on
TAC results.
TRACKS
TAC 2017 has six tracks in two major areas:
1) Adverse Drug Reaction Extraction from Drug Labels (ADR)
* Home page: https://bionlp.nlm.nih.gov/tac2017adversereactions/
* Mailing list: tac-adr [at] googlegroups.com
The purpose of the ADR track is to test various natural language
approaches for their information extraction performance on adverse
drug reactions (ADR). Participants will be provided with mention-,
relation-, and document-level annotations in order to extract
structured ADR information from the Food and Drug Administration's
(FDA) official pharmaceutical knowledge base.
2) Knowledge Base Population (KBP)
* Home page: https://tac.nist.gov/2017/KBP/
* Mailing list: tac-kbp [at] nist.gov
KBP tracks develop technologies for building and populating knowledge
bases (KBs) from unstructured text. In addition to the main end-to-end
KB construction task in the Cold Start KB track, component tasks are
offered in 4 tracks that focus on specific components of the KB.
i) Cold Start KB (CSKB)
The Cold Start KB track builds a knowledge base from scratch using a
given document collection and knowledge base schema. The KB schema
includes entities, events, and relations involving entities and
events, including entity attributes (aka slots), event arguments, and
sentiment between entities.
ii) Entity Discovery and Linking (EDL)
The Entity Discovery and Linking track aims to extract entity mentions
from a source collection of textual documents and link them to a
reference KB; an EDL system is also required to cluster mentions for
those entities that don't have corresponding KB entries.
iii) Slot Filling (SF)
The Slot Filling task is to search a document collection to fill in
values for predefined slots (attributes) for a given entity.
iv) Event
The goal of the Event track is to extract information about events
such that the information would be suitable as input to a knowledge
base. The track includes Event Nugget (EN) tasks to detect and link
events, and Event Argument (EAL) tasks to extract event arguments and
link arguments that belong to the same event.
v) Belief and Sentiment (BeSt)
The Belief and Sentiment track detects belief and sentiment of an
entity toward another entity, relation, or event.
REGISTRATION
Organizations wishing to participate in any of the TAC 2017 tracks are
invited to register online by June 15, 2017. Participants are advised
to register and submit all required agreement forms as soon as
possible in order to receive timely access to track resources,
including any sample and training data. Participants should also make
sure they are subscribed to the mailing lists for the tracks for which
they are registered. Registration for a track does not commit you to
participating in the track, but is helpful to know for planning. Late
registration will be permitted only if resources allow. Any questions
about conference participation may be sent to the TAC project manager:
tac-manager [at] nist.gov.
Track registration: https://tac.nist.gov/2017/registration.html
WORKSHOP
The TAC 2017 workshop will be held November 13-14, 2017, in
Gaithersburg, Maryland, USA. The TAC workshop is a forum both for
presentation of results (including failure analyses and system
comparisons), and for more lengthy system presentations describing
techniques used, experiments run on the data, and other issues of
interest to researchers in NLP.
SCHEDULE
June 15 Deadline for registration for track participation
June - October Track evaluation windows (varies by track)
By mid October Release of individual evaluated results to participants (most tracks)
October 15 Deadline for short system descriptions
October 15 Deadline for workshop presentation proposals
October 20 Notification of acceptance of presentation proposals
November 1 Deadline for system reports (workshop notebook version)
November 13-14 TAC 2017 workshop in Gaithersburg, Maryland, USA
February 2018 Deadline for system reports (final proceedings version)
ORGANIZING COMMITTEE
Hoa Trang Dang (U.S. National Institute of Standards and Techonology)
Dina Demner-Fushman (U.S. National Library of Medicine)
Jason Duncan (MITRE)
Joe Ellis (Linguistic Data Consortium)
Marjorie Freedman (BBN Technologies)
Ralph Grishman (New York University)
Eduard Hovy (Carnegie Mellon University)
Heng Ji (Rensselaer Polytechnic Institute)
James Mayfield (Johns Hopkins University)
Teruko Mitamura (Carnegie Mellon University)
Boyan Onyshkevych (U.S. Department of Defense)
Shahzad Rajput (U.S. National Institute of Standards and Techonology)
Kirk Roberts (The University of Texas Health Science Center)
Owen Rambow (Columbia University)
Zhiyi Song (Linguistic Data Consortium)
Stephanie Strassel (Linguistic Data Consortium)
Joseph Tonning (U.S. Food and Drug Administration)