TAC 2018 Final Call for Track Proposals
(Proposal Deadline: October 20, 2017)
The Text Analysis Conference (TAC) is a series of evaluations and
workshops organized by the U.S. National Institute of Standards and
Technology (NIST) to promote research in Natural Language Processing
and text analysis, by providing a large test collection, common
evaluation procedures, and a forum for dissemination and discussion of
results. TAC comprises multiple tracks, each of which focuses on a
particular subproblem of NLP. TAC tracks aim to improve performance
on end-user tasks, but also include diagnostic and component
evaluations situated within the context of end-user tasks.
NIST invites proposals for tracks to be run in TAC 2018. Past TAC
tracks have included Question Answering (QA), Knowledge Base
Population (KBP), Text Summarization, and Recognizing Textual
Entailment (RTE). For TAC 2018, NIST invites track proposals that
continue or build on previous TAC tracks, as well as proposals for new
tracks in NLP and related areas of research.
Proposed TAC tracks should aim for the following timeline:
by February: track home page and mailing list set up
by March: initial task definition(s) released
by May: task definition(s) finalized
July - early September: test data released
by late September: evaluation results released
mid November: TAC workshop at NIST in Gaithersburg, Maryland, USA
(co-located with Text REtrieval Conference and TRECVid)
Criteria for selection of tracks include:
+ a strong advocate who is willing to volunteer as track
coordinator;
+ a large enough core of interested researchers to make the
track viable;
+ the availability of sufficient resources such as appropriate
corpora, funding, and assessors/annotators with expertise in the
area; and
+ the fit with other tracks.
Proposals should be no longer than 4 pages (PDF, Word, or ASCII) and include:
1. An explicit statement of the goals of the track (i.e., what is
expected to be learned and/or what infrastructure would be created if
the track were run).
2. A description of the task(s) to be run in the track, and
possible methods of evaluation, including availability of any required
tools.
3. What corpora would be necessary for the track, and whether there
are any special constraints on distribution of the corpora.
4. What manual assessment or annotation is required (if any).
* Where the assessing/annotation would occur (NIST or elsewhere).
* Who will provide funding for the assessment/annotation.
* Whether there are any special qualifications or domain expertise
required of the assessors/annotators.
* An estimate of the amount of time required for such
assessing/annotation.
5. Full contact details of the organizing committee, including
designation of a track coordinator.
Please email track proposals to tac-manager [at] nist.gov by October 20,
2017. Selection will be completed by November 1, 2017. Selected
track proposals will be presented at the TAC 2017 workshop, November
13-14, 2017, in Gaithersburg, MD, USA. Questions regarding track
proposals may be directed to tac-manager [at] nist.gov.