GENERATION CHALLENGES 2012
--------------------------
To be held in conjunction with INLG 2012, 30 May-01 June 2012, Starved Rock, IL, USA.
Over the past five years, there has been a lot of activity in
connection with shared tasks in Natural Language Generation (NLG).
Six separate sets of shared tasks each with its own data and team of
organisers have so far been run: TUNA (Gatt et al.), GREC (Belz et
al.), GIVE (Koller et al.), QG (Rus et al.), HOO (Dale and Kilgarriff)
SR (Belz, White et al.). The Pilot Attribute Selection for
Generating Referring Expressions (ASGRE) Challenge took place in 2007
(TUNA); the Referring Expression Generation (REG) Challenge in 2008
(TUNA, GREC); the first, second and third Generation Challenges umbrella
events encompassing diverse sets of shared tasks, took place in 2009
(TUNA, GREC and GIVE), 2010 (QG, GREC and GIVE) and 2011 (HOO, SR);
other shared tasks are being prepared. More information about all of these activities can be found via the links on the Generation Challenges 2012 homepage:
http://www.nltg.brighton.ac.uk/research/genchal12.
In order to continue to provide a common forum for these activities,
we are organising Generation Challenges 2012 (GenChal'12), an umbrella
event designed to bring together a variety of shared-task evaluation
efforts that involve the generation of natural language. For some of the
tasks, evaluation results and participating systems will be presented at
the Generation Challenges 2012 Special Session at the 7TH International
Natural Language Generation Conference (INLG'12). The session will follow the
format of previous GenChal results sessions, with presentations of
results by the organisers of the different shared tasks, as well as
presentations of proposals for new shared tasks in the Task Proposals
Track. As at GenChal'10 and GenChal'11, we envisage having a poster session where
participants in the various shared tasks will have the opportunity to
present their approaches and results. In addition, we are planning to
have break-out sessions for individual tasks, providing further
opportunity for current and future task participants to exchange ideas
and feed back to the organisers.
1. Shared Tasks:
- - - - - - - -
The GenChal'12 meeting will be held at INLG'12 in Starved Rock, IL, USA,
30 May-01 June 2012, but the timetables of the individual tasks are not
completely aligned, and they will be organised independently (for
details, please refer to the Call for Participation for each
task). The following is the list of tasks that are running in 2012:
1. Surface Realisation Task 2012: (White, Wanner, Bohnet, Hogan, Stent,
Belz et al.): Generation of sentences from shallow and deep input
represenations. There will be an update report presented at GenChal'12.
2. Helping Our Own (HOO) (Dale): Text-to-text generation
task aiming to improve the quality of texts, in particular those
written by non-native speakers. Call for Registration has been
posted: http://clt.mq.edu.au/research/projects/hoo/hoo2012/callforregistration.html.
The official HOO results session will be hosted at BEA'12. A summary report
will be presented at GenChal'12.
2. QG'12 (Rus et al.): Question Generation task aiming to generate questions
given answers in the form of Natural Language text. Although the QG task will not
be running in 2012, there will be an update report presented at GenChal'12.
2. Track on Proposals for Future Tasks:
- - - - - - - - - - - - - - - - - - - -
We invite submissions of papers describing ideas for future shared
tasks in the general area of language generation. Proposed tasks can
be in the area of core NLG, or in other research areas in which
language is generated, e.g. text-to-text generation (including MT and
summarisation), combining core NLG and MT, or combining core NLG and
text summarisation. Submissions should describe possible future tasks
in detail, including information regarding organisers, task
description, motivating theoretical interest and/or application
context, size and state of completion of data to be used, and
evaluation plans.
3. Instructions for Paper Submissions:
- - - - - - - - - - - - - - - - - - -
Submissions in the Task Proposals Track should be no more than 4 (four)
pages long excluding citations, and should
follow the ACL'12 guidelines using the style files provided via the
ACL 2012 homepage. Papers should be sent in PDF format by email to
nlg-stec [at] itri.brighton.ac.uk.
Task Proposal papers should be submitted no later than 15 April 2012.
Note that this deadline has been extended.
Submissions will be reviewed by at least 3 members of the Generation
Challenges Steering Committee (see below). As reviewing will not be
blind, there is no need to anonymise papers. This is not intended to be
a selective process, but the organisers reserve the right to reject papers
which do not fall within the scope of the GenChal initiative, or which do
not follow guidelines.
Accepted submissions will be included in the INLG'12 proceedings, but
the page limit for camera-ready versions has not yet been finalised
(the final limit will depend on how many submissions there are).
6. Dates:
- - - - -
Paper submission 15th April 2012
Notification of acceptance 20th April 2012
Submission of camera-ready papers 27th April 2012
INLG'12 30 May-01 June 2012
NB: Submission deadlines for the GenChal'12 Shared Tasks differ and can be
found in the calls for participation for the tasks.
7. Generation Challenges Steering Committee:
- - - - - - - - - - - - - - - - - - - - - -
Anja Belz, NLTG, University of Brighton, UK
Albert Gatt, University of Malta and University of Aberdeen, UK
Alexander Koller, Saarland University, Germany
Robert Dale, Macquarie University, Australia
Kevin Knight, ISI, University of Southern California, USA
Chris Mellish, University of Aberdeen, UK
Johanna Moore, University of Edinburgh, UK
Amanda Stent, AT&T Labs, USA
Kristina Striegnitz, Union College, USA
8. Organisation:
- - - - - - - - -
Anja Belz, NLTG, University of Brighton, UK
Albert Gatt, University of Malta and University of Aberdeen, UK
Alexander Koller, Saarland University, Germany
Kristina Striegnitz, Union College, US
Generation Challenges 2012 homepage: http://www.nltg.brighton.ac.uk/research/genchal12
Queries can be emailed directly to Albert Gatt (albert.gatt [at] um.edu.mt)
or Anja Belz (a.s.belz [at] brighton.ac.uk).