COLING/ACL 2006 Program Co-Chairs:
Claire Cardie
Pierre Isabelle
- Pierre and I began talking by phone last May of 2005 to determine
various aspects of the joint conference.
- PLANNING MEETING AT ACL-2005. We met with Nicoletta Calzolari
(general chair), and Robert Dale and Cecile Paris (local
arrangements) at ACL-2005 in Ann Arbor to discuss the goals and
general structure of the conference. In addition, we presented our
ideas for the conference at a lunch meeting with representatives
from the ACL executive committee and the ICCL, which runs COLING
conferences. The main conference program will be fairly standard for
an ACL conference:
- parallel sessions
- invited speakers or panels, one per day
- poster sessions
- demos (demo chair: James Curran)
At these meetings, it was decided that we would try to take
advantage of the conference's location in Asia to highlight papers
focusing on Asian language processing, possibly in a separate
session of the main conference.
- PAPERS+POSTERS. In order to meet the combined desires of the ICCL
and the ACL executive committee, we opted for a single submission
and reviewing process for papers and posters. Papers were submitted
to one of two categories: the REGULAR PAPER category or the POSTER
category. Authors had to designate one of these categories at
submission time. Although both categories of submission were the
same maximum length, the presentation format would vary --- regular
papers would be presented during one of the parallel paper sessions
at the conference, and poster presentations would be repeated
several times before small groups of people at one of the conference
poster sessions. Regular papers are most appropriate for presenting
substantial research results, while posters are more appropriate for
presenting an ongoing research effort. Importantly for ACL, regular
papers and poster papers would need to appear in separate volumes of
the proceedings; and the acceptance rate for regular papers would
need to stay below 25%. Importantly for ICCL, the combined
acceptance rate among regular and poster papers should be around
40%.
In spite of the guidelines we placed in the call for papers and on
the review form for each category of submission , the distinction
caused some confusion among both authors and reviewers. In
addition, the vast majority of authors opted for paper presentations
(see below).
- CALL FOR PAPERS AND POSTERS. The call for papers and posters went
out in early October. Handouts were included in conference materials
at HLT-EMNLP in Vancouver. Richard Power offered to coordinate a
mentoring service for authors from regions of the world where
English is not the language of scientific exchange.
- SCHEDULE. We created and basically stayed on the following
schedule:
February 15 (weds) reviewers recruited
February 28 (tues) submissions arrive
March 1 (weds) papers assigned to primary track/area
March 1-4 (weds-sun) reviewer bidding phase
March 5 (mon) initial assignment of reviewers to papers using START
March 7 (weds) final assignment of reviewers to papers by area chairs
March 8 (thurs) reviewing stage begins
April 17 (mon) reviews due
April 17-24 (tues-mon) e-mail/START discussion amongst reviewers and
area chairs on "disagreement" papers
April 25-27 (tues-thurs) area chair discussion/decisions on
accept/reject; paper/poster
Fri, April 28 author notification
- AREA CHAIRS. Based on submission numbers from ACL 2005 (436
submissions) and ACL 2004 (351 submissions), we prepared for 500
submissions. We establised 19 areas and recruited 20 area chairs
(two for the machine translation area, which had 56 submissions in
2005).
Johan Bos (University of Edinburgh)
Jason Chang (National Tsing Hua University)
David Chiang (USC Information Sciences Institute)
Eva Hajicova (Charles University)
Chu-Ren Huang (Academia Sinica)
Martin Kay (Stanford University)
Emiel Krahmer (Tilburg University)
Roland Kuhn (National Research Council of Canada)
Lillian Lee (Cornell University)
Yuji Matsumoto (Nara Institute of Technology)
Dan Moldovan (University of Texas)
Mark-Jan Nederhof (University of Groningen)
Hwee Tou Ng (National University of Singapore)
John Prager (IBM Watson Research Center)
Anoop Sarkar (Simon Fraser University
Donia Scott (Open University UK)
Simone Teufel (University of Cambridge)
Benjamin Tsou (City University, Hong Kong)
Ming Zhou (Microsoft Beijing)
ChengXiang Zhai (University of Illinois)
- SUBMISSIONS. We got more submissions than expected --- 628 vs. ~500.
# paper submissions: 558 (88.9%)
# poster submissions: 70 (11.1%)
Submission stats by area, showing the expected vs. the actual number
of submissions are below.
AREA expected # actual #
of subs of subs
Phonology, Word Segmentation, Morphology;
POS tagging, chunking 40 35
Grammars/syntax 15 16
Parsing 25 45
Lexical Semantics 10 28
WSD 30 20
Inference, pragmatics 20 47
Coreference, Discourse, Dialog, Prosody
and Multi-modality 45 38
Speech and Language Modeling 40 25
Machine Learning methods 50 22
Language resources, corpus annotation 10 22
Machine Translation and Multilinguality 60 81
Information retrieval and text classification,
including sentiment analysis 10 56
Information Extraction 45 56
Question-Answering 20 19
Summarization 25 20
Generation 25 12
NLP applications and tools (e.g. tutoring) 20 39
Asian languages 15 47
Totals 505 628
As a result, we made IR its own area, and recruited ChengXiang Zhai
(University of Illinois) as its area chair. In addition, many more
reviewers needed to be recruited, 384 in all.
- SUBMISSION STATS BY COUNTRY/REGION. Note that these numbers were
computed AFTER a number of papers were withdrawn or rejected without
review.
We received submissions from 40+ countries: 39% from 13 countries in
Asia, 29% from 17 countries in Europe, 25% from Canada and the
United States, 4% from Australia and New Zealand, less than 1%
from South America (Brazil) and from Africa (South Africa and
Tunisia), 3% from 4 countries in the Middle East.
Asia==================================================
15 countries
266/616: 43% of submissions
CHINA: 55 (8.93%)
HONGKONG: 8 (1.30%)
INDIA: 11 (1.79%)
JAPAN: 107 (17.37%)
MALAYSIA: 2 (0.32%)
PAKISTAN: 1 (0.16%)
PHILIPPINES: 1 (0.16%)
REPUBLIC-OF-KOREA: 13 (2.11%)
SINGAPORE: 20 (3.25%)
SRILANKA: 3 (0.49%)
TAIWAN: 18 (2.92%)
THAILAND: 4 (0.65%)
TURKEY: 1 (0.16%)
AUSTRALIA: 22 (3.57%)
NEWZEALAND: 1 (0.16%)
Europe==================================================
17 countries
177/616= 29%
AUSTRIA: 3 (0.49%)
CZECHREPUBLIC: 7 (1.14%)
FINLAND: 2 (0.32%)
FRANCE: 34 (5.52%)
GERMANY: 36 (5.84%)
GREECE: 3 (0.49%)
HUNGARY: 1 (0.16%)
IRELAND: 8 (1.30%)
ITALY: 9 (1.46%)
NETHERLANDS: 2 (0.32%)
PORTUGAL: 4 (0.65%)
ROMANIA: 1 (0.16%)
RUSSIANFEDERATION: 1 (0.16%)
SPAIN: 17 (2.76%)
SWEDEN: 6 (0.97%)
SWITZERLAND: 4 (0.65%)
UNITED-KINGDOM: 6 (0.97%)
UNITEDKINGDOM: 32 (5.19%) ***dup
South America =============================================
2/616= <1%
BRAZIL: 2 (0.32%)
North America =============================================
153/616= 25%
CANADA: 13 (2.11%)
UNITED-STATES: 140 (22.73%)
Middle East ================================================
4 countries
16/616= 3%
IRAN: 3 (0.49%)
ISRAEL: 11 (1.79%)
SAUDIARABIA: 1 (0.16%)
UNITED-ARAB-EMIRATES: 1 (0.16%)
Africa =====================================================
2 countries
2/616= <1%
SOUTHAFRICA: 1 (0.16%)
TUNISIA: 1 (0.16%)
- CONFERENCE MANAGEMENT SYSTEM. As in previous ACL conferences, we
used the START system to manage submissions and reviews. Things
generally went well and we got good support from Rich Gerber.
- BIDDING. One new component of the reviewing process for Coling/ACL
was "bidding" for papers by reviewers. This had worked well for
other conferences (AAAI, ICML, KDD, etc.). In general, it also
worked well for us. In particular, reviewers generally received the
papers that they preferred via bidding; they only needed to bid on
papers for the areas that they were reviewers for; and START makes
automatic assignments (which can be modified) based on the bids, so
that, in general, it is easy for area chairs to assign papers to
reviewers. We had hoped that bidding would give us greater
flexibility in assigning reviewers to papers, e.g. for papers that
span more than one area or if we wanted to share reviewers across
areas. But START did not have a reasonable mechanism for doing
this. Some reviewers (e.g. Peter Turney) were disappointed by that
and expressed a strong preference for the way these matters were
handled by CyberChairPRO at ICML and other conferences.
Nevertheless, we were happy with the bidding process overall.
- REVIEWING. The reviewing process went very smoothly. As indicated in
the schedule presented above, it included a discussion period for
papers on which reviewers strongly disagreed. 100% of reviews
were turned in on or nearly on the due date!
- ACCEPTANCE RATES AND STATS. Paper presentation: 23%; Poster
presentation: 20%; Overall: 43%
N.B. (1) Since only 11% of the submissions were in the poster
category, the majority of submissions that were accepted as posters
had in fact been submitted in the paper category.
N.B. (2) The numbers above are in fact the ratios between the number
of papers/posters accepted and the total number of submissions. The
proportion of positive decisions from the PC was slightly higher,
but some papers were pulled out, especially for paper submissions
that were being redirected to the poster category.
For more details, see the .htm file.
- DOUBLE SUBMISSIONS. Double submissions were a bit of a problem. In
spite of the wording in the call for proposals, some (30+?) authors
failed to note parallel submissions. The current wording is the
following:
"Papers that are being submitted in parallel to other conferences
or workshops must indicate this on the title page, as must papers
that contain significant overlap with previously published work."
We suggest that the wording be stronger next year. The new wording
should probably make it clear that if the authors decide to
submit to another conference only after they have completed their
submission to the ACL conference, then they must update their
submission info with the ACL.
There was a particular problem with EMNLP submissions; we needed
confirmation on accepted Coling/ACL papers and posters BEFORE the
EMNLP decision notification date. Authors of 8-10 EMNLP submissions
that we had accepted as posters, wanted to wait until the EMNLP
decisions were made before committing to attend Coling/ACL for a
poster presentation.
- INVITED SPEAKERS. With input from the area chairs, we decided on
two invited speakers, one from within the Coling/ACL community and
one from a field traditionally outside the purview of NLP/CL.
- ASIAN LANGUAGE EVENTS. In honor of the joint conference's location
in Asia, we planned a special Asian language event that consists of
the presentation of the top four Asian language papers in a parallel
session and a plenary talk/panel focusing on issues in Asian
language processing, followed by the presentation of the Best Asian
Language Paper Award. The panel was organized by Aravind Joshi. The
Asian language best paper was selected by the area chairs and
program chairs.
- BEST REVIEWER AWARD. In order to acknowledge the crucial role from
our reviewers, we were planning to grant a Best Reviewer Awards at
the Closing Session. We asked our area chairs to designate some
contenders for the title. Our plan was to randomly draw 5 names from
the resulting list. However, we realized that some people were
taking this matter too seriously ("this is stuff for beefing up
one's CV") for the kind of lightweight informal process we had set
up. This is why we decided to deemphasize this by dropping the name
"Best Reviewer Award". Instead as part of the general "thanking"
episode we will draw the names of 5 great reviewers.