|
|
| Line 1: |
Line 1: |
| ACL Program Chair Report
| |
| Sandra Carberry and Stephen Clark
| |
| June 21, 2010
| |
|
| |
|
| ACL 2010 received 987 submissions, a record number for the
| |
| conference. This was a surprise, since the submission deadline was
| |
| identical for both long and short papers. Thus we expected that the
| |
| total number of submissions would be lower than for ACL 2009, where
| |
| authors of rejected long papers had the opportunity to resubmit them
| |
| as revised short papers.
| |
|
| |
| 655 papers were submitted to ACL 2010 as long papers, and 332
| |
| papers were submitted as short papers. After filtering out those
| |
| papers that did not satify the submission requirements, for example
| |
| exceeded the length limitations or were not anonymous, 956 papers
| |
| were distributed to the Area Chairs for reviewing, of which 646 were
| |
| long submissions and 310 were short submissions. Due to the large
| |
| number of submissions, a fifth parallel session was added to the
| |
| program. We accepted 7 long papers as short papers, since these were
| |
| considered worthy of acceptance, but not as long papers, and it was
| |
| thought that the content could be presented as short papers. Counting
| |
| these 7 submissions as short papers, the overall acceptance rate was
| |
| 25% for long papers and 22% for short papers. The submissions roughly
| |
| fell into the following categories, with an attempt made to prevent
| |
| some categories, for example Machine Translation, from becoming too
| |
| large (which would have placed an unreasonable burden on the MT Area
| |
| Chairs):
| |
|
| |
| Long Paper Submissions:
| |
| AREA SUBMITTED ACCEPT-LONG ACCEPT-SHORT %-ACCEPTED
| |
| ---- --------- ----------- ------------ ----------
| |
| Bioinformatics 10 1 10.0%
| |
| Discourse 38 11 28.9%
| |
| Formal semantics 19 6 31.6%
| |
| Generation/summarization 39 10 1 28.2%
| |
| Information extraction 44 8 18.2%
| |
| Information retrieval 24 6 25.0%
| |
| Lexical semantics 59 16 27.1%
| |
| Machine learning 57 13 22.8%
| |
| Machine translation 64 15 23.4%
| |
| Mathematical linguistics 23 10 43.5%
| |
| Multimodal 13 4 30.8%
| |
| Parsing 68 16 2 26.5%
| |
| Psycholinguistics 14 5 35.7%
| |
| Question answering 22 5 2 31.8%
| |
| Resources and evaluation 28 8 28.6%
| |
| Sentiment analysis 46 9 1 21.7%
| |
| Speech 18 4 22.2%
| |
| Tagging 37 10 27.0%
| |
| Text mining 23 7 1 34.8%
| |
|
| |
|
| |
| Short Paper Submissions:
| |
| AREA SUBMITTED ACCEPTED %-ACCEPTED
| |
| ---- --------- -------- ----------
| |
| Bioinformatics 2 0 0.0%
| |
| Discourse 21 5 23.8%
| |
| Formal semantics 4 1 25.0%
| |
| Generation/summarization 17 2 11.8%
| |
| Information extraction 11 3 27.3%
| |
| Information retrieval 12 1 8.3%
| |
| Lexical semantics 30 6 20.0%
| |
| Machine learning 30 8 26.7%
| |
| Machine translation 38 10 26.3%
| |
| Mathematical linguistics 6 0 0.0%
| |
| Multimodal 5 1 20.0%
| |
| Parsing 24 5 20.8%
| |
| Psycholinguistics 5 1 20.0%
| |
| Question answering 18 2 11.1%
| |
| Resources and evaluation 25 5 20.0%
| |
| Sentiment analysis 26 6 23.1%
| |
| Speech 15 3 20.0%
| |
| Tagging 14 3 21.4%
| |
| Text mining 7 1 14.3%
| |
|
| |
| The ACL 2010 program contains a wide variety of papers, ranging
| |
| from theoretical papers to analysis papers to empirical papers. Of
| |
| particular interest is the presence of 3 challenge papers and 3 survey
| |
| papers on the ACL 2010 program; unfortunately, although we sought
| |
| position papers, none of the submissions in this category were judged
| |
| to warrant acceptance.
| |
|
| |
| ACL 2010 will award 3 best paper prizes: best long paper, best long
| |
| paper by a student author, and best short paper. The selection of the
| |
| prize recipients was made by a small panel consisting of selected area
| |
| chairs and other senior members of the research community. Although
| |
| the selection of a single paper in each prize category was difficult,
| |
| we chose to award only one prize in each category since we felt that
| |
| it lends more prestige to the prize than if the prize were shared by
| |
| several papers.
| |
|
| |
| All submissions that were accepted as long papers were allocated 9
| |
| pages of content in the proceedings, with the authors being granted an
| |
| extra page for the final version compared to the 8 pages of content
| |
| allowed at submission time. All submissions that were accepted as
| |
| short papers were allocated 5 pages of content in the proceedings,
| |
| again with an extra page allowed for the final version compared to the
| |
| 4 pages at submission time. In both cases, authors were allowed an
| |
| unlimited number of extra pages for references. The extra page of
| |
| content in the final papers was an experiment this year, in order to
| |
| allow authors to better improve their papers by addressing the
| |
| comments and suggestions of the reviewers, without having to cut
| |
| essential parts of their original submissions. Long papers will be
| |
| presented either as 25 minute oral talks or as 10 minute oral talks
| |
| followed by a poster presentation. Short papers will be presented
| |
| either as 10 minute oral talks followed by a poster presentation or
| |
| just as a poster presentation. The decision about presentation mode
| |
| was made by the program chairs based on the quality of the paper,
| |
| input from the area chairs, and our own judgement about how the paper
| |
| might best be presented.
| |
|
| |
| In order to attract and appropriately review a wider variety of
| |
| papers, we experimented this year with different review forms for
| |
| the different categories of papers. For example, the review criteria
| |
| (and thus the questions on the review form) for theoretical papers were
| |
| different from those for empirical papers. There were 10 different
| |
| review categories (analysis, challenge, empirical, negative result,
| |
| paradigms, position, resources, survey, systems, and theoretical), which,
| |
| combined with the long/short distinction resulted in 20 different review
| |
| forms; the set of review forms can be found at:
| |
| http://acl2010.org/reviewforms.html
| |
| In retrospect, we believe that the different review forms helped
| |
| immensely in obtaining appropriate reviews for the different types
| |
| of papers.
| |
|
| |
| However, despite the submission page explicitly directing authors to
| |
| examine the review forms posted on the ACL web site before selecting their
| |
| review category, it is clear that authors did not do so. Thus the program
| |
| chairs and area chairs examined every paper and changed the review
| |
| category for papers where an inappropriate category had been selected.
| |
| (Note that the Call for Papers stated that the Program Chairs reserved
| |
| the right to change the review category.) This was an enormous amount
| |
| of work. If this experiment is continued next year, we recommend that
| |
| the submission web page do one of the following:
| |
| 1. contain a question associated with each category (such
| |
| as "Does this paper present a system that has been deployed
| |
| in an industrial or research setting and includes reports
| |
| of tests with actual users?" for a systems paper), where
| |
| the user must reply "Yes" in order to select that submission
| |
| category.
| |
| 2. pop up the review form for the category that the author selects,
| |
| along with the statement "I have read this entire review form and
| |
| believe that it is appropriate for this submission". The author
| |
| would need to respond "Yes" in order to finalize the category
| |
| selection.
| |
| We would like to note that Rich Gerber was extremely helpful
| |
| in modifying the START system to accommodate our needs. For example,
| |
| the START system has been modified so that when a reviewer clicks
| |
| on the review form for a paper, he or she gets the appropriate
| |
| review form for that category of paper.
| |
|
| |
| Most of our Area Chairs and reviewers were outstanding. However,
| |
| a few reviewers did not provide the high-quality reviews that we
| |
| expect for ACL submissions. It appears that these individuals often
| |
| review for ACL. Although we do not have a solution, we do believe
| |
| that some mechanism should be developed for keeping track of
| |
| reviewers whose work is below norm so that future program chairs
| |
| and area chairs avoid inviting them to be part of the program
| |
| committee.
| |
|
| |
| While we hope that at least one author of each accepted paper will
| |
| attend the conference and present the paper, it appears that in a few
| |
| cases, the authors may not be making a serious attempt to ensure that
| |
| at least one author is present at the conference. Some conferences
| |
| require that at least one author register for the conference in order
| |
| for a paper to appear in the proceedings; other conferences prohibit
| |
| authors from submitting to the conference in the future if at least
| |
| one author of an accepted paper does not attend the conference. We
| |
| recognize that these are either difficult to enforce or somewhat
| |
| draconian measures, we do suggest that ACL adopt the policy that
| |
| the submission page contain the following statement:
| |
| If this submission is accepted for the ACL conference, we
| |
| commit to at least one of the authors attending the conference
| |
| and presenting the paper during the main technical session
| |
| on <dates>.
| |
| with the requirement that the authors click "YES" in order for
| |
| the submission to be successful.
| |
|
| |
| We also encountered problems with authors expecting that the
| |
| submission deadline would not be enforced. We had several authors
| |
| (some who are senior researchers and active in ACL) who were very
| |
| unhappy that the START system refused their papers after the
| |
| submission deadline. One spoke of a "traditional 1 hour or more grace
| |
| period" for submissions. After consultation with the ACL Exec, we did
| |
| not accept these late submissions since we felt it was the only way to
| |
| be fair to all authors. (We did keep the START system open for 15
| |
| minutes after the deadline in order to avoid cutting off authors who
| |
| were in the process of submitting prior to the deadline.) We most
| |
| strongly recommend that ACL adopt the following (or something similar)
| |
| as policy, that it be included in the Call for Papers in future years,
| |
| and that it be added to the ACL conference handbook:
| |
| "The ACL submission deadline will be extended only in the event
| |
| that the START system crashes near the deadline. The START system
| |
| will automatically shut down at the deadline, and it is ACL policy
| |
| that late submissions will not be allowed."
| |
| Without such a policy, exceptions that are informally granted by
| |
| Program Chairs are unfair to other authors (who are not aware of the
| |
| possibility of an exception) and cause problems for subsequent Program
| |
| Chairs.
| |
|
| |
| ACL 2010 will have two outstanding invited talks:
| |
| Andrei Broder: vice-president of Yahoo and both an ACM Fellow
| |
| and an IEEE Fellow. He will present a talk on
| |
| the emerging field of computational advertising,
| |
| with an emphasis on issues relevant to computational
| |
| linguistics and natural language processing.
| |
| Zenzi Griffin: professor of psychology at the University of Texas
| |
| at Austin. She will present a talk on the
| |
| psycholinguistics of social interaction, with an
| |
| emphasis on issues in language processing.
| |
| Our goal was to select invited speakers whose talks would be related
| |
| to computational linguistics, but would broaden the perspective of
| |
| the conference attendees. In addition, we sought individuals who
| |
| had a reputation for excellent presentational skills. We believe
| |
| that both of these individuals satisfy these criteria and will
| |
| present excellent and exciting invited talks at ACL.
| |