@@ Line 1: / Line 1: @@
-NAACL HLT Conference Chair Report
+This report can be found at http://www.cs.cmu.edu/~lsl/acl/NAACL-HLT-2009-report.pdf
-As General Chair, my goal for the 2009 NAACL HLT Conference was to have high
-quality technical presentations in all areas, but particularly to increase the number and
-quality of papers from the speech and IR areas. In addition, I was encouraged by the
-NAACL Board to explore ways to better engage researchers from industry.
-The ACL-HLT 2008 conference chair raised the issue of imbalance in the speech/IR/NLP
-fields. There were many more submissions in NLP than in the other two areas. She
-recommended either changing the structure of committees to reflect the actual balance of
-papers or having an “additional coordinated event that involves invitations to members of
-the community in order to draw more participation.” For NAACL HLT 2009, we
-maintained the area distribution for the area chairs, publicity and tutorials, but not for
-other committees. For committees with fewer chairs, we tried to choose people whose
-expertise spanned multiple disciplines, which seemed to work well. It would probably be
-reasonable to take a similar strategy for the PC chairs next year, i.e. have only 3 PC
-chairs with 2 from NLP and one from speech, with one of the three having a tie to IR.
-(We had one person serve as lead for long papers (Mike Collins) and one as lead for short
-papers (Lucy Vanderwende), and this worked very well.)
-In order to try to attract more papers in the speech and IR areas, we included two special
-sessions targeting the cross-cutting topics of large scale language processing (IR and
-NLP) and speech indexing and retrieval (speech and IR). The area chairs for the speech
-indexing session actively recruited paper submissions, which contributed to the session’s
-success. The large scale language processing area had several submissions without active
-recruiting other than our highlighting it in the call for papers.
-In an attempt to better engage industry researchers, we organized a lunchtime panel
-discussion on “Emerging Application Areas in Computational Linguistics,” which
-included representatives from different application areas and different size companies.
-Thanks to Bill Dolan for organizing and moderating the discussion. Box lunches were
-available for purchase. The panel discussion was very well attended. One problem was
-that the panel started late because of the long line for getting lunches, which involved
-participants composing their lunches. Possibilities for improving this include having the
-lunches pre-boxed to simplify pickup logistics and moving the lunch pickup outside of
-the panel discussion room. In addition to the industry panel, another success with
-industry is that the tutorial chairs actively recruited some more practically oriented
-panels, which were very well received.
-Two web pages were available to provide guidance:
-http://aclweb.org/adminwiki/index.php?title=Conference_Handbook
-http://www.cs.jhu.edu/~jason/advice/how-to-chair-a-conference
-While I should have done a better job in making all chairs aware of the guidelines, our
-biggest problem (coordinating schedules with other conferences) is not addressed in these
-guidelines. In addition, it would be nice to have guidance on other HLT-specific issues
-(e.g. demos). NAACL might want to develop an additional web page covering special
-considerations for NAACL HLT.
-Several changes were instituted this year, including:
-* Multi-conference coordination of sponsorship
-* Multi-conference coordination of workshops
-* New format for short paper reviewing and explicit call for different types of short
-papers
-* Including the student research workshop as a parallel session within the main
-conference
-* Allowing students whose papers were accepted to the student session to also
-(optionally) present a poster in the main conference poster session.
-All changes seemed to work well with a couple of exceptions. Most importantly, there
-was a lack of clarity of responsibility among the different sponsorship chairs and a lack
-of connection of the regional sponsorship chairs to the NAACL HLT 2009 meeting. The
-general idea of multi-conference coordination makes sense, but it is necessary to clarify
-the role of the local chair and to make this person the main point of contact. Second, as
-noted in the PC chairs’ report, there were not many papers in the “negative result” and
-“opinion piece” categories of short papers. The program committee also felt that it would
-be better to have the author indicate the type of short paper, in addition to the reviewers.
-The total funds raised through sponsorship efforts was $27,350, which includes one
-$4000 contribution that resulted from the multi-conference sponsorship coordination. The
-Local Arrangements Chairs were very helpful in this effort in addition to the Local
-Sponsorship Chair. Companies contributing included: Rosetta Stone, CNGL, Google,
-AT&T, Language Weaver, JD Powers, IBM Research, CLEAR, HLT, LDC, and John
-Benjamins. One difficulty that arose was a last minute decision that funds from a sponsor
-would go to a student travel award, which caused some minor program glitches. We
-recommend that an advance deadline be set for designating donated funds for new
-awards.
-Based on discussions during and after the conference, additional areas where we think
-there could be improvements include:
-* Parallel review of short papers: A possible reason for serial review was to allow
-papers submitted as long papers and rejected to be resubmitted as short papers.
-Very few short papers are accepted this way, and authors can always submit to
-other conferences. Parallel review would make scheduling much easier and
-would make the reviewing process less complicated for the program committee.
-* Demos: We had very few demos this year, but many people like the tradition.
-Ideas for addressing this include eingither assign the Demo Chairs the task of
-actively recruiting demos, or giving authors of accepted papers the opportunity to
-present as a demo.
-* Poster session: The evening poster session combined with a reception was well
-received by poster viewers, but less so by poster presenters. People who had
-posters in the first session didn’t get much food, and people who had their posters
-in the later session had a smaller audience. Some ideas for improving this include:
-having a better enforced time period for poster presenters to eat, having some
-overlap of the two time slots so the second session runs less late, grouping posters
-that are on related topics in the same area, and including an introductory session
-where poster presenters give a 1-minute pitch on their poster.
-* Workshops: One of the workshop chairs should be affiliated with the local
-(hosting) institution, since there are a lot of local arrangements issues that arise
-with the workshops.
-Other suggestions are included in the reports from other chairs.
-While there were several areas for improvement, overall, I consider the conference to be
-a success. There were roughly 700 participants, and the quality of the tutorials,
-presentations and workshops was high. The local arrangements were terrific. I am
-indebted to all the chairs involved in the organization and to the NAACL Board for their
-support. While there remains an imbalance between NLP, speech and IR, I am
-encouraged by the quality of the papers that were included. I strongly support continued
-efforts to include these different areas of HLT and make it possible for researchers to
-benefit from the insights of these related field.
-Mari Ostendorf, University of Washington
-General Chair
-NAACL 2009 Program Chairs Report
-In 2009 the NAACL HLT program continued to include high-quality work in the areas of
-computational linguistics, information retrieval, and speech technology. The program included
-full papers, short papers, demonstrations, a student research workshop, pre-conference
-tutorials, and post-conference workshops. The call for papers included solicitation of papers
-for 2 special sessions, “Large-Scale Language Processing” and “Speech Indexing and Retrieval”.
-This year, 260 full papers were submitted, of which 75 papers were accepted (giving a 29%
-acceptance rate); and 178 short papers were submitted, of which 71 were accepted (giving
-a 40% acceptance rate). All full papers were presented as talks; this contrasts with some
-previous years, e.g., ACL-08 HLT, where some full papers were presented as posters. Of the
-short papers, 35 were presented as talks, with the remainder being presented as posters. A
-full breakdown of the statistics by area is presented at the end of this report.
-This year, short papers of five types were solicited: “a small, focused contribution”, “work in
-progress”, “a negative result”, “an opinion piece”, or “an interesting application note”; it was
-a reviewer task to determine which paper type a short paper best belonged to, alternatively,
-this could be a check-box at submission time. In practice, the largest agreement among
-reviewers was found in the “small, focused contribution” category, the traditional type of short
-paper submitted to NAACL HLT (119/178). A majority of reviewers thought that 38/178
-papers were “work in progress”, and that 10/180 were “interesting application note”. There
-were only a handful of papers submitted that any of the reviewers considered to be a “negative
-result” or “opinion piece”. It will take more than one conference cycle to determine the field’s
-interest in writing, and then accepting, such paper types.
-Reviewing was organised in a two-tier system, with eighteen senior program committee
-(SPC) members (“area chairs”), who in turn recruited 352 reviewers. The SPC members
-managed the review process for both the full and short paper submissions: each full paper
-received at least three reviews, and each short paper received at least two reviews. Similar to
-recent years, we did not have a face-to-face meeting of the area chairs, instead we held a series
-of tele-conferences between individual area chairs and the PC chairs. The START conference
-management system was used to manage paper submissions and the review process—Rich
-Gerber and the START team gave invaluable help with the system.
-Two best paper awards were given at the conference. The senior program committee members
-for the conference nominated an initial set of papers that were candidates for the awards; the
-final decisions were then made by a committee chaired by Candace Sidner, and with Hal
-Daume III, Roland Kuhn, Ryan McDonald, and Mark Steedman as its other members.
-Michael Collins, Massachusetts Institute of Technology
-Shri Narayanan, University of Southern California
-Douglas W. Oard, University of Maryland
-Lucy Vanderwende, Microsoft Research
-Table 1: Statistics for full paper submissions.
-Area                                                Submissions      Acceptances (Talk)
-Sentiment/Information Extraction                     34               8 (24%)
-Discourse                                            10               4 (40%)
-Generation/Summarization                             22               2 (9%)
-Machine learning                                     28               8 (29%)
-Phonology/Morphology/Language acquisition            14               5 (36%)
-Semantics                                            32               7 (22%)
-Syntax                                               33              11 (33%)
-Machine translation                                  37              13 (35%)
-Dialog                                                9               3 (33%)
-IR/Question answering                                16               4 (25%)
-Large-scale language processing                      11               3 (27%)
-Speech indexing and retrieval                         1               0 (0%)
-Speech/Spoken Language Processing Algorithms          9               4 (44%)
-Speech/Spoken Language Processing Applications        8               3 (38%)
-Area Submissions Acceptances Acceptances
-(talk) (poster)
-Sentiment/Information Extraction 22 3 (14%) 4 (18%)
-Discourse 7 - 2 (29%)
-Generation/Summarization 10 2 (20%) 3 (30%)
-Machine Learning 14 2 (14%) 2 (14%)
-Phonology/Morphology/Language Aquisition 4 - 1 (25%)
-Semantics 17 4 (24%) -
-Syntax 16 3 (19%) 4 (25%)
-Machine Translation 30 8 (27%) 5 (17%)
-Dialog 11 2 (18%) 3 (27%)
-IR/Question answering 16 3 (19%) 3 (19%)
-Large Scale Processing 6 1 (17%) 2 (33%)
-Speech Indexing and Retrieval 7 5 (71%) -
-Speech/Spoken Language Algorithms 9 1 (11%) 5 (56%)
-Speech/Spoken Language Applications 9 1 (11%) 2 (22%)
-Table 2: Statistics for short paper submissions.
-Report from Student Research Workshop/ Doctoral Consortium
-The Student Research Workshop provided a venue for student researchers investigating topics
-in the broad fields of Computational Linguistics and Language Technologies to present their
-work and receive feedback from the community. The workshop was composed of three parallel
-tracks in Natural Language Processing, Information Retrieval, and Speech. We received a total
-of 29 submissions (4 IR, 4 Speech, 21 NLP) from 11 countries. Submissions were up from last
-year, although that was not until after we extended the deadline twice, so we wonder if there
-may be too many similar venues competing for submissions, such as the ACL SRW and the
-EACL SRW. Considering the uneven distribution of submissions to the three tracks, the topical
-organization of the workshop into tracks should also be re-thought.
-Of the 29 submission, we accepted 9 as oral presentations (one of which withdrew from the
-workshop) and another 9 as poster presentations. Accepted oral presentations and posters
-came form 9 different countries. Both oral presentation and poster presentation sessions were
-scheduled during the main conference; each paper accepted for oral presentation was also
-given a slot in the poster session. We made a special effort to schedule the sessions at times
-when many senior people in the field would be able to attend and offer their valuable wisdom
-from years. A total of 86 students and senior researchers agreed to serve on the program
-committee, which allowed us to assign 4 to 6 reviewers per paper.
-During the workshop, each oral presentation was followed by a brief panel discussion by two
-panelists per paper. Despite the extra effort of having to recruit panelists, we believe that the
-panels added considerable extra value to the workshop. Not only did it ensure good feedback to
-the presenters, it also helped the audience put the papers into perspective within the respective
-research fields. Each of the three oral presentation sessions drew an audience of 30 to 50
-people.
-All presenters received financial support from the U.S. National Science Foundation to assist
-them in their travel to Boulder for the conference. Altogether we received $21,000 from the
-National Science Foundation to fund the workshop, which included support for student
-participants, student co-chairs, and the cost of the student lunch. Oral presenters were offered
-$400 to defray the cost of registration and hotel as well as $500 to cover travel from within North
-America, or $1000 if they were traveling internationally. Poster presenters were offered $300.00
-total for reimbursement. We also budgeted a small amount of money for materials, such as
-poster boards for the poster presentations.
-At the student lunch on the day of the SRW, we had a group discussion to get feedback from
-the student community on how the SRW could be improved, and in general what could be done
-to offer mentoring to the students in our community. One issue that was raised is that it is not
-very clear what sets the SRW apart from the main conference or exactly what types of
-submissions are desired. Students felt that some of the feedback they received from reviewers
-wasn’t consistent with what the call for participation described as the target submissions. In
-response to this, we may need a more structured review form. One participant in the discussion
-pointed out that the form used for ACL short papers this year was a particularly good example of
-how to keep reviewers thinking along the right lines for review. Another issue that was raised is
-that students are not getting encouragement from their advisors to submit to the SRW. So we
-may need to go back to the faculty segment of the ACL community to find out why and what we
-can do about it.
-Students expressed a desire for more networking opportunities at ACL conferences, especially
-to help the shyer students come out of their shells. One idea was to organize topic specific
-round tables where selected faculty would attend, but which would be mainly students
-interested in similar topics. Other ideas included websites to help students find roommates for
-conferences and distributing contact information for all people who are registered by a particular
-date, along with their affiliations and research interests, to help students plan for who they want
-to try to set up meetings with, etc. during the conference.
-Student Research Workshop Faculty Chairs and Student Co-Chairs
-Anoop Sarkar (Faculty Chair, Simon Fraser University)
-Carolyn Rose (Faculty Chair, CMU)
-Svetlana Stenchikova (Student Co-Chair, Stony Brook University) - Speech
-Ulrich Germann (Student Co-Chair, University of Toronto) - NLP
-Chirag Shah (Student Co-Chair, University of North Carolina) - Information Retrieval
-Brief Reports from Other NAACL HLT Chairs
-Publicitiy Chairs
- Matthew Stone (Rutgers)
- Gokhan Tur (SRI)
- Diana Inkpen (U Ottawa)
-The Publicity Chairs were chosen from each of the three HLT areas -- IR, speech and
-NLP – in order to ensure good connections to the communities. The forwarded all
-announcements to mailing lists in their respective fields, including the main conference
-Call for Papers and Call for Short Papers, the Calls for Workshop and Tutorial Proposals,
-the Call for Demos, and the Doctoral Consortium Call for Papers. The lists and websites
-used included: corpora, elsenet, acl, asis-l, linguist, webir, ISCA ISCAPad, IEEE
-eNewsletter, AI Magazine, and the cognitive science society website.
-Future organizers should bear in mind that many organizations produce bimonthly or
-quarterly newsletters for conference announcements (e.g., AAAI's AI magazine, IEEE
-signal processing speech & language technical committee), which require CFPs to be
-distributed at least 2-3 months in advance of submission deadlines.
-Publications Chairs
- Christy Doran (MITRE)
- Eric Ringger (BYU)
-At the recommendation of the ACL-HLT 2008 chair, we continued the tradition from
-ACL 2008 of having two Publication Chairs, which seemed to work well. The Chairs
-followed the recipe written for publications chairs by Joakim Nivre and Noah Smith,
-located here:
-http://stp.lingfil.uu.se/~nivre/how-to-pub.html
-This includes the updated recipe for using ACLPUB to assemble the actual proceedings:
-http://faculty.cs.byu.edu/~ringger/naacl09/howto.html
-Several improvements are in the queue for both documents as well as the ACLPUB tools.
-Notes for improvement and discussion:
-o Better publicizing and enforcement of publications-related deadlines.
-o Mailing lists for relevant subsets of organizing committee, including: PC cochairs,
-local organizers, workshop chairs and sponsorship chairs (There are no
-sponsor logos in the proceedings, since the sponsorship chairs did not know about
-the deadline.)
-o Improved documentation of pre-requisites for hand-off to OmniPress (e.g. file
-formats), especially regarding book covers.
-o Per Jan Hajic, there is an opportunity to integrate some of ACLPUB into START
-o Numbers of needed printed volumes continue to drop. It may be a good time to
-consider going digital only.
-o Recommendations for shared documentation on aclweb.org?
-Tutorials Chairs
- Ciprian Chelba (Google)
- Paul Kantor (Rutgers)
- Brian Roark (OHSU)
-The tutorial chairs actively recruited submissions, received 12, and accepted 8. They
-erred on the side of accepting rather than rejecting because of “ties” in the reviews, and
-felt that it worked out quite well. Even though 8 tutorials is more than in most years
-(typically 6), we ended up with sufficient enrollment in all 8 accepted proposals. The
-complete list of tutorials is given below.
-. Data-Intensive Text Processing with MapReduce -- Jimmy Lin and Chris Dyer
-participants
-. Distributed Language Models -- Thorsten Brants and Peng Xu
-participants
-. Search Algorithms in NLP: Theory and Practice with Dynamic Programming --
-Liang Huang
-participants
-. Extracting world/linguistic knowledge from Wikipedia -- Simone Paolo Ponzetto
-and Michael Strube
-participants
-. OpenFst: An Open-Source, Weighted FST Library -- Martin Jansche/Cyril
-Allauzen/Michael Riley
-participants
-. OntoNotes: The 90% Solution -- Sameer Pradhan and Nianwen Xue
-participants
-. VerbNet overview, extensions, mappings and apps -- Karin Kipper Schuler, Anna
-Korhonen, Susan W. Brown
-participants
-. Writing Systems, Transliteration and Decipherment -- Richard Sproat and Kevin
-Knight
-participants
-Workshops Chairs
- Mark Hasegawa-Johnson (UIUC),
- Nizar Habash (Columbia)
-There were 41 workshop submissions jointly to ACL, EACL, and NAACL. ACL
-accepted 12, EACL accepted 10, and we accepted 11. Eleven plus the student workshop
-and CoNLL gave a total of 13 workshops, listed below. Number of participants listed is
-the final estimate from Priscilla Rasmussen as of May 26. For more information, see
-http://isle.uiuc.edu/hltnaacl2009/.
-. Semantic Evaluations: Recent Achievements and Future Directions
-Organizers: Eneko Agirre, LluÌs Marquez, Richard Wicentowski
-participants
-. BioNLP 2009
-Organizers: Sophia Ananiadou, K. Bretonnel Cohen, Dina Demner-Fushman,
-John Pestian, Jun'ichi Tsujii, Bonnie Webber
-participants
-. Third International Workshop on Cross Lingual Information Access: Addressing
-the Information Need of Multilingual Societies
-Organizers: Sivaji Bandyopadhyay, Pushpak Bhattacharya, Vasudeva Varma,
-Sudeshna Sarkar, A Kumaran
-participants
-. Workshop on Integer Linear Programming for Natural Language Processing
-Organizers: James Clarke, Sebastian Riedel
-participants
-. Software engineering, testing, and quality assurance for natural language
-processing
-Organizers: Kevin Bretonnel Cohen, Marc Light
-participants
-. Computational Approaches to Linguistic Creativity
-Organizers: Birte Loenneker-Rodman, Anna Feldman
-participants
-. Unsupervised and minimally supervised learning of lexical semantics
-Organizers: Suresh Manandhar, Ioannis Klapaftis
-participants
-. Semi-supervised Learning for NLP
-Organizers: Qin Wang, Kevin Duh, Dekang Lin
-participants
-. Active Learning for NLP
-Organizers: Eric Ringger, Robbie Haertel, Katrin Tomanek
-participants
-. Innovative Use of NLP for Building Educational Applications
-Organizers: Joel Tetreault, Jill Burstein, Claudia Leacock
-participants
-. Third Workshop on Syntax and Structure in Statistical Translation
-Organizers: Dekai Wu, David Chiang
-participants
-. Thirteenth Conference on Computational Natural Language Learning (CoNLL)
-Organizers: Suzanne Stevenson and Xavier Carreras
-participants
-The multi-conference proposal system seemed to work very well for all concerned: three
-workshops that would not have been offered (because they were rejected by their firstchoice
-conference) were, instead, offered at NAACL. All of the workshops had full
-schedules, as indicated in the online schedule.
-Demo Chairs
- Fred Popowich (Simon Fraser University)
- Michael Johnston (AT&T)
-Six demos were submitted, and five were accepted. The demo chairs did not actively
-recruit demos from specific research groups, and relied on the general publicity efforts.
-Future organizers might consider opening the demos up to allow people who have
-accepted papers to give demos in the demo session. From attending the demo session, the
-presenters seemed to be happy with how it went (and they were glad that they were in the
-high traffic area).

Difference between revisions of "2009Q3 Reports: NAACL 2009"

Revision as of 13:30, 11 July 2009

Navigation menu

Search