Program chair suggestions
=== START - something to do after submissions
After submissions, approach START for them to run a process to make the papers anonymous - this has to do with some embedded author information inside the PDF files.
=== area chairs
In choosing the area chairs, we tried to avoid asking people who had been area chairs for ACL in the last 2-3 years. We strove to balance the area chairs according to geographic region.
We asked the area chairs to send us acceptance-recommendation reports in which they put papers in bins; give us their recommendation for how many bins to accept; and include any relevant discussion. We communicated with them about these reports in making our final acceptance decisions.
One instruction you may want to give the area chairs: your report should not be surprising, given the reviews and the ratings. The discussion with/among the reviewers should have clarified issues before this point.
=== Conflicts of Interest
For the program chair COI: the other program co-chair completely handled the paper. The author program chair did not know where the paper was at all.
For the area chair COIs: In general, the strategy is to create new tracks and assign the COI papers to them. If there is a co-chair who is not conflicted, we made him or her the chair of that track. If there is no appropriate co-chair, the program co-chairs handled them in their own track.
START allows reviewers to be in more than one track, so the chairs of the special tracks simply asked reviewers and added them to their special tracks.
This process does make the blindness in one direction less strong - the fact that the reviewer is in a separate track might suggest to the reviewer that the author is one of the area chairs. We feel that blindness in that direction is not as important as authors not knowing who the reviewers are. After all, proposals and journal articles are only single blind.
===Best paper awards
It turns out there are some choices in how to handle the best paper awards.
Here is what we did for 2009:
After the acceptance decisions, we asked the area chairs to nominate papers for the best paper awards. We also looked at other papers that had high ratings. After discussion between the program chairs, we ending up nominating exactly the papers that were nominated by the area chairs.
Then, the program chairs formed a committee. Most are area chairs, but we needed to bring in a couple more people due to conflict of interests and unavailability of the area chairs.
The committee opted to remain anonymous. Though, the chairs of the committee were the program chairs, and we announced that we were the chairs.
Based on the ratings of the committee, the program chairs decided to award 3 best paper awards. We checked with Priscilla and she said it would be fine - each of the papers is awarded $1000.
Since the first authors of all the best papers are students, we did not have a separate student best paper competition - we just awarded three best student paper awards. The awards are supported by IBM. We checked with them and they were fine with us dividing the award into 3 (1000/3).
Since this process was after the camera ready copies are in (we wanted the committee to consider the camera ready copies), there was no time to schedule a plenary session (that is, the program/schedule was due before the decision was made). So, we simply announced the best paper awards at the final session of the conference. To us, we would have felt rushed if we had to complete the process before the program was due.
There is another model, which is to determine the best paper winners before the program is created, and have a special plenary session. This was the ACL 2008 model.
We don't argue for one or the other. We did what felt right to us as things were unfolding.
=== To emphasize to the reviewers
== All communication about papers must be through the start system. This includes confidential comments to program committee, and discussion among reviewers, area chairs, and program chairs on the paper discussion board. Why?
= Otherwise, the conflict of interest protections may be violated.
= Everything should be out in the open to the appropriate people. Why should one area chair and not the other or one reviewer and not the others discuss the papers privately.
== The final reviews must make it plain to the authors why their paper was rejected. We had to chase down several reviewers who agreed in the discussion board that the paper should be not accepted, but they did not change their comments/ratings. The reviewers have to update their final reviews to reflect the actual decision that was made.
== Poster versus paper. Our policy, continued from ACL08, was that posters and paper presentations are not distinguished by quality. As it happened, we had enough oral slots so that we didn't have to decide which long papers would be posters.
However, several people ignored this - we saw several comments that this paper is ok, maybe can be a poster, but isn't good enough for an oral presentation. So, if you want their feedback with respect this issue, you'll need to somehow highlight that information.
== Currently, communication to the reviewers is via email. It might be more workable to post reviewing guidelines/deadlines/etc. so the information is not spread among several emails.
===Short papers - something to add to the CFP
Consider this case: a paper accepted to EMNLP, where an accepted short paper is a subset of that EMNLP paper. Actually, the short paper was not strictly a subset. Even so, we made them withdraw the short paper. A test we used is this: considering that the EMNLP paper is published, is there a reason to publish the short paper? We decided the answer was no because most of the material in the short paper was also in the EMNLP paper; the new work was not as substantial as in the typical short paper.
Some policy about what is considered "unpublished" wrt short papers needs to be added to the CFP. Short papers can be work in progress, but it seems wrong to have another paper published, during the same time period, that completes the work, for example.
=== Abstract submissions
We requested authors to submit abstracts 1 week before. We suggest you require this. However, ask START not to run the mapping-papers-to -tracks process more than once, or the program chairs will be confused (You don't want the papers to jump automatically to different tracks - you'll want to manually handle that after the initial automatic mapping).
=== Keywords
The program chairs use the keywords for two purposes:
== communicating with the area chairs what types of papers the program chairs plan to send the track, the the area chairs can invite appropriate reviewers. For example, this year, we decided that the co-reference papers would go to discourse, not to IE (in past years, they went to IE).
== authors check off keywords - this is used to assign papers to track. However, many authors checked off any keyword they thought was relevant, which made the initial automatic assignment of papers to tracks noisier than it needed to be.
We suggest that you ask START to have an entry for the primary keyword (maybe two), and use that for the initial automatic assignment to tracks.
The person below contacted Priscilla who contacted us about Google sponsoring some kind of student or diversity award. However, it was too late for us to do anything. Perhaps you want to contact them for 2010.
From: Caryn Epstein <carynepstein@google.com>
=== ACL-ASIA researcher database
Just in case that you're not aware of it, there's a database of Asian researchers for ACL reference at http://www.bdc.com.tw/kysu/ACL_ASIA_Reference/index.htm . It's organized according to both research areas and regions. It's refered by both ACL 2008 and ACL-IJCNLP 2009. In ACL-IJCNLP 2009, we refered it for the area chair candidates from this region. Some area chairs who might not know Asian researchers were forwarded with the link so that they can find out suitable reviewers in this region as well. - Hide quoted text -
== Display Switches to help smoothing the switch over process between different laptops
This is not implemented this time. Things went on fine. Yet it may futher smooth the process if the Local Organizing Team can facilitate in the conference rooms.
=== Poster Session
This time the poster session is arranged during the lunch. But some presentors came to prepare their poster earlier in the morning. So you might contact Local Organizing Team to make the mounting items available earlier to these presentors. Besides, have the posters seperated in each row, so that it won't be too crowded for the poster in the middle.
You may also require the poster session chairs to walk around the posters to see any help needed.