2025Q3 Reports: Peer Review Committee

ACL Standing Committee on Peer Review (COPR) 2025q3 report

Committee members:

Roy Schwartz (co-chair)
Anna Rogers (co-chair)
Steven Bethard
Roi Reichart
Yuki Arase
Vivian Chen
Barbara Plank
Thamar Solorio
Noah A. Smith

Abstract

The ACL peer-review committee was established in mid 2024 to devise ideas on improving the peer-review process in *ACL conferences. In the year that has passed, we ran a survey on various ideas for the community. We compiled the survey answers and based on them have made several actionable recommendations, which we describe below. For more details on the recommendations below, see our February and May 2025 reports. The recommendations below were all accepted by the ACL exec and are now implemented in the ARR May 2025 cycle.

Moving from 6 ARR cycles per year to 5 cycles:

We recommend moving to 5 annual ARR cycles, each 10 weeks long. This would increase the average cycle length by 10 days. The extra days will be allocated by ARR so as to allow for more time in the current pain points, including chasing late reviewers, author-reviewer discussion, AC writing meta-reviews and SACs chasing late ACs. This solution also allows for starting each cycle, and each corresponding task on Monday to allow time to fulfill it during weekdays. Finally, it leaves two weeks of downtime (typically the final two weeks of the calendar year, during the December-January holiday) without any active ARR cycle, which can be used for improving ARR infrastructure.

Systematically provide incentives for great reviewers.

We propose incentivizing great reviewers using various mechanisms, e.g., great reviewer/AC/SAC recognition and awards, and a system for promoting reviewers.

Create a clear protocol for handling deeply problematic reviewers.

We propose adding a mechanism, process and criteria for reporting 'highly irresponsible' reviewers/chairs. This mechanism can be used to penalize 'highly irresponsible' reviewers/chairs.

We are also currently considering other mechanisms for improving the peer review mechanism:

Allow ARR to make Findings decisions

The motivation comes from a frustration within the community of the discrepancy between ARR and the actual conferences. In particular, authors are surprised to receive a high score in their ARR meta review, but end up getting their paper rejected altogether. To mitigate this, as well as provide decisions earlier, we are considering shifting part of the conference decisions to ARR: ARR will decide for each paper whether or not it is accepted to Findings. As a result, papers with high scores will be guaranteed at least a Findings acceptance. This proposal is currently still under consideration by the committee. In particular, we are currently analyzing ACL 2025 data to try to estimate the implications of such a proposal. We plan to decide on whether to recommend it or not in the coming months.