ACL Policies for Review and Citation

The following policies are adopted for *ACL conferences (ACL, NAACL, EACL) and the TACL journal in the interest of maintaining and protecting double-blind review without sacrificing the positive effects of preprint publishing. The policies are in effect for all *ACL conferences starting with NAACL 2018 and for TACL from the submission deadline of January 1, 2018. We recommend conferences and workshops that are co-located with *ACL conferences and use double-blind review to adopt the same policies unless this conflicts with other policies they have.

• For further motivation and discussion, we refer to the report of the working group appointed by the ACL Executive Committee: File:ACL Guidelines for Submission, Review and Citation.pdf.
• For guidelines to be used by conferences and journals adopting these policies, see:
  • ACL Author Guidelines
  • ACL Reviewer Guidelines

Submission

*ACL conferences and TACL require that submissions be anonymized. A submission will not be considered anonymized if the authors post (or update) a non-anonymized preprint version within an anonymity period lasting from 1 month before the submission deadline until the time of notification (or withdrawal). Submissions will be rejected if not properly anonymized.

• Anonymized preprints within the anonymity period are allowed. This is currently only possible on certain platforms but ACL may consider using such a platform for all submissions in the future.
• Non-anonymized preprints before the anonymity period are allowed, although we encourage authors to wait to post them until after the anonymity period.
• If a non-anonymized preprint version exists, authors must declare its existence at submission time but should not cite it and are asked not to publicize it further during the anonymity period – the submitted paper should be as anonymous as possible.

The notion of preprint is understood broadly to refer to any non-refereed paper posted online, including but not limited to preprint servers such as arXiv. Note that the rule applies only to preprints that authors post themselves, so it does not apply to (say) non-refereed proceedings volumes. The restriction on updating is to prevent authors from circumventing these rules by "flag planting" with a placeholder version over 1 month in advance.

Review

To preserve the merits of double-blind review, reviewers for *ACL conferences and TACL should be instructed not to actively try to discover the identity of the authors and not to let any suspicion or knowledge of author identity affect their judgment of the paper. This should not prevent them from searching for related work, but they should take special care and postpone such searches until after they have read the paper completely and formed an initial impression of it. If a reviewer nevertheless uncovers the identity of the authors (or believes they can identify the authors for other reasons), they should inform the area chair or action editor (but not other reviewers) so that this information can be taken into account when making the final acceptance decision. To be able to track how often this happens, we also recommend adding a question to the review form about whether reviewers think they can identify the authors and who they think the authors are.

Citation

Papers submitted to *ACL conferences and TACL should in principle cite and compare to all relevant prior work, regardless of when and how that work was presented to the community, and must credit work that influenced them. To this general rule there are two important qualifications:

• For citation, refereed publications take priority over preprints. Specifically:
  • Authors are expected to cite all refereed publications relevant to their submission, but may be excused for not knowing about all unpublished work (especially work that has been recently posted and/or is not widely cited).
  • If a preprint has been superseded by a refereed publication, the refereed publication should be cited in addition to or instead of the preprint version.
• For comparison, papers (whether refereed or not) appearing less than 3 months before the submission deadline should be considered contemporaneous to the submission. This relieves authors from the obligation to make detailed comparisons that require additional experimentation and/or in-depth analysis, but they are still expected to cite and discuss contemporaneous work to the degree feasible.

It follows from our recommendations about citation and comparison that failure to cite relevant prior work or failure to compare to (non-contemporaneous) empirical results may affect the assessment of a submission regardless of how the prior work was published. However, reviewers should be instructed to give authors the benefit of the doubt in cases where the work appears in preprints with no corresponding refereed publication, especially preprints that are recent and/or not widely cited. In such cases, reviewers should point authors to the non-cited work (so that they can discuss it in the camera-ready version) but not penalize the authors for missing the citation.