ACL Conference Awards Policy - Revision history

Yukiar: ACL conference award policy update on March 2025

2025-04-06T07:12:39Z

ACL conference award policy update on March 2025

Yukiar: Add non-publicized paper awards and updates on conference paper awards

2024-03-16T10:46:44Z

Add non-publicized paper awards and updates on conference paper awards

← Older revision		Revision as of 10:46, 16 March 2024
Line 7:		Line 7:
	We define "Best" as work that is particularly fascinating, controversial, surprising, impressive, and/or potentially field-changing.		We define "Best" as work that is particularly fascinating, controversial, surprising, impressive, and/or potentially field-changing.

	* "Best Paper Award": ~~Typically no~~ more than 6 papers receive ~~this~~ award. These papers are presented in a plenary session at the conference. PCs should consider scheduling this plenary session early in the conference, to encourage discussion of the papers during the conference.		* "Best Paper Award" & "Best Non-publicized Paper Award": No more than 0.25% of accepted papers, counting all accepted main conference and findings papers, receive the "Best Paper Award". For smaller conferences with less than 2,000 accepted papers, the maximum number of "Best Paper Award" slots should be kept at 6. Additional "Best Non-publicized Paper Award" winners are selected from the submissions that remained anonymous to the public during the whole process. See [[#Additional Award Slots for Unpublicized Work]] for more details, including the award slots. These papers are presented in a plenary session at the conference. PCs should consider scheduling this plenary session early in the conference, to encourage discussion of the papers during the conference.
	* "Outstanding Paper Award": 1.5–2.5% of accepted papers, selected as part of the same process. This allows for broader recognition of work that meets the criteria.		* "Outstanding Paper Award" & "Outstanding Non-publicized Paper Award": 1.5–2.5% of accepted papers (counting all accepted main conference and findings papers), selected as part of the same process for the "Outstanding Paper Award". This allows for broader recognition of work that meets the criteria. Additional "Outstanding Non-publicized Paper Award" winners are selected from the submissions that remained anonymous to the public during the whole process. See [[#Additional Award Slots for Unpublicized Work]] for more details.
	* "Area Chair's Award": Up to one paper per track. These are selected by the SACs for each track.		* "Area Chair's Award": Up to one paper per track. These are selected by the SACs for each track.
	* Short and long papers are considered together for these awards.		* Short and long papers are considered together for these awards.
Line 36:		Line 36:
	In January 2024, the ACL Executive Committee voted to adopt [[Media:ACL_Anonymity_Policy.pdf\|recommendations]] including the following:		In January 2024, the ACL Executive Committee voted to adopt [[Media:ACL_Anonymity_Policy.pdf\|recommendations]] including the following:

	~~<blockquote>~~Add new best and outstanding paper awards for submissions that remained anonymous to the public during the whole process, to ameliorate the ~~“publicity deficit”~~ for such submissions. (The precise eligibility criteria for these awards would be at the discretion of the awards committee. Of course, such submissions would remain eligible also for all other awards.)</~~blockquote>~~		* Add new best and outstanding paper awards for submissions that remained anonymous to the public during the whole process, to ameliorate the "publicity deficit" for such submissions. (The precise eligibility criteria for these awards would be at the discretion of the awards committee. Of course, such submissions would remain eligible also for all other awards.)
			* Non-publicized submissions are eligible for both "Best/Outstanding Paper Award" and "Best/Outstanding Non-publicized Paper Award". There will be one award per paper, and if a paper happens to win both publicized and non-publicized awards, it will remain with the general (non-publicized) award, and an additional non-publicized paper will be sought for the non-publicized award slot.
			* All papers are eligible for the regular awards, and only non-publicized papers are eligible for the non-publicized awards. The numbers of non-publicized award slots are calculated based on the number of accepted papers that remained anonymous (e.g., 1.5-2.5% of accepted papers that remained anonymous will receive Outstanding Non-publicized Paper awards). The method of identifying the total number of papers that remained anonymous will be determined by the PCs.

	== Selection Process ==		== Selection Process ==
Line 53:		Line 55:
	The best paper committee should consider any paper that was marked "Yes" or "Maybe" by any reviewer (Note: PCs may choose whether or not consider papers that were labeled 'maybe' by a reviewer and not nominated by the AE/AC.), AE, AC, or SAC (Note: This requirement is included to clarify how this policy interacts with ACL Rolling Review.).		The best paper committee should consider any paper that was marked "Yes" or "Maybe" by any reviewer (Note: PCs may choose whether or not consider papers that were labeled 'maybe' by a reviewer and not nominated by the AE/AC.), AE, AC, or SAC (Note: This requirement is included to clarify how this policy interacts with ACL Rolling Review.).

	SACs may also choose a paper in their area to receive the ~~“Area Chair’s Award”~~. This will encourage diversity in the papers that are highlighted. The Best Paper Committee will not be told which papers have received this award, to avoid biasing their choices.		SACs may also choose a paper in their area to receive the "Area Chair's Award". This will encourage diversity in the papers that are highlighted. SACs may not give this award to a paper that one of the SACs in the track has a conflict-of-interest (COI) on, but any other paper in their track can be chosen. Papers that SACs have COIs on can be nominated for other awards. If a conference has a COI track, that track does not select an Area Chair's award. The process for SACs to choose the award is up to the SACs (e.g., whether and how to involve ACs). The Best Paper Committee will not be told which papers have received this award, to avoid biasing their choices.

	Papers are not eligible for awards if their authors include Program Chairs. The PCs may also choose to specify that authors in other senior organisational roles are not eligible for awards.		Papers are not eligible for awards if their authors include Program Chairs. The PCs may also choose to specify that authors in other senior organisational roles are not eligible for awards.

	Special category awards can either be handled in the same way, by editing the question above to include them, or through a separate process defined by the PCs.		Special category awards can either be handled in the same way, by editing the question above to include them, or through a separate process defined by the PCs.

			The eligibility for non-publicized paper awards should be confirmed before notification, i.e., in-between the final acceptance decision by PCs and notification. The best paper committee chairs or program chairs search for paper titles and authors of the candidates at preprint servers, popular social media, or on the web in general.

	=== Selection ===		=== Selection ===
Line 63:		Line 67:
	(Note, this process was developed based on the assumption that the nomination process leads to 80-100 papers being considered for awards. If the number of nominated papers turns out to be much higher or lower then the selection process should be adapted as necessary.)		(Note, this process was developed based on the assumption that the nomination process leads to 80-100 papers being considered for awards. If the number of nominated papers turns out to be much higher or lower then the selection process should be adapted as necessary.)

	A best paper committee will be selected by the program chairs. The committee size should be large enough to keep the load to around 10-15 papers per member. The committee should be diverse in composition in terms of research areas and demographics.		Program chairs should select and appoint a Best Paper Committee Chair(s) early in the process (it is recommended well before the submission deadline), who will manage the entire process. A best paper committee will be selected by the program chairs together with the Best Paper Committee Chair(s). The committee size should be large enough to keep the load to around 10-15 papers per member. The committee should be diverse in composition in terms of research areas and demographics. It is best to form the committee before the review process commences, otherwise it may be hard to recruit members.

	The committee will follow a multistage process to determine the final awards. The committee will receive the ~~final~~ camera-ready version of the paper ~~(or its anonymized version if possible)~~, anonymized reviews, and associated supplementary materials.		The committee will follow a multistage process to determine the final awards. First, the program chairs and committee chair(s) collect anonymized final camera-ready versions of the candidates. If the papers are not anonymized then the committee chair is encouraged to manually redact author information on the first page of the paper. The committee will receive the (anonymized) camera-ready version of the paper, anonymized reviews, and associated supplementary materials. If the papers are not anonymized then the committee chair is encouraged to redact author information on the first page of the paper. Also, the chair should encourage the committee to not look for revealing author information elsewhere (e.g. acceptance list on the conference page).
	# Papers are divided between the committee members based on research areas for a first pass in which each paper is read by at least 2 committee members. They independently place the papers into three groups: (1) consider for best paper, (2) consider for outstanding paper, (3) do not consider further.		# Papers are divided between the committee members based on research areas for a first pass in which each paper is read by at least 2 committee members. They independently place the papers into three groups: (1) consider for best paper, (2) consider for outstanding paper, (3) do not consider further.
	# All committee members read the papers that ~~were identified as~~ under consideration for 'best paper'.		# All committee members read the papers that both readers placed under consideration for 'best paper'.
	# The committee meets to make the final selections.		# The committee meets to make the final selections.

Line 79:		Line 83:
	* Do the awards include papers that show excellence in potential positive social impact?		* Do the awards include papers that show excellence in potential positive social impact?

	Special awards may be selected by the best paper committee or by separate committees.		Special awards may be selected by the best paper committee or by separate committees. All award decisions may be overruled by the PCs.

	== Announcement, Certificates, and Financial Awards ==		== Announcement, Certificates, and Financial Awards ==

	All awardees will be announced on the conference website or blog before the conference. All awardees will receive a certificate with the name of the conference, name of the award, title of the paper, and author names, signed by the Program Chair(s) and/or General Chair(s). Conferences may choose to have a financial component for any of the awards, which may be sponsored, and may be split between the authors as the authors choose.		Announcement: All awardees will be announced on the conference website or blog before the conference. Titles and authors do not need to be announced during a plenary session.

			Certificates: All awardees will receive a certificate with the name of the conference, name of the award, title of the paper, and author names, signed by the Program Chair(s) and/or General Chair(s).

			Financial Awards: Conferences may choose to have a financial component for any of the awards, which may be sponsored, and may be split between the authors as the authors choose.

	== Documentation ==		== Documentation ==

Iryna Gurevych: /* Additional Award Slots for Unpublicized Work */ full name of Exec

2024-01-12T13:50:02Z

Additional Award Slots for Unpublicized Work: full name of Exec

← Older revision		Revision as of 13:50, 12 January 2024
Line 34:		Line 34:
	=== Additional Award Slots for Unpublicized Work ===		=== Additional Award Slots for Unpublicized Work ===

	In January 2024, the ACL Executive voted to adopt [[Media:ACL_Anonymity_Policy.pdf\|recommendations]] including the following:		In January 2024, the ACL Executive Committee voted to adopt [[Media:ACL_Anonymity_Policy.pdf\|recommendations]] including the following:

	<blockquote>Add new best and outstanding paper awards for submissions that remained anonymous to the public during the whole process, to ameliorate the “publicity deficit” for such submissions. (The precise eligibility criteria for these awards would be at the discretion of the awards committee. Of course, such submissions would remain eligible also for all other awards.)</blockquote>		<blockquote>Add new best and outstanding paper awards for submissions that remained anonymous to the public during the whole process, to ameliorate the “publicity deficit” for such submissions. (The precise eligibility criteria for these awards would be at the discretion of the awards committee. Of course, such submissions would remain eligible also for all other awards.)</blockquote>

Iryna Gurevych: /* Award Types and Criteria */ added subhead

2024-01-12T13:39:32Z

Award Types and Criteria: added subhead

← Older revision		Revision as of 13:39, 12 January 2024
Line 31:		Line 31:
	* Interdisciplinary Research Award - For papers that contribute to NLP and another field in new and interesting ways.		* Interdisciplinary Research Award - For papers that contribute to NLP and another field in new and interesting ways.
	* Reproduction Award - For papers that reproduce prior work in a particularly enlightening way, revealing additional features of the prior work.		* Reproduction Award - For papers that reproduce prior work in a particularly enlightening way, revealing additional features of the prior work.

			=== Additional Award Slots for Unpublicized Work ===

	In January 2024, the ACL Executive voted to adopt [[Media:ACL_Anonymity_Policy.pdf\|recommendations]] including the following:		In January 2024, the ACL Executive voted to adopt [[Media:ACL_Anonymity_Policy.pdf\|recommendations]] including the following:

Iryna Gurevych: /* Special Categories */ deleted stray characters

2024-01-12T13:38:30Z

Special Categories: deleted stray characters

← Older revision		Revision as of 13:38, 12 January 2024
Line 30:		Line 30:
	* Low-Resource Paper Award - For papers that contribute to work on languages or domains with limited data.		* Low-Resource Paper Award - For papers that contribute to work on languages or domains with limited data.
	* Interdisciplinary Research Award - For papers that contribute to NLP and another field in new and interesting ways.		* Interdisciplinary Research Award - For papers that contribute to NLP and another field in new and interesting ways.
	* Reproduction Award -~~{{Quote\|text=~~ For papers that reproduce prior work in a particularly enlightening way, revealing additional features of the prior work.		* Reproduction Award - For papers that reproduce prior work in a particularly enlightening way, revealing additional features of the prior work.

	In January 2024, the ACL Executive voted to adopt [[Media:ACL_Anonymity_Policy.pdf\|recommendations]] including the following:		In January 2024, the ACL Executive voted to adopt [[Media:ACL_Anonymity_Policy.pdf\|recommendations]] including the following:

Iryna Gurevych: /* Award Types and Criteria */ deleted stray characters

2024-01-12T13:38:03Z

Award Types and Criteria: deleted stray characters

← Older revision		Revision as of 13:38, 12 January 2024
Line 25:		Line 25:
	of additional special categories include:		of additional special categories include:
	* Software Award - For papers that describe and share (ideally via source code) a fascinating, valuable, or potentially field-changing new piece of software. Note that this is not the same as the Demonstration Paper Award since there could be papers outside of that track that meet this criterion.		* Software Award - For papers that describe and share (ideally via source code) a fascinating, valuable, or potentially field-changing new piece of software. Note that this is not the same as the Demonstration Paper Award since there could be papers outside of that track that meet this criterion.
	* Demonstration Paper Award - For papers in ~~th{{Quote\|text=e~~ Demonstration Track.		* Demonstration Paper Award - For papers in the Demonstration Track.
	* Theme Paper Award - For papers on the conference’s theme.		* Theme Paper Award - For papers on the conference’s theme.
	* Linguistic Insight Award - For papers that make a particularly significant contribution to our understanding of language.		* Linguistic Insight Award - For papers that make a particularly significant contribution to our understanding of language.

Iryna Gurevych: /* Award Types and Criteria */ new publicity-deficit awards

2024-01-12T13:36:45Z

Award Types and Criteria: new publicity-deficit awards

← Older revision		Revision as of 13:36, 12 January 2024
Line 25:		Line 25:
	of additional special categories include:		of additional special categories include:
	* Software Award - For papers that describe and share (ideally via source code) a fascinating, valuable, or potentially field-changing new piece of software. Note that this is not the same as the Demonstration Paper Award since there could be papers outside of that track that meet this criterion.		* Software Award - For papers that describe and share (ideally via source code) a fascinating, valuable, or potentially field-changing new piece of software. Note that this is not the same as the Demonstration Paper Award since there could be papers outside of that track that meet this criterion.
	* Demonstration Paper Award - For papers in ~~the~~ Demonstration Track.		* Demonstration Paper Award - For papers in th{{Quote\|text=e Demonstration Track.
	* Theme Paper Award - For papers on the conference’s theme.		* Theme Paper Award - For papers on the conference’s theme.
	* Linguistic Insight Award - For papers that make a particularly significant contribution to our understanding of language.		* Linguistic Insight Award - For papers that make a particularly significant contribution to our understanding of language.
	* Low-Resource Paper Award - For papers that contribute to work on languages or domains with limited data.		* Low-Resource Paper Award - For papers that contribute to work on languages or domains with limited data.
	* Interdisciplinary Research Award - For papers that contribute to NLP and another field in new and interesting ways.		* Interdisciplinary Research Award - For papers that contribute to NLP and another field in new and interesting ways.
	* Reproduction Award - For papers that reproduce prior work in a particularly enlightening way, revealing additional features of the prior work.		* Reproduction Award -{{Quote\|text= For papers that reproduce prior work in a particularly enlightening way, revealing additional features of the prior work.

			In January 2024, the ACL Executive voted to adopt [[Media:ACL_Anonymity_Policy.pdf\|recommendations]] including the following:

			<blockquote>Add new best and outstanding paper awards for submissions that remained anonymous to the public during the whole process, to ameliorate the “publicity deficit” for such submissions. (The precise eligibility criteria for these awards would be at the discretion of the awards committee. Of course, such submissions would remain eligible also for all other awards.)</blockquote>

	== Selection Process ==		== Selection Process ==

YusukeMiyao at 13:23, 20 July 2022

2022-07-20T13:23:47Z

← Older revision		Revision as of 13:23, 20 July 2022
Line 3:		Line 3:
	== Award Types and Criteria ==		== Award Types and Criteria ==

	~~'''~~Best Paper Award~~'''~~		=== Best Paper Award ===

	We define "Best" as work that is particularly fascinating, controversial, surprising, impressive, and/or potentially field-changing.		We define "Best" as work that is particularly fascinating, controversial, surprising, impressive, and/or potentially field-changing.
Line 12:		Line 12:
	* Short and long papers are considered together for these awards.		* Short and long papers are considered together for these awards.

	~~'''~~Test of Time Award~~'''~~		=== Test of Time Award ===

	See the policy at [[Test-of-Time Papers Award]].		See the policy at [[Test-of-Time Papers Award]].

	~~'''~~Special Categories~~'''~~		=== Special Categories ===

	Two special awards will recognise work with a particular focus as described below. Papers that receive an award in a special category are also eligible to receive a general award.		Two special awards will recognise work with a particular focus as described below. Papers that receive an award in a special category are also eligible to receive a general award.
Line 34:		Line 34:
	== Selection Process ==		== Selection Process ==

	~~'''~~Nominations~~'''~~		=== Nominations ===

	In the review process, reviewers (Note: PCs may choose whether or not to include reviewers in the nomination process.), AEs, ACs, and SACs will be asked to answer the following questions:		In the review process, reviewers (Note: PCs may choose whether or not to include reviewers in the nomination process.), AEs, ACs, and SACs will be asked to answer the following questions:
Line 53:		Line 53:
	Special category awards can either be handled in the same way, by editing the question above to include them, or through a separate process defined by the PCs.		Special category awards can either be handled in the same way, by editing the question above to include them, or through a separate process defined by the PCs.

	~~'''~~Selection~~'''~~		=== Selection ===

	(Note, this process was developed based on the assumption that the nomination process leads to 80-100 papers being considered for awards. If the number of nominated papers turns out to be much higher or lower then the selection process should be adapted as necessary.)		(Note, this process was developed based on the assumption that the nomination process leads to 80-100 papers being considered for awards. If the number of nominated papers turns out to be much higher or lower then the selection process should be adapted as necessary.)
Line 97:		Line 97:
	Edits were made by Jonathan Kummerfeld based on discussion with the ACL Executive. Feedback was solicited from the community and informed further improvements.		Edits were made by Jonathan Kummerfeld based on discussion with the ACL Executive. Feedback was solicited from the community and informed further improvements.

	~~'''~~Anonymity~~'''~~		=== Anonymity ===

	We discussed requiring authors to be anonymous to avoid potential a source of bias, but realised it was (1) not easy to do with camera-ready versions of papers, and (2) would be difficult to maintain while also handling conflict-of-interests on the best paper committee (COIs will be more likely than in the past because more papers are being considered). Given those concerns, we have made it optional.		We discussed requiring authors to be anonymous to avoid potential a source of bias, but realised it was (1) not easy to do with camera-ready versions of papers, and (2) would be difficult to maintain while also handling conflict-of-interests on the best paper committee (COIs will be more likely than in the past because more papers are being considered). Given those concerns, we have made it optional.

	~~'''~~Choice of terms~~'''~~		=== Choice of terms ===

	The committee had mixed opinions on the term 'best'. On one hand, it is common/standard and expected, on the other hand, it implies a metric with a clear ranking of papers, which we do not have. After discussing a range of alternatives we returned to 'best' because it is such a well established term. Several options for the second category were discussed, including 'noteworthy', 'honourable mention', and 'outstanding'. The final proposal uses 'outstanding' as it		The committee had mixed opinions on the term 'best'. On one hand, it is common/standard and expected, on the other hand, it implies a metric with a clear ranking of papers, which we do not have. After discussing a range of alternatives we returned to 'best' because it is such a well established term. Several options for the second category were discussed, including 'noteworthy', 'honourable mention', and 'outstanding'. The final proposal uses 'outstanding' as it
	is a recognisable term.		is a recognisable term.

	~~'''~~Choice of numbers~~'''~~		=== Choice of numbers ===

	* "Typically no more than 6" best papers. This value was chosen based on what we thought could be included in a single plenary session. However, the language is intentionally flexible.		* "Typically no more than 6" best papers. This value was chosen based on what we thought could be included in a single plenary session. However, the language is intentionally flexible.
	* "1.5–2.5% of accepted papers" receive the outstanding paper award. This range was chosen to balance prestige (which favors a lower number) and the goal to recognize more work (which favors a higher number).		* "1.5–2.5% of accepted papers" receive the outstanding paper award. This range was chosen to balance prestige (which favors a lower number) and the goal to recognize more work (which favors a higher number).
Line 112:		Line 114:
	[[File: papers receiving awards at ACL.png\|800px]]		[[File: papers receiving awards at ACL.png\|800px]]

	~~'''~~Several ideas came up that are worth considering in the future:~~'''~~		=== Several ideas came up that are worth considering in the future: ===

	* Dissertation Awards. These exist in other communities (e.g. AAAI) and are intended to recognise a body of work. We did not include them as they are outside of the conference awards process. They would need a separate process that could, for example, be conducted by each organisation / society separately. We also discussed awards for thesis proposals, which would be suitable for the Student Research Workshop.		* Dissertation Awards. These exist in other communities (e.g. AAAI) and are intended to recognise a body of work. We did not include them as they are outside of the conference awards process. They would need a separate process that could, for example, be conducted by each organisation / society separately. We also discussed awards for thesis proposals, which would be suitable for the Student Research Workshop.

ShiqiZhao at 01:10, 16 July 2022

2022-07-16T01:10:44Z

← Older revision		Revision as of 01:10, 16 July 2022
Line 111:		Line 111:

	[[File: papers receiving awards at ACL.png\|800px]]		[[File: papers receiving awards at ACL.png\|800px]]

			'''Several ideas came up that are worth considering in the future:'''

			* Dissertation Awards. These exist in other communities (e.g. AAAI) and are intended to recognise a body of work. We did not include them as they are outside of the conference awards process. They would need a separate process that could, for example, be conducted by each organisation / society separately. We also discussed awards for thesis proposals, which would be suitable for the Student Research Workshop.
			* Journal Paper Awards. Other communities have these (e.g. ISCA for papers in CSL). We discussed including them here by saying CL and TACL papers presented at a conference are eligible for awards, but then the awards committee needs to compare quite different papers (e.g. a 30+ page journal article and a 4 page short conference paper). The approach used elsewhere is that each journal chooses a 'best paper', but that is then outside the scope of this proposal regarding conferences.
			* Requirements for papers to be well-presented. We discussed the idea that papers that receive awards should be examples of well-presented ideas that students can look to for understanding how to write a good paper. On the other hand, it would seem unfair to not reward an innovative idea just because the author is inexperienced and did not present it perfectly. We tried to come up with language to balance these factors, but decided there was no simple solution, so this requirement was dropped.
			* Advice/guidance for reviewers on when to select a paper for an award. This is a good idea that is outside the scope of this policy.

ShiqiZhao at 01:08, 16 July 2022

2022-07-16T01:08:15Z

← Older revision		Revision as of 01:08, 16 July 2022
Line 110:		Line 110:
	* For context, the graph below shows historical trends in awards at ACL going back to the very first year a best paper award was announced (note, these values include all award types together in the count for each year):		* For context, the graph below shows historical trends in awards at ACL going back to the very first year a best paper award was announced (note, these values include all award types together in the count for each year):

	[[File: papers receiving awards at ACL.png]]		[[File: papers receiving awards at ACL.png\|800px]]