ACL Conference Reviewer Awards Policy - Revision history

Y.feng: /* Notes on the development of this proposal */

2025-05-02T12:52:17Z

Notes on the development of this proposal

← Older revision		Revision as of 12:52, 2 May 2025
Line 66:		Line 66:
	1.5-2.5% target awards were chosen to match the ACL awards policy. https://www.aclweb.org/adminwiki/index.php/ACL_Conference_Awards_Policy		1.5-2.5% target awards were chosen to match the ACL awards policy. https://www.aclweb.org/adminwiki/index.php/ACL_Conference_Awards_Policy

	For conferences colocated with a non-ACL event there may need to be some adjustments depending on		For conferences colocated with a non-ACL event there may need to be some adjustments depending on the policies of the other event. This process was not developed for workshops or journals.

Y.feng: /* Award Criteria */

2025-05-02T12:50:44Z

Award Criteria

← Older revision		Revision as of 12:50, 2 May 2025
Line 18:		Line 18:
	* open-minded reviewers, who significantly changed their mind and assessment in the light of evidence brought by the authors or other reviewers;		* open-minded reviewers, who significantly changed their mind and assessment in the light of evidence brought by the authors or other reviewers;
	* patient reviewers, who were able to provide unusually kind and helpful feedback even to low-quality submissions.		* patient reviewers, who were able to provide unusually kind and helpful feedback even to low-quality submissions.


	''' Great Area Chairs. ''' All area chairs are expected to follow their guidelines, provide clear recommendations, and perform all their tasks on time. Above that, the senior area chairs may nominate the 'great' area chairs for a number of criteria, including but not limited to the following:		''' Great Area Chairs. ''' All area chairs are expected to follow their guidelines, provide clear recommendations, and perform all their tasks on time. Above that, the senior area chairs may nominate the 'great' area chairs for a number of criteria, including but not limited to the following:
Line 26:		Line 25:
	* above-average proactive engagement with the authors of low-quality reviews, helping to improve the reviews and train the reviewers for the future review cycles;		* above-average proactive engagement with the authors of low-quality reviews, helping to improve the reviews and train the reviewers for the future review cycles;
	* managing an unusually high load of unresponsive reviewers and finding emergency substitutes on time.		* managing an unusually high load of unresponsive reviewers and finding emergency substitutes on time.


	''' Great Senior Area Chairs. ''' The program chairs may choose to nominate the senior area chairs who were the most on-time with their duties, dealt with an unusual number of unresponsive ACs, spotted serious publication ethics issues, helped to catch early any ambiguities in the instructions or potential issues in the overall process, suggested useful improvements, etc.		''' Great Senior Area Chairs. ''' The program chairs may choose to nominate the senior area chairs who were the most on-time with their duties, dealt with an unusual number of unresponsive ACs, spotted serious publication ethics issues, helped to catch early any ambiguities in the instructions or potential issues in the overall process, suggested useful improvements, etc.

Y.feng: /* Selection process */

2025-05-02T12:50:23Z

Selection process

← Older revision		Revision as of 12:50, 2 May 2025
Line 45:		Line 45:

	Ideally, all reviewers/chairs who perform great service should be at least recognized as great reviewers/chairs, and hence there is no cap on the number of such nominations.		Ideally, all reviewers/chairs who perform great service should be at least recognized as great reviewers/chairs, and hence there is no cap on the number of such nominations.


	Per agreement with the ACL exec, 1.5-2.5% of total number of chairs/reviewers can be selected for receiving a monetary award. If there are more nominations for reviewers or area chairs than 2.5% of their total number in a given review cycle, a subset of nominees for receiving monetary awards will be randomly selected. The more nominations a given reviewer/chair received in a given cycle, the higher their chances for receiving the award.		Per agreement with the ACL exec, 1.5-2.5% of total number of chairs/reviewers can be selected for receiving a monetary award. If there are more nominations for reviewers or area chairs than 2.5% of their total number in a given review cycle, a subset of nominees for receiving monetary awards will be randomly selected. The more nominations a given reviewer/chair received in a given cycle, the higher their chances for receiving the award.

Y.feng: /* Nominations */

2025-05-02T12:50:12Z

Nominations

← Older revision		Revision as of 12:50, 2 May 2025
Line 35:		Line 35:

	The reviewers can be nominated by the area chairs by marking the great reviews in the peer review system, after the reviews have been finalized. The instructions for meta-review should include the instructions for entering reviewer nominations. The interface for collecting nominations should include a field for a short justification for the nomination.		The reviewers can be nominated by the area chairs by marking the great reviews in the peer review system, after the reviews have been finalized. The instructions for meta-review should include the instructions for entering reviewer nominations. The interface for collecting nominations should include a field for a short justification for the nomination.


	The area chairs can be nominated by the senior area chairs at the stage of entering the final recommendations. Likewise, the interface for entering the nominations should include a short justification or selection from pre-defined categories. The instructions for final recommendations should include the instructions for entering area chair nominations.		The area chairs can be nominated by the senior area chairs at the stage of entering the final recommendations. Likewise, the interface for entering the nominations should include a short justification or selection from pre-defined categories. The instructions for final recommendations should include the instructions for entering area chair nominations.


	The program chairs can nominate the senior area chairs of one track directly.		The program chairs can nominate the senior area chairs of one track directly.


	The award justification text should be short (1-3 sentences) and not include any information that could directly identify the papers or the reviewers.		The award justification text should be short (1-3 sentences) and not include any information that could directly identify the papers or the reviewers.

Y.feng at 12:48, 2 May 2025

2025-05-02T12:48:02Z

← Older revision		Revision as of 12:48, 2 May 2025
Line 71:		Line 71:

	1.5-2.5% target awards were chosen to match the ACL awards policy. https://www.aclweb.org/adminwiki/index.php/ACL_Conference_Awards_Policy		1.5-2.5% target awards were chosen to match the ACL awards policy. https://www.aclweb.org/adminwiki/index.php/ACL_Conference_Awards_Policy

	For conferences colocated with a non-ACL event there may need to be some adjustments depending on		For conferences colocated with a non-ACL event there may need to be some adjustments depending on

Y.feng at 12:47, 2 May 2025

2025-05-02T12:47:36Z

← Older revision		Revision as of 12:47, 2 May 2025
Line 61:		Line 61:

	The awardees selected for the monetary awards will receive a free attendee registration to a virtual ACL event, or a corresponding discount to the on-site attendance. These awards can be used within a year of the award, at an event where the awardee is not a paper presenter. The rationale for this choice is that many institutions do not sponsor the attendance of the events for their students and employees, if they do not have a paper to present at that event. At the same time, when they do have a paper to present, awarding a registration fee waiver actually benefits the institution rather than the awardee. With these flexible-date attended fee waivers we hope that the awardees would be able to attend more events than they would otherwise. It will also be possible to exchange the virtual registration for a discount to the on-site attendance of an ACL event (also to be used within a year of the award).		The awardees selected for the monetary awards will receive a free attendee registration to a virtual ACL event, or a corresponding discount to the on-site attendance. These awards can be used within a year of the award, at an event where the awardee is not a paper presenter. The rationale for this choice is that many institutions do not sponsor the attendance of the events for their students and employees, if they do not have a paper to present at that event. At the same time, when they do have a paper to present, awarding a registration fee waiver actually benefits the institution rather than the awardee. With these flexible-date attended fee waivers we hope that the awardees would be able to attend more events than they would otherwise. It will also be possible to exchange the virtual registration for a discount to the on-site attendance of an ACL event (also to be used within a year of the award).

			== Logistics of the flexi-date fee waiver awards ==

			The ARR editors-in-chief will be responsible for maintaining and updating a running record of awardees, with the following information: Award Type, First Name, Last Name, Affiliation, Country, Email, Backup Email, Date of Award, Date of Award Expiration. The possible values for "Award Type" are: Great Reviewer, Great Area Chair, Great Senior Area Chair.

			When there is an upcoming conference, the ACL director of events will communicate to the ARR EiCs and program chairs the registration code that the awardees can use for that event. The chairs will communicate that code and registration instructions to the awardees who have an unused fee waiver, awarded within the year prior to the event. After the event, the list of used awards will be updated based on the list of registrations. With each subsequent review cycle, the list will be updated with new awards.

			== Notes on the development of this proposal ==

			1.5-2.5% target awards were chosen to match the ACL awards policy. https://www.aclweb.org/adminwiki/index.php/ACL_Conference_Awards_Policy
			For conferences colocated with a non-ACL event there may need to be some adjustments depending on

Y.feng: /* Announcement, Certificates, and Awards */

2025-05-02T10:34:05Z

Announcement, Certificates, and Awards

← Older revision		Revision as of 10:34, 2 May 2025
Line 55:		Line 55:

	The reviewer and area chair awards will be announced on the ARR website for each cycle. For cycles associated with conferences, the conference website will also copy the list of nominees to the conference website or link to the ARR list. The ARR editors responsible for the given review cycle will communicate the award(s) to the reviewers/area chairs, ideally before the closest conference (in case they would like to use the prize to attend that same conference).		The reviewer and area chair awards will be announced on the ARR website for each cycle. For cycles associated with conferences, the conference website will also copy the list of nominees to the conference website or link to the ARR list. The ARR editors responsible for the given review cycle will communicate the award(s) to the reviewers/area chairs, ideally before the closest conference (in case they would like to use the prize to attend that same conference).


	All awardees will receive an electronic certificate with the name of the ARR cycle, name of the award, and their name, signed by the ARR EiCs and/or conference Program Chair(s). These certificates will be generated and distributed by ARR in each cycle. Pending the development of reviewer history tools, all nominated reviewers/chairs should also have the record of their awards.		All awardees will receive an electronic certificate with the name of the ARR cycle, name of the award, and their name, signed by the ARR EiCs and/or conference Program Chair(s). These certificates will be generated and distributed by ARR in each cycle. Pending the development of reviewer history tools, all nominated reviewers/chairs should also have the record of their awards.


	For the cycles directly associated with a conference, the program chairs will work with ACL director of events to coordinate the availability of 'great reviewer/chair' stickers for conference badges and posters, and the logistics of their distribution, based on a list of awardees provided by ARR. Similarly to the CVPR initiative, the papers of 'great reviewers/chairs' (if any) can display a special 'great reviewer/chair' sticker in the poster sessions and the programs. A comparable design should be developed for virtual participation.		For the cycles directly associated with a conference, the program chairs will work with ACL director of events to coordinate the availability of 'great reviewer/chair' stickers for conference badges and posters, and the logistics of their distribution, based on a list of awardees provided by ARR. Similarly to the CVPR initiative, the papers of 'great reviewers/chairs' (if any) can display a special 'great reviewer/chair' sticker in the poster sessions and the programs. A comparable design should be developed for virtual participation.

	The awardees selected for the monetary awards will receive a free attendee registration to a virtual ACL event, or a corresponding discount to the on-site attendance. These awards can be used within a year of the award, at an event where the awardee is not a paper presenter. The rationale for this choice is that many institutions do not sponsor the attendance of the events for their students and employees, if they do not have a paper to present at that event. At the same time, when they do have a paper to present, awarding a registration fee waiver actually benefits the institution rather than the awardee. With these flexible-date attended fee waivers we hope that the awardees would be able to attend more events than they would otherwise. It will also be possible to exchange the virtual registration for a discount to the on-site attendance of an ACL event (also to be used within a year of the award).		The awardees selected for the monetary awards will receive a free attendee registration to a virtual ACL event, or a corresponding discount to the on-site attendance. These awards can be used within a year of the award, at an event where the awardee is not a paper presenter. The rationale for this choice is that many institutions do not sponsor the attendance of the events for their students and employees, if they do not have a paper to present at that event. At the same time, when they do have a paper to present, awarding a registration fee waiver actually benefits the institution rather than the awardee. With these flexible-date attended fee waivers we hope that the awardees would be able to attend more events than they would otherwise. It will also be possible to exchange the virtual registration for a discount to the on-site attendance of an ACL event (also to be used within a year of the award).

Y.feng at 10:33, 2 May 2025

2025-05-02T10:33:22Z

@@ Line 29: / Line 29: @@
 ''' Great Senior Area Chairs. ''' The program chairs may choose to nominate the senior area chairs who were the most on-time with their duties, dealt with an unusual number of unresponsive ACs, spotted serious publication ethics issues, helped to catch early any ambiguities in the instructions or potential issues in the overall process, suggested useful improvements, etc.
 == Selection Process ==
@@ Line 46: / Line 45: @@
 The award justification text should be short (1-3 sentences) and not include any information that could directly identify the papers or the reviewers.
-=== Nominations ===
+=== Selection process ===

Y.feng: /* Nominations */

2025-05-02T10:30:02Z

Nominations

← Older revision		Revision as of 10:30, 2 May 2025
Line 44:		Line 44:


	The award justification text should be short (1-3 sentences)		The award justification text should be short (1-3 sentences) and not include any information that could directly identify the papers or the reviewers.

	=== Nominations ===		=== Nominations ===

Y.feng at 10:28, 2 May 2025

2025-05-02T10:28:58Z

← Older revision		Revision as of 10:28, 2 May 2025
Line 31:		Line 31:


	=== ~~Best Paper Award~~ ===		== Selection Process ==

			=== Nominations ===

			The reviewers can be nominated by the area chairs by marking the great reviews in the peer review system, after the reviews have been finalized. The instructions for meta-review should include the instructions for entering reviewer nominations. The interface for collecting nominations should include a field for a short justification for the nomination.


			The area chairs can be nominated by the senior area chairs at the stage of entering the final recommendations. Likewise, the interface for entering the nominations should include a short justification or selection from pre-defined categories. The instructions for final recommendations should include the instructions for entering area chair nominations.


			The program chairs can nominate the senior area chairs of one track directly.


			The award justification text should be short (1-3 sentences)

			=== Nominations ===