2016Q3 Reports: TACL Journal Editor - Revision history

LillianLee: /* Decision statistics */ typo

2016-07-28T02:40:17Z

Decision statistics: typo

← Older revision		Revision as of 02:40, 28 July 2016
Line 69:		Line 69:
	= 32% rejected based on full review, 6% rejected without full review (but received editorial consideration)		= 32% rejected based on full review, 6% rejected without full review (but received editorial consideration)

	72 (=12%) ~~some~~ of these papers are revised versions of (c) papers: TACL treats these as new submissions. Of these 72 papers, the first-decision breakdown is:		72 (=12%) of these papers are revised versions of (c) papers: TACL treats these as new submissions. Of these 72 papers, the first-decision breakdown is:
	44% (a)		44% (a)
	11% (b)		11% (b)

LillianLee: minor updates

2016-07-28T02:19:17Z

minor updates

← Older revision		Revision as of 02:19, 28 July 2016
Line 3:		Line 3:
	== Personnel updates ==		== Personnel updates ==

	The ACL Exec and Lillian Lee have jointly agreed to give her an extra half-term lasting up to but not including September 2018, when she goes on sabbatical. Editorial Assistant Cindy Robinson ~~does~~ not ~~currently plan to~~ stay on past that time.		The ACL Exec and Lillian Lee have jointly agreed to give her an extra half-term lasting up to but not including September 2018, when she goes on sabbatical. Editorial Assistant Cindy Robinson may or may not stay on past that time.

	We have reached the Grand Term Expiration Date of July 2016, where out of our 34 action editors, the terms of 16 have ended. We thank our departing AEs Regina Barzilay, Chris Callison-Burch, and Joakim Nivre for all their help over these pivotal years for TACL, and thank our renewing AEs David Chiang, Alexander Clark, Hal Daume III, Katrin Erk, Philipp Koehn, Hwee Tou Ng, Patrick Pantel, and Chris Potts for being willing and able to stay on board. (Other AEs with expiring terms are still pondering the question.)		We have reached the Grand Term Expiration Date of July 2016, where out of our 34 action editors, the terms of 16 have ended. We thank our departing AEs Regina Barzilay, Chris Callison-Burch, and Joakim Nivre for all their help over these pivotal years for TACL, and thank our renewing AEs David Chiang, Alexander Clark, Hal Daume III, Katrin Erk, Philipp Koehn, Hwee Tou Ng, Patrick Pantel, and Chris Potts for being willing and able to stay on board. (Other AEs with expiring terms are still pondering the question.)
Line 77:		Line 77:
	=== Publishing statistics ===		=== Publishing statistics ===

	27 papers have been published so far in 2016.		29 papers have been published so far in 2016.

	Another 10 are in the publication queue.		Another 8 are in the publication queue.

LillianLee: minor updates

2016-07-23T13:18:40Z

minor updates

← Older revision		Revision as of 13:18, 23 July 2016
Line 20:		Line 20:


	Counts of TACL papers (to be) presented at this year's conferences: 10 at NAACL 2016, 25 at ACL 2016, 10 or so at EMNLP 2016. Conference program chairs have been making different decisions with respect to the poster-vs-oral presentation question for TACL papers. NAACL and ACL essentially gave TACL authors their choice (a very few do choose poster), a decision perhaps ~~aided~~ by the conferences' decisions to have shorter talk slots and hence more talk slots available.		Counts of TACL papers (to be) presented at this year's conferences: 10 at NAACL 2016, 25 at ACL 2016, 10 or so at EMNLP 2016. Conference program chairs have been making different decisions with respect to the poster-vs-oral presentation question for TACL papers. NAACL and ACL essentially gave TACL authors their choice (a very few do choose poster), a decision perhaps influenced by the conferences' decisions to have shorter talk slots and hence more talk slots available. However, it's not clear whether this is consistent with prior conference's policies that the poster/oral decision be based on suitability of topic to presentation type.

	== Interface with preprint servers/workshop proceedings ==		== Interface with preprint servers/workshop proceedings ==

	In the copy-editing phase, we are requesting that authors replace references to preprints such as arXiv papers and other non-refereed papers with references to peer-reviewed versions, if available.		In the copy-editing phase, we are requesting that authors replace references to preprints such as arXiv papers and other non-refereed papers with references to peer-reviewed versions, if available. We understand that this decision could influence authors' choices about whether to post preprints.

	We have agreed that shared tasks familiar to the ACL community whose papers are put in a separate proceedings instead of the main meeting proceedings will be treated as workshop papers, and thus not subject to the multiple-submission restrictions.		We have agreed that papers for shared tasks familiar to the ACL community whose papers are put in a separate proceedings instead of the main meeting proceedings will be treated as workshop papers, and thus not subject to the multiple-submission restrictions.


Line 37:		Line 37:

	=== number of submissions and timing of first decision ===		=== number of submissions and timing of first decision ===
	Depicted below is the history of the number of distinct submission IDs that have received a decision, and the average time to first decision, grouped by round (=nearest first-of-the-month).		Depicted below is the history of the number of distinct submission IDs that have received a decision, and the average time to first decision, grouped by round (=nearest first-of-the-month). The decision time for papers submitted within in a month is counted as starting from the first of the next month.

	[[File:First-decision-to-post.png]]		[[File:First-decision-to-post.png]]

	The drop in the last two months is due at least in part to 13 papers submitted in that period that have not yet received a decision. Also, not counted: papers handled using START (the early days of TACL), papers, and the 106 papers archived for some technical or formatting problem. ~~The decision time for papers submitted late in a month is counted as starting from the first of the next month.~~		The drop in the last two months is due at least in part to 13 papers submitted in that period that have not yet received a decision. Also not counted: papers handled using START (the early days of TACL), papers, and the 106 papers archived for some technical or formatting problem.



	''Observations'':		''Observations'':

	While we don't count technical/format problems, we "do" count papers that receive an editorial decision of rejection without going out for full review. One way we EiCs are trying to keep reviewing loads down and submissions interesting for our reviewers is sometimes doing a preliminary consultation with an action editor or highly relevant reviewer for submissions that seem highly unlikely to be accepted. Such consultations and subsequent discussion can take two weeks ~~or so~~, since we strive to be cautious about making such decisions, but also want to factor in the value of conserving reviewer effort.		While we don't count technical/format problems, we "do" count papers that receive an editorial decision of rejection without going out for full review. One way we EiCs are trying to keep reviewing loads down and submissions interesting for our reviewers is sometimes doing a preliminary consultation with an action editor or highly relevant reviewer for submissions that seem highly unlikely to be accepted. Such consultations and subsequent discussion can take two-three weeks, since we strive to be cautious about making such decisions, but also want to factor in the value of conserving reviewer effort.

	Waves in the number of submissions seem correlated with upcoming conference deadlines.		Waves in the number of submissions seem correlated with upcoming conference deadlines.
Line 59:		Line 58:
	which would yield an ideal target of 45 days turnaround.		which would yield an ideal target of 45 days turnaround.

	For reviewers that complete their reviewers, the average completion time is in fact 23 days!		For reviewers that complete their reviewers, the average completion time is in fact 23 days.

	=== Decision statistics ===		=== Decision statistics ===
Line 78:		Line 77:
	=== Publishing statistics ===		=== Publishing statistics ===

	26 papers have been published so far in 2016.		27 papers have been published so far in 2016.

	Another 11 are in the publication queue.		Another 10 are in the publication queue.

KristinaToutanova at 00:08, 23 July 2016

2016-07-23T00:08:06Z

← Older revision		Revision as of 00:08, 23 July 2016
Line 20:		Line 20:


	Counts of TACL papers (to be) presented at this year's conferences: 12 at NAACL 2016, 25 at ACL 2016, 10 or so at EMNLP 2016. Conference program chairs have been making different decisions with respect to the poster-vs-oral presentation question for TACL papers. NAACL and ACL essentially gave TACL authors their choice (a very few do choose poster), a decision perhaps aided by the conferences' decisions to have shorter talk slots and hence more talk slots available.		Counts of TACL papers (to be) presented at this year's conferences: 10 at NAACL 2016, 25 at ACL 2016, 10 or so at EMNLP 2016. Conference program chairs have been making different decisions with respect to the poster-vs-oral presentation question for TACL papers. NAACL and ACL essentially gave TACL authors their choice (a very few do choose poster), a decision perhaps aided by the conferences' decisions to have shorter talk slots and hence more talk slots available.

	== Interface with preprint servers/workshop proceedings ==		== Interface with preprint servers/workshop proceedings ==

LillianLee: added about withdrawal to avoid rejection decision

2016-07-22T15:25:27Z

added about withdrawal to avoid rejection decision

LillianLee: added number of papers published and in the publshing queue

2016-07-22T08:55:26Z

added number of papers published and in the publshing queue

← Older revision		Revision as of 08:55, 22 July 2016
Line 72:		Line 72:
	14% (c)		14% (c)
	30% (d)		30% (d)

			=== Publishing statistics ===

			26 papers have been published so far in 2016.

			Another 11 are in the publication queue.

LillianLee: minor additions

2016-07-22T08:05:08Z

minor additions

← Older revision		Revision as of 08:05, 22 July 2016
Line 9:		Line 9:
	218 of our elite reviewing team also had expiring terms. Renewal invitations have been issued: of these, there have been 10 declines, 19 requests pending, and --- a fact we are deepy, truly grateful for --- the rest have agreed to rejoin.		218 of our elite reviewing team also had expiring terms. Renewal invitations have been issued: of these, there have been 10 declines, 19 requests pending, and --- a fact we are deepy, truly grateful for --- the rest have agreed to rejoin.

	''We sometimes joke that the "A" in TACL stands for "artisanal". The reviewers and action editors are, on the whole, putting a huge amount of time and craft into their "pieces", broadly construed. Whether this "personal touch" can (or should) be scaled to a conference-size production line is an important question.''		We anticipating issuing invitations to new people as well soon.

			''We sometimes joke that the "A" in TACL stands for "artisanal". The reviewers and action editors are, on the whole, putting a huge amount of time and craft into their "pieces", broadly construed. (For one bit of proof, see the statistics below.) Whether this "personal touch" can (or should) be scaled to a conference-size production line is an important question.''

	== Interface with conferences ==		== Interface with conferences ==
Line 53:		Line 55:
	* ideally 10 days for AE to coordinate discussion among reviewers and come to a decision		* ideally 10 days for AE to coordinate discussion among reviewers and come to a decision
	which would yield an ideal target of 45 days turnaround.		which would yield an ideal target of 45 days turnaround.

			For reviewers that complete their reviewers, the average completion time is in fact 23 days!

	=== Decision statistics ===		=== Decision statistics ===

LillianLee: this should be considered a submission version of the TACL 2016Q3 report

2016-07-22T07:35:12Z

this should be considered a submission version of the TACL 2016Q3 report

@@ Line 3: / Line 3: @@
 == Personnel updates ==
-The ACL Exec and Lillian Lee have jointly agreed to give her an extra half-term lasting up to but not including September 2018, when she goes on sabbatical.  Editorial Assistant Cindy Robinson does not currently plan on staying on part that time.
+The ACL Exec and Lillian Lee have jointly agreed to give her an extra half-term lasting up to but not including September 2018, when she goes on sabbatical.  Editorial Assistant Cindy Robinson does not currently plan on staying on past that time.
 We have reached the Grand Term Expiration Date of July 2016, where  out of our 34 action editors, the terms of 16 have ended.  We thank our departing AEs Regina Barzilay, Chris Callison-Burch, and Joakim Nivre for all their help over these pivotal years for TACL, and thank our renewing AEs David Chiang, Alexander Clark, Hal Daume III, Katrin Erk, Philipp Koehn, Hwee You Ng, Patrick Pantel, and Chris Potts for being willing and able to stay on board.  (Other AEs with expiring terms  are still pondering the question.)
@@ Line 9: / Line 9: @@
 of our elite reviewing team also had expiring terms.  Renewal invitations have been issued: of these, there have been 10 declines, 19 requests pending, and --- a fact we are deepy, truly grateful for --- the rest have agreed to rejoin.

LillianLee at 07:09, 22 July 2016

2016-07-22T07:09:10Z

← Older revision		Revision as of 07:09, 22 July 2016
Line 1:		Line 1:
	(We are grateful to have received an extension on filing our report until July 22 from ACL Secretary Shiqi Zhao. --the TACL editors in chief)		(We are very grateful to have received an extension on filing our report until July 22 from ACL Secretary Shiqi Zhao. --the TACL editors in chief)

			== Personnel updates ==

			The ACL Exec and Lillian Lee have jointly agreed to give her an extra half-term lasting up to but not including September 2018, when she goes on sabbatical. Editorial Assistant Cindy Robinson does not currently plan on staying on part that time.

			We have reached the Grand Term Expiration Date of July 2016, where out of our 34 action editors, the terms of 16 have ended. We thank our departing AEs Regina Barzilay, Chris Callison-Burch, and Joakim Nivre for all their help over these pivotal years for TACL, and thank our renewing AEs David Chiang, Alexander Clark, Hal Daume III, Katrin Erk, Philipp Koehn, Hwee You Ng, Patrick Pantel, and Chris Potts for being willing and able to stay on board. (Other AEs with expiring terms are still pondering the question.)

			218 of our elite reviewing team also had expiring terms. Renewal invitations have been issued: of these, there have been 10 declines, 19 requests pending, and --- a fact we are deepy, truly grateful for --- the rest have agreed to rejoin.



Line 9:		Line 18:
	[[File:First-decision-to-post.png]]		[[File:First-decision-to-post.png]]

	The drop in the last two months is due at least in part to 13 papers submitted in that period that have not yet received a decision. Also, not counted: papers handled using START (the early days of TACL), papers, papers archived for some technical or formatting problem. The decision time for papers submitted late in a month is counted as starting from the first of the next month.		The drop in the last two months is due at least in part to 13 papers submitted in that period that have not yet received a decision. Also, not counted: papers handled using START (the early days of TACL), papers, and the 106 papers archived for some technical or formatting problem. The decision time for papers submitted late in a month is counted as starting from the first of the next month.

LillianLee at 06:40, 22 July 2016

2016-07-22T06:40:50Z

← Older revision		Revision as of 06:40, 22 July 2016
Line 28:		Line 28:

	=== Decision statistics ===		=== Decision statistics ===

			First decision breakdown for our 589 submissions.
			9% (a) = accepted as is
			22% (b) = conditional accept: acceptance guaranteed if conditions met
			31% (c) = rejected, encourage resubmission but no guarantee of acceptance
			38% (d) = rejected with 1-year moratorium on TACL submission.
			= 32% rejected based on full review, 6% rejected without full review (but received editorial consideration)

			72 (=12%) some of these papers are revised versions of (c) papers: TACL treats these as new submissions. Of these 72 papers, the first-decision breakdown is:
			44% (a)
			11% (b)
			14% (c)
			30% (d)

← Older revision		Revision as of 15:25, 22 July 2016
Line 3:		Line 3:
	== Personnel updates ==		== Personnel updates ==

	The ACL Exec and Lillian Lee have jointly agreed to give her an extra half-term lasting up to but not including September 2018, when she goes on sabbatical. Editorial Assistant Cindy Robinson does not currently plan ~~on staying~~ on past that time.		The ACL Exec and Lillian Lee have jointly agreed to give her an extra half-term lasting up to but not including September 2018, when she goes on sabbatical. Editorial Assistant Cindy Robinson does not currently plan to stay on past that time.

	We have reached the Grand Term Expiration Date of July 2016, where out of our 34 action editors, the terms of 16 have ended. We thank our departing AEs Regina Barzilay, Chris Callison-Burch, and Joakim Nivre for all their help over these pivotal years for TACL, and thank our renewing AEs David Chiang, Alexander Clark, Hal Daume III, Katrin Erk, Philipp Koehn, Hwee ~~You~~ Ng, Patrick Pantel, and Chris Potts for being willing and able to stay on board. (Other AEs with expiring terms are still pondering the question.)		We have reached the Grand Term Expiration Date of July 2016, where out of our 34 action editors, the terms of 16 have ended. We thank our departing AEs Regina Barzilay, Chris Callison-Burch, and Joakim Nivre for all their help over these pivotal years for TACL, and thank our renewing AEs David Chiang, Alexander Clark, Hal Daume III, Katrin Erk, Philipp Koehn, Hwee Tou Ng, Patrick Pantel, and Chris Potts for being willing and able to stay on board. (Other AEs with expiring terms are still pondering the question.)

	218 of our elite reviewing team also had expiring terms. Renewal invitations have been issued: of these, there have been 10 declines, 19 requests pending, and --- a fact we are ~~deepy~~, truly grateful for --- the rest have agreed to rejoin.		218 of our elite reviewing team also had expiring terms. Renewal invitations have been issued: of these, there have been 10 declines, 19 requests pending, and --- a fact we are deeply, truly grateful for --- the rest have agreed to rejoin.

	We ~~anticipating~~ issuing invitations to new people as well soon.		We anticipate issuing invitations to new people as well soon.

	''We sometimes joke that the "A" in TACL stands for "artisanal". The reviewers and action editors are, on the whole, putting a huge amount of time and craft into their "pieces", broadly construed. (For one bit of proof, see the statistics below.) Whether this "personal touch" can (or should) be scaled to a conference-size production line is an important question.''		''We sometimes joke that the "A" in TACL stands for "artisanal". The reviewers and action editors are, on the whole, putting a huge amount of time and craft into their "pieces", broadly construed. (For one bit of proof, see the statistics below.) Whether this "personal touch" can (or should) be scaled to a conference-size production line is an important question.''
Line 15:		Line 15:
	== Interface with conferences ==		== Interface with conferences ==

	Our checks for undeclared multiple/overlapping submissions with main conferences are surprisingly time-consuming but unfortunately do turn up problematic cases. ~~The~~ policy language ~~has been clarified accordingly~~.		Our checks for undeclared multiple/overlapping submissions with main conferences are surprisingly time-consuming but unfortunately do turn up problematic cases. As one consequence, we have clarified our policy language to state that ''no'' material in a TACL submission can be under consideration elsewhere while under review by TACL (see website for exact policy).

	Counts of TACL papers (to be) presented at this year's conferences: 12 at NAACL 2016, 25 at ACL 2016, 10 or so at EMNLP 2016. Conference program chairs have been making different decisions with respect to the poster-vs-oral presentation question for TACL papers. NAACL and ACL ~~essential~~ gave TACL authors their choice (a very few do choose poster), a decision perhaps aided by ~~their~~ decisions to have shorter talk slots and hence more talk slots available.		Author-response periods for conferences --- so that authors see reviews before they receive an official decision --- raise the question from authors, "After getting negative reviews from a conference, if we withdraw the paper from the conference (and revise it), can we submit it to TACL?" We consider a paper that has received reviews and is not accepted by a conference to be "rejected" from TACL's point of view. Hence, submission of a heavily revised version to TACL is allowed ''if'' the TACL submission follows the rules for papers rejected from *ACL/EMNLP; but undeclared (revisions of) such submissions will be rejected from TACL.


			Counts of TACL papers (to be) presented at this year's conferences: 12 at NAACL 2016, 25 at ACL 2016, 10 or so at EMNLP 2016. Conference program chairs have been making different decisions with respect to the poster-vs-oral presentation question for TACL papers. NAACL and ACL essentially gave TACL authors their choice (a very few do choose poster), a decision perhaps aided by the conferences' decisions to have shorter talk slots and hence more talk slots available.

	== Interface with preprint servers/workshop proceedings ==		== Interface with preprint servers/workshop proceedings ==
Line 28:		Line 31:
	== Transition to new (old) domain name ==		== Transition to new (old) domain name ==

	We are ~~transition~~ from using URLs referencing tacl2013.cs.columbia.edu to transacl.org. Old links (for both http and https) should still work, but we ask that people try to use the transacl.org ~~form~~ and update links they control, if possible. The plan is to get DOIs that are linked to transacl.org URLs.		We are transitioning from using URLs referencing tacl2013.cs.columbia.edu to transacl.org ([https://transacl.org]) . Old links (for both http and https) should still work, but we ask that people try to use the transacl.org format and update links they control, if possible. The plan is to get DOIs that are linked to transacl.org URLs.


Line 44:		Line 47:
	''Observations'':		''Observations'':

	While we don't count technical/format problems, we "do" count papers that receive an editorial decision of rejection without going out for full review. One way we EiCs trying to keep reviewing loads down and ~~keep~~ submissions interesting for our reviewers is sometimes doing a preliminary consultation with an action editor or highly relevant reviewer for submissions that seem highly unlikely to be accepted. Such consultations and subsequent discussion can take two weeks or so, since we strive to be cautious about making such decisions, but also want to factor in the value of conserving reviewer effort.		While we don't count technical/format problems, we "do" count papers that receive an editorial decision of rejection without going out for full review. One way we EiCs are trying to keep reviewing loads down and submissions interesting for our reviewers is sometimes doing a preliminary consultation with an action editor or highly relevant reviewer for submissions that seem highly unlikely to be accepted. Such consultations and subsequent discussion can take two weeks or so, since we strive to be cautious about making such decisions, but also want to factor in the value of conserving reviewer effort.

	Waves in the number of submissions seem correlated with upcoming conference deadlines.		Waves in the number of submissions seem correlated with upcoming conference deadlines.

	Average decision times per month appear to correspond well with an ideal but attainable schedule of		Average decision times per month appear to correspond well with an ideal but attainable schedule of:

	* ideally 7 days to make a load-balanced assignment of ~~submission~~ to action editors (AEs)		: ideally 7 days to make a load-balanced assignment of submissions to action editors (AEs)
	* ideally 7 days for AEs to get three reviewers confirmed to review		: ideally 7 days for AEs to get three reviewers confirmed to review
	* ideally 21 days for reviewers to review		: ideally 21 days for reviewers to review
	* ideally 10 days for AE to coordinate discussion among reviewers and come to a decision		: ideally 10 days for AE to coordinate discussion among reviewers and come to a decision
	which would yield an ideal target of 45 days turnaround.		which would yield an ideal target of 45 days turnaround.