2013Q3 Reports: TACL Journal Editor - Revision history

DekangLin at 21:05, 27 July 2013

2013-07-27T21:05:04Z

← Older revision		Revision as of 21:05, 27 July 2013
Line 26:		Line 26:
	Jul 1st 2013: 10		Jul 1st 2013: 10

	== ~~[2]~~ Review Process/Benefits of "Revise and Resubmit ==		== Review Process/Benefits of "Revise and Resubmit ==


Line 48:		Line 48:


	== ~~[3]~~ Appearance of TACL papers at conferences ==		== Appearance of TACL papers at conferences ==


Line 58:		Line 58:


	== ~~[4]~~ Outstanding issues ==		== Outstanding issues ==

DekangLin: /* [1] Submission numbers */

2013-07-27T21:04:36Z

[1] Submission numbers

← Older revision		Revision as of 21:04, 27 July 2013
Line 3:		Line 3:


	== ~~[1]~~ Submission numbers ==		== Submission numbers ==


Line 25:		Line 25:
	Jun 1st 2013: 6 <br>		Jun 1st 2013: 6 <br>
	Jul 1st 2013: 10		Jul 1st 2013: 10



	== [2] Review Process/Benefits of "Revise and Resubmit ==		== [2] Review Process/Benefits of "Revise and Resubmit ==

DekangLin at 21:03, 27 July 2013

2013-07-27T21:03:52Z

← Older revision		Revision as of 21:03, 27 July 2013
Line 2:		Line 2:
	This report gives some key points regarding the experience with TACL over the last year.		This report gives some key points regarding the experience with TACL over the last year.

	~~'''~~[1] Submission numbers~~:'''~~
			== [1] Submission numbers ==


	TACL has now had 111 submissions in total since its launch in May 2012. Of those papers, 20 are still in review, and 91 have had decisions. Of those 91 papers, 28 papers have been accepted and published. Some of the remaining 63 papers fall into one of the "revise and resubmit" categories, so they may still be published after revisions.		TACL has now had 111 submissions in total since its launch in May 2012. Of those papers, 20 are still in review, and 91 have had decisions. Of those 91 papers, 28 papers have been accepted and published. Some of the remaining 63 papers fall into one of the "revise and resubmit" categories, so they may still be published after revisions.
Line 25:		Line 27:


	~~'''~~[2] Review Process/Benefits of "Revise and Resubmit~~"'''~~
			== [2] Review Process/Benefits of "Revise and Resubmit ==


	There are 35 action editors for TACL. Each paper is assigned to an action editor shortly after submission. The action editor then chooses 3 reviewers for the paper; the reviewers have 3 weeks to review the paper. Once the reviews are in, a decision is made by the action editor for the paper.		There are 35 action editors for TACL. Each paper is assigned to an action editor shortly after submission. The action editor then chooses 3 reviewers for the paper; the reviewers have 3 weeks to review the paper. Once the reviews are in, a decision is made by the action editor for the paper.
Line 45:		Line 49:
	Lastly, there are obvious benefits in reviewing efficiency, in that borderline papers have required corrections made, and are sent back to the same set of reviewers, rather than being resubmitted to another conference (potentially with recommended changes from reviewers being ignored).		Lastly, there are obvious benefits in reviewing efficiency, in that borderline papers have required corrections made, and are sent back to the same set of reviewers, rather than being resubmitted to another conference (potentially with recommended changes from reviewers being ignored).

	~~'''~~[3] Appearance of TACL papers at conferences~~'''~~
			== [3] Appearance of TACL papers at conferences ==


	Of the 28 accepted papers, 6 were presented at NAACL 2013, and 16 will be presented at ACL 2013. The other 6 papers are TBD.		Of the 28 accepted papers, 6 were presented at NAACL 2013, and 16 will be presented at ACL 2013. The other 6 papers are TBD.
Line 53:		Line 59:
	One issue arose at NAACL 2013 (and may arise at ACL 2013). The agreement with conferences was that TACL papers should have roughly the same proportion of posters vs talks as regular papers accepted to the conference. Some TACL authors were initially surprised that their paper received a poster slot rather than a talk (although we believe that they did not have a reason to be sure of a talk); we received some suggestions from various people in the community (including past ACL presidents) that all TACL papers should get talks, the danger being that otherwise people would not want to submit to TACL. We are not at all sure that this is feasible or desirable given the increasing necessity for poster slots at ACL and other conferences. But this issue should be discussed with the ACL exec.		One issue arose at NAACL 2013 (and may arise at ACL 2013). The agreement with conferences was that TACL papers should have roughly the same proportion of posters vs talks as regular papers accepted to the conference. Some TACL authors were initially surprised that their paper received a poster slot rather than a talk (although we believe that they did not have a reason to be sure of a talk); we received some suggestions from various people in the community (including past ACL presidents) that all TACL papers should get talks, the danger being that otherwise people would not want to submit to TACL. We are not at all sure that this is feasible or desirable given the increasing necessity for poster slots at ACL and other conferences. But this issue should be discussed with the ACL exec.

	~~'''~~[4] Outstanding issues~~'''~~
			== [4] Outstanding issues ==


	We believe that TACL has been a real success. However given what we have learned in this first year or so, there are some significant issues - which may require additional resources - which we believe need to be addressed. These issues will only become more pressing as TACL grows. Specifically:		We believe that TACL has been a real success. However given what we have learned in this first year or so, there are some significant issues - which may require additional resources - which we believe need to be addressed. These issues will only become more pressing as TACL grows. Specifically:
Line 77:		Line 85:


	~~'''~~Appendix: Time to first decision, in sorted order~~'''~~
			== Appendix: Time to first decision, in sorted order ==


	Here we show stats for the 51 papers submitted Nov 1st or later, for which a decision has been made. For each paper we track the number of months and days until the first decision made by the action editor. We show the papers in order sorted from fastest decision to slowest. For example 10 papers have decisions made in 1 month 8 days or less, 20 papers have decisions made in 1 month and 12 days, and so on.		Here we show stats for the 51 papers submitted Nov 1st or later, for which a decision has been made. For each paper we track the number of months and days until the first decision made by the action editor. We show the papers in order sorted from fastest decision to slowest. For example 10 papers have decisions made in 1 month 8 days or less, 20 papers have decisions made in 1 month and 12 days, and so on.

DekangLin at 21:00, 27 July 2013

2013-07-27T21:00:49Z

← Older revision		Revision as of 21:00, 27 July 2013
Line 31:		Line 31:
	The appendix shows the time taken to first decision for the 51 papers submitted Nov 1st 2012 or later, that now have decisions made. Some key statistics are as follows:		The appendix shows the time taken to first decision for the 51 papers submitted Nov 1st 2012 or later, that now have decisions made. Some key statistics are as follows:

	20% (10 papers): 1 months 08 days or less		20% (10 papers): 1 months 08 days or less <br>
	40% (20 papers): 1 months 12 days or less		40% (20 papers): 1 months 12 days or less <br>
	60% (30 papers): 1 months 26 days or less		60% (30 papers): 1 months 26 days or less <br>
	80% (40 papers): 2 months 08 days or less		80% (40 papers): 2 months 08 days or less <br>
	100% (51 papers): 3 months 16 days or less		100% (51 papers): 3 months 16 days or less

Line 81:		Line 81:
	Here we show stats for the 51 papers submitted Nov 1st or later, for which a decision has been made. For each paper we track the number of months and days until the first decision made by the action editor. We show the papers in order sorted from fastest decision to slowest. For example 10 papers have decisions made in 1 month 8 days or less, 20 papers have decisions made in 1 month and 12 days, and so on.		Here we show stats for the 51 papers submitted Nov 1st or later, for which a decision has been made. For each paper we track the number of months and days until the first decision made by the action editor. We show the papers in order sorted from fastest decision to slowest. For example 10 papers have decisions made in 1 month 8 days or less, 20 papers have decisions made in 1 month and 12 days, and so on.

	1 0 months 26 days		1 0 months 26 days <br>
	2 0 months 29 days		2 0 months 29 days <br>
	3 1 months 03 days		3 1 months 03 days <br>
	4 1 months 03 days		4 1 months 03 days <br>
	5 1 months 05 days		5 1 months 05 days <br>
	6 1 months 06 days		6 1 months 06 days <br>
	7 1 months 07 days		7 1 months 07 days <br>
	8 1 months 08 days		8 1 months 08 days <br>
	9 1 months 08 days		9 1 months 08 days <br>
	10 1 months 08 days		10 1 months 08 days <br>
	11 1 months 08 days		11 1 months 08 days <br>
	12 1 months 09 days		12 1 months 09 days <br>
	13 1 months 09 days		13 1 months 09 days <br>
	14 1 months 10 days		14 1 months 10 days <br>
	15 1 months 11 days		15 1 months 11 days <br>
	16 1 months 12 days		16 1 months 12 days <br>
	17 1 months 12 days		17 1 months 12 days <br>
	18 1 months 12 days		18 1 months 12 days <br>
	19 1 months 12 days		19 1 months 12 days <br>
	20 1 months 12 days		20 1 months 12 days <br>
	21 1 months 13 days		21 1 months 13 days <br>
	22 1 months 13 days		22 1 months 13 days <br>
	23 1 months 16 days		23 1 months 16 days <br>
	24 1 months 17 days		24 1 months 17 days <br>
	25 1 months 18 days		25 1 months 18 days <br>
	26 1 months 19 days		26 1 months 19 days <br>
	27 1 months 21 days		27 1 months 21 days <br>
	28 1 months 22 days		28 1 months 22 days <br>
	29 1 months 22 days		29 1 months 22 days <br>
	30 1 months 26 days		30 1 months 26 days <br>
	31 1 months 27 days		31 1 months 27 days <br>
	32 1 months 27 days		32 1 months 27 days <br>
	33 2 months 03 days		33 2 months 03 days <br>
	34 2 months 03 days		34 2 months 03 days <br>
	35 2 months 05 days		35 2 months 05 days <br>
	36 2 months 06 days		36 2 months 06 days <br>
	37 2 months 06 days		37 2 months 06 days <br>
	38 2 months 07 days		38 2 months 07 days <br>
	39 2 months 07 days		39 2 months 07 days <br>
	40 2 months 08 days		40 2 months 08 days <br>
	41 2 months 08 days		41 2 months 08 days <br>
	42 2 months 09 days		42 2 months 09 days <br>
	43 2 months 09 days		43 2 months 09 days <br>
	44 2 months 16 days		44 2 months 16 days <br>
	45 2 months 16 days		45 2 months 16 days <br>
	46 2 months 22 days		46 2 months 22 days <br>
	47 2 months 22 days		47 2 months 22 days <br>
	48 2 months 23 days		48 2 months 23 days <br>
	49 3 months 01 days		49 3 months 01 days <br>
	50 3 months 03 days		50 3 months 03 days <br>
	51 3 months 16 days		51 3 months 16 days

DekangLin: New page: This report gives some key points regarding the experience with TACL over the last year. '''[1] Submission numbers:''' TACL has now had 111 submissi...

2013-07-27T20:59:34Z

New page: This report gives some key points regarding the experience with TACL over the last year. '''[1] Submission numbers:''' TACL has now had 111 submissi...

New page

This report gives some key points regarding the experience with TACL over the last year.

'''[1] Submission numbers:'''

TACL has now had 111 submissions in total since its launch in May 2012. Of those papers, 20 are still in review, and 91 have had decisions. Of those 91 papers, 28 papers have been accepted and published. Some of the remaining 63 papers fall into one of the "revise and resubmit" categories, so they may still be published after revisions.

The distribution of submissions by month is as follows:

May 1st 2012: 6 
Jun 1st 2012: 6 
Jul 1st 2012: 3 
Aug 1st 2012: 3 
Sep 1st 2012: 8 
Oct 1st 2012: 15 
Nov 1st 2012: 17 
Dec 1st 2012: 11 
Jan 1st 2013: 8 
Feb 1st 2013: 5 
Mar 1st 2013: 8 
Apr 1st 2013: 2 
May 1st 2013: 3 
Jun 1st 2013: 6 
Jul 1st 2013: 10

'''[2] Review Process/Benefits of "Revise and Resubmit"'''

There are 35 action editors for TACL. Each paper is assigned to an action editor shortly after submission. The action editor then chooses 3 reviewers for the paper; the reviewers have 3 weeks to review the paper. Once the reviews are in, a decision is made by the action editor for the paper.

The appendix shows the time taken to first decision for the 51 papers submitted Nov 1st 2012 or later, that now have decisions made. Some key statistics are as follows:

20% (10 papers): 1 months 08 days or less
40% (20 papers): 1 months 12 days or less
60% (30 papers): 1 months 26 days or less
80% (40 papers): 2 months 08 days or less
100% (51 papers): 3 months 16 days or less

The maximum time (3 months 16 days) is a little misleading, as there are a couple of papers in the system that have taken over 3 months and do not yet have a final decision. However the statistics do show relatively fast turnaround in reviewing.

We view the "revise and resubmit" options within TACL as particularly beneficial. Of the 28 papers accepted so far, 9 papers were immediate accepts, and 19 fell into one of the "revise and resubmit" categories. Most of these fell into the fast revise and resubmit category, with specific revisions required within 2 months; but some fell within the category of major revisions in the 3-6 month time period.

There are in our view multiple benefits for revise and resubmit options. From the reviewer's perspective, it allows a reviewer to specify what they see as necessary revisions before publication. From the author's perspective, a paper which may have been borderline - and thus at risk or rejection, followed by resubmission to a new conference, with a new set of reviewers - becomes an acceptance once specified revisions have been made. From the point of view of the community, the revise and resubmit cycle is likely to produce higher quality final papers.

Lastly, there are obvious benefits in reviewing efficiency, in that borderline papers have required corrections made, and are sent back to the same set of reviewers, rather than being resubmitted to another conference (potentially with recommended changes from reviewers being ignored).

'''[3] Appearance of TACL papers at conferences'''

Of the 28 accepted papers, 6 were presented at NAACL 2013, and 16 will be presented at ACL 2013. The other 6 papers are TBD.

In addition to agreements with ACL 2013 and NAACL 2013, TACL has reached agreements with EMNLP 2013 and EACL 2014, with acceptance deadlines of August 31st 2013 and November 30th 2013 respectively.

One issue arose at NAACL 2013 (and may arise at ACL 2013). The agreement with conferences was that TACL papers should have roughly the same proportion of posters vs talks as regular papers accepted to the conference. Some TACL authors were initially surprised that their paper received a poster slot rather than a talk (although we believe that they did not have a reason to be sure of a talk); we received some suggestions from various people in the community (including past ACL presidents) that all TACL papers should get talks, the danger being that otherwise people would not want to submit to TACL. We are not at all sure that this is feasible or desirable given the increasing necessity for poster slots at ACL and other conferences. But this issue should be discussed with the ACL exec.

'''[4] Outstanding issues'''

We believe that TACL has been a real success. However given what we have learned in this first year or so, there are some significant issues - which may require additional resources - which we believe need to be addressed. These issues will only become more pressing as TACL grows. Specifically:

(a) Administrative support

Having a few hours of admin support each week, for the day to day running of the journal, would be a great help. It is possible that we could find help for this from Columbia; this would require funding from the ACL exec. We would prefer to pay a professional admin person for this (rather than for example hiring a student).

(b) Publication process

There are many nuances to the publication process. At a low level, each camera ready paper needs to be prepared and published - this is a time consuming process, but could potentially be handled by a student helping with the journal. More significantly, there are many strategic/higher level issues such as: the design of style files for the journal; posting papers in a way such that they are indexed correctly by Google scholar; handling DOIs correctly; assigning ISSN numbers; having the journal registered with Thomson Reuters; interacting correctly with the ACL anthology; ensuring that the journal is archived correctly; monitoring statistics for the journal such as impact factor; and so on.

For this reason it may make sense for TACL to assign somebody to the role of production or publications editor. We have some potential names in mind for this role.

(c) Interaction with the conferences

The interaction with conferences has in general been very productive. There are just a few issues.

One is the issue of acceptance dates for TACL. These dates have to be negotiated with the various conferences, and must synchronize with reviewing schedules. The danger here is that this takes time, and that acceptance deadlines are announced late (as an example, the EMNLP 2013 acceptance date of August 31st 2013 was announced in mid April, which is rather late).

Another is that TACL papers have to be handled essentially manually with respect to the conference program. For example the titles/authors were first sent to the ACL PC chairs; the PC chairs made decisions about posters vs talks; these decisions were then communicated to the TACL editors who passed on those decisions to the TACL authors. It would be helpful if TACL papers could be more closely integrated within the ACL process, although it is not entirely clear how to do this.

'''Appendix: Time to first decision, in sorted order'''

Here we show stats for the 51 papers submitted Nov 1st or later, for which a decision has been made. For each paper we track the number of months and days until the first decision made by the action editor. We show the papers in order sorted from fastest decision to slowest. For example 10 papers have decisions made in 1 month 8 days or less, 20 papers have decisions made in 1 month and 12 days, and so on.

1 0 months 26 days
2 0 months 29 days
3 1 months 03 days
4 1 months 03 days
5 1 months 05 days
6 1 months 06 days
7 1 months 07 days
8 1 months 08 days
9 1 months 08 days
10 1 months 08 days
11 1 months 08 days
12 1 months 09 days
13 1 months 09 days
14 1 months 10 days
15 1 months 11 days
16 1 months 12 days
17 1 months 12 days
18 1 months 12 days
19 1 months 12 days
20 1 months 12 days
21 1 months 13 days
22 1 months 13 days
23 1 months 16 days
24 1 months 17 days
25 1 months 18 days
26 1 months 19 days
27 1 months 21 days
28 1 months 22 days
29 1 months 22 days
30 1 months 26 days
31 1 months 27 days
32 1 months 27 days
33 2 months 03 days
34 2 months 03 days
35 2 months 05 days
36 2 months 06 days
37 2 months 06 days
38 2 months 07 days
39 2 months 07 days
40 2 months 08 days
41 2 months 08 days
42 2 months 09 days
43 2 months 09 days
44 2 months 16 days
45 2 months 16 days
46 2 months 22 days
47 2 months 22 days
48 2 months 23 days
49 3 months 01 days
50 3 months 03 days
51 3 months 16 days