2014Q3 Reports: Program Chairs - Revision history

KristinaToutanova: /* Recommendations */

2014-06-05T20:42:46Z

Recommendations

← Older revision		Revision as of 20:42, 5 June 2014
Line 93:		Line 93:
	==Recommendations==		==Recommendations==
	Based on our workthis year, we would like to make the following recommendations for future years:		Based on our workthis year, we would like to make the following recommendations for future years:
	# ~~Tool~~ for allocating reviewers to areas.		# Recommendations for the tool for allocating reviewers to areas: the tool was very useful to ensure a balanced and sufficient number of reviewers were assigned to each area; we recommend using it in future years. (i) Integrate the tool in START: it would make things easier and less confusing for area chairs and reviewers. (ii) Enable area chairs to bid on reviewers after the submissions are in since there are some very narrow areas for which there are only a handful of expert reviewers.
	~~# It would be great if~~ the tool ~~is implemented~~ in START: it would make things easier and less confusing for area chairs and reviewers.		# Add tools for checking non-anonymity and formatting issues to START. We rejected 15 long and 19 short papers for such issues which could have been prevented if the system could automatically alert the authors of such issues. If there are large technical problems with implementing this, there could be a student volunteer on the organizing committee who could help alert authors that submit at least a few hours before the deadline.
	~~# It would be great if the~~ area chairs ~~can~~ bid on reviewers after the submissions are in since there are some very narrow areas ~~where~~ only a handful of reviewers ~~are experts in~~.		# Enable merging of the schedules for long, short, and TACL papers in START, as well as uploading metadata and papers for TACL papers, to simplify the process of deriving the conference program for the website, the conference handbook, and the downloadable proceedings.
	# ~~It would be good if the Softconf has~~ tools for checking non-anonymity and formatting issues. We rejected 15 long and 19 short papers for such issues which could have been prevented if the system could automatically alert the authors of such issues. If there are large technical problems with implementing this, there could be a student volunteer on the organizing committee who could help ~~alter~~ authors that submit at least a few hours before the deadline.		# Enable the persistence of reviewers and their assignment completion rate in START.
	# Enable merging of the schedules for short and ~~long~~ papers in START, as well as uploading metadata and ~~text of~~ for TACL papers, to simplify the process of deriving the conference program for the website, the conference handbook, and the downloadable proceedings.		# Issue a policy on whether self-published papers (published on ArXiv or the authors’ web pages) should constitute previously published work. This year we published a policy applicable to ACL-14 only, but this changes from year to year and CL conference to conference.
	# Enable the persistence of ~~reviewer lists~~ in START~~, as well as persist the completion rate of assignments~~.
	# Issue a policy on whether self-published papers (published on ~~Arxiv~~ or the authors’ web pages) should constitute previously published work. This year we published a policy applicable to ACL-14 only, but this changes from year to year and CL conference to conference.
	# Issue a policy on whether supplementary materials should be anonymized.		# Issue a policy on whether supplementary materials should be anonymized.

KristinaToutanova: /* Presentations */

2014-06-05T20:35:04Z

Presentations

← Older revision		Revision as of 20:35, 5 June 2014
Line 88:		Line 88:
	The oral presentations are arranged in five parallel sessions. There are two large poster sessions including dinner on two evenings of the conference.		The oral presentations are arranged in five parallel sessions. There are two large poster sessions including dinner on two evenings of the conference.

	We optimized the conference schedule based on feedback from attendees on the talks they would like to see. We collected attendee responses using a scheduling survey developed with the help of David Yarowsky and Svitlana Volkova, and we optimized the conference schedule to assign popular sessions to large conference rooms, and to reduce the chance that two talks that an attendee is interested in are scheduled at the same time. The number of responses collected by the time the schedule was due was 307. The top five topics with highest interest per paper were Semantics, Lexical Semantics, Information Extraction, Question Answering, and Discourse, Dialogue, Coreference and Pragmatics.		We optimized the conference schedule based on feedback from attendees on the talks they would like to see. We collected attendee responses using a scheduling survey developed with the help of David Yarowsky and Svitlana Volkova, and we optimized the conference schedule to assign popular sessions to large conference rooms, and to reduce the chance that two talks that an attendee is interested in are scheduled at the same time. The number of responses collected by the time the schedule was due was 307. The top five topics with highest interest per paper were Semantics, Lexical Semantics, Information Extraction, Question Answering, and Discourse, Dialogue, Coreference and Pragmatics.
			We started with a manual grouping of papers in groups for sessions. We then optimized the assignments of groups to rooms/times, and the order of papers within a group automatically. Compared to the original manually created schedule, the optimized schedule had 644 fewer instances where an attendee was interested in seeing two papers scheduled at the same time. More thorough and exact optimization methods could be applied in the future.
	We started with a manual grouping of papers in groups for sessions. We then optimized the assignments of groups to rooms/times, and the order of papers within a group automatically. ~~In our initial~~ manually ~~constructed~~ schedule, ~~there were X~~ instances where an attendee is interested in seeing two papers scheduled at the same time and ~~after~~ optimization~~, this number was reduced to Y~~.

	==Recommendations==		==Recommendations==

KristinaToutanova: /* Innovations */

2014-06-05T20:28:31Z

Innovations

← Older revision		Revision as of 20:28, 5 June 2014
Line 4:		Line 4:

	As compared to ACL conferences in prior years, the main innovations this year were:		As compared to ACL conferences in prior years, the main innovations this year were:
	# We ~~allocatedreviewers~~ to areas based on the reviewers’ preferences over areas, the preferences of area chairs over reviewers, and the number of submissions in each area (using a tool developed by Mark Dredze and applied to ACL with the help of Jiang Guo; this tool had been applied to NAACL and EMNLP in the past but was applied at ACL for the first time this year).		# We allocated reviewers to areas based on the reviewers’ preferences over areas, the preferences of area chairs over reviewers, and the number of submissions in each area (using a tool developed by Mark Dredze and applied to ACL with the help of Jiang Guo; this tool had been applied to NAACL and EMNLP in the past but was applied at ACL for the first time this year).
	# We optimized the conference schedule based on feedback from attendees on the talks they would like to see.		# We optimized the conference schedule based on feedback from attendees on the talks they would like to see.
	# All talks will be recorded at ACL 2014, as in NAACL 2013.		# All talks will be recorded at ACL 2014, as in NAACL 2013.
	# We scheduled two large poster sessions on two evenings of the conference, to accommodate the large number of poster presentations. Instead of a banquet, we will have is a no-~~feelight~~ social event, and the president’s talk will be given on the first morning of the conference.		# We scheduled two large poster sessions on two evenings of the conference, to accommodate the large number of poster presentations. Instead of a banquet, we will have is a no-fee light social event, and the president’s talk will be given on the first morning of the conference.
	# We grouped oral TACL papers in thematic sessions together with ACL paper presentations, instead of in separate TACL-only sessions.		# We grouped oral TACL papers in thematic sessions together with ACL paper presentations, instead of in separate TACL-only sessions.
	# We solicited nominations for outstanding reviewers and acknowledged them in the proceedings (around 14% of reviewers were acknowledged).		# We solicited nominations for outstanding reviewers and acknowledged them in the proceedings (around 14% of reviewers were acknowledged).


	== Submissions and Presentations ==		== Submissions and Presentations ==

HuaWu at 13:20, 5 June 2014

2014-06-05T13:20:48Z

← Older revision		Revision as of 13:20, 5 June 2014
Line 69:		Line 69:
	\|}		\|}

	The top five areas for the highest number of submissions this year were Machine Translation, Information Extraction, Document Categorization/Sentiment analysis /Topic Models, Semantics, and Tagging/Chunking/Syntax/Parsing.		The top five areas for the highest number of submissions this year were Machine Translation, Information Extraction, Document Categorization/ Sentiment analysis/Topic Models, Semantics, and Tagging/Chunking/Syntax/Parsing.

	===Review process===		===Review process===

HuaWu at 13:16, 5 June 2014

2014-06-05T13:16:33Z

← Older revision		Revision as of 13:16, 5 June 2014
Line 64:		Line 64:
	\| Phonology, Morphology and Word Segmentation \|\|10 \|\|4 \|\|18 \|\|4 \|\|28 \|\|2.57% \|\|8 \|\|2.81% \|\|28.57%		\| Phonology, Morphology and Word Segmentation \|\|10 \|\|4 \|\|18 \|\|4 \|\|28 \|\|2.57% \|\|8 \|\|2.81% \|\|28.57%
	\|-		\|-
	\| Withdrawn \|\|11 \|\| \|\|34 \|\| \|\|45 \|\|4.13% \|\| \|\| \|\|		\| Withdrawn \|\|11 \|\|0 \|\|34 \|\|0 \|\|45 \|\|4.13% \|\|0 \|\|0% \|\|0%
	\|-		\|-
	\| Total \|\|557 \|\|146 \|\|532 \|\|139 \|\|1089 \|\|100~~.00~~% \|\|285 \|\|100~~.00~~% \|\|26.17%		\| Total \|\|557 \|\|146 \|\|532 \|\|139 \|\|1089 \|\|100% \|\|285 \|\|100% \|\|26.17%
	\|}		\|}

HuaWu at 13:14, 5 June 2014

2014-06-05T13:14:58Z

← Older revision		Revision as of 13:14, 5 June 2014
Line 82:		Line 82:

	We did not reject papers for non-anonymity of their supplementary data, software, or notes, since the call for papers did not say anything about whether such materials should be anonymized.		We did not reject papers for non-anonymity of their supplementary data, software, or notes, since the call for papers did not say anything about whether such materials should be anonymized.
	Best paper awards
			===Best paper awards===
	The area chairs nominated sixteen papers for the long best paper award. We selected a list of seven candidates which were ranked by a specially formed best paper committee, consisting of seven members --- six area chairs and one external member. Based on the ranking, we selected a long best paper and a long best student paper winner. The selection of a best short paper is in process. The best long papers will be presented in a plenary session, whereas the best short paper will be presented in one of the parallel sessions.		The area chairs nominated sixteen papers for the long best paper award. We selected a list of seven candidates which were ranked by a specially formed best paper committee, consisting of seven members --- six area chairs and one external member. Based on the ranking, we selected a long best paper and a long best student paper winner. The selection of a best short paper is in process. The best long papers will be presented in a plenary session, whereas the best short paper will be presented in one of the parallel sessions.

HuaWu: /* Program Chairs (Kristina Toutanova and Hua Wu) */

2014-06-05T13:12:18Z

Program Chairs (Kristina Toutanova and Hua Wu)

← Older revision		Revision as of 13:12, 5 June 2014
Line 1:		Line 1:
	==Program Chairs (Kristina Toutanova and Hua Wu)==		===Program Chairs (Kristina Toutanova and Hua Wu)===

	== Innovations ==		== Innovations ==

HuaWu at 13:10, 5 June 2014

2014-06-05T13:10:03Z

@@ Line 64: / Line 64: @@
 | Phonology, Morphology and Word Segmentation	||10	||4	||18	||4	||28	||2.57%	||8	||2.81%	||28.57%
 |-
-| Withdrawn	||11	 ||NA	||34	||NA	||45	||4.13%	||NA	||NA	||NA
+| Withdrawn	||11	 ||	||34	||	||45	||4.13%	||	||	||
 |-
 | Total	||557	||146	||532	||139	||1089	||100.00%	||285	||100.00%	||26.17%
 |}

HuaWu at 13:07, 5 June 2014

2014-06-05T13:07:00Z

← Older revision		Revision as of 13:07, 5 June 2014
Line 64:		Line 64:
	\| Phonology, Morphology and Word Segmentation \|\|10 \|\|4 \|\|18 \|\|4 \|\|28 \|\|2.57% \|\|8 \|\|2.81% \|\|28.57%		\| Phonology, Morphology and Word Segmentation \|\|10 \|\|4 \|\|18 \|\|4 \|\|28 \|\|2.57% \|\|8 \|\|2.81% \|\|28.57%
	\|-		\|-
	\| Withdrawn \|\|11 \|\|34 \|\|45 \|\|4.13% \|\|0 \|\|~~0.00%~~ \|\|~~0.00%~~		\| Withdrawn \|\|11 \|\|NA \|\|34 \|\|NA \|\|45 \|\|4.13% \|\|NA \|\|NA \|\|NA
	\|-		\|-
	\| Total \|\|557 \|\|146 \|\|532 \|\|139 \|\|1089 \|\|100.00% \|\|285 \|\|100.00% \|\|26.17%		\| Total \|\|557 \|\|146 \|\|532 \|\|139 \|\|1089 \|\|100.00% \|\|285 \|\|100.00% \|\|26.17%
	\|}		\|}

HuaWu at 12:57, 5 June 2014

2014-06-05T12:57:52Z

← Older revision		Revision as of 12:57, 5 June 2014
Line 23:		Line 23:
	{\| class="wikitable" cellpadding="2" border="1"		{\| class="wikitable" cellpadding="2" border="1"
	\| Areas\|\|Long Received\|\|Long Accepted\|\|Short Received\|\|Short Accepted\|\|Total Submissions \|\|Percent of Total Submission\|\|Total Accepts\|\|Percent of Total Accepts\|\|Area Accept Rate		\| Areas\|\|Long Received\|\|Long Accepted\|\|Short Received\|\|Short Accepted\|\|Total Submissions \|\|Percent of Total Submission\|\|Total Accepts\|\|Percent of Total Accepts\|\|Area Accept Rate
	\|-
	~~\| Dialogue & Discourse \|\|2 \|\|39 \|\|39 \|\|5 \|\|2 \|\|7 \|\|0 \|\|32 \|\|12.82% \|\|5.13% \|\|17.95%~~
	\|-		\|-
	\| Cognitive Modeling and Psycholinguistics \|\|9 \|\|3 \|\|14 \|\|4 \|\|23 \|\|2.11% \|\|7 \|\|2.46% \|\|30.43%		\| Cognitive Modeling and Psycholinguistics \|\|9 \|\|3 \|\|14 \|\|4 \|\|23 \|\|2.11% \|\|7 \|\|2.46% \|\|30.43%
Line 34:		Line 32:
	\| Document Categorization, Sentiment Analysis, and Topic Models \|\|53 \|\|14 \|\|48 \|\|13 \|\|101 \|\|9.27% \|\|27 \|\|9.47% \|\|26.73%		\| Document Categorization, Sentiment Analysis, and Topic Models \|\|53 \|\|14 \|\|48 \|\|13 \|\|101 \|\|9.27% \|\|27 \|\|9.47% \|\|26.73%
	\|-		\|-
	\| Generation 13 \|\|6 \|\|7 \|\|4 \|\|20 \|\|1.84% \|\|10 \|\|3.51% \|\|50.00%		\| Generation \|\|13 \|\|6 \|\|7 \|\|4 \|\|20 \|\|1.84% \|\|10 \|\|3.51% \|\|50.00%
	\|-		\|-
	\| Information Extraction and Text Mining \|\|54 \|\|13 \|\|49 \|\|14 \|\|103 \|\|9.46% \|\|27 \|\|9.47% \|\|26.21%		\| Information Extraction and Text Mining \|\|54 \|\|13 \|\|49 \|\|14 \|\|103 \|\|9.46% \|\|27 \|\|9.47% \|\|26.21%