Admin Wiki - User contributions [en]

Tutorial chair handbook

2021-08-19T16:06:54Z

AgataSavary: /* Organization of past tutorials */

== Duties ==
Tutorial chairs are nominated by the general chair of the main conference. Their duties are described in [[Tutorial chair duties|here]].

== Timeline ==
The timeline for the organisation of tutorials should follow the general [[conference planning schedule]]. Below, we complete this schedule with more detailed tasks, as defined for the ACL 2020 tutorials. The dates from the [[conference planning schedule]] are highlighted in '''bold'''. 
In recent years, calls and selection of tutorial proposals have been coordinated among some major CL conferences (usually a subset of ACL, NAACL, EMNLP, AACL and COLING). Note that if your conference is the first in the year to come, you will need to be the major driving force in the initial steps described below. 
Tutorial proposals are usually reviewed by the tutorial chairs of the coordinating conferences, but it may be a good idea to also rectruit a small group of external reviewers. They should have a good experience in reviewing and a large understanding of the recent progress in the CL domain.

* [T-12]
** Learn from the general chair which conferences will have their tutorials coordinated with yours.
** Establish contact with the tutorial chairs of these other conferences, and discuss
*** Recruitment of external reviewers
*** Proposal evaluation criteria
*** Soliciting tutorial proposals from particular potential lecturers on particular topics
*** Organizing a public poll about interest in the submitted tutorial
** Discuss general strategies of the main conference with the general chair and the program chairs
***promotion of some topics
***mailing lists and conference alert portals to be used for all announcements
** Set up a SoftConf space for proposal submission
*** Solicit the creation of a SoftConf space from [mailto:support@softconf.com support] (currently Rich Gerber). Note that submissions will be joint for several conferences but the proceedings will be edited for each conference separately. Therefore, a new SoftConf space will have to be created later for the selected proposals of each conference.
*** If you plan to have external tutorial reviewers, invite them via SoftConf.
*** Set up a submission page.
** Set up a joint email for tutorial chairs.
** Confirm the number (usually 6-8) and length (usually 3 hours) of tutorial slots with the ACL Business Manager (currently Priscilla Rasmussen).
** Draft the joint call for tutorials (based on [[#Past calls for tutorials|past calls]]).

* '''[T-11]''' ''(in the future, it would be good to extend the time slot between the call and the deadline to 2 months at least, especially if July/August is included)''
** Send out the call for tutorials.
** For posting the call on the ACL website, contact the ACL Business Manager (currently Priscilla Rasmussen).
** If you planned to encourage submission of proposals from particular potential lecturers, contact them now.
** Set up a SoftConf page for proposal reviews.

* [T-10]
** Deadline for tutorial proposal submission.
** If you decided to run a public poll about interest in the submitted tutorial, set it up and send out a call for expression of interest.
** Assign proposals to reviewers, send out invitations to review (via SoftConf).

* '''[T-9]'''
** Select tutorials jointly with other conference chairs.
** Notify the authors (via SoftConf).
** Liaise with local chair for space needs and scheduling.
** Send instructions to tutorial teachers about printed material, agenda/time management, payments, fees, etc.
** Liaise the conference website chair for updates about tutorials and authors' guidelines.
** Liaise with the ACL Office for the registration form.

* '''[T-4]'''
** Collect short descriptions for proceedings from tutorial teachers.
** Provide them to the webmaster and the general and program chairs .

* '''[T-3]'''
** Confirm the tutorials with authors.
** Authors submit the final tutorial descriptions for the proceedings.
** Arrange for printing of tutorial materials.

* [T-2]
** Arrange with the [mailto:support@softconf.com SoftConf support] the creation of a separate SoftConf space for the selected tutorials of your conference only. Indicate which tutorial submissions should be copied there.
** Set up the final submission form in the new SoftConf space. Include the submissions of slides, which will be published on the ACL Anthology. Include signing the CC-BY-4.0 license agreement.

* '''[T-1]'''
** Collect and check final tutorial materials.
** Create the tutorial proceedings, including slides.

* '''[T-0]''' Conference and tutorials

* [T+1]
** Invite the lecturers to update the slides at the ACL Anthology via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form] (under CC-BY-4.0).
** Update this tutorial page and the list of [[Past tutorials|past tutorials]] with the data from your conference. For access to the ACL [[Conference Handbook]], contact the ACL Secretary (currently Shiqi Zhou).

== Tutorials in a virtual conference ==
ACL 2020 was transformed into a fully virtual conference due to the COVID-19 pandemic. This decision was taken only 3 months before the conference and added a bunch of new tasks on top of the usual organization. This new organization was largely to be invented from scratch, additionally to the previous experience from [https://icml.cc/ ICML]. SlideLive was the subcontractor for the virtual infrastructure.

The tasks for tutorials included:
* collecting slides and pre-recordings from the tutorial teachers (recorded with the SlideLive online software)
* defining [https://acl2020.org/_pages/docs/ACL_2020_Tutorial_Timeslots.pdf time slots] which would increase participation across various time zones
* collecting teachers' preferences for various time slots and encouraging them to give their tutorials twice
* setting up the [https://acl2020.org/program/tutorials/ schedule]
* informing the participants about [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ virtual participation] and about the [https://acl2020.org/blog/detailed-modalities-of-tutorials/ detailed modalities] for each tutorial
* defining the contents of live sessions (a mixture of pre-recordings, question-answering, break-out rooms for one tutorial, etc.)
* creating automatic captioning, sending the captions to the teachers and uploading the edited versions
* dry runs with teachers and SlidesLive people
* archiving pre-recordings or recordings of the live sessions additionally to slides
More details are given in the [https://www.aclweb.org/adminwiki/index.php?title=2020Q3_Reports:_Tutorial_Chairs final report].

== Publication of tutorials ==
For NAACL 2019, ACL Anthology stores the tutorial descriptions, slides and video recordings. The same should go for ACL 2020 (which went fully virtual due to the COVID pandemic), except that recordings of live sessions are sometimes replaced with pre-recordings.

In tutorials organized until 2019, there was no homogeneous policy as to the publication of tutorial material. Data (abstracts, slides, code, etc.) from selected tutorials were usually scattered in 2 or 3 places:
* conference website
* ACL Anthology
* private repositories of the authors
The sustainability of these varies from venue to venue. The main websites of some conferences are not maintained, therefore ACL collects [https://www.aclweb.org/portal/node/2481 mirrors] of some of them (but not all).
The tutorial proceedings, published in the ACL Anthology, until recently included only abstracts but no tutorial slides.

As a result, for many [[past tutorials]], no slides are available online, so the community cannot benefit from these assets. It is, therefore, important to promote '''Open Access''' of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation '''slides in the ACL Anthology'''. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form].

It is recommended for tutorial chairs to:
* announce the publication of the tutorial slides under the CC-BY-4.0 license (used in the ACL Anthology)
* encourage the authors to also publish, under open licenses, their code and data used in the tutorials
* include the initial versions of tutorial slides, together with abstract, in the proceedings
* encourage the tutorial authors to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]

== Organization of past tutorials ==
For the last years, the call and selection of tutorials has been coordinated among several major CL conferences. The table below summarizes some data collected from existing reports and from direct contacts with past tutorial chairs.

{| border="1" cellspacing="0" cellpadding="5"
|'''Year'''
|'''Coordinated conferences'''
|'''Tutorial chairs'''
|'''Submission platform'''
|'''# Submissions'''
|'''Reviewers'''
|'''# Reviews per proposal'''
|'''# Selected tutorials'''
|'''Selection criteria'''
|'''Useful links'''
|-
|2020
|ACL EMNLP AACL-IJCNLP COLING
|ACL: Agata Savary (France), Yue Zhang (China) EMNLP: Benjamin Van Durme (USA), Aline Villavicencio (UK & Brazil) AACL-IJCNLP: Tim Baldwin (Australia), Fei Xia (USA) COLING: Daniel Beck (Australia), Lucia Specia (UK)
|SoftConf
|43
|Tutorial chairs + external experts
|3
|ACL: 8 EMNLP: 7 ACCL-IJCNLP: 6 COLING: 7
|Clarity and preparedness Novelty or timely character of the topic Lecturers' experience Likely audience interest Open access of the teaching material Diversity aspects (multilingualism, gender, age and country of the lecturers) Compatibility with the required venues
|[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 Call] [https://www.aclweb.org/adminwiki/index.php?title=2020Q1_Reports:_ACL_2020#Tutorial_Chairs Report] [https://acl2020.org/program/tutorials/ ACL tutorials] [https://2020.emnlp.org/tutorials EMNLP tutorials] [http://aacl2020.org/ ACCL-IJCNLP tutorials] [https://coling2020.org/pages/tutorials COLING tutorials]
|
|-
|2019
|NAACL ACL EMNLP
|ACL: Preslav Nakov (Qatar), Alexis Palmer (USA) NAACL: Anoop Sarkar (Canda), Michael Strube (Germany) EMNLP-IJCNLP: Marine Carpuat (USA), Tim Baldwin (Australia)
|SoftConf
|45
|Tutorial chairs
|2-3
|ACL: 9 NAACL: 6 EMNLP: 7
|Presenters (1-5) Description (1-5) Interest Level (1-5) Preparedness (1-5)
|[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials Call] [http://www.acl2019.org/EN/tutorials.xhtml ACL tutorials] [https://naacl2019.org/program/tutorials/ NAACL tutorials]
|-
|2018
|ACL NAACL EMNLP COLING
|ACL: Yoav Artzi (USA), Jacob Eisenstein (USA) COLING: Pascale Fung (Hong Kong), Donia Scott (UK), Marilyn Walker (USA) EMNLP: Mausam (India), Lu Wang (USA) NAACL: Mohit Bansal (USA), Rebecca Passonneau (USA)
|SoftConf
|49
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6 COLING: 6
|Quality of plan Diversity of topics Qualifications of presenter likely audience interest
|[https://acl2018.org/2017/09/11/joint-call-for-tutorials/ Call] 
[https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs Report] 
[https://acl2018.org/tutorials/ ACL tutorials] 
[https://emnlp2018.org/program/tutorials/ EMNLP tutorials] 
[https://coling2018.org/tutorial-list/ COLING tutorials]
|-
|2017
|ACL EACL EMNLP
|ACL: Maja Popović (Germany), Jordan Boyd-Graber (USA) EACL: Alex Klementiev (Germany), Lucia Specia (UK) EMNLP: Nathan Schneider (USA), Alexandra Birch (UK)
|SoftConf
|26
|Tutorial chairs
|3
|ACL: 6 EACL: 6 EMNLP: 7
|Clarity (description and outline) Topic
|[https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs Report] [http://acl2017.org/tutorials/ ACL tutorials] [http://eacl2017.org/index.php/tutorials EACL tutorials]
|-
|2016
|ACL NAACL EMNLP
|ACL: Alexandra Birch (UK), Willem Zuidema (Netherlands) NAACL: Alexander Rush and Bishan Yang EMNLP: Rebecca Hwa and Mohit Bansal
|SoftConf
|32
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6
|Relevance to ACL community (15) Quality of instructor (15) Outline and depth/breadth of the proposal (15) 
Potential attendance Newly emerging area not previously covered in an ACL-related tutorial (yes or no) Introduction into related fields (yes or no) 
Overall score (15) 1st preference for venue as indicated in proposal comments
|[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 Call] [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs Report] [http://naacl.org/naacl-hlt-2016/tutorials.html NAACL tutorials] [https://www.aclweb.org/mirror/emnlp2016/tutorials.html EMNLP tutorials]
|}

== Past calls for tutorials ==
*[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 ACL/AACL-IJCNLP/EMNLP/COLING 2020 joint call for tutorials]
*[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials NAACL/ACL/EMNLP 2019 joint call for tutorials]
*[https://www.aclweb.org/portal/content/final-joint-call-tutorial-proposals-aclcolingemnlpnaacl-2018 ACL/COLING/EMNLP/NAACL 2018 joint call for tutorials]
*[https://www.aclweb.org/portal/content/eacl-2017-call-tutorial-proposals EACL 2017 call for tutorials]
*[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 NAACL/ACL/EMNLP 2016 joint call for tutorials]
*[https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-naacl-2015-acl-2015-emnlp-2015-0 NAACL/ACL/EMNLP 2015 joint call for tutorials]

== Reports from past tutorial chairs ==
*[[2020Q3_Reports:_Tutorial_Chairs|ACL 2020 report]] (by Agata Savary and Yue Zhang)
*[[2019Q1_Reports:_ACL_2019#Tutorial_Co-Chairs|ACL 2019 report]] (by Preslav Nakov and Alexis Palmer)
*[[2018Q3 Reports: Tutorial Chairs|ACL 2018 report]] (by Yoav Artzi and Jacob Eisenstein)
*[[2017Q3 Reports: Tutorial Chairs|ACL 2017 report]] (by Maja Popović and Jordan Boyd-Graber)
*[[2016Q3 Reports: Tutorial Chairs|ACL 2016 report]] (by Alexandra Birch and Willem Zuidema)

== Lessons learned by past tutorial chairs ==
* [Agata Savary, 2020] If the conference is virtual:
** Remember that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]).
** Announce the individual modalities with the attendees early enough (for instance, if watching pre-recordings in advance is necessary for some tutorials because their live sessions only included question answering)
** If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to Zoom rooms only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.
* [Agata Savary, 2020] Use a common SoftConf space for joint tutorial submissions for all coordinated conference. Once the proposals have been selected and assigned to conferences, use one track per conference in the same SoftConf space.
* [Agata Savary, 2020] Publish the list of recommended readings on each tutorial page
* [Agata Savary, 2020] A good way to proceed while selecting tutorials for several venues is to have a common online meeting, in which
** Tutorials are clustered per themes (to distribute similar tutorials over different conferences)
** The chairs for each venue select their best choices in a round robin manner.
** Generate csv files with reviews and scores: Manager -> Other tools -> Spreadsheet maker -> Reports
* [Agata Savary, 2020] While setting up a review form on SoftConf, include:
** All evaluation criteria, including diversity, in the review form
** Cutting edge vs. introductory category
** Duration (3 or 6 hours)
* [Agata Savary, 2020] Currently, the generic conference schedule assumes only 2 months between the call for tutorials and notification to proposers. If a solid 1-month review procedure is to take place, this means, that proposers only have 1 month between the call and the submission deadline. It would be good to shift the call to [T-12].
* [Agata Savary, 2020] For many past tutorials, no slides are available online. It is, therefore, important to promote Open Access of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation slides on the ACL Anthology. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]. As a tutorial chair, make sure that
** initial versions of tutorial slides are submitted to the proceedings together with the tutorial abstracts
** tutorial authors are encouraged to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]
* [Marine Capuat, 2019] Continue soliciting both cutting edge and introductory tutorials
* [Marine Capuat, 2019] Ask proposers to express their preferred location/conference in one place (i.e. either in proposal pdf, or in softconf form, not both) to avoid inconsistencies
* [ACL 2018 chairs] Issues with proceedings edition, when one joint SoftConf space is used for many conferences (see the [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs report])
* [ACL 2016 chairs] Avoid confusion about the submission procedures for the final tutorial descriptions, those were submitted by email in many different formats

Author: Agata Savary, July 2019-July 2020

Updates are welcome

[[Category:Conference Handbook]]

Past tutorials

2020-12-31T09:30:37Z

AgataSavary: /* 2016 tutorials */

This page belongs to the [[tutorial chair handbook]]. It summarizes data on tutorials which took place at some recent ACL, EACL, NAACL, EMNLP and COLING conferences.

==2020 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''Virtual conference link'''
|'''ACL Anthology link'''
|-
|Interpretability and Analysis in Neural NLP
|Yonatan Belinkov, Sebastian Gehrmann and Ellie Pavlick
|ACL 2020
|[https://acl2020.org/program/tutorials/#t1-interpretability-and-analysis-in-neural-nlp-cutting-edge-]
|[https://virtual.acl2020.org/tutorial_T1.html]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.1/]
|-
|Multi-modal Information Extraction from Text, Semi-structured, and Tabular Data on the Web
|Xin Luna Dong, Hannaneh Hajishirzi, Colin Lockard and Prashant Shiralkar
|ACL 2020
|[https://acl2020.org/program/tutorials/#t2-multi-modal-information-extraction-from-text-semi-structured-and-tabular-data-on-the-web-cutting-edge-]
|[https://virtual.acl2020.org/tutorial_T2.html]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.6/]
|-
|Reviewing Natural Language Processing Research
|Kevin Cohen, Karën Fort, Margot Mieskes and Aurélie Névéol
|ACL 2020
|[https://acl2020.org/program/tutorials/#t3-reviewing-natural-language-processing-research-introductory-]
|[https://virtual.acl2020.org/tutorial_T3.html]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.4/]
|-
|Stylized Text Generation: Approaches and Applications
|Lili Mou and Olga Vechtomova
|ACL 2020
|[https://acl2020.org/program/tutorials/#t4-stylized-text-generation-approaches-and-applications-cutting-edge-]
|[https://virtual.acl2020.org/tutorial_T4.html]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.5/]
|-
|Achieving Common Ground in Multi-modal Dialogue
|Malihe Alikhani and Matthew Stone
|ACL 2020
|[https://acl2020.org/program/tutorials/#t5-achieving-common-ground-in-multi-modal-dialogue-cutting-edge-]
|[https://virtual.acl2020.org/tutorial_T5.html]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.3/]
|-
|Commonsense Reasoning for Natural Language Processing
|Maarten Sap, Vered Shwartz, Antoine Bosselut, Yejin Choi and Dan Roth
|ACL 2020
|[https://acl2020.org/program/tutorials/#t6-commonsense-reasoning-for-natural-language-processing-introductory-]
|[https://virtual.acl2020.org/tutorial_T6.html]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.7/]
|-
|Integrating Ethics into the NLP Curriculum
|Emily M. Bender, Dirk Hovy and Alexandra Schofield
|ACL 2020
|[https://acl2020.org/program/tutorials/#t7-integrating-ethics-into-the-nlp-curriculum-introductory-]
|[https://virtual.acl2020.org/tutorial_T7.html]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.2/]
|-
|Open-Domain Question Answering
|Danqi Chen and Scott Wen-tau Yih
|ACL 2020
|[https://acl2020.org/program/tutorials/#t8-open-domain-question-answering-cutting-edge-]
|[https://virtual.acl2020.org/tutorial_T8.html]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.8/]
|-
|TO BE COMPLETED for AACL, EMNLP and COLING 2020
|
|
|
|
|
|}

==2019 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Latent Structure Models for Natural Language Processing
|André F. T. Martins, Tsvetomila Mihaylova, Nikita Nangia and Vlad Niculae
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T1]
|[https://www.aclweb.org/anthology/P19-4001/]
|-
|Graph-Based Meaning Representations: Design and Processing
|Alexander Koller, Stephan Oepen and Weiwei Sun
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T2]
|[https://www.aclweb.org/anthology/P19-4002/]
|-
|Discourse Analysis and Its Applications
|Shafiq Joty, Giuseppe Carenini, Raymond Ng and Gabriel Murray
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T3]
|[https://www.aclweb.org/anthology/P19-4003/]
|-
|Computational Analysis of Political Texts: Bridging Research Efforts Across Communities
|Goran Glavaš, Federico Nanni and Simone Paolo Ponzetto
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T4]
|[https://www.aclweb.org/anthology/P19-4004/]
|-
|Wikipedia as a Resource for Text Analysis and Retrieval
|Marius Pasca
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T5]
|[https://www.aclweb.org/anthology/P19-4005/]
|-
|Deep Bayesian Natural Language Processing
|Jen-Tzung Chien
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T6]
|[https://www.aclweb.org/anthology/P19-4006/]
|-
|Unsupervised Cross-Lingual Representation Learning
|Sebastian Ruder, Anders Søgaard and Ivan Vulić
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T7]
|[https://www.aclweb.org/anthology/P19-4007/]
|-
|Advances in Argument Mining
|Katarzyna Budzynska and Chris Reed
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T8]
|[https://www.aclweb.org/anthology/P19-4008/]
|-
|Storytelling from Structured Data and Knowledge Graphs : An NLG Perspective
|Abhijit Mishra, Anirban Laha, Karthik Sankaranarayanan, Parag Jain and Saravanan Krishnan
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T9]
|[https://www.aclweb.org/anthology/P19-4009/]
|-
|Deep Adversarial Learning for NLP
|William Yang Wang, Sameer Singh and Jiwei Li
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t1-deep-adversarial-learning-for-nlp]
|[https://aclweb.org/anthology/N19-5001/]
|-
|Deep Learning for Natural Language Inference
|Samuel Bowman and Xiaodan Zhu
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t2-deep-learning-for-natural-language-inference]
|[https://aclweb.org/anthology/N19-5002/]
|-
|Measuring and Modeling Language Change
|Jacob Eisenstein
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t3-measuring-and-modeling-language-change]
|[https://aclweb.org/anthology/N19-5003/]
|-
|Transfer Learning in Natural Language Processing
|Sebastian Ruder, Matthew Peters, Swabha Swayamdipta and Thomas Wolf
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t4-transfer-learning-in-natural-language-processing]
|[https://aclweb.org/anthology/N19-5004/]
|-
|Language Learning and Processing in People and Machines
|Aida Nematzadeh, Richard Futrell and Roger Levy
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t5-language-learning-and-processing-in-people-and-machines]
|[https://aclweb.org/anthology/N19-5005/]
|-
|Applications of Natural Language Processing in Clinical Research and Practice
|Yanshan Wang, Ahmad Tafti, Sunghwan Sohn and Rui Zhang
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t6-applications-of-natural-language-processing-in-clinical-research-and-practice]
|[https://aclweb.org/anthology/N19-5006/]
|}

==2018 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Joint models for NLP
|Yue Zhang
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Graph Formalisms for Meaning Representations
|Adam Lopez and Sorcha Gilroy
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Writing Code for NLP Research
|Matt Gardner, Mark Neumann, Joel Grus, and Nicholas Lourie
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Deep Latent Variable Models of Natural Language
|Alexander Rush, Yoon Kim, and Sam Wiseman
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Standardized Tests as benchmarks for Artificial Intelligence
|Mrinmaya Sachan, Minjoon Seo, Hannaneh Hajishirzi, and Eric Xing
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Deep Chit-Chat: Deep Learning for ChatBots
|Wei Wu and Rui Yan
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|100 Things You Always Wanted to Know about Semantics & Pragmatics But Were Afraid to Ask
|Emily M. Bender
|ACL 2018
|[https://acl2018.org/tutorials/#100-things-you-always-wante]
|[https://aclweb.org/anthology/P18-5001/]
|-
|Neural Approaches to Conversational AI
|Jianfeng Gao, Michel Galley and Lihong Li
|ACL 2018
|[https://acl2018.org/tutorials/#neural-approaches-to-conver]
|[https://aclweb.org/anthology/P18-5002/]
|-
|Variational Inference and Deep Generative Models
|Wilker Aziz and Philip Schulz
|ACL 2018
|[https://acl2018.org/tutorials/#variational-inference-and-d]
|[https://aclweb.org/anthology/P18-5003/]
|-
|Connecting Language and Vision to Actions
|Peter Anderson, Abhishek Das and Qi Wu
|ACL 2018
|[https://acl2018.org/tutorials/#connecting-language-and-vis]
|[https://aclweb.org/anthology/P18-5004/]
|-
|Beyond Multiword Expressions: Processing Idioms and Metaphors
|Valia Kordoni
|ACL 2018
|[https://acl2018.org/tutorials/#beyond-multiword-expression]
|[https://aclweb.org/anthology/P18-5005/]
|-
|Neural Semantic Parsing
|Luke Zettlemoyer, Matt Gardner, Pradeep Dasigi, Srinivasan Iyer and Alane Suhr
|ACL 2018
|[https://acl2018.org/tutorials/#neural-semantic-parsing]
|[https://aclweb.org/anthology/P18-5006/]
|-
|Deep Reinforcement Learning for NLP
|William Yang Wang, Jiwei Li and Xiaodong He
|ACL 2018
|[https://acl2018.org/tutorials/#deep-reinforcement-learning]
|[https://aclweb.org/anthology/P18-5007/]
|-
|Multi-lingual Entity Discovery and Linking
|Avirup Sil, Heng Ji, Dan Roth and Silviu-Petru Cucerzan
|ACL 2018
|[https://acl2018.org/tutorials/#multi-lingual-entity-discov]
|[https://aclweb.org/anthology/P18-5008/]
|-
|Modelling Natural Language, Programs, and their Intersection
|Graham Neubig and Miltiadis Allamanis
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6001/]
|-
|Deep Learning Approaches to Text Production
|Claire Gardent and Shashi Narayan
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6002/]
|-
|Scalable Construction and Reasoning of Massive Knowledge Bases
|Xiang Ren, Nanyun Peng and William Yang Wang
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6003/]
|-
|The interplay between lexical resources and Natural Language Processing
|Jose Camacho-Collados, Luis Espinosa Anke and Mohammad Taher Pilehvar
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6004/]
|-
|Socially Responsible NLP
|Yulia Tsvetkov, Vinodkumar Prabhakaran and Rob Voigt
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6005/]
|-
|Deep Learning for Conversational AI
|Pei-Hao Su, Nikola Mrkšić, Iñigo Casanueva, Ivan Vulić
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6006/]
|-
|NLP for Conversations: Sentiment, Summarization, and Group Dynamics
|Gabriel Murray, Giuseppe Carenini and Shafiq Joty
|COLING 2018
|[https://sites.google.com/view/nlpforconversations]
|[https://aclweb.org/anthology/C18-3001/]
|-
|Practical Parsing for Downstream Applications
|Daniel Dakota and Sandra Kübler
|COLING 2018
|[http://cl.indiana.edu/colingParsing18/]
|[https://aclweb.org/anthology/C18-3002/]
|-
|Frame Semantics across Languages: Towards a Multilingual FrameNet
|Collin Baker, Michael Ellsworth, Miriam R L Petruck and Swabha Swayamdipta
|COLING 2018
|[https://framenet.icsi.berkeley.edu/fndrupal/node/5552/]
|[https://aclweb.org/anthology/C18-3003/]
|-
|Deep Bayesian Learning and Understanding
|Jen-Tzung Chien
|COLING 2018
|[http://chien.cm.nctu.edu.tw/home/coling/]
|[https://aclweb.org/anthology/C18-3004/]
|-
|Data-Driven Text Simplification
|Sanja Štajner and Horacio Saggion
|COLING 2018
|[http://taln.upf.edu/pages/coling2018simplification/]
|[https://aclweb.org/anthology/C18-3005/]
|-
|Deep Learning for Dialogue Systems
|Yun-Nung Chen, Asli Celikyilmaz and Dilek Hakkani-Tur
|COLING 2018
|[http://deepdialogue.miulab.tw/]
|[https://aclweb.org/anthology/C18-3006/]
|}

==2017 tutorials==

EMNLP 2017 website is no longer available. There are no traces of the AMNLP 2017 tutorials, except this [https://www.facebook.com/emnlp2017/posts/well-have-7-tutorials-at-emnlp2017-concepthier-sarcasm-graphrep-srl-memaugnn-str/1860468284191872/ Facebook post].

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Universal Dependencies
|Joakim Nivre, Daniel Zeman, Filip Ginter, and Francis Tyers
|EACL 2017
|[http://universaldependencies.org/eacl17tutorial/]
|
|-
|Practical Neural Machine Translation
|Rico Sennrich and Barry Haddow
|EACL 2017
|[http://www.statmt.org/eacl2017/practical-nmt.pdf]
|
|-
|Imitation learning for structured prediction in natural language processing
|Andreas Vlachos, Gerasimos Lampouras and Sebastian Riedel
|EACL 2017
|[https://sheffieldnlp.github.io/ImitationLearningTutorialEACL2017/]
|
|-
|Word Vector Space Specialisation
|Ivan Vulić, Nikola Mrkšić, and Mohammad Taher Pilehvar
|EACL 2017
|[http://people.ds.cam.ac.uk/iv250/tutorial/wv-tutorial.pdf]
|
|-
|Integer Linear Programming formulations in Natural Language Processing
|Dan Roth and Vivek Srikumar
|EACL 2017
|[https://sheffieldnlp.github.io/ImitationLearningTutorialEACL2017/]
|
|-
|Building Multimodal Simulations for Natural Language
|James Pustejovsky and Nikhil Krishnaswamy
|EACL 2017
|[http://www.voxicon.net]
|
|-
|Natural Language Processing for Precision Medicine
|Hoifung Poon, Chris Quirk, Kristina Toutanova, and Wen-tau Yih
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5001/]
|-
|Multimodal Machine Learning
|Louis-Philippe Morency and Tadas Baltrusaitis
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5002/]
|-
|Deep Learning for Semantic Composition
|Xiaodan Zhu and Edward Grefenstette
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5003/]
|-
|Deep Learning for Dialogue Systems
|Yun-Nung Chen, Asli Celikyilmaz, and Dilek Hakkani-Tur
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5004/]
|-
|Beyond Words: Deep Learning for Multi-word Expressions and Collocations
|Valia Kordoni
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5005/]
|-
|Making Better Use of the Crowd
|Jennifer Wortman Vaughan
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5006/]
|}

==2016 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Multimodal Learning and Reasoning
|Desmond Elliott, Douwe Kiela and Angeliki Lazaridou
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index4d04.html?article_id=59]
|[https://www.aclweb.org/anthology/P16-5001/]
|-
|NLP Approaches to Computational Argumentation
|Noam Slonim, Iryna Gurevych, Chris Reed and Benno Stein
|ACL 2016
|[http://mirror.aclweb.org/acl2016/indexfc69.html?article_id=56]
|
|-
|Computer Aided Translation
|Philipp Koehn
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index6ea2.html?article_id=62]
|
|-
|Semantic Representations of Word Senses and Concepts
|José Camacho-Collados, Ignacio Iacobacci, Roberto Navigli and Mohammad Taher Pilehvar
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index6b9c.html?article_id=58]
|
|-
|Neural Machine Translation
|Thang Luong, Kyunghyun Cho and Christopher D. Manning
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index2e51.html?article_id=55]
|
|-
|Game Theory and Natural Language: Origin, Evolution and Processing
|Rocco Tripodi and Marcello Pelillo
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index56cf.html?article_id=57]
|
|-
|Understanding Short Texts
|Zhongyuan Wang and Haixun Wang
|ACL 2016
|[http://mirror.aclweb.org/acl2016/indexedf3.html?article_id=60]
|
|-
|MetaNet: Repository, Identification System, and Applications
|Miriam R L Petruck and Ellen K Dodge
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index6a61.html?article_id=61]
|
|-
|English Resource Semantics
|Dan Flickinger, Emily M. Bender, and Woodley Packard
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t1.html]
|[https://aclweb.org/anthology/N16-4001/]
|-
|Multilingual Multimodal Language Processing Using Neural Networks
|Mitesh M Khapra and Sarath Chandar
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t2.html]
|[https://aclweb.org/anthology/N16-4002/]
|-
|Question Answering with Knowledge Base, Web and Beyond
|Scott Wen-tau Yih & Hao Ma
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t3.html]
|[https://aclweb.org/anthology/N16-4003/]
|-
|Recent Progress in Deep Learning for NLP
|Zhengdong Lu and Hang Li
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t4.html]
|[https://aclweb.org/anthology/N16-4004/]
|-
|Scalable Statistical Relational Learning for NLP
|William Yang Wang and William W. Cohen
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t5.html]
|[https://aclweb.org/anthology/N16-4005/]
|-
|Statistical Machine Translation between Related Languages
|Pushpak Bhattacharyya, Mitesh Khapra, and Anoop Kunchukuttan
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t6.html]
|[https://aclweb.org/anthology/N16-4006/]
|-
|Practical Neural Networks for NLP: From Theory to Code
|Chris Dyer, Yoav Goldberg and Graham Neubig
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#practical]
|
|-
|Advanced Markov Logic Techniques for Scalable Joint Inference in NLP
|Deepak Venugopal, Vibhav Gogate and Vincent Ng
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#advanced]
|
|-
|Lifelong Machine Learning for Natural Language Processing
|Zhiyuan Chen and Bing Liu
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#lifelong]
|
|-
|Neural Networks for Sentiment Analysis
|Yue Zhang and Duy Tin Vo
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#neural]
|
|-
|Continuous Vector Spaces for Cross-language NLP Applications
|Rafael E. Banchs
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#continuous]
|
|-
|Methods and Theories for Large-scale Structured Prediction
|Xu Sun and Yansong Feng
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#methods]
|
|-
|Compositional Distributional Models of Meaning
|Mehrnoosh Sadrzadeh and Dimitri Kartsaklis
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T1/]
|[https://aclweb.org/anthology/C16-3001/]
|-
|Chinese Textual Sentiment Analysis: Datasets, Resources and Tools
|Lun-Wei Ku and Wei-Fan Chen
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T2]
|[https://aclweb.org/anthology/C16-3002/]
|-
|Natural Language Processing for Intelligent Access to Scientific Information
|Horacio Saggion and Francesco Ronzano
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T3]
|[https://aclweb.org/anthology/C16-3003/]
|-
|Quality Estimation for Language Output Applications
|Carolina Scarton, Gustavo Henrique Paetzold, and Lucia Specia
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T4]
|[https://aclweb.org/anthology/C16-3004/]
|-
|Translationese: Between Human and Machine Translation
|Shuly Wintner
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T5/]
|[https://aclweb.org/anthology/C16-3005/]
|-
|Succinct Data Structures for NLP-at-Scale
|Matthias Petri and Trevor Cohn
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T6/]
|[https://aclweb.org/anthology/C16-3006/]
|-
|The Role of Wikipedia in Text Analysis and Retrieval
|Marius Pasca
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T7]
|[https://aclweb.org/anthology/C16-3007/]
|}

Author: Agata Savary, July 2019

Updates are welcome

Past tutorials

2020-12-08T09:02:12Z

AgataSavary:

This page belongs to the [[tutorial chair handbook]]. It summarizes data on tutorials which took place at some recent ACL, EACL, NAACL, EMNLP and COLING conferences.

==2020 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''Virtual conference link'''
|'''ACL Anthology link'''
|-
|Interpretability and Analysis in Neural NLP
|Yonatan Belinkov, Sebastian Gehrmann and Ellie Pavlick
|ACL 2020
|[https://acl2020.org/program/tutorials/#t1-interpretability-and-analysis-in-neural-nlp-cutting-edge-]
|[https://virtual.acl2020.org/tutorial_T1.html]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.1/]
|-
|Multi-modal Information Extraction from Text, Semi-structured, and Tabular Data on the Web
|Xin Luna Dong, Hannaneh Hajishirzi, Colin Lockard and Prashant Shiralkar
|ACL 2020
|[https://acl2020.org/program/tutorials/#t2-multi-modal-information-extraction-from-text-semi-structured-and-tabular-data-on-the-web-cutting-edge-]
|[https://virtual.acl2020.org/tutorial_T2.html]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.6/]
|-
|Reviewing Natural Language Processing Research
|Kevin Cohen, Karën Fort, Margot Mieskes and Aurélie Névéol
|ACL 2020
|[https://acl2020.org/program/tutorials/#t3-reviewing-natural-language-processing-research-introductory-]
|[https://virtual.acl2020.org/tutorial_T3.html]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.4/]
|-
|Stylized Text Generation: Approaches and Applications
|Lili Mou and Olga Vechtomova
|ACL 2020
|[https://acl2020.org/program/tutorials/#t4-stylized-text-generation-approaches-and-applications-cutting-edge-]
|[https://virtual.acl2020.org/tutorial_T4.html]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.5/]
|-
|Achieving Common Ground in Multi-modal Dialogue
|Malihe Alikhani and Matthew Stone
|ACL 2020
|[https://acl2020.org/program/tutorials/#t5-achieving-common-ground-in-multi-modal-dialogue-cutting-edge-]
|[https://virtual.acl2020.org/tutorial_T5.html]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.3/]
|-
|Commonsense Reasoning for Natural Language Processing
|Maarten Sap, Vered Shwartz, Antoine Bosselut, Yejin Choi and Dan Roth
|ACL 2020
|[https://acl2020.org/program/tutorials/#t6-commonsense-reasoning-for-natural-language-processing-introductory-]
|[https://virtual.acl2020.org/tutorial_T6.html]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.7/]
|-
|Integrating Ethics into the NLP Curriculum
|Emily M. Bender, Dirk Hovy and Alexandra Schofield
|ACL 2020
|[https://acl2020.org/program/tutorials/#t7-integrating-ethics-into-the-nlp-curriculum-introductory-]
|[https://virtual.acl2020.org/tutorial_T7.html]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.2/]
|-
|Open-Domain Question Answering
|Danqi Chen and Scott Wen-tau Yih
|ACL 2020
|[https://acl2020.org/program/tutorials/#t8-open-domain-question-answering-cutting-edge-]
|[https://virtual.acl2020.org/tutorial_T8.html]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.8/]
|-
|TO BE COMPLETED for AACL, EMNLP and COLING 2020
|
|
|
|
|
|}

==2019 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Latent Structure Models for Natural Language Processing
|André F. T. Martins, Tsvetomila Mihaylova, Nikita Nangia and Vlad Niculae
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T1]
|[https://www.aclweb.org/anthology/P19-4001/]
|-
|Graph-Based Meaning Representations: Design and Processing
|Alexander Koller, Stephan Oepen and Weiwei Sun
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T2]
|[https://www.aclweb.org/anthology/P19-4002/]
|-
|Discourse Analysis and Its Applications
|Shafiq Joty, Giuseppe Carenini, Raymond Ng and Gabriel Murray
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T3]
|[https://www.aclweb.org/anthology/P19-4003/]
|-
|Computational Analysis of Political Texts: Bridging Research Efforts Across Communities
|Goran Glavaš, Federico Nanni and Simone Paolo Ponzetto
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T4]
|[https://www.aclweb.org/anthology/P19-4004/]
|-
|Wikipedia as a Resource for Text Analysis and Retrieval
|Marius Pasca
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T5]
|[https://www.aclweb.org/anthology/P19-4005/]
|-
|Deep Bayesian Natural Language Processing
|Jen-Tzung Chien
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T6]
|[https://www.aclweb.org/anthology/P19-4006/]
|-
|Unsupervised Cross-Lingual Representation Learning
|Sebastian Ruder, Anders Søgaard and Ivan Vulić
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T7]
|[https://www.aclweb.org/anthology/P19-4007/]
|-
|Advances in Argument Mining
|Katarzyna Budzynska and Chris Reed
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T8]
|[https://www.aclweb.org/anthology/P19-4008/]
|-
|Storytelling from Structured Data and Knowledge Graphs : An NLG Perspective
|Abhijit Mishra, Anirban Laha, Karthik Sankaranarayanan, Parag Jain and Saravanan Krishnan
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T9]
|[https://www.aclweb.org/anthology/P19-4009/]
|-
|Deep Adversarial Learning for NLP
|William Yang Wang, Sameer Singh and Jiwei Li
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t1-deep-adversarial-learning-for-nlp]
|[https://aclweb.org/anthology/N19-5001/]
|-
|Deep Learning for Natural Language Inference
|Samuel Bowman and Xiaodan Zhu
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t2-deep-learning-for-natural-language-inference]
|[https://aclweb.org/anthology/N19-5002/]
|-
|Measuring and Modeling Language Change
|Jacob Eisenstein
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t3-measuring-and-modeling-language-change]
|[https://aclweb.org/anthology/N19-5003/]
|-
|Transfer Learning in Natural Language Processing
|Sebastian Ruder, Matthew Peters, Swabha Swayamdipta and Thomas Wolf
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t4-transfer-learning-in-natural-language-processing]
|[https://aclweb.org/anthology/N19-5004/]
|-
|Language Learning and Processing in People and Machines
|Aida Nematzadeh, Richard Futrell and Roger Levy
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t5-language-learning-and-processing-in-people-and-machines]
|[https://aclweb.org/anthology/N19-5005/]
|-
|Applications of Natural Language Processing in Clinical Research and Practice
|Yanshan Wang, Ahmad Tafti, Sunghwan Sohn and Rui Zhang
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t6-applications-of-natural-language-processing-in-clinical-research-and-practice]
|[https://aclweb.org/anthology/N19-5006/]
|}

==2018 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Joint models for NLP
|Yue Zhang
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Graph Formalisms for Meaning Representations
|Adam Lopez and Sorcha Gilroy
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Writing Code for NLP Research
|Matt Gardner, Mark Neumann, Joel Grus, and Nicholas Lourie
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Deep Latent Variable Models of Natural Language
|Alexander Rush, Yoon Kim, and Sam Wiseman
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Standardized Tests as benchmarks for Artificial Intelligence
|Mrinmaya Sachan, Minjoon Seo, Hannaneh Hajishirzi, and Eric Xing
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Deep Chit-Chat: Deep Learning for ChatBots
|Wei Wu and Rui Yan
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|100 Things You Always Wanted to Know about Semantics & Pragmatics But Were Afraid to Ask
|Emily M. Bender
|ACL 2018
|[https://acl2018.org/tutorials/#100-things-you-always-wante]
|[https://aclweb.org/anthology/P18-5001/]
|-
|Neural Approaches to Conversational AI
|Jianfeng Gao, Michel Galley and Lihong Li
|ACL 2018
|[https://acl2018.org/tutorials/#neural-approaches-to-conver]
|[https://aclweb.org/anthology/P18-5002/]
|-
|Variational Inference and Deep Generative Models
|Wilker Aziz and Philip Schulz
|ACL 2018
|[https://acl2018.org/tutorials/#variational-inference-and-d]
|[https://aclweb.org/anthology/P18-5003/]
|-
|Connecting Language and Vision to Actions
|Peter Anderson, Abhishek Das and Qi Wu
|ACL 2018
|[https://acl2018.org/tutorials/#connecting-language-and-vis]
|[https://aclweb.org/anthology/P18-5004/]
|-
|Beyond Multiword Expressions: Processing Idioms and Metaphors
|Valia Kordoni
|ACL 2018
|[https://acl2018.org/tutorials/#beyond-multiword-expression]
|[https://aclweb.org/anthology/P18-5005/]
|-
|Neural Semantic Parsing
|Luke Zettlemoyer, Matt Gardner, Pradeep Dasigi, Srinivasan Iyer and Alane Suhr
|ACL 2018
|[https://acl2018.org/tutorials/#neural-semantic-parsing]
|[https://aclweb.org/anthology/P18-5006/]
|-
|Deep Reinforcement Learning for NLP
|William Yang Wang, Jiwei Li and Xiaodong He
|ACL 2018
|[https://acl2018.org/tutorials/#deep-reinforcement-learning]
|[https://aclweb.org/anthology/P18-5007/]
|-
|Multi-lingual Entity Discovery and Linking
|Avirup Sil, Heng Ji, Dan Roth and Silviu-Petru Cucerzan
|ACL 2018
|[https://acl2018.org/tutorials/#multi-lingual-entity-discov]
|[https://aclweb.org/anthology/P18-5008/]
|-
|Modelling Natural Language, Programs, and their Intersection
|Graham Neubig and Miltiadis Allamanis
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6001/]
|-
|Deep Learning Approaches to Text Production
|Claire Gardent and Shashi Narayan
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6002/]
|-
|Scalable Construction and Reasoning of Massive Knowledge Bases
|Xiang Ren, Nanyun Peng and William Yang Wang
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6003/]
|-
|The interplay between lexical resources and Natural Language Processing
|Jose Camacho-Collados, Luis Espinosa Anke and Mohammad Taher Pilehvar
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6004/]
|-
|Socially Responsible NLP
|Yulia Tsvetkov, Vinodkumar Prabhakaran and Rob Voigt
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6005/]
|-
|Deep Learning for Conversational AI
|Pei-Hao Su, Nikola Mrkšić, Iñigo Casanueva, Ivan Vulić
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6006/]
|-
|NLP for Conversations: Sentiment, Summarization, and Group Dynamics
|Gabriel Murray, Giuseppe Carenini and Shafiq Joty
|COLING 2018
|[https://sites.google.com/view/nlpforconversations]
|[https://aclweb.org/anthology/C18-3001/]
|-
|Practical Parsing for Downstream Applications
|Daniel Dakota and Sandra Kübler
|COLING 2018
|[http://cl.indiana.edu/colingParsing18/]
|[https://aclweb.org/anthology/C18-3002/]
|-
|Frame Semantics across Languages: Towards a Multilingual FrameNet
|Collin Baker, Michael Ellsworth, Miriam R L Petruck and Swabha Swayamdipta
|COLING 2018
|[https://framenet.icsi.berkeley.edu/fndrupal/node/5552/]
|[https://aclweb.org/anthology/C18-3003/]
|-
|Deep Bayesian Learning and Understanding
|Jen-Tzung Chien
|COLING 2018
|[http://chien.cm.nctu.edu.tw/home/coling/]
|[https://aclweb.org/anthology/C18-3004/]
|-
|Data-Driven Text Simplification
|Sanja Štajner and Horacio Saggion
|COLING 2018
|[http://taln.upf.edu/pages/coling2018simplification/]
|[https://aclweb.org/anthology/C18-3005/]
|-
|Deep Learning for Dialogue Systems
|Yun-Nung Chen, Asli Celikyilmaz and Dilek Hakkani-Tur
|COLING 2018
|[http://deepdialogue.miulab.tw/]
|[https://aclweb.org/anthology/C18-3006/]
|}

==2017 tutorials==

EMNLP 2017 website is no longer available. There are no traces of the AMNLP 2017 tutorials, except this [https://www.facebook.com/emnlp2017/posts/well-have-7-tutorials-at-emnlp2017-concepthier-sarcasm-graphrep-srl-memaugnn-str/1860468284191872/ Facebook post].

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Universal Dependencies
|Joakim Nivre, Daniel Zeman, Filip Ginter, and Francis Tyers
|EACL 2017
|[http://universaldependencies.org/eacl17tutorial/]
|
|-
|Practical Neural Machine Translation
|Rico Sennrich and Barry Haddow
|EACL 2017
|[http://www.statmt.org/eacl2017/practical-nmt.pdf]
|
|-
|Imitation learning for structured prediction in natural language processing
|Andreas Vlachos, Gerasimos Lampouras and Sebastian Riedel
|EACL 2017
|[https://sheffieldnlp.github.io/ImitationLearningTutorialEACL2017/]
|
|-
|Word Vector Space Specialisation
|Ivan Vulić, Nikola Mrkšić, and Mohammad Taher Pilehvar
|EACL 2017
|[http://people.ds.cam.ac.uk/iv250/tutorial/wv-tutorial.pdf]
|
|-
|Integer Linear Programming formulations in Natural Language Processing
|Dan Roth and Vivek Srikumar
|EACL 2017
|[https://sheffieldnlp.github.io/ImitationLearningTutorialEACL2017/]
|
|-
|Building Multimodal Simulations for Natural Language
|James Pustejovsky and Nikhil Krishnaswamy
|EACL 2017
|[http://www.voxicon.net]
|
|-
|Natural Language Processing for Precision Medicine
|Hoifung Poon, Chris Quirk, Kristina Toutanova, and Wen-tau Yih
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5001/]
|-
|Multimodal Machine Learning
|Louis-Philippe Morency and Tadas Baltrusaitis
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5002/]
|-
|Deep Learning for Semantic Composition
|Xiaodan Zhu and Edward Grefenstette
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5003/]
|-
|Deep Learning for Dialogue Systems
|Yun-Nung Chen, Asli Celikyilmaz, and Dilek Hakkani-Tur
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5004/]
|-
|Beyond Words: Deep Learning for Multi-word Expressions and Collocations
|Valia Kordoni
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5005/]
|-
|Making Better Use of the Crowd
|Jennifer Wortman Vaughan
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5006/]
|}

==2016 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Multimodal Learning and Reasoning
|Desmond Elliott, Douwe Kiela and Angeliki Lazaridou
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index4d04.html?article_id=59]
|
|-
|NLP Approaches to Computational Argumentation
|Noam Slonim, Iryna Gurevych, Chris Reed and Benno Stein
|ACL 2016
|[http://mirror.aclweb.org/acl2016/indexfc69.html?article_id=56]
|
|-
|Computer Aided Translation
|Philipp Koehn
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index6ea2.html?article_id=62]
|
|-
|Semantic Representations of Word Senses and Concepts
|José Camacho-Collados, Ignacio Iacobacci, Roberto Navigli and Mohammad Taher Pilehvar
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index6b9c.html?article_id=58]
|
|-
|Neural Machine Translation
|Thang Luong, Kyunghyun Cho and Christopher D. Manning
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index2e51.html?article_id=55]
|
|-
|Game Theory and Natural Language: Origin, Evolution and Processing
|Rocco Tripodi and Marcello Pelillo
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index56cf.html?article_id=57]
|
|-
|Understanding Short Texts
|Zhongyuan Wang and Haixun Wang
|ACL 2016
|[http://mirror.aclweb.org/acl2016/indexedf3.html?article_id=60]
|
|-
|MetaNet: Repository, Identification System, and Applications
|Miriam R L Petruck and Ellen K Dodge
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index6a61.html?article_id=61]
|
|-
|English Resource Semantics
|Dan Flickinger, Emily M. Bender, and Woodley Packard
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t1.html]
|[https://aclweb.org/anthology/N16-4001/]
|-
|Multilingual Multimodal Language Processing Using Neural Networks
|Mitesh M Khapra and Sarath Chandar
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t2.html]
|[https://aclweb.org/anthology/N16-4002/]
|-
|Question Answering with Knowledge Base, Web and Beyond
|Scott Wen-tau Yih & Hao Ma
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t3.html]
|[https://aclweb.org/anthology/N16-4003/]
|-
|Recent Progress in Deep Learning for NLP
|Zhengdong Lu and Hang Li
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t4.html]
|[https://aclweb.org/anthology/N16-4004/]
|-
|Scalable Statistical Relational Learning for NLP
|William Yang Wang and William W. Cohen
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t5.html]
|[https://aclweb.org/anthology/N16-4005/]
|-
|Statistical Machine Translation between Related Languages
|Pushpak Bhattacharyya, Mitesh Khapra, and Anoop Kunchukuttan
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t6.html]
|[https://aclweb.org/anthology/N16-4006/]
|-
|Practical Neural Networks for NLP: From Theory to Code
|Chris Dyer, Yoav Goldberg and Graham Neubig
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#practical]
|
|-
|Advanced Markov Logic Techniques for Scalable Joint Inference in NLP
|Deepak Venugopal, Vibhav Gogate and Vincent Ng
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#advanced]
|
|-
|Lifelong Machine Learning for Natural Language Processing
|Zhiyuan Chen and Bing Liu
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#lifelong]
|
|-
|Neural Networks for Sentiment Analysis
|Yue Zhang and Duy Tin Vo
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#neural]
|
|-
|Continuous Vector Spaces for Cross-language NLP Applications
|Rafael E. Banchs
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#continuous]
|
|-
|Methods and Theories for Large-scale Structured Prediction
|Xu Sun and Yansong Feng
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#methods]
|
|-
|Compositional Distributional Models of Meaning
|Mehrnoosh Sadrzadeh and Dimitri Kartsaklis
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T1/]
|[https://aclweb.org/anthology/C16-3001/]
|-
|Chinese Textual Sentiment Analysis: Datasets, Resources and Tools
|Lun-Wei Ku and Wei-Fan Chen
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T2]
|[https://aclweb.org/anthology/C16-3002/]
|-
|Natural Language Processing for Intelligent Access to Scientific Information
|Horacio Saggion and Francesco Ronzano
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T3]
|[https://aclweb.org/anthology/C16-3003/]
|-
|Quality Estimation for Language Output Applications
|Carolina Scarton, Gustavo Henrique Paetzold, and Lucia Specia
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T4]
|[https://aclweb.org/anthology/C16-3004/]
|-
|Translationese: Between Human and Machine Translation
|Shuly Wintner
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T5/]
|[https://aclweb.org/anthology/C16-3005/]
|-
|Succinct Data Structures for NLP-at-Scale
|Matthias Petri and Trevor Cohn
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T6/]
|[https://aclweb.org/anthology/C16-3006/]
|-
|The Role of Wikipedia in Text Analysis and Retrieval
|Marius Pasca
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T7]
|[https://aclweb.org/anthology/C16-3007/]
|}

Author: Agata Savary, July 2019

Updates are welcome

Past tutorials

2020-12-08T09:01:38Z

AgataSavary:

This page belongs to the [[tutorial chair handbook]]. It summarizes data on tutorials which took place at some recent ACL, EACL, NAACL, EMNLP and COLING conferences.

==2020 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''Virtual conference link'''
|'''ACL Anthology link'''
|-
|Interpretability and Analysis in Neural NLP
|Yonatan Belinkov, Sebastian Gehrmann and Ellie Pavlick
|ACL 2020
|[https://acl2020.org/program/tutorials/#t1-interpretability-and-analysis-in-neural-nlp-cutting-edge-]
|[https://virtual.acl2020.org/tutorial_T1.html]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.1/]
|-
|Multi-modal Information Extraction from Text, Semi-structured, and Tabular Data on the Web
|Xin Luna Dong, Hannaneh Hajishirzi, Colin Lockard and Prashant Shiralkar
|ACL 2020
|[https://acl2020.org/program/tutorials/#t2-multi-modal-information-extraction-from-text-semi-structured-and-tabular-data-on-the-web-cutting-edge-]
|[https://virtual.acl2020.org/tutorial_T2.html]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.6/]
|-
|Reviewing Natural Language Processing Research
|Kevin Cohen, Karën Fort, Margot Mieskes and Aurélie Névéol
|ACL 2020
|[https://acl2020.org/program/tutorials/#t3-reviewing-natural-language-processing-research-introductory-]
|[https://virtual.acl2020.org/tutorial_T3.html]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.4/]
|-
|Stylized Text Generation: Approaches and Applications
|Lili Mou and Olga Vechtomova
|ACL 2020
|[https://acl2020.org/program/tutorials/#t4-stylized-text-generation-approaches-and-applications-cutting-edge-]
|[https://virtual.acl2020.org/tutorial_T4.html]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.5/]
|-
|Achieving Common Ground in Multi-modal Dialogue
|Malihe Alikhani and Matthew Stone
|ACL 2020
|[https://acl2020.org/program/tutorials/#t5-achieving-common-ground-in-multi-modal-dialogue-cutting-edge-]
|[https://virtual.acl2020.org/tutorial_T5.html]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.3/]
|-
|Commonsense Reasoning for Natural Language Processing
|Maarten Sap, Vered Shwartz, Antoine Bosselut, Yejin Choi and Dan Roth
|ACL 2020
|[https://acl2020.org/program/tutorials/#t6-commonsense-reasoning-for-natural-language-processing-introductory-]
|[https://virtual.acl2020.org/tutorial_T6.html]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.7/]
|-
|Integrating Ethics into the NLP Curriculum
|Emily M. Bender, Dirk Hovy and Alexandra Schofield
|ACL 2020
|[https://acl2020.org/program/tutorials/#t7-integrating-ethics-into-the-nlp-curriculum-introductory-]
|[https://virtual.acl2020.org/tutorial_T7.html]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.2/]
|-
|Open-Domain Question Answering
|Danqi Chen and Scott Wen-tau Yih
|ACL 2020
|[https://acl2020.org/program/tutorials/#t8-open-domain-question-answering-cutting-edge-]
|[https://virtual.acl2020.org/tutorial_T8.html]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.8/]
|-
|TO BE COMPLETED for AACL, EMNLP and COLING 2020
|
|
|
|
|}

==2019 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Latent Structure Models for Natural Language Processing
|André F. T. Martins, Tsvetomila Mihaylova, Nikita Nangia and Vlad Niculae
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T1]
|[https://www.aclweb.org/anthology/P19-4001/]
|-
|Graph-Based Meaning Representations: Design and Processing
|Alexander Koller, Stephan Oepen and Weiwei Sun
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T2]
|[https://www.aclweb.org/anthology/P19-4002/]
|-
|Discourse Analysis and Its Applications
|Shafiq Joty, Giuseppe Carenini, Raymond Ng and Gabriel Murray
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T3]
|[https://www.aclweb.org/anthology/P19-4003/]
|-
|Computational Analysis of Political Texts: Bridging Research Efforts Across Communities
|Goran Glavaš, Federico Nanni and Simone Paolo Ponzetto
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T4]
|[https://www.aclweb.org/anthology/P19-4004/]
|-
|Wikipedia as a Resource for Text Analysis and Retrieval
|Marius Pasca
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T5]
|[https://www.aclweb.org/anthology/P19-4005/]
|-
|Deep Bayesian Natural Language Processing
|Jen-Tzung Chien
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T6]
|[https://www.aclweb.org/anthology/P19-4006/]
|-
|Unsupervised Cross-Lingual Representation Learning
|Sebastian Ruder, Anders Søgaard and Ivan Vulić
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T7]
|[https://www.aclweb.org/anthology/P19-4007/]
|-
|Advances in Argument Mining
|Katarzyna Budzynska and Chris Reed
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T8]
|[https://www.aclweb.org/anthology/P19-4008/]
|-
|Storytelling from Structured Data and Knowledge Graphs : An NLG Perspective
|Abhijit Mishra, Anirban Laha, Karthik Sankaranarayanan, Parag Jain and Saravanan Krishnan
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T9]
|[https://www.aclweb.org/anthology/P19-4009/]
|-
|Deep Adversarial Learning for NLP
|William Yang Wang, Sameer Singh and Jiwei Li
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t1-deep-adversarial-learning-for-nlp]
|[https://aclweb.org/anthology/N19-5001/]
|-
|Deep Learning for Natural Language Inference
|Samuel Bowman and Xiaodan Zhu
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t2-deep-learning-for-natural-language-inference]
|[https://aclweb.org/anthology/N19-5002/]
|-
|Measuring and Modeling Language Change
|Jacob Eisenstein
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t3-measuring-and-modeling-language-change]
|[https://aclweb.org/anthology/N19-5003/]
|-
|Transfer Learning in Natural Language Processing
|Sebastian Ruder, Matthew Peters, Swabha Swayamdipta and Thomas Wolf
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t4-transfer-learning-in-natural-language-processing]
|[https://aclweb.org/anthology/N19-5004/]
|-
|Language Learning and Processing in People and Machines
|Aida Nematzadeh, Richard Futrell and Roger Levy
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t5-language-learning-and-processing-in-people-and-machines]
|[https://aclweb.org/anthology/N19-5005/]
|-
|Applications of Natural Language Processing in Clinical Research and Practice
|Yanshan Wang, Ahmad Tafti, Sunghwan Sohn and Rui Zhang
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t6-applications-of-natural-language-processing-in-clinical-research-and-practice]
|[https://aclweb.org/anthology/N19-5006/]
|}

==2018 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Joint models for NLP
|Yue Zhang
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Graph Formalisms for Meaning Representations
|Adam Lopez and Sorcha Gilroy
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Writing Code for NLP Research
|Matt Gardner, Mark Neumann, Joel Grus, and Nicholas Lourie
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Deep Latent Variable Models of Natural Language
|Alexander Rush, Yoon Kim, and Sam Wiseman
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Standardized Tests as benchmarks for Artificial Intelligence
|Mrinmaya Sachan, Minjoon Seo, Hannaneh Hajishirzi, and Eric Xing
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Deep Chit-Chat: Deep Learning for ChatBots
|Wei Wu and Rui Yan
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|100 Things You Always Wanted to Know about Semantics & Pragmatics But Were Afraid to Ask
|Emily M. Bender
|ACL 2018
|[https://acl2018.org/tutorials/#100-things-you-always-wante]
|[https://aclweb.org/anthology/P18-5001/]
|-
|Neural Approaches to Conversational AI
|Jianfeng Gao, Michel Galley and Lihong Li
|ACL 2018
|[https://acl2018.org/tutorials/#neural-approaches-to-conver]
|[https://aclweb.org/anthology/P18-5002/]
|-
|Variational Inference and Deep Generative Models
|Wilker Aziz and Philip Schulz
|ACL 2018
|[https://acl2018.org/tutorials/#variational-inference-and-d]
|[https://aclweb.org/anthology/P18-5003/]
|-
|Connecting Language and Vision to Actions
|Peter Anderson, Abhishek Das and Qi Wu
|ACL 2018
|[https://acl2018.org/tutorials/#connecting-language-and-vis]
|[https://aclweb.org/anthology/P18-5004/]
|-
|Beyond Multiword Expressions: Processing Idioms and Metaphors
|Valia Kordoni
|ACL 2018
|[https://acl2018.org/tutorials/#beyond-multiword-expression]
|[https://aclweb.org/anthology/P18-5005/]
|-
|Neural Semantic Parsing
|Luke Zettlemoyer, Matt Gardner, Pradeep Dasigi, Srinivasan Iyer and Alane Suhr
|ACL 2018
|[https://acl2018.org/tutorials/#neural-semantic-parsing]
|[https://aclweb.org/anthology/P18-5006/]
|-
|Deep Reinforcement Learning for NLP
|William Yang Wang, Jiwei Li and Xiaodong He
|ACL 2018
|[https://acl2018.org/tutorials/#deep-reinforcement-learning]
|[https://aclweb.org/anthology/P18-5007/]
|-
|Multi-lingual Entity Discovery and Linking
|Avirup Sil, Heng Ji, Dan Roth and Silviu-Petru Cucerzan
|ACL 2018
|[https://acl2018.org/tutorials/#multi-lingual-entity-discov]
|[https://aclweb.org/anthology/P18-5008/]
|-
|Modelling Natural Language, Programs, and their Intersection
|Graham Neubig and Miltiadis Allamanis
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6001/]
|-
|Deep Learning Approaches to Text Production
|Claire Gardent and Shashi Narayan
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6002/]
|-
|Scalable Construction and Reasoning of Massive Knowledge Bases
|Xiang Ren, Nanyun Peng and William Yang Wang
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6003/]
|-
|The interplay between lexical resources and Natural Language Processing
|Jose Camacho-Collados, Luis Espinosa Anke and Mohammad Taher Pilehvar
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6004/]
|-
|Socially Responsible NLP
|Yulia Tsvetkov, Vinodkumar Prabhakaran and Rob Voigt
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6005/]
|-
|Deep Learning for Conversational AI
|Pei-Hao Su, Nikola Mrkšić, Iñigo Casanueva, Ivan Vulić
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6006/]
|-
|NLP for Conversations: Sentiment, Summarization, and Group Dynamics
|Gabriel Murray, Giuseppe Carenini and Shafiq Joty
|COLING 2018
|[https://sites.google.com/view/nlpforconversations]
|[https://aclweb.org/anthology/C18-3001/]
|-
|Practical Parsing for Downstream Applications
|Daniel Dakota and Sandra Kübler
|COLING 2018
|[http://cl.indiana.edu/colingParsing18/]
|[https://aclweb.org/anthology/C18-3002/]
|-
|Frame Semantics across Languages: Towards a Multilingual FrameNet
|Collin Baker, Michael Ellsworth, Miriam R L Petruck and Swabha Swayamdipta
|COLING 2018
|[https://framenet.icsi.berkeley.edu/fndrupal/node/5552/]
|[https://aclweb.org/anthology/C18-3003/]
|-
|Deep Bayesian Learning and Understanding
|Jen-Tzung Chien
|COLING 2018
|[http://chien.cm.nctu.edu.tw/home/coling/]
|[https://aclweb.org/anthology/C18-3004/]
|-
|Data-Driven Text Simplification
|Sanja Štajner and Horacio Saggion
|COLING 2018
|[http://taln.upf.edu/pages/coling2018simplification/]
|[https://aclweb.org/anthology/C18-3005/]
|-
|Deep Learning for Dialogue Systems
|Yun-Nung Chen, Asli Celikyilmaz and Dilek Hakkani-Tur
|COLING 2018
|[http://deepdialogue.miulab.tw/]
|[https://aclweb.org/anthology/C18-3006/]
|}

==2017 tutorials==

EMNLP 2017 website is no longer available. There are no traces of the AMNLP 2017 tutorials, except this [https://www.facebook.com/emnlp2017/posts/well-have-7-tutorials-at-emnlp2017-concepthier-sarcasm-graphrep-srl-memaugnn-str/1860468284191872/ Facebook post].

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Universal Dependencies
|Joakim Nivre, Daniel Zeman, Filip Ginter, and Francis Tyers
|EACL 2017
|[http://universaldependencies.org/eacl17tutorial/]
|
|-
|Practical Neural Machine Translation
|Rico Sennrich and Barry Haddow
|EACL 2017
|[http://www.statmt.org/eacl2017/practical-nmt.pdf]
|
|-
|Imitation learning for structured prediction in natural language processing
|Andreas Vlachos, Gerasimos Lampouras and Sebastian Riedel
|EACL 2017
|[https://sheffieldnlp.github.io/ImitationLearningTutorialEACL2017/]
|
|-
|Word Vector Space Specialisation
|Ivan Vulić, Nikola Mrkšić, and Mohammad Taher Pilehvar
|EACL 2017
|[http://people.ds.cam.ac.uk/iv250/tutorial/wv-tutorial.pdf]
|
|-
|Integer Linear Programming formulations in Natural Language Processing
|Dan Roth and Vivek Srikumar
|EACL 2017
|[https://sheffieldnlp.github.io/ImitationLearningTutorialEACL2017/]
|
|-
|Building Multimodal Simulations for Natural Language
|James Pustejovsky and Nikhil Krishnaswamy
|EACL 2017
|[http://www.voxicon.net]
|
|-
|Natural Language Processing for Precision Medicine
|Hoifung Poon, Chris Quirk, Kristina Toutanova, and Wen-tau Yih
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5001/]
|-
|Multimodal Machine Learning
|Louis-Philippe Morency and Tadas Baltrusaitis
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5002/]
|-
|Deep Learning for Semantic Composition
|Xiaodan Zhu and Edward Grefenstette
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5003/]
|-
|Deep Learning for Dialogue Systems
|Yun-Nung Chen, Asli Celikyilmaz, and Dilek Hakkani-Tur
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5004/]
|-
|Beyond Words: Deep Learning for Multi-word Expressions and Collocations
|Valia Kordoni
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5005/]
|-
|Making Better Use of the Crowd
|Jennifer Wortman Vaughan
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5006/]
|}

==2016 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Multimodal Learning and Reasoning
|Desmond Elliott, Douwe Kiela and Angeliki Lazaridou
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index4d04.html?article_id=59]
|
|-
|NLP Approaches to Computational Argumentation
|Noam Slonim, Iryna Gurevych, Chris Reed and Benno Stein
|ACL 2016
|[http://mirror.aclweb.org/acl2016/indexfc69.html?article_id=56]
|
|-
|Computer Aided Translation
|Philipp Koehn
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index6ea2.html?article_id=62]
|
|-
|Semantic Representations of Word Senses and Concepts
|José Camacho-Collados, Ignacio Iacobacci, Roberto Navigli and Mohammad Taher Pilehvar
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index6b9c.html?article_id=58]
|
|-
|Neural Machine Translation
|Thang Luong, Kyunghyun Cho and Christopher D. Manning
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index2e51.html?article_id=55]
|
|-
|Game Theory and Natural Language: Origin, Evolution and Processing
|Rocco Tripodi and Marcello Pelillo
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index56cf.html?article_id=57]
|
|-
|Understanding Short Texts
|Zhongyuan Wang and Haixun Wang
|ACL 2016
|[http://mirror.aclweb.org/acl2016/indexedf3.html?article_id=60]
|
|-
|MetaNet: Repository, Identification System, and Applications
|Miriam R L Petruck and Ellen K Dodge
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index6a61.html?article_id=61]
|
|-
|English Resource Semantics
|Dan Flickinger, Emily M. Bender, and Woodley Packard
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t1.html]
|[https://aclweb.org/anthology/N16-4001/]
|-
|Multilingual Multimodal Language Processing Using Neural Networks
|Mitesh M Khapra and Sarath Chandar
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t2.html]
|[https://aclweb.org/anthology/N16-4002/]
|-
|Question Answering with Knowledge Base, Web and Beyond
|Scott Wen-tau Yih & Hao Ma
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t3.html]
|[https://aclweb.org/anthology/N16-4003/]
|-
|Recent Progress in Deep Learning for NLP
|Zhengdong Lu and Hang Li
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t4.html]
|[https://aclweb.org/anthology/N16-4004/]
|-
|Scalable Statistical Relational Learning for NLP
|William Yang Wang and William W. Cohen
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t5.html]
|[https://aclweb.org/anthology/N16-4005/]
|-
|Statistical Machine Translation between Related Languages
|Pushpak Bhattacharyya, Mitesh Khapra, and Anoop Kunchukuttan
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t6.html]
|[https://aclweb.org/anthology/N16-4006/]
|-
|Practical Neural Networks for NLP: From Theory to Code
|Chris Dyer, Yoav Goldberg and Graham Neubig
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#practical]
|
|-
|Advanced Markov Logic Techniques for Scalable Joint Inference in NLP
|Deepak Venugopal, Vibhav Gogate and Vincent Ng
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#advanced]
|
|-
|Lifelong Machine Learning for Natural Language Processing
|Zhiyuan Chen and Bing Liu
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#lifelong]
|
|-
|Neural Networks for Sentiment Analysis
|Yue Zhang and Duy Tin Vo
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#neural]
|
|-
|Continuous Vector Spaces for Cross-language NLP Applications
|Rafael E. Banchs
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#continuous]
|
|-
|Methods and Theories for Large-scale Structured Prediction
|Xu Sun and Yansong Feng
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#methods]
|
|-
|Compositional Distributional Models of Meaning
|Mehrnoosh Sadrzadeh and Dimitri Kartsaklis
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T1/]
|[https://aclweb.org/anthology/C16-3001/]
|-
|Chinese Textual Sentiment Analysis: Datasets, Resources and Tools
|Lun-Wei Ku and Wei-Fan Chen
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T2]
|[https://aclweb.org/anthology/C16-3002/]
|-
|Natural Language Processing for Intelligent Access to Scientific Information
|Horacio Saggion and Francesco Ronzano
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T3]
|[https://aclweb.org/anthology/C16-3003/]
|-
|Quality Estimation for Language Output Applications
|Carolina Scarton, Gustavo Henrique Paetzold, and Lucia Specia
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T4]
|[https://aclweb.org/anthology/C16-3004/]
|-
|Translationese: Between Human and Machine Translation
|Shuly Wintner
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T5/]
|[https://aclweb.org/anthology/C16-3005/]
|-
|Succinct Data Structures for NLP-at-Scale
|Matthias Petri and Trevor Cohn
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T6/]
|[https://aclweb.org/anthology/C16-3006/]
|-
|The Role of Wikipedia in Text Analysis and Retrieval
|Marius Pasca
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T7]
|[https://aclweb.org/anthology/C16-3007/]
|}

Author: Agata Savary, July 2019

Updates are welcome

Past tutorials

2020-12-08T09:00:41Z

AgataSavary: /* 2020 tutorials */

This page belongs to the [[tutorial chair handbook]]. It summarizes data on tutorials which took place at some recent ACL, EACL, NAACL, EMNLP and COLING conferences.

==2020 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''Virtual conference link'''
|'''ACL Anthology link'''
|-
|Interpretability and Analysis in Neural NLP
|Yonatan Belinkov, Sebastian Gehrmann and Ellie Pavlick
|ACL 2020
|[https://acl2020.org/program/tutorials/#t1-interpretability-and-analysis-in-neural-nlp-cutting-edge-]
|[https://virtual.acl2020.org/tutorial_T1.html]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.1/]
|-
|Multi-modal Information Extraction from Text, Semi-structured, and Tabular Data on the Web
|Xin Luna Dong, Hannaneh Hajishirzi, Colin Lockard and Prashant Shiralkar
|ACL 2020
|[https://acl2020.org/program/tutorials/#t2-multi-modal-information-extraction-from-text-semi-structured-and-tabular-data-on-the-web-cutting-edge-]
|[https://virtual.acl2020.org/tutorial_T2.html]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.6/]
|-
|Reviewing Natural Language Processing Research
|Kevin Cohen, Karën Fort, Margot Mieskes and Aurélie Névéol
|ACL 2020
|[https://acl2020.org/program/tutorials/#t3-reviewing-natural-language-processing-research-introductory-]
|[https://virtual.acl2020.org/tutorial_T3.html]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.4/]
|-
|Stylized Text Generation: Approaches and Applications
|Lili Mou and Olga Vechtomova
|ACL 2020
|[https://acl2020.org/program/tutorials/#t4-stylized-text-generation-approaches-and-applications-cutting-edge-]
|[https://virtual.acl2020.org/tutorial_T4.html]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.5/]
|-
|Achieving Common Ground in Multi-modal Dialogue
|Malihe Alikhani and Matthew Stone
|ACL 2020
|[https://acl2020.org/program/tutorials/#t5-achieving-common-ground-in-multi-modal-dialogue-cutting-edge-]
|[https://virtual.acl2020.org/tutorial_T5.html]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.3/]
|-
|Commonsense Reasoning for Natural Language Processing
|Maarten Sap, Vered Shwartz, Antoine Bosselut, Yejin Choi and Dan Roth
|ACL 2020
|[https://acl2020.org/program/tutorials/#t6-commonsense-reasoning-for-natural-language-processing-introductory-]
|[https://virtual.acl2020.org/tutorial_T6.html]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.7/]
|-
|Integrating Ethics into the NLP Curriculum
|Emily M. Bender, Dirk Hovy and Alexandra Schofield
|ACL 2020
|[https://acl2020.org/program/tutorials/#t7-integrating-ethics-into-the-nlp-curriculum-introductory-]
|[https://virtual.acl2020.org/tutorial_T7.html]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.2/]
|-
|Open-Domain Question Answering
|Danqi Chen and Scott Wen-tau Yih
|ACL 2020
|[https://acl2020.org/program/tutorials/#t8-open-domain-question-answering-cutting-edge-]
|[https://virtual.acl2020.org/tutorial_T8.html]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.8/]
|-
|TO BE COMPLETED for AACL, EMNLP and COLING 2020
|
|
|
|}

==2019 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Latent Structure Models for Natural Language Processing
|André F. T. Martins, Tsvetomila Mihaylova, Nikita Nangia and Vlad Niculae
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T1]
|[https://www.aclweb.org/anthology/P19-4001/]
|-
|Graph-Based Meaning Representations: Design and Processing
|Alexander Koller, Stephan Oepen and Weiwei Sun
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T2]
|[https://www.aclweb.org/anthology/P19-4002/]
|-
|Discourse Analysis and Its Applications
|Shafiq Joty, Giuseppe Carenini, Raymond Ng and Gabriel Murray
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T3]
|[https://www.aclweb.org/anthology/P19-4003/]
|-
|Computational Analysis of Political Texts: Bridging Research Efforts Across Communities
|Goran Glavaš, Federico Nanni and Simone Paolo Ponzetto
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T4]
|[https://www.aclweb.org/anthology/P19-4004/]
|-
|Wikipedia as a Resource for Text Analysis and Retrieval
|Marius Pasca
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T5]
|[https://www.aclweb.org/anthology/P19-4005/]
|-
|Deep Bayesian Natural Language Processing
|Jen-Tzung Chien
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T6]
|[https://www.aclweb.org/anthology/P19-4006/]
|-
|Unsupervised Cross-Lingual Representation Learning
|Sebastian Ruder, Anders Søgaard and Ivan Vulić
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T7]
|[https://www.aclweb.org/anthology/P19-4007/]
|-
|Advances in Argument Mining
|Katarzyna Budzynska and Chris Reed
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T8]
|[https://www.aclweb.org/anthology/P19-4008/]
|-
|Storytelling from Structured Data and Knowledge Graphs : An NLG Perspective
|Abhijit Mishra, Anirban Laha, Karthik Sankaranarayanan, Parag Jain and Saravanan Krishnan
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T9]
|[https://www.aclweb.org/anthology/P19-4009/]
|-
|Deep Adversarial Learning for NLP
|William Yang Wang, Sameer Singh and Jiwei Li
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t1-deep-adversarial-learning-for-nlp]
|[https://aclweb.org/anthology/N19-5001/]
|-
|Deep Learning for Natural Language Inference
|Samuel Bowman and Xiaodan Zhu
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t2-deep-learning-for-natural-language-inference]
|[https://aclweb.org/anthology/N19-5002/]
|-
|Measuring and Modeling Language Change
|Jacob Eisenstein
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t3-measuring-and-modeling-language-change]
|[https://aclweb.org/anthology/N19-5003/]
|-
|Transfer Learning in Natural Language Processing
|Sebastian Ruder, Matthew Peters, Swabha Swayamdipta and Thomas Wolf
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t4-transfer-learning-in-natural-language-processing]
|[https://aclweb.org/anthology/N19-5004/]
|-
|Language Learning and Processing in People and Machines
|Aida Nematzadeh, Richard Futrell and Roger Levy
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t5-language-learning-and-processing-in-people-and-machines]
|[https://aclweb.org/anthology/N19-5005/]
|-
|Applications of Natural Language Processing in Clinical Research and Practice
|Yanshan Wang, Ahmad Tafti, Sunghwan Sohn and Rui Zhang
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t6-applications-of-natural-language-processing-in-clinical-research-and-practice]
|[https://aclweb.org/anthology/N19-5006/]
|}

==2018 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Joint models for NLP
|Yue Zhang
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Graph Formalisms for Meaning Representations
|Adam Lopez and Sorcha Gilroy
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Writing Code for NLP Research
|Matt Gardner, Mark Neumann, Joel Grus, and Nicholas Lourie
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Deep Latent Variable Models of Natural Language
|Alexander Rush, Yoon Kim, and Sam Wiseman
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Standardized Tests as benchmarks for Artificial Intelligence
|Mrinmaya Sachan, Minjoon Seo, Hannaneh Hajishirzi, and Eric Xing
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Deep Chit-Chat: Deep Learning for ChatBots
|Wei Wu and Rui Yan
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|100 Things You Always Wanted to Know about Semantics & Pragmatics But Were Afraid to Ask
|Emily M. Bender
|ACL 2018
|[https://acl2018.org/tutorials/#100-things-you-always-wante]
|[https://aclweb.org/anthology/P18-5001/]
|-
|Neural Approaches to Conversational AI
|Jianfeng Gao, Michel Galley and Lihong Li
|ACL 2018
|[https://acl2018.org/tutorials/#neural-approaches-to-conver]
|[https://aclweb.org/anthology/P18-5002/]
|-
|Variational Inference and Deep Generative Models
|Wilker Aziz and Philip Schulz
|ACL 2018
|[https://acl2018.org/tutorials/#variational-inference-and-d]
|[https://aclweb.org/anthology/P18-5003/]
|-
|Connecting Language and Vision to Actions
|Peter Anderson, Abhishek Das and Qi Wu
|ACL 2018
|[https://acl2018.org/tutorials/#connecting-language-and-vis]
|[https://aclweb.org/anthology/P18-5004/]
|-
|Beyond Multiword Expressions: Processing Idioms and Metaphors
|Valia Kordoni
|ACL 2018
|[https://acl2018.org/tutorials/#beyond-multiword-expression]
|[https://aclweb.org/anthology/P18-5005/]
|-
|Neural Semantic Parsing
|Luke Zettlemoyer, Matt Gardner, Pradeep Dasigi, Srinivasan Iyer and Alane Suhr
|ACL 2018
|[https://acl2018.org/tutorials/#neural-semantic-parsing]
|[https://aclweb.org/anthology/P18-5006/]
|-
|Deep Reinforcement Learning for NLP
|William Yang Wang, Jiwei Li and Xiaodong He
|ACL 2018
|[https://acl2018.org/tutorials/#deep-reinforcement-learning]
|[https://aclweb.org/anthology/P18-5007/]
|-
|Multi-lingual Entity Discovery and Linking
|Avirup Sil, Heng Ji, Dan Roth and Silviu-Petru Cucerzan
|ACL 2018
|[https://acl2018.org/tutorials/#multi-lingual-entity-discov]
|[https://aclweb.org/anthology/P18-5008/]
|-
|Modelling Natural Language, Programs, and their Intersection
|Graham Neubig and Miltiadis Allamanis
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6001/]
|-
|Deep Learning Approaches to Text Production
|Claire Gardent and Shashi Narayan
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6002/]
|-
|Scalable Construction and Reasoning of Massive Knowledge Bases
|Xiang Ren, Nanyun Peng and William Yang Wang
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6003/]
|-
|The interplay between lexical resources and Natural Language Processing
|Jose Camacho-Collados, Luis Espinosa Anke and Mohammad Taher Pilehvar
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6004/]
|-
|Socially Responsible NLP
|Yulia Tsvetkov, Vinodkumar Prabhakaran and Rob Voigt
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6005/]
|-
|Deep Learning for Conversational AI
|Pei-Hao Su, Nikola Mrkšić, Iñigo Casanueva, Ivan Vulić
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6006/]
|-
|NLP for Conversations: Sentiment, Summarization, and Group Dynamics
|Gabriel Murray, Giuseppe Carenini and Shafiq Joty
|COLING 2018
|[https://sites.google.com/view/nlpforconversations]
|[https://aclweb.org/anthology/C18-3001/]
|-
|Practical Parsing for Downstream Applications
|Daniel Dakota and Sandra Kübler
|COLING 2018
|[http://cl.indiana.edu/colingParsing18/]
|[https://aclweb.org/anthology/C18-3002/]
|-
|Frame Semantics across Languages: Towards a Multilingual FrameNet
|Collin Baker, Michael Ellsworth, Miriam R L Petruck and Swabha Swayamdipta
|COLING 2018
|[https://framenet.icsi.berkeley.edu/fndrupal/node/5552/]
|[https://aclweb.org/anthology/C18-3003/]
|-
|Deep Bayesian Learning and Understanding
|Jen-Tzung Chien
|COLING 2018
|[http://chien.cm.nctu.edu.tw/home/coling/]
|[https://aclweb.org/anthology/C18-3004/]
|-
|Data-Driven Text Simplification
|Sanja Štajner and Horacio Saggion
|COLING 2018
|[http://taln.upf.edu/pages/coling2018simplification/]
|[https://aclweb.org/anthology/C18-3005/]
|-
|Deep Learning for Dialogue Systems
|Yun-Nung Chen, Asli Celikyilmaz and Dilek Hakkani-Tur
|COLING 2018
|[http://deepdialogue.miulab.tw/]
|[https://aclweb.org/anthology/C18-3006/]
|}

==2017 tutorials==

EMNLP 2017 website is no longer available. There are no traces of the AMNLP 2017 tutorials, except this [https://www.facebook.com/emnlp2017/posts/well-have-7-tutorials-at-emnlp2017-concepthier-sarcasm-graphrep-srl-memaugnn-str/1860468284191872/ Facebook post].

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Universal Dependencies
|Joakim Nivre, Daniel Zeman, Filip Ginter, and Francis Tyers
|EACL 2017
|[http://universaldependencies.org/eacl17tutorial/]
|
|-
|Practical Neural Machine Translation
|Rico Sennrich and Barry Haddow
|EACL 2017
|[http://www.statmt.org/eacl2017/practical-nmt.pdf]
|
|-
|Imitation learning for structured prediction in natural language processing
|Andreas Vlachos, Gerasimos Lampouras and Sebastian Riedel
|EACL 2017
|[https://sheffieldnlp.github.io/ImitationLearningTutorialEACL2017/]
|
|-
|Word Vector Space Specialisation
|Ivan Vulić, Nikola Mrkšić, and Mohammad Taher Pilehvar
|EACL 2017
|[http://people.ds.cam.ac.uk/iv250/tutorial/wv-tutorial.pdf]
|
|-
|Integer Linear Programming formulations in Natural Language Processing
|Dan Roth and Vivek Srikumar
|EACL 2017
|[https://sheffieldnlp.github.io/ImitationLearningTutorialEACL2017/]
|
|-
|Building Multimodal Simulations for Natural Language
|James Pustejovsky and Nikhil Krishnaswamy
|EACL 2017
|[http://www.voxicon.net]
|
|-
|Natural Language Processing for Precision Medicine
|Hoifung Poon, Chris Quirk, Kristina Toutanova, and Wen-tau Yih
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5001/]
|-
|Multimodal Machine Learning
|Louis-Philippe Morency and Tadas Baltrusaitis
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5002/]
|-
|Deep Learning for Semantic Composition
|Xiaodan Zhu and Edward Grefenstette
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5003/]
|-
|Deep Learning for Dialogue Systems
|Yun-Nung Chen, Asli Celikyilmaz, and Dilek Hakkani-Tur
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5004/]
|-
|Beyond Words: Deep Learning for Multi-word Expressions and Collocations
|Valia Kordoni
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5005/]
|-
|Making Better Use of the Crowd
|Jennifer Wortman Vaughan
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5006/]
|}

==2016 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Multimodal Learning and Reasoning
|Desmond Elliott, Douwe Kiela and Angeliki Lazaridou
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index4d04.html?article_id=59]
|
|-
|NLP Approaches to Computational Argumentation
|Noam Slonim, Iryna Gurevych, Chris Reed and Benno Stein
|ACL 2016
|[http://mirror.aclweb.org/acl2016/indexfc69.html?article_id=56]
|
|-
|Computer Aided Translation
|Philipp Koehn
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index6ea2.html?article_id=62]
|
|-
|Semantic Representations of Word Senses and Concepts
|José Camacho-Collados, Ignacio Iacobacci, Roberto Navigli and Mohammad Taher Pilehvar
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index6b9c.html?article_id=58]
|
|-
|Neural Machine Translation
|Thang Luong, Kyunghyun Cho and Christopher D. Manning
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index2e51.html?article_id=55]
|
|-
|Game Theory and Natural Language: Origin, Evolution and Processing
|Rocco Tripodi and Marcello Pelillo
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index56cf.html?article_id=57]
|
|-
|Understanding Short Texts
|Zhongyuan Wang and Haixun Wang
|ACL 2016
|[http://mirror.aclweb.org/acl2016/indexedf3.html?article_id=60]
|
|-
|MetaNet: Repository, Identification System, and Applications
|Miriam R L Petruck and Ellen K Dodge
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index6a61.html?article_id=61]
|
|-
|English Resource Semantics
|Dan Flickinger, Emily M. Bender, and Woodley Packard
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t1.html]
|[https://aclweb.org/anthology/N16-4001/]
|-
|Multilingual Multimodal Language Processing Using Neural Networks
|Mitesh M Khapra and Sarath Chandar
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t2.html]
|[https://aclweb.org/anthology/N16-4002/]
|-
|Question Answering with Knowledge Base, Web and Beyond
|Scott Wen-tau Yih & Hao Ma
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t3.html]
|[https://aclweb.org/anthology/N16-4003/]
|-
|Recent Progress in Deep Learning for NLP
|Zhengdong Lu and Hang Li
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t4.html]
|[https://aclweb.org/anthology/N16-4004/]
|-
|Scalable Statistical Relational Learning for NLP
|William Yang Wang and William W. Cohen
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t5.html]
|[https://aclweb.org/anthology/N16-4005/]
|-
|Statistical Machine Translation between Related Languages
|Pushpak Bhattacharyya, Mitesh Khapra, and Anoop Kunchukuttan
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t6.html]
|[https://aclweb.org/anthology/N16-4006/]
|-
|Practical Neural Networks for NLP: From Theory to Code
|Chris Dyer, Yoav Goldberg and Graham Neubig
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#practical]
|
|-
|Advanced Markov Logic Techniques for Scalable Joint Inference in NLP
|Deepak Venugopal, Vibhav Gogate and Vincent Ng
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#advanced]
|
|-
|Lifelong Machine Learning for Natural Language Processing
|Zhiyuan Chen and Bing Liu
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#lifelong]
|
|-
|Neural Networks for Sentiment Analysis
|Yue Zhang and Duy Tin Vo
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#neural]
|
|-
|Continuous Vector Spaces for Cross-language NLP Applications
|Rafael E. Banchs
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#continuous]
|
|-
|Methods and Theories for Large-scale Structured Prediction
|Xu Sun and Yansong Feng
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#methods]
|
|-
|Compositional Distributional Models of Meaning
|Mehrnoosh Sadrzadeh and Dimitri Kartsaklis
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T1/]
|[https://aclweb.org/anthology/C16-3001/]
|-
|Chinese Textual Sentiment Analysis: Datasets, Resources and Tools
|Lun-Wei Ku and Wei-Fan Chen
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T2]
|[https://aclweb.org/anthology/C16-3002/]
|-
|Natural Language Processing for Intelligent Access to Scientific Information
|Horacio Saggion and Francesco Ronzano
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T3]
|[https://aclweb.org/anthology/C16-3003/]
|-
|Quality Estimation for Language Output Applications
|Carolina Scarton, Gustavo Henrique Paetzold, and Lucia Specia
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T4]
|[https://aclweb.org/anthology/C16-3004/]
|-
|Translationese: Between Human and Machine Translation
|Shuly Wintner
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T5/]
|[https://aclweb.org/anthology/C16-3005/]
|-
|Succinct Data Structures for NLP-at-Scale
|Matthias Petri and Trevor Cohn
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T6/]
|[https://aclweb.org/anthology/C16-3006/]
|-
|The Role of Wikipedia in Text Analysis and Retrieval
|Marius Pasca
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T7]
|[https://aclweb.org/anthology/C16-3007/]
|}

Author: Agata Savary, July 2019

Updates are welcome

2020Q3 Reports: Tutorial Chairs

2020-07-24T08:32:34Z

AgataSavary: /* Virtual Conference Organization */

Tutorial Chairs:

<blockquote>Agata Savary, University of Tours, France

Yue Zhang, Westlake University, Hangzhou, China
</blockquote>

=== Preparatory Work, Call, Reviewing and Decision-Making ===
The call, submission, reviewing and selection of tutorials was coordinated jointly for 4 conferences: ACL, AACL-IJCNLP, COLING and EMNLP.

Before drafting the call, we collected lists of tutorials offered within the past 4 years. We analysed previous calls for tutorials and reports from tutorial chairs (from [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs 2016], [https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs 2017], [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs 2018] and [http://aclweb.org/adminwiki/index.php?title=2019Q1_Reports:_ACL_2019 2019]). We consulted previous tutorial chairs with a questionnaire including questions about: the number of submissions, encouraging submissions on specific topics or from specific lecturers, the review procedure, the evaluation criteria, the post-tutorial availability of the slides/codes, and lessons learned from tutorial coordination. We also discussed the publication of slides and video recordings from future tutorials with the persons in charge of the ACL Anthology. As a result of these steps, we created two new sections for the [https://www.aclweb.org/adminwiki/index.php?title=Conference_Handbook ACL Conference Handbook] (future chairs might consider updating these documents yearly):

* the list of [https://www.aclweb.org/adminwiki/index.php?title=Past_tutorials past tutorials] at ACL, COLING, EACL, EMNLP, and NAACL in 2016-2019
* a [https://www.aclweb.org/adminwiki/index.php?title=Tutorial_chair_handbook tutorial chair handbook]

The final [https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020 call] differs from previous calls in several aspects: (i) the expectations about tutorial proposals were made clearer, (ii) following the central ACL decision, the teachers' payment policy was replaced by a fee-waiving policy, (iii) the required submission details include two new items: diversity considerations and agreement for open access publication of slides, codes, data and video recordings, (iv) the evaluation criteria (see below) are announced.

We recruited a review committee of 19 members, including the 8 tutorial chairs and 11 external members selected for their large understanding of the NLP domain and a good experience in reviewing and/or tutorial teaching:

'''Review Committee'''

* Timothy Baldwin (University of Melbourne, Australia) - AACL-IJCNLP 2020 tutorial chair
* Daniel Beck (University of Melbourne, Australia) - COLING 2020 tutorial chair
* Emily M. Bender (University of Washington, WA, USA)
* Erik Cambria (Nanyang Technological University, Singapore)
* Gaël Dias (University of Caen Normandie, France)
* Stefan Evert (Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany)
* Yang Liu (Tsinghua University, Beijing, China)
* Agata Savary (University of Tours, France) - ACL 2020 tutorial chair
* João Sedoc (Johns Hopkins University, Baltimore, MD, USA)
* Lucia Specia (Sheffield University, UK) - COLING 2020 tutorial chair
* Xu Sun (Peking University, China)
* Yulia Tsvetkov (Carnegie Mellon University, Pittsburgh, PA, USA)
* Benjamin Van Durme (Johns Hopkins University, Baltimore, MD, USA) - EMNLP 2020 tutorial chair
* Aline Villavicencio (University of Sheffield, UK and Federal University of Rio Grande do Sul, Brazil) - EMNLP 2020 tutorial chair
* Taro Watanabe (Google, Inc., Tokyo, Japan)
* Aaron Steven White (University of Rochester, NY, USA)
* Fei Xia (University of Washington, WA, USA) - AACL-IJCNLP 2020 tutorial chair
* Yue Zhang (Westlake University, Hangzhou, China) - ACL 2020 tutorial chair
* Meishan Zhang (Tianjin University, China)

In total, we received 43 submissions for the 4 conferences. Each reviewer was assigned 6-7 proposals and each proposal received 3 reviews. The selection criteria included: clarity and preparedness, novelty or timely character of the topic, lecturers' experience, likely audience interest, open access of the teaching material, diversity aspects (multilingualism, gender, age and country of the lecturers), and compatibility with the preferred venues. We accepted 31 proposals.

The decision making was handled via an online meeting of the 8 tutorial chairs. In particular, the selection of tutorials for each conference was done via the expression of interest of the tutorial chairs on a round-robin basis. Some slight adjustments were also performed after the meeting to better fit the authors' preferences. In total, 8, 8, 8 and 7 proposals were selected for ACL, AACL-IJCNLP, COLING and EMNLP, respectively. Upon the announcement the results, 2 of the proposals accepted for AACL-IJCNLP were withdrawn.

The submission, review, selection and collection of final descriptions for all tutorials was handled via a dedicated SoftConf space, shared by the 4 coordinating conferences. After the selection of proposals, a separate track was created on SoftConf for each conference. The final submission page (one per conference) was set up so as to collect all the necessary data including notably: the tutorial slides, URLs for course material (if any), printable material (if any) and agreement for open access publication.

===Tutorials Selected for ACL 2020===

The final selection for ACL 2020 consists of the following 8 tutorials of 3 hours each (each of them had ACL as the preferred or the second preferred venue):

'''T1: Interpretability and Analysis in Neural NLP''' (cutting-edge) 
Yonatan Belinkov, Sebastian Gehrmann and Ellie Pavlick 
While deep learning has transformed the NLP field and impacted the larger computational linguistics community, the rise of neural networks is stained by their opaque nature: It is challenging to interpret the inner workings of neural network models, and explicate their behavior. Therefore, in the last few years, an increasingly large body of work has been devoted to the analysis and interpretation of neural network models in NLP. 
This body of work is so far lacking a common framework and methodology. Moreover, approaching the analysis of modern neural networks can be difficult for newcomers to the field. This tutorial aims to fill this gap and introduce the nascent field of interpretability and analysis of neural networks in NLP. 
The tutorial covers the main lines of analysis work, such as probing classifier, behavior studies and test suites, psycholinguistic methods, visualizations, adversarial examples, and other methods. We highlight not only the most commonly applied analysis methods, but also the specific limitations and shortcomings of current approaches, in order to inform participants where to focus future efforts.

'''T2: Multi-modal Information Extraction from Text, Semi-structured, and Tabular Data on the Web''' (cutting-edge) 
Xin Luna Dong, Hannaneh Hajishirzi, Colin Lockard and Prashant Shiralkar 
The World Wide Web contains vast quantities of textual information in several forms: unstructured text, template-based semi-structured webpages (which present data in key-value pairs and lists), and tables. Methods for extracting information from these sources and converting it to a structured form have been a target of research from the natural language processing (NLP), data mining, and database communities. While these researchers have largely separated extraction from web data into different problems based on the modality of the data, they have faced similar problems such as learning with limited labeled data, defining (or avoiding defining) ontologies, making use of prior knowledge, and scaling solutions to deal with the size of the Web. 
In this tutorial we take a holistic view toward information extraction, exploring the commonalities in the challenges and solutions developed to address these different forms of text. We will explore the approaches targeted at unstructured text that largely rely on learning syntactic or semantic textual patterns, approaches targeted at semi-structured documents that learn to identify structural patterns in the template, and approaches targeting web tables which rely heavily on entity linking and type information. 
While these different data modalities have largely been considered separately in the past, recent research has started taking a more inclusive approach toward textual extraction, in which the multiple signals offered by textual, layout, and visual clues are combined into a single extraction model made possible by new deep learning approaches. At the same time, trends within purely textual extraction have shifted toward full-document understanding rather than considering sentences as independent units. With this in mind, it is worth considering the information extraction problem as a whole to motivate solutions that harness textual semantics along with visual and semi-structured layout information. We will discuss these approaches and suggest avenues for future work.

'''T3: Reviewing Natural Language Processing Research''' (introductory) 
Kevin Cohen, Karën Fort, Margot Mieskes and Aurélie Névéol 
As the demand for reviewing grows, so must the pool of reviewers. As the [http://www.livecongress.it/aol/indexSA.php?id=E2EAED7D&ticket= survey] presented by Graham Neubig at the 2019 ACL showed, a considerable number of reviewers are junior researchers, who might lack the experience and expertise necessary for high-quality reviews. Some of them might not have the environment or lack opportunities that allow them to learn the skills necessary. A tutorial on reviewing for the NLP community might increase reviewers’ confidence, as well as the quality of the reviews. This introductory tutorial will cover the goals, processes, and evaluation of reviewing research papers in natural language processing.

'''T4: Stylized Text Generation: Approaches and Applications''' (cutting-edge) 
Lili Mou and Olga Vechtomova 
Text generation has played an important role in various applications of natural language processing (NLP), and kn recent studies, researchers are paying increasing attention to modeling and manipulating the style of the generation text, which we call stylized text generation. In this tutorial, we will provide a comprehensive literature review in this direction. We start from the definition of style and different settings of stylized text generation, illustrated with various applications. Then, we present different settings of stylized generation, such as parallel supervised, style label-supervised, and unsupervised. In each setting, we delve deep into machine learning methods, including embedding learning techniques to represent style}, adversarial learning and reinforcement learning with cycle consistency to match content but to distinguish different styles. We also introduce current approaches of evaluating stylized text generation systems. We conclude our tutorial by presenting the challenges of stylized text generation and discussing future directions, such as small-data training, non-categorical style modeling, and a generalized scope of style transfer (e.g., controlling the syntax as a style).

'''T5: Achieving Common Ground in Multi-modal Dialogue''' (cutting-edge) 
Malihe Alikhani and Matthew Stone 
All communication aims at achieving common ground (grounding): interlocutors can work together effectively only with mutual beliefs about what the state of the world is, about what their goals are, and about how they plan to make their goals a reality. Computational dialogue research offers some classic results on grouding, which unfortunately offer scant guidance to the design of grounding modules and behaviors in cutting-edge systems. In this tutorial, we focus on three main topic areas: 1) grounding in human-human communication; 2) grounding in dialogue systems; and 3) grounding in multi-modal interactive systems, including image-oriented conversations and human-robot interactions. We highlight a number of achievements of recent computational research in coordinating complex content, show how these results lead to rich and challenging opportunities for doing grounding in more flexible and powerful ways, and canvass relevant insights from the literature on human--human conversation. We expect that the tutorial will be of interest to researchers in dialogue systems, computational semantics and cognitive modeling, and hope that it will catalyze research and system building that more directly explores the creative, strategic ways conversational agents might be able to seek and offer evidence about their understanding of their interlocutors.

'''T6: Commonsense Reasoning for Natural Language Processing''' (introductory) 
Maarten Sap, Vered Shwartz, Antoine Bosselut, Dan Roth and Yejin Choi 
In our tutorial, we (1) outline the various types of commonsense (e.g., physical, social), and (2) discuss techniques to gather and represent commonsense knowledge, while highlighting the challenges specific to this type of knowledge (e.g., reporting bias). We will then (3) discuss the types of commonsense knowledge captured by modern NLP systems (e.g., large pretrained language models), and (4) present ways to measure systems' commonsense reasoning abilities. We finish with (5) a discussion of various ways in which commonsense reasoning can be used to improve performance on NLP tasks, exemplified by an (6) interactive session on integrating commonsense into a downstream task.

'''T7: Integrating Ethics into the NLP Curriculum''' (introductory) 
Emily M. Bender, Dirk Hovy and Alexandra Schofield 
Our goal in this tutorial is to empower NLP researchers and practitioners with tools and resources to teach others about how to ethically apply NLP techniques. Our tutorial will present both high-level strategies for developing an ethics-oriented curriculum, based on experience and best practices, as well as specific sample exercises that can be brought to a classroom. We plan to make this a highly interactive work session culminating in a shared online resource page that pools lesson plans, assignments, exercise ideas, reading suggestions, and ideas from the attendees. We consider three primary topics with our session that frequently underlie ethical issues in NLP research: Dual use, bias and privacy. 
In this setting, a key lesson is that there is no single approach to ethical NLP: each project requires thoughtful consideration about what steps can be taken to best support people affected by that project. However, we can learn (and teach) what kinds of issues to be aware of and what kinds of strategies are available for mitigating harm. To teach this process, we apply and promote interactive exercises that provide an opportunity to ideate, discuss, and reflect. We plan to facilitate this in a way that encourages positive discussion, emphasizing the creation of ideas for the future instead of negative opinions of previous work.

'''T8: Recent Advances in Open-Domain Question Answering''' (cutting-edge) 
Danqi Chen and Scott Wen-tau Yih 
Open-domain (textual) question answering (QA), the task of finding answers to open-domain questions by searching a large collection of documents, has been a long-standing problem in NLP, information retrieval (IR) and related fields (Voorhees et al., 1999; Moldovan et al., 2000; Brill et al.,2002; Ferrucci et al., 2010). Traditional QA systems were usually constructed as a pipeline, consisting of many different components such as question processing, document/passage retrieval and answer processing. With the rapid development of neural reading comprehension (Chen, 2018), modern open-domain QA systems have been restructured by combining traditional IR techniques and neural reading comprehension models (Chen et al., 2017; Yang et al., 2019) or even implemented in a fully end-to-end fashion (Lee et al., 2019; Seo et al., 2019). While the system architecture has been drastically simplified, two technical challenges remain critical:(1) “Retriever”: finding documents that (might)contain an answer from a large collection of documents; (2) “Reader”: finding the answer in a given paragraph or a document. 
In this tutorial, we aim to provide a comprehensive and coherent overview of recent advances in this line of research. We will start by first giving a brief historical background of open-domain question answering, discussing the basic setup and core technical challenges of the research problem.The focus will then shift to modern techniques and resources proposed for open-domain QA, including the basics of latest neural reading comprehension systems, new datasets and models. The scope will also be broadened to cover the information retrieval component on how to effectively identify passages relevant to the questions. Moreover, in-depth discussions will be given on the use of traditional / neural IR modules, as well as the trade-offs between modular design and end-to-end training. If time permits, we also plan to discuss some hybrid approaches for answering questions using both text and large knowledge bases (e.g. (Sun et al., 2018)) and give a critical review on how structured data complements the information from unstructured text. 
At the end of our tutorial, we will discuss some important questions, including (1) How much progress have we made compared to the QA systems developed in the last decade?(2) What are the main challenges and limitations of cur-rent approaches? (3) How to trade off the efficiency (computational time and memory requirements) and accuracy in the deep learning era? We hope that our tutorial will not only serve as a useful resource for the audience to efficiently acquire the up-to-date knowledge, but also provide new perspectives to stimulate the advances of open-domain QA research in the next phase.

===Virtual Conference Organization===

In April the whole conference was decided to be held in virtual. We carried on the preparations jointly with the tutorial instructors, the virtual infrastructure chairs, the website chairs and the SlidesLive team to set up the virtual conference infrastructure, and to publicize it for each tutorial.

We first made a decision on the time slots for tutorial presentations. Five major time zones were considered, including US west coast, US east coast, central Europe, China/Asia and Australia (India was also considered). Our aim was to offer at least two convenient time slots (i.e. within regular working hours) for each of these zones.

The final time slots include (Jul 5 Pacific Time):

* slot 1 – 7:00 PM (-1D) to 10:30 PM (-1D)
* slot 2 – 12:00 AM to 3:30 AM
* slot 3 – 6:00 AM to 9:30 AM
* slot 4 – 10:30 AM to 2:00 PM
* slot 5 – 3:00 PM to 6:30 PM

We asked the tutorial teachers to indicate their preferences among the time slots, and – if possible – to agree to give their tutorial twice, so as to increase accessibility. Our final decisions on slot assignments traded off the teacher preferences and slot availability. In the end, the assignments were the following (see also the [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ blog post] and [https://acl2020.org/program/tutorials/ online schedule]):

* T1 – twice on slots 3 and 4
* T2 – once on slot 5
* T3 – once on slot 4
* T4 – once on slot 4
* T5 – twice on slots 3 and 5
* T6 – once on slot 5
* T7 – twice on slots 3 and 4
* T8 – once on slot 5

Unfortunately, the tutorial teachers proved hardly available for slots 1 and 2.

In addition to time slot arrangements, we gave the tutorial teacher two options on presentation, namely a pre-recorded and a fully interactive form of tutorials. The former allows the teachers to pre-record the main content of their tutorials, and use the time slots mainly for question answering and detailed discussions. The latter is more interactive, the teachers use the online sessions for both giving the lecture and other activities such as active discussions. All these arrangements were made for taking advantage of the virtual conference. In the end the choices were the following:

* T1 – pre-recorded
* T2 – interactive (the initial decision was pre-recorded but a change was made in June)
* T3 – pre-recorded
* T4 – pre-recorded
* T5 – pre-recorded
* T6 – interactive
* T7 – interactive
* T8 – interactive

After making decisions on time slots and teaching forms, we started to work on the technical details of the virtual conference infrastructure jointly with the infrastructure chairs, the website chairs, the SlidesLive support team and individual tutorial teachers. We collected needs from each tutorial from time to time, inquired information from the SlidesLive team, and tried to accommodate specific requirement for each tutorial. Since this is the first time ACL was virtual, it typically took several rounds of information exchange for making a certain technical feature work. Major things discussed included:

* Sorting out how to pre-record a tutorial using the system
* Recording of interactive sessions
* Additional RocketChat channel
* Virtual tutorial webpage setup
* Coordinators for tutorial sessions and SlidesLive team participation
* Rehearsal of interactive sessions
* Captioning of pre-recorded and interactive tutorials
* Privacy issues
* Special needs from particular tutorials
** Integrating [https://www.dory.app/ Dory] for question answering (in T1)
** Fully interactive sessions, with occasional split into break-out rooms (in T7)

For the technical details about the SlidesLive infrastructure and detailed planning of each live session, the teachers were directly in contact with the SlidesLive team. Two online demos were made available by SlidesLive: about [https://slideslive.com/38931377/how-to-use-zoom-for-live-qa using Zoom] for QA sessions and about how SlidesLive coordinates a live sessions [https://slideslive.com/38931376/slideslive-virtual-conference-demo behind the scene]. SlidesLive also organized a dry run of the live session with the teachers of each tutorial.

To ensure smooth running of the sessions, we sent two rounds of reminders to each tutorial to ensure that each teacher is registered to the conference. We collected rough timelines of each tutorial so that the SlidesLive team can work closely during the live sessions. To further help the teachers prepare their live session content we additionally collected the lists of registered attendees twice before the conference, and shared them with the teachers.

In the end, the numbers of registrations for each tutorial were:
* T1 – 1762
* T2 – 793
* T3 – 503
* T4 – 571
* T5 – 641
* T6 – 1288
* T7 – 378
* T8 – 1087
This sums up to 7023 registrations.

===Archiving and Publicity===

There are three main channels for publicity, namely the static [http://www.acl2020.org/program/tutorials/ website], the website of the [https://virtual.acl2020.org/tutorials.html virtual conference] (restricted to the registered participants) and the social media (including [https://twitter.com/aclmeeting/ Twitter]). For the former, we collected photos from each instructor, which are shown with the title and abstract of their tutorials. Following NAACL 2019, we also displayed the time slots as a right-hand-side bar on the tutorial website. Due to special arrangements of the virtual conference, we marked the time zones explicitly. In addition, we asked the tutorial teachers to provide URL links to teaching materials (e.g., at GitHub) during their final submission, and included these links to their tutorial information. For social media, we drafted two blog posts (i.e., [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ this post] and [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]) for advertising the tutorials.

The archiving consists of three main items, namely the tutorial proceedings, the slides for each tutorial and the videos. For the tutorial proceedings, we collected final abstracts from the authors using the START system. We asked the instructors to sign the CC BY v4 agreement when submitting their final materials on SoftConf. For the slides, we asked the tutorial teachers to provide a first version of the slides by June 17 regardless whether they are pre-recorded or interactive. Some tutorials made further updates to their slides before ACL. For the video recordings, we worked with the ACL Anthology team discussing the technical details since this is the first time the tutorial videos are shared in the website. [Note: this is currently undergoing at the time of writing of this summary] One important aspect of for inclusiveness and accessibility are tutorial captions. With the help of SlidesLive, automatic captions generated for pre-recording were collected and sent to the teachers for edition. They should be included in the tutorial archives.

===Lessons Learned===

While organizing a virtual conference it is important to stress that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]).

It is also important to announce the individual modalities with the attendees early enough. For instance, watching pre-recordings in advance was necessary for some tutorials (since their live sessions only included question answering) and such constrains should have been known more in advance.

If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to virtual meeting tools like Zoom only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.

One thing to remember which we missed is to publish the reading list for each tutorial (which the teachers submit with the proposal) on the tutorial website.

2020Q3 Reports: Tutorial Chairs

2020-07-24T08:31:10Z

AgataSavary: /* Virtual Conference Organization */

Tutorial Chairs:

<blockquote>Agata Savary, University of Tours, France

Yue Zhang, Westlake University, Hangzhou, China
</blockquote>

=== Preparatory Work, Call, Reviewing and Decision-Making ===
The call, submission, reviewing and selection of tutorials was coordinated jointly for 4 conferences: ACL, AACL-IJCNLP, COLING and EMNLP.

Before drafting the call, we collected lists of tutorials offered within the past 4 years. We analysed previous calls for tutorials and reports from tutorial chairs (from [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs 2016], [https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs 2017], [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs 2018] and [http://aclweb.org/adminwiki/index.php?title=2019Q1_Reports:_ACL_2019 2019]). We consulted previous tutorial chairs with a questionnaire including questions about: the number of submissions, encouraging submissions on specific topics or from specific lecturers, the review procedure, the evaluation criteria, the post-tutorial availability of the slides/codes, and lessons learned from tutorial coordination. We also discussed the publication of slides and video recordings from future tutorials with the persons in charge of the ACL Anthology. As a result of these steps, we created two new sections for the [https://www.aclweb.org/adminwiki/index.php?title=Conference_Handbook ACL Conference Handbook] (future chairs might consider updating these documents yearly):

* the list of [https://www.aclweb.org/adminwiki/index.php?title=Past_tutorials past tutorials] at ACL, COLING, EACL, EMNLP, and NAACL in 2016-2019
* a [https://www.aclweb.org/adminwiki/index.php?title=Tutorial_chair_handbook tutorial chair handbook]

The final [https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020 call] differs from previous calls in several aspects: (i) the expectations about tutorial proposals were made clearer, (ii) following the central ACL decision, the teachers' payment policy was replaced by a fee-waiving policy, (iii) the required submission details include two new items: diversity considerations and agreement for open access publication of slides, codes, data and video recordings, (iv) the evaluation criteria (see below) are announced.

We recruited a review committee of 19 members, including the 8 tutorial chairs and 11 external members selected for their large understanding of the NLP domain and a good experience in reviewing and/or tutorial teaching:

'''Review Committee'''

* Timothy Baldwin (University of Melbourne, Australia) - AACL-IJCNLP 2020 tutorial chair
* Daniel Beck (University of Melbourne, Australia) - COLING 2020 tutorial chair
* Emily M. Bender (University of Washington, WA, USA)
* Erik Cambria (Nanyang Technological University, Singapore)
* Gaël Dias (University of Caen Normandie, France)
* Stefan Evert (Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany)
* Yang Liu (Tsinghua University, Beijing, China)
* Agata Savary (University of Tours, France) - ACL 2020 tutorial chair
* João Sedoc (Johns Hopkins University, Baltimore, MD, USA)
* Lucia Specia (Sheffield University, UK) - COLING 2020 tutorial chair
* Xu Sun (Peking University, China)
* Yulia Tsvetkov (Carnegie Mellon University, Pittsburgh, PA, USA)
* Benjamin Van Durme (Johns Hopkins University, Baltimore, MD, USA) - EMNLP 2020 tutorial chair
* Aline Villavicencio (University of Sheffield, UK and Federal University of Rio Grande do Sul, Brazil) - EMNLP 2020 tutorial chair
* Taro Watanabe (Google, Inc., Tokyo, Japan)
* Aaron Steven White (University of Rochester, NY, USA)
* Fei Xia (University of Washington, WA, USA) - AACL-IJCNLP 2020 tutorial chair
* Yue Zhang (Westlake University, Hangzhou, China) - ACL 2020 tutorial chair
* Meishan Zhang (Tianjin University, China)

In total, we received 43 submissions for the 4 conferences. Each reviewer was assigned 6-7 proposals and each proposal received 3 reviews. The selection criteria included: clarity and preparedness, novelty or timely character of the topic, lecturers' experience, likely audience interest, open access of the teaching material, diversity aspects (multilingualism, gender, age and country of the lecturers), and compatibility with the preferred venues. We accepted 31 proposals.

The decision making was handled via an online meeting of the 8 tutorial chairs. In particular, the selection of tutorials for each conference was done via the expression of interest of the tutorial chairs on a round-robin basis. Some slight adjustments were also performed after the meeting to better fit the authors' preferences. In total, 8, 8, 8 and 7 proposals were selected for ACL, AACL-IJCNLP, COLING and EMNLP, respectively. Upon the announcement the results, 2 of the proposals accepted for AACL-IJCNLP were withdrawn.

The submission, review, selection and collection of final descriptions for all tutorials was handled via a dedicated SoftConf space, shared by the 4 coordinating conferences. After the selection of proposals, a separate track was created on SoftConf for each conference. The final submission page (one per conference) was set up so as to collect all the necessary data including notably: the tutorial slides, URLs for course material (if any), printable material (if any) and agreement for open access publication.

===Tutorials Selected for ACL 2020===

The final selection for ACL 2020 consists of the following 8 tutorials of 3 hours each (each of them had ACL as the preferred or the second preferred venue):

'''T1: Interpretability and Analysis in Neural NLP''' (cutting-edge) 
Yonatan Belinkov, Sebastian Gehrmann and Ellie Pavlick 
While deep learning has transformed the NLP field and impacted the larger computational linguistics community, the rise of neural networks is stained by their opaque nature: It is challenging to interpret the inner workings of neural network models, and explicate their behavior. Therefore, in the last few years, an increasingly large body of work has been devoted to the analysis and interpretation of neural network models in NLP. 
This body of work is so far lacking a common framework and methodology. Moreover, approaching the analysis of modern neural networks can be difficult for newcomers to the field. This tutorial aims to fill this gap and introduce the nascent field of interpretability and analysis of neural networks in NLP. 
The tutorial covers the main lines of analysis work, such as probing classifier, behavior studies and test suites, psycholinguistic methods, visualizations, adversarial examples, and other methods. We highlight not only the most commonly applied analysis methods, but also the specific limitations and shortcomings of current approaches, in order to inform participants where to focus future efforts.

'''T2: Multi-modal Information Extraction from Text, Semi-structured, and Tabular Data on the Web''' (cutting-edge) 
Xin Luna Dong, Hannaneh Hajishirzi, Colin Lockard and Prashant Shiralkar 
The World Wide Web contains vast quantities of textual information in several forms: unstructured text, template-based semi-structured webpages (which present data in key-value pairs and lists), and tables. Methods for extracting information from these sources and converting it to a structured form have been a target of research from the natural language processing (NLP), data mining, and database communities. While these researchers have largely separated extraction from web data into different problems based on the modality of the data, they have faced similar problems such as learning with limited labeled data, defining (or avoiding defining) ontologies, making use of prior knowledge, and scaling solutions to deal with the size of the Web. 
In this tutorial we take a holistic view toward information extraction, exploring the commonalities in the challenges and solutions developed to address these different forms of text. We will explore the approaches targeted at unstructured text that largely rely on learning syntactic or semantic textual patterns, approaches targeted at semi-structured documents that learn to identify structural patterns in the template, and approaches targeting web tables which rely heavily on entity linking and type information. 
While these different data modalities have largely been considered separately in the past, recent research has started taking a more inclusive approach toward textual extraction, in which the multiple signals offered by textual, layout, and visual clues are combined into a single extraction model made possible by new deep learning approaches. At the same time, trends within purely textual extraction have shifted toward full-document understanding rather than considering sentences as independent units. With this in mind, it is worth considering the information extraction problem as a whole to motivate solutions that harness textual semantics along with visual and semi-structured layout information. We will discuss these approaches and suggest avenues for future work.

'''T3: Reviewing Natural Language Processing Research''' (introductory) 
Kevin Cohen, Karën Fort, Margot Mieskes and Aurélie Névéol 
As the demand for reviewing grows, so must the pool of reviewers. As the [http://www.livecongress.it/aol/indexSA.php?id=E2EAED7D&ticket= survey] presented by Graham Neubig at the 2019 ACL showed, a considerable number of reviewers are junior researchers, who might lack the experience and expertise necessary for high-quality reviews. Some of them might not have the environment or lack opportunities that allow them to learn the skills necessary. A tutorial on reviewing for the NLP community might increase reviewers’ confidence, as well as the quality of the reviews. This introductory tutorial will cover the goals, processes, and evaluation of reviewing research papers in natural language processing.

'''T4: Stylized Text Generation: Approaches and Applications''' (cutting-edge) 
Lili Mou and Olga Vechtomova 
Text generation has played an important role in various applications of natural language processing (NLP), and kn recent studies, researchers are paying increasing attention to modeling and manipulating the style of the generation text, which we call stylized text generation. In this tutorial, we will provide a comprehensive literature review in this direction. We start from the definition of style and different settings of stylized text generation, illustrated with various applications. Then, we present different settings of stylized generation, such as parallel supervised, style label-supervised, and unsupervised. In each setting, we delve deep into machine learning methods, including embedding learning techniques to represent style}, adversarial learning and reinforcement learning with cycle consistency to match content but to distinguish different styles. We also introduce current approaches of evaluating stylized text generation systems. We conclude our tutorial by presenting the challenges of stylized text generation and discussing future directions, such as small-data training, non-categorical style modeling, and a generalized scope of style transfer (e.g., controlling the syntax as a style).

'''T5: Achieving Common Ground in Multi-modal Dialogue''' (cutting-edge) 
Malihe Alikhani and Matthew Stone 
All communication aims at achieving common ground (grounding): interlocutors can work together effectively only with mutual beliefs about what the state of the world is, about what their goals are, and about how they plan to make their goals a reality. Computational dialogue research offers some classic results on grouding, which unfortunately offer scant guidance to the design of grounding modules and behaviors in cutting-edge systems. In this tutorial, we focus on three main topic areas: 1) grounding in human-human communication; 2) grounding in dialogue systems; and 3) grounding in multi-modal interactive systems, including image-oriented conversations and human-robot interactions. We highlight a number of achievements of recent computational research in coordinating complex content, show how these results lead to rich and challenging opportunities for doing grounding in more flexible and powerful ways, and canvass relevant insights from the literature on human--human conversation. We expect that the tutorial will be of interest to researchers in dialogue systems, computational semantics and cognitive modeling, and hope that it will catalyze research and system building that more directly explores the creative, strategic ways conversational agents might be able to seek and offer evidence about their understanding of their interlocutors.

'''T6: Commonsense Reasoning for Natural Language Processing''' (introductory) 
Maarten Sap, Vered Shwartz, Antoine Bosselut, Dan Roth and Yejin Choi 
In our tutorial, we (1) outline the various types of commonsense (e.g., physical, social), and (2) discuss techniques to gather and represent commonsense knowledge, while highlighting the challenges specific to this type of knowledge (e.g., reporting bias). We will then (3) discuss the types of commonsense knowledge captured by modern NLP systems (e.g., large pretrained language models), and (4) present ways to measure systems' commonsense reasoning abilities. We finish with (5) a discussion of various ways in which commonsense reasoning can be used to improve performance on NLP tasks, exemplified by an (6) interactive session on integrating commonsense into a downstream task.

'''T7: Integrating Ethics into the NLP Curriculum''' (introductory) 
Emily M. Bender, Dirk Hovy and Alexandra Schofield 
Our goal in this tutorial is to empower NLP researchers and practitioners with tools and resources to teach others about how to ethically apply NLP techniques. Our tutorial will present both high-level strategies for developing an ethics-oriented curriculum, based on experience and best practices, as well as specific sample exercises that can be brought to a classroom. We plan to make this a highly interactive work session culminating in a shared online resource page that pools lesson plans, assignments, exercise ideas, reading suggestions, and ideas from the attendees. We consider three primary topics with our session that frequently underlie ethical issues in NLP research: Dual use, bias and privacy. 
In this setting, a key lesson is that there is no single approach to ethical NLP: each project requires thoughtful consideration about what steps can be taken to best support people affected by that project. However, we can learn (and teach) what kinds of issues to be aware of and what kinds of strategies are available for mitigating harm. To teach this process, we apply and promote interactive exercises that provide an opportunity to ideate, discuss, and reflect. We plan to facilitate this in a way that encourages positive discussion, emphasizing the creation of ideas for the future instead of negative opinions of previous work.

'''T8: Recent Advances in Open-Domain Question Answering''' (cutting-edge) 
Danqi Chen and Scott Wen-tau Yih 
Open-domain (textual) question answering (QA), the task of finding answers to open-domain questions by searching a large collection of documents, has been a long-standing problem in NLP, information retrieval (IR) and related fields (Voorhees et al., 1999; Moldovan et al., 2000; Brill et al.,2002; Ferrucci et al., 2010). Traditional QA systems were usually constructed as a pipeline, consisting of many different components such as question processing, document/passage retrieval and answer processing. With the rapid development of neural reading comprehension (Chen, 2018), modern open-domain QA systems have been restructured by combining traditional IR techniques and neural reading comprehension models (Chen et al., 2017; Yang et al., 2019) or even implemented in a fully end-to-end fashion (Lee et al., 2019; Seo et al., 2019). While the system architecture has been drastically simplified, two technical challenges remain critical:(1) “Retriever”: finding documents that (might)contain an answer from a large collection of documents; (2) “Reader”: finding the answer in a given paragraph or a document. 
In this tutorial, we aim to provide a comprehensive and coherent overview of recent advances in this line of research. We will start by first giving a brief historical background of open-domain question answering, discussing the basic setup and core technical challenges of the research problem.The focus will then shift to modern techniques and resources proposed for open-domain QA, including the basics of latest neural reading comprehension systems, new datasets and models. The scope will also be broadened to cover the information retrieval component on how to effectively identify passages relevant to the questions. Moreover, in-depth discussions will be given on the use of traditional / neural IR modules, as well as the trade-offs between modular design and end-to-end training. If time permits, we also plan to discuss some hybrid approaches for answering questions using both text and large knowledge bases (e.g. (Sun et al., 2018)) and give a critical review on how structured data complements the information from unstructured text. 
At the end of our tutorial, we will discuss some important questions, including (1) How much progress have we made compared to the QA systems developed in the last decade?(2) What are the main challenges and limitations of cur-rent approaches? (3) How to trade off the efficiency (computational time and memory requirements) and accuracy in the deep learning era? We hope that our tutorial will not only serve as a useful resource for the audience to efficiently acquire the up-to-date knowledge, but also provide new perspectives to stimulate the advances of open-domain QA research in the next phase.

===Virtual Conference Organization===

In April the whole conference was decided to be held in virtual. We carried on the preparations jointly with the tutorial instructors, the virtual infrastructure chairs, the website chairs and the SlidesLive team to set up the virtual conference infrastructure, and to publicize it for each tutorial.

We first made a decision on the time slots for tutorial presentations. Five major time zones were considered, including US west coast, US east coast, central Europe, China/Asia and Australia (India was also considered). Our aim was to offer at least two convenient time slots (i.e. within regular working hours) for each of these zones.

The final time slots include (Jul 5 Pacific Time):

* slot 1 – 7:00 PM (-1D) to 10:30 PM (-1D)
* slot 2 – 12:00 AM to 3:30 AM
* slot 3 – 6:00 AM to 9:30 AM
* slot 4 – 10:30 AM to 2:00 PM
* slot 5 – 3:00 PM to 6:30 PM

We asked the tutorial teachers to indicate their preferences among the time slots, and – if possible – to agree to give their tutorial twice, so as to increase accessibility. Our final decisions on slot assignments traded off the teacher preferences and slot availability. In the end, the assignments were the following (see also the [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ blog post] and [https://acl2020.org/program/tutorials/ online schedule]):

* T1 – twice on slots 3 and 4
* T2 – once on slot 5
* T3 – once on slot 4
* T4 – once on slot 4
* T5 – twice on slots 3 and 5
* T6 – once on slot 5
* T7 – twice on slots 3 and 4
* T8 – once on slot 5

Unfortunately, the tutorial teachers proved hardly available for slots 1 and 2.

In addition to time slot arrangements, we gave the tutorial teacher two options on presentation, namely a pre-recorded and a fully interactive form of tutorials. The former allows the teachers to pre-record the main content of their tutorials, and use the time slots mainly for question answering and detailed discussions. The latter is more interactive, the teachers use the online sessions for both giving the lecture and other activities such as active discussions. All these arrangements were made for taking advantage of the virtual conference. In the end the choices were the following:

* T1 – pre-recorded
* T2 – interactive (the initial decision was pre-recorded but a change was made in June)
* T3 – pre-recorded
* T4 – pre-recorded
* T5 – pre-recorded
* T6 – interactive
* T7 – interactive
* T8 – interactive

After making decisions on time slots and teaching forms, we started to work on the technical details of the virtual conference infrastructure jointly with the infrastructure chairs, the website chairs, the SlidesLive support team and individual tutorial teachers. We collected needs from each tutorial from time to time, inquired information from the SlidesLive team, and tried to accommodate specific requirement for each tutorial. Since this is the first time ACL was virtual, it typically took several rounds of information exchange for making a certain technical feature work. Major things discussed included:

* Sorting out how to pre-record a tutorial using the system
* Recording of interactive sessions
* Additional RocketChat channel
* Virtual tutorial webpage setup
* Coordinators for tutorial sessions and SlidesLive team participation
* Rehearsal of interactive sessions
* Captioning of pre-recorded and interactive tutorials
* Privacy issues
* Special needs from particular tutorials
** Integrating [https://www.dory.app/ Dory] for question answering (in T1)
** Fully interactive sessions, with occasional split into break-out rooms (in T7)

For the technical details about using the SlidesLive and detailed schedules of each session, the teachers were directly in contact with the SlidesLive team. Two online demos were made available by SlidesLive: about [https://slideslive.com/38931377/how-to-use-zoom-for-live-qa using Zoom] for QA sessions and about how SlidesLive coordinates a live sessions [https://slideslive.com/38931376/slideslive-virtual-conference-demo behind the scene].

To ensure smooth running of the sessions, we sent two rounds of reminders to each tutorial to ensure that each teacher is registered to the conference. We collected rough timelines of each tutorial so that the SlidesLive team can work closely during the live sessions. To further help the teachers prepare their live session content we additionally collected the lists of registered attendees twice before the conference, and shared them with the teachers.

In the end, the numbers of registrations for each tutorial were:
* T1 – 1762
* T2 – 793
* T3 – 503
* T4 – 571
* T5 – 641
* T6 – 1288
* T7 – 378
* T8 – 1087
This sums up to 7023 registrations.

===Archiving and Publicity===

There are three main channels for publicity, namely the static [http://www.acl2020.org/program/tutorials/ website], the website of the [https://virtual.acl2020.org/tutorials.html virtual conference] (restricted to the registered participants) and the social media (including [https://twitter.com/aclmeeting/ Twitter]). For the former, we collected photos from each instructor, which are shown with the title and abstract of their tutorials. Following NAACL 2019, we also displayed the time slots as a right-hand-side bar on the tutorial website. Due to special arrangements of the virtual conference, we marked the time zones explicitly. In addition, we asked the tutorial teachers to provide URL links to teaching materials (e.g., at GitHub) during their final submission, and included these links to their tutorial information. For social media, we drafted two blog posts (i.e., [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ this post] and [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]) for advertising the tutorials.

The archiving consists of three main items, namely the tutorial proceedings, the slides for each tutorial and the videos. For the tutorial proceedings, we collected final abstracts from the authors using the START system. We asked the instructors to sign the CC BY v4 agreement when submitting their final materials on SoftConf. For the slides, we asked the tutorial teachers to provide a first version of the slides by June 17 regardless whether they are pre-recorded or interactive. Some tutorials made further updates to their slides before ACL. For the video recordings, we worked with the ACL Anthology team discussing the technical details since this is the first time the tutorial videos are shared in the website. [Note: this is currently undergoing at the time of writing of this summary] One important aspect of for inclusiveness and accessibility are tutorial captions. With the help of SlidesLive, automatic captions generated for pre-recording were collected and sent to the teachers for edition. They should be included in the tutorial archives.

===Lessons Learned===

While organizing a virtual conference it is important to stress that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]).

It is also important to announce the individual modalities with the attendees early enough. For instance, watching pre-recordings in advance was necessary for some tutorials (since their live sessions only included question answering) and such constrains should have been known more in advance.

If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to virtual meeting tools like Zoom only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.

One thing to remember which we missed is to publish the reading list for each tutorial (which the teachers submit with the proposal) on the tutorial website.

2020Q3 Reports: Tutorial Chairs

2020-07-23T11:38:38Z

AgataSavary: /* Virtual Conference Organization */

Tutorial Chairs:

<blockquote>Agata Savary, University of Tours, France

Yue Zhang, Westlake University, Hangzhou, China
</blockquote>

=== Preparatory Work, Call, Reviewing and Decision-Making ===
The call, submission, reviewing and selection of tutorials was coordinated jointly for 4 conferences: ACL, AACL-IJCNLP, COLING and EMNLP.

Before drafting the call, we collected lists of tutorials offered within the past 4 years. We analysed previous calls for tutorials and reports from tutorial chairs (from [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs 2016], [https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs 2017], [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs 2018] and [http://aclweb.org/adminwiki/index.php?title=2019Q1_Reports:_ACL_2019 2019]). We consulted previous tutorial chairs with a questionnaire including questions about: the number of submissions, encouraging submissions on specific topics or from specific lecturers, the review procedure, the evaluation criteria, the post-tutorial availability of the slides/codes, and lessons learned from tutorial coordination. We also discussed the publication of slides and video recordings from future tutorials with the persons in charge of the ACL Anthology. As a result of these steps, we created two new sections for the [https://www.aclweb.org/adminwiki/index.php?title=Conference_Handbook ACL Conference Handbook] (future chairs might consider updating these documents yearly):

* the list of [https://www.aclweb.org/adminwiki/index.php?title=Past_tutorials past tutorials] at ACL, COLING, EACL, EMNLP, and NAACL in 2016-2019
* a [https://www.aclweb.org/adminwiki/index.php?title=Tutorial_chair_handbook tutorial chair handbook]

The final [https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020 call] differs from previous calls in several aspects: (i) the expectations about tutorial proposals were made clearer, (ii) following the central ACL decision, the teachers' payment policy was replaced by a fee-waiving policy, (iii) the required submission details include two new items: diversity considerations and agreement for open access publication of slides, codes, data and video recordings, (iv) the evaluation criteria (see below) are announced.

We recruited a review committee of 19 members, including the 8 tutorial chairs and 11 external members selected for their large understanding of the NLP domain and a good experience in reviewing and/or tutorial teaching:

'''Review Committee'''

* Timothy Baldwin (University of Melbourne, Australia) - AACL-IJCNLP 2020 tutorial chair
* Daniel Beck (University of Melbourne, Australia) - COLING 2020 tutorial chair
* Emily M. Bender (University of Washington, WA, USA)
* Erik Cambria (Nanyang Technological University, Singapore)
* Gaël Dias (University of Caen Normandie, France)
* Stefan Evert (Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany)
* Yang Liu (Tsinghua University, Beijing, China)
* Agata Savary (University of Tours, France) - ACL 2020 tutorial chair
* João Sedoc (Johns Hopkins University, Baltimore, MD, USA)
* Lucia Specia (Sheffield University, UK) - COLING 2020 tutorial chair
* Xu Sun (Peking University, China)
* Yulia Tsvetkov (Carnegie Mellon University, Pittsburgh, PA, USA)
* Benjamin Van Durme (Johns Hopkins University, Baltimore, MD, USA) - EMNLP 2020 tutorial chair
* Aline Villavicencio (University of Sheffield, UK and Federal University of Rio Grande do Sul, Brazil) - EMNLP 2020 tutorial chair
* Taro Watanabe (Google, Inc., Tokyo, Japan)
* Aaron Steven White (University of Rochester, NY, USA)
* Fei Xia (University of Washington, WA, USA) - AACL-IJCNLP 2020 tutorial chair
* Yue Zhang (Westlake University, Hangzhou, China) - ACL 2020 tutorial chair
* Meishan Zhang (Tianjin University, China)

In total, we received 43 submissions for the 4 conferences. Each reviewer was assigned 6-7 proposals and each proposal received 3 reviews. The selection criteria included: clarity and preparedness, novelty or timely character of the topic, lecturers' experience, likely audience interest, open access of the teaching material, diversity aspects (multilingualism, gender, age and country of the lecturers), and compatibility with the preferred venues. We accepted 31 proposals.

The decision making was handled via an online meeting of the 8 tutorial chairs. In particular, the selection of tutorials for each conference was done via the expression of interest of the tutorial chairs on a round-robin basis. Some slight adjustments were also performed after the meeting to better fit the authors' preferences. In total, 8, 8, 8 and 7 proposals were selected for ACL, AACL-IJCNLP, COLING and EMNLP, respectively. Upon the announcement the results, 2 of the proposals accepted for AACL-IJCNLP were withdrawn.

The submission, review, selection and collection of final descriptions for all tutorials was handled via a dedicated SoftConf space, shared by the 4 coordinating conferences. After the selection of proposals, a separate track was created on SoftConf for each conference. The final submission page (one per conference) was set up so as to collect all the necessary data including notably: the tutorial slides, URLs for course material (if any), printable material (if any) and agreement for open access publication.

===Tutorials Selected for ACL 2020===

The final selection for ACL 2020 consists of the following 8 tutorials of 3 hours each (each of them had ACL as the preferred or the second preferred venue):

'''T1: Interpretability and Analysis in Neural NLP''' (cutting-edge) 
Yonatan Belinkov, Sebastian Gehrmann and Ellie Pavlick 
While deep learning has transformed the NLP field and impacted the larger computational linguistics community, the rise of neural networks is stained by their opaque nature: It is challenging to interpret the inner workings of neural network models, and explicate their behavior. Therefore, in the last few years, an increasingly large body of work has been devoted to the analysis and interpretation of neural network models in NLP. 
This body of work is so far lacking a common framework and methodology. Moreover, approaching the analysis of modern neural networks can be difficult for newcomers to the field. This tutorial aims to fill this gap and introduce the nascent field of interpretability and analysis of neural networks in NLP. 
The tutorial covers the main lines of analysis work, such as probing classifier, behavior studies and test suites, psycholinguistic methods, visualizations, adversarial examples, and other methods. We highlight not only the most commonly applied analysis methods, but also the specific limitations and shortcomings of current approaches, in order to inform participants where to focus future efforts.

'''T2: Multi-modal Information Extraction from Text, Semi-structured, and Tabular Data on the Web''' (cutting-edge) 
Xin Luna Dong, Hannaneh Hajishirzi, Colin Lockard and Prashant Shiralkar 
The World Wide Web contains vast quantities of textual information in several forms: unstructured text, template-based semi-structured webpages (which present data in key-value pairs and lists), and tables. Methods for extracting information from these sources and converting it to a structured form have been a target of research from the natural language processing (NLP), data mining, and database communities. While these researchers have largely separated extraction from web data into different problems based on the modality of the data, they have faced similar problems such as learning with limited labeled data, defining (or avoiding defining) ontologies, making use of prior knowledge, and scaling solutions to deal with the size of the Web. 
In this tutorial we take a holistic view toward information extraction, exploring the commonalities in the challenges and solutions developed to address these different forms of text. We will explore the approaches targeted at unstructured text that largely rely on learning syntactic or semantic textual patterns, approaches targeted at semi-structured documents that learn to identify structural patterns in the template, and approaches targeting web tables which rely heavily on entity linking and type information. 
While these different data modalities have largely been considered separately in the past, recent research has started taking a more inclusive approach toward textual extraction, in which the multiple signals offered by textual, layout, and visual clues are combined into a single extraction model made possible by new deep learning approaches. At the same time, trends within purely textual extraction have shifted toward full-document understanding rather than considering sentences as independent units. With this in mind, it is worth considering the information extraction problem as a whole to motivate solutions that harness textual semantics along with visual and semi-structured layout information. We will discuss these approaches and suggest avenues for future work.

'''T3: Reviewing Natural Language Processing Research''' (introductory) 
Kevin Cohen, Karën Fort, Margot Mieskes and Aurélie Névéol 
As the demand for reviewing grows, so must the pool of reviewers. As the [http://www.livecongress.it/aol/indexSA.php?id=E2EAED7D&ticket= survey] presented by Graham Neubig at the 2019 ACL showed, a considerable number of reviewers are junior researchers, who might lack the experience and expertise necessary for high-quality reviews. Some of them might not have the environment or lack opportunities that allow them to learn the skills necessary. A tutorial on reviewing for the NLP community might increase reviewers’ confidence, as well as the quality of the reviews. This introductory tutorial will cover the goals, processes, and evaluation of reviewing research papers in natural language processing.

'''T4: Stylized Text Generation: Approaches and Applications''' (cutting-edge) 
Lili Mou and Olga Vechtomova 
Text generation has played an important role in various applications of natural language processing (NLP), and kn recent studies, researchers are paying increasing attention to modeling and manipulating the style of the generation text, which we call stylized text generation. In this tutorial, we will provide a comprehensive literature review in this direction. We start from the definition of style and different settings of stylized text generation, illustrated with various applications. Then, we present different settings of stylized generation, such as parallel supervised, style label-supervised, and unsupervised. In each setting, we delve deep into machine learning methods, including embedding learning techniques to represent style}, adversarial learning and reinforcement learning with cycle consistency to match content but to distinguish different styles. We also introduce current approaches of evaluating stylized text generation systems. We conclude our tutorial by presenting the challenges of stylized text generation and discussing future directions, such as small-data training, non-categorical style modeling, and a generalized scope of style transfer (e.g., controlling the syntax as a style).

'''T5: Achieving Common Ground in Multi-modal Dialogue''' (cutting-edge) 
Malihe Alikhani and Matthew Stone 
All communication aims at achieving common ground (grounding): interlocutors can work together effectively only with mutual beliefs about what the state of the world is, about what their goals are, and about how they plan to make their goals a reality. Computational dialogue research offers some classic results on grouding, which unfortunately offer scant guidance to the design of grounding modules and behaviors in cutting-edge systems. In this tutorial, we focus on three main topic areas: 1) grounding in human-human communication; 2) grounding in dialogue systems; and 3) grounding in multi-modal interactive systems, including image-oriented conversations and human-robot interactions. We highlight a number of achievements of recent computational research in coordinating complex content, show how these results lead to rich and challenging opportunities for doing grounding in more flexible and powerful ways, and canvass relevant insights from the literature on human--human conversation. We expect that the tutorial will be of interest to researchers in dialogue systems, computational semantics and cognitive modeling, and hope that it will catalyze research and system building that more directly explores the creative, strategic ways conversational agents might be able to seek and offer evidence about their understanding of their interlocutors.

'''T6: Commonsense Reasoning for Natural Language Processing''' (introductory) 
Maarten Sap, Vered Shwartz, Antoine Bosselut, Dan Roth and Yejin Choi 
In our tutorial, we (1) outline the various types of commonsense (e.g., physical, social), and (2) discuss techniques to gather and represent commonsense knowledge, while highlighting the challenges specific to this type of knowledge (e.g., reporting bias). We will then (3) discuss the types of commonsense knowledge captured by modern NLP systems (e.g., large pretrained language models), and (4) present ways to measure systems' commonsense reasoning abilities. We finish with (5) a discussion of various ways in which commonsense reasoning can be used to improve performance on NLP tasks, exemplified by an (6) interactive session on integrating commonsense into a downstream task.

'''T7: Integrating Ethics into the NLP Curriculum''' (introductory) 
Emily M. Bender, Dirk Hovy and Alexandra Schofield 
Our goal in this tutorial is to empower NLP researchers and practitioners with tools and resources to teach others about how to ethically apply NLP techniques. Our tutorial will present both high-level strategies for developing an ethics-oriented curriculum, based on experience and best practices, as well as specific sample exercises that can be brought to a classroom. We plan to make this a highly interactive work session culminating in a shared online resource page that pools lesson plans, assignments, exercise ideas, reading suggestions, and ideas from the attendees. We consider three primary topics with our session that frequently underlie ethical issues in NLP research: Dual use, bias and privacy. 
In this setting, a key lesson is that there is no single approach to ethical NLP: each project requires thoughtful consideration about what steps can be taken to best support people affected by that project. However, we can learn (and teach) what kinds of issues to be aware of and what kinds of strategies are available for mitigating harm. To teach this process, we apply and promote interactive exercises that provide an opportunity to ideate, discuss, and reflect. We plan to facilitate this in a way that encourages positive discussion, emphasizing the creation of ideas for the future instead of negative opinions of previous work.

'''T8: Recent Advances in Open-Domain Question Answering''' (cutting-edge) 
Danqi Chen and Scott Wen-tau Yih 
Open-domain (textual) question answering (QA), the task of finding answers to open-domain questions by searching a large collection of documents, has been a long-standing problem in NLP, information retrieval (IR) and related fields (Voorhees et al., 1999; Moldovan et al., 2000; Brill et al.,2002; Ferrucci et al., 2010). Traditional QA systems were usually constructed as a pipeline, consisting of many different components such as question processing, document/passage retrieval and answer processing. With the rapid development of neural reading comprehension (Chen, 2018), modern open-domain QA systems have been restructured by combining traditional IR techniques and neural reading comprehension models (Chen et al., 2017; Yang et al., 2019) or even implemented in a fully end-to-end fashion (Lee et al., 2019; Seo et al., 2019). While the system architecture has been drastically simplified, two technical challenges remain critical:(1) “Retriever”: finding documents that (might)contain an answer from a large collection of documents; (2) “Reader”: finding the answer in a given paragraph or a document. 
In this tutorial, we aim to provide a comprehensive and coherent overview of recent advances in this line of research. We will start by first giving a brief historical background of open-domain question answering, discussing the basic setup and core technical challenges of the research problem.The focus will then shift to modern techniques and resources proposed for open-domain QA, including the basics of latest neural reading comprehension systems, new datasets and models. The scope will also be broadened to cover the information retrieval component on how to effectively identify passages relevant to the questions. Moreover, in-depth discussions will be given on the use of traditional / neural IR modules, as well as the trade-offs between modular design and end-to-end training. If time permits, we also plan to discuss some hybrid approaches for answering questions using both text and large knowledge bases (e.g. (Sun et al., 2018)) and give a critical review on how structured data complements the information from unstructured text. 
At the end of our tutorial, we will discuss some important questions, including (1) How much progress have we made compared to the QA systems developed in the last decade?(2) What are the main challenges and limitations of cur-rent approaches? (3) How to trade off the efficiency (computational time and memory requirements) and accuracy in the deep learning era? We hope that our tutorial will not only serve as a useful resource for the audience to efficiently acquire the up-to-date knowledge, but also provide new perspectives to stimulate the advances of open-domain QA research in the next phase.

===Virtual Conference Organization===

In April the whole conference was decided to be held in virtual. We carried on the preparations jointly with the tutorial instructors, the virtual infrastructure chairs, the website chairs and the SlidesLive team to set up the virtual conference infrastructure, and to publicize it for each tutorial.

We first made a decision on the time slots for tutorial presentations. Five major time zones were considered, including US west coast, US east coast, central Europe, China/Asia and Australia (India was also considered). Our aim was to offer at least two convenient time slots (i.e. within regular working hours) for each of these zones.

The final time slots include (Jul 5 Pacific Time):

* slot 1 – 7:00 PM (-1D) to 10:30 PM (-1D)
* slot 2 – 12:00 AM to 3:30 AM
* slot 3 – 6:00 AM to 9:30 AM
* slot 4 – 10:30 AM to 2:00 PM
* slot 5 – 3:00 PM to 6:30 PM

We asked the tutorial teachers to indicate their preferences among the time slots, and – if possible – to agree to give their tutorial twice, so as to increase accessibility. Our final decisions on slot assignments traded off the teacher preferences and slot availability. In the end, the assignments were the following (see also the [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ blog post] and [https://acl2020.org/program/tutorials/ online schedule]):

* T1 – twice on slots 3 and 4
* T2 – once on slot 5
* T3 – once on slot 4
* T4 – once on slot 4
* T5 – twice on slots 3 and 5
* T6 – once on slot 5
* T7 – twice on slots 3 and 4
* T8 – once on slot 5

Unfortunately, the tutorial teachers proved hardly available for slots 1 and 2.

In addition to time slot arrangements, we gave the tutorial teacher two options on presentation, namely a pre-recorded and a fully interactive form of tutorials. The former allows the teachers to pre-record the main content of their tutorials, and use the time slots mainly for question answering and detailed discussions. The latter is more interactive, the teachers use the online sessions for both giving the lecture and other activities such as active discussions. All these arrangements were made for taking advantage of the virtual conference. In the end the choices were the following:

* T1 – pre-recorded
* T2 – interactive (the initial decision was pre-recorded but a change was made in June)
* T3 – pre-recorded
* T4 – pre-recorded
* T5 – pre-recorded
* T6 – interactive
* T7 – interactive
* T8 – interactive

After making decisions on time slots and teaching forms, we started to work on the technical details of the virtual conference infrastructure jointly with the infrastructure chairs, the website chairs, the SlidesLive support team and individual tutorial teachers. We collected needs from each tutorial from time to time, inquired information from the SlidesLive team, and tried to accommodate specific requirement for each tutorial. Since this is the first time ACL was virtual, it typically took several rounds of information exchange for making a certain technical feature work. Major things discussed included:

* Sorting out how to pre-record a tutorial using the system
* Recording of interactive sessions
* Additional RocketChat channel
* Virtual tutorial webpage setup
* Coordinators for tutorial sessions and SlidesLive team participation
* Rehearsal of interactive sessions
* Captioning of pre-recorded and interactive tutorials
* Privacy issues
* Special needs from particular tutorials
** Integrating [https://www.dory.app/ Dory] for question answering (in T1)
** Fully interactive sessions, with occasional split into break-out rooms (in T7)

To ensure smooth running of the sessions, we sent two rounds of reminders to each tutorial to ensure that each teacher is registered to the conference. We collected rough timelines of each tutorial so that the SlidesLive team can work closely during the live sessions. To further help the teachers prepare their live session content we additionally collected the lists of registered attendees twice before the conference, and shared them with the teachers.

In the end, the numbers of registrations for each tutorial were:
* T1 – 1762
* T2 – 793
* T3 – 503
* T4 – 571
* T5 – 641
* T6 – 1288
* T7 – 378
* T8 – 1087
This sums up to 7023 registrations.

===Archiving and Publicity===

There are three main channels for publicity, namely the static [http://www.acl2020.org/program/tutorials/ website], the website of the [https://virtual.acl2020.org/tutorials.html virtual conference] (restricted to the registered participants) and the social media (including [https://twitter.com/aclmeeting/ Twitter]). For the former, we collected photos from each instructor, which are shown with the title and abstract of their tutorials. Following NAACL 2019, we also displayed the time slots as a right-hand-side bar on the tutorial website. Due to special arrangements of the virtual conference, we marked the time zones explicitly. In addition, we asked the tutorial teachers to provide URL links to teaching materials (e.g., at GitHub) during their final submission, and included these links to their tutorial information. For social media, we drafted two blog posts (i.e., [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ this post] and [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]) for advertising the tutorials.

The archiving consists of three main items, namely the tutorial proceedings, the slides for each tutorial and the videos. For the tutorial proceedings, we collected final abstracts from the authors using the START system. We asked the instructors to sign the CC BY v4 agreement when submitting their final materials on SoftConf. For the slides, we asked the tutorial teachers to provide a first version of the slides by June 17 regardless whether they are pre-recorded or interactive. Some tutorials made further updates to their slides before ACL. For the video recordings, we worked with the ACL Anthology team discussing the technical details since this is the first time the tutorial videos are shared in the website. [Note: this is currently undergoing at the time of writing of this summary] One important aspect of for inclusiveness and accessibility are tutorial captions. With the help of SlidesLive, automatic captions generated for pre-recording were collected and sent to the teachers for edition. They should be included in the tutorial archives.

===Lessons Learned===

While organizing a virtual conference it is important to stress that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]).

It is also important to announce the individual modalities with the attendees early enough. For instance, watching pre-recordings in advance was necessary for some tutorials (since their live sessions only included question answering) and such constrains should have been known more in advance.

If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to virtual meeting tools like Zoom only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.

One thing to remember which we missed is to publish the reading list for each tutorial (which the teachers submit with the proposal) on the tutorial website.

2020Q3 Reports: Tutorial Chairs

2020-07-23T11:36:39Z

AgataSavary: /* Archiving and Publicity */

Tutorial Chairs:

<blockquote>Agata Savary, University of Tours, France

Yue Zhang, Westlake University, Hangzhou, China
</blockquote>

=== Preparatory Work, Call, Reviewing and Decision-Making ===
The call, submission, reviewing and selection of tutorials was coordinated jointly for 4 conferences: ACL, AACL-IJCNLP, COLING and EMNLP.

Before drafting the call, we collected lists of tutorials offered within the past 4 years. We analysed previous calls for tutorials and reports from tutorial chairs (from [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs 2016], [https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs 2017], [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs 2018] and [http://aclweb.org/adminwiki/index.php?title=2019Q1_Reports:_ACL_2019 2019]). We consulted previous tutorial chairs with a questionnaire including questions about: the number of submissions, encouraging submissions on specific topics or from specific lecturers, the review procedure, the evaluation criteria, the post-tutorial availability of the slides/codes, and lessons learned from tutorial coordination. We also discussed the publication of slides and video recordings from future tutorials with the persons in charge of the ACL Anthology. As a result of these steps, we created two new sections for the [https://www.aclweb.org/adminwiki/index.php?title=Conference_Handbook ACL Conference Handbook] (future chairs might consider updating these documents yearly):

* the list of [https://www.aclweb.org/adminwiki/index.php?title=Past_tutorials past tutorials] at ACL, COLING, EACL, EMNLP, and NAACL in 2016-2019
* a [https://www.aclweb.org/adminwiki/index.php?title=Tutorial_chair_handbook tutorial chair handbook]

The final [https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020 call] differs from previous calls in several aspects: (i) the expectations about tutorial proposals were made clearer, (ii) following the central ACL decision, the teachers' payment policy was replaced by a fee-waiving policy, (iii) the required submission details include two new items: diversity considerations and agreement for open access publication of slides, codes, data and video recordings, (iv) the evaluation criteria (see below) are announced.

We recruited a review committee of 19 members, including the 8 tutorial chairs and 11 external members selected for their large understanding of the NLP domain and a good experience in reviewing and/or tutorial teaching:

'''Review Committee'''

* Timothy Baldwin (University of Melbourne, Australia) - AACL-IJCNLP 2020 tutorial chair
* Daniel Beck (University of Melbourne, Australia) - COLING 2020 tutorial chair
* Emily M. Bender (University of Washington, WA, USA)
* Erik Cambria (Nanyang Technological University, Singapore)
* Gaël Dias (University of Caen Normandie, France)
* Stefan Evert (Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany)
* Yang Liu (Tsinghua University, Beijing, China)
* Agata Savary (University of Tours, France) - ACL 2020 tutorial chair
* João Sedoc (Johns Hopkins University, Baltimore, MD, USA)
* Lucia Specia (Sheffield University, UK) - COLING 2020 tutorial chair
* Xu Sun (Peking University, China)
* Yulia Tsvetkov (Carnegie Mellon University, Pittsburgh, PA, USA)
* Benjamin Van Durme (Johns Hopkins University, Baltimore, MD, USA) - EMNLP 2020 tutorial chair
* Aline Villavicencio (University of Sheffield, UK and Federal University of Rio Grande do Sul, Brazil) - EMNLP 2020 tutorial chair
* Taro Watanabe (Google, Inc., Tokyo, Japan)
* Aaron Steven White (University of Rochester, NY, USA)
* Fei Xia (University of Washington, WA, USA) - AACL-IJCNLP 2020 tutorial chair
* Yue Zhang (Westlake University, Hangzhou, China) - ACL 2020 tutorial chair
* Meishan Zhang (Tianjin University, China)

In total, we received 43 submissions for the 4 conferences. Each reviewer was assigned 6-7 proposals and each proposal received 3 reviews. The selection criteria included: clarity and preparedness, novelty or timely character of the topic, lecturers' experience, likely audience interest, open access of the teaching material, diversity aspects (multilingualism, gender, age and country of the lecturers), and compatibility with the preferred venues. We accepted 31 proposals.

The decision making was handled via an online meeting of the 8 tutorial chairs. In particular, the selection of tutorials for each conference was done via the expression of interest of the tutorial chairs on a round-robin basis. Some slight adjustments were also performed after the meeting to better fit the authors' preferences. In total, 8, 8, 8 and 7 proposals were selected for ACL, AACL-IJCNLP, COLING and EMNLP, respectively. Upon the announcement the results, 2 of the proposals accepted for AACL-IJCNLP were withdrawn.

The submission, review, selection and collection of final descriptions for all tutorials was handled via a dedicated SoftConf space, shared by the 4 coordinating conferences. After the selection of proposals, a separate track was created on SoftConf for each conference. The final submission page (one per conference) was set up so as to collect all the necessary data including notably: the tutorial slides, URLs for course material (if any), printable material (if any) and agreement for open access publication.

===Tutorials Selected for ACL 2020===

The final selection for ACL 2020 consists of the following 8 tutorials of 3 hours each (each of them had ACL as the preferred or the second preferred venue):

'''T1: Interpretability and Analysis in Neural NLP''' (cutting-edge) 
Yonatan Belinkov, Sebastian Gehrmann and Ellie Pavlick 
While deep learning has transformed the NLP field and impacted the larger computational linguistics community, the rise of neural networks is stained by their opaque nature: It is challenging to interpret the inner workings of neural network models, and explicate their behavior. Therefore, in the last few years, an increasingly large body of work has been devoted to the analysis and interpretation of neural network models in NLP. 
This body of work is so far lacking a common framework and methodology. Moreover, approaching the analysis of modern neural networks can be difficult for newcomers to the field. This tutorial aims to fill this gap and introduce the nascent field of interpretability and analysis of neural networks in NLP. 
The tutorial covers the main lines of analysis work, such as probing classifier, behavior studies and test suites, psycholinguistic methods, visualizations, adversarial examples, and other methods. We highlight not only the most commonly applied analysis methods, but also the specific limitations and shortcomings of current approaches, in order to inform participants where to focus future efforts.

'''T2: Multi-modal Information Extraction from Text, Semi-structured, and Tabular Data on the Web''' (cutting-edge) 
Xin Luna Dong, Hannaneh Hajishirzi, Colin Lockard and Prashant Shiralkar 
The World Wide Web contains vast quantities of textual information in several forms: unstructured text, template-based semi-structured webpages (which present data in key-value pairs and lists), and tables. Methods for extracting information from these sources and converting it to a structured form have been a target of research from the natural language processing (NLP), data mining, and database communities. While these researchers have largely separated extraction from web data into different problems based on the modality of the data, they have faced similar problems such as learning with limited labeled data, defining (or avoiding defining) ontologies, making use of prior knowledge, and scaling solutions to deal with the size of the Web. 
In this tutorial we take a holistic view toward information extraction, exploring the commonalities in the challenges and solutions developed to address these different forms of text. We will explore the approaches targeted at unstructured text that largely rely on learning syntactic or semantic textual patterns, approaches targeted at semi-structured documents that learn to identify structural patterns in the template, and approaches targeting web tables which rely heavily on entity linking and type information. 
While these different data modalities have largely been considered separately in the past, recent research has started taking a more inclusive approach toward textual extraction, in which the multiple signals offered by textual, layout, and visual clues are combined into a single extraction model made possible by new deep learning approaches. At the same time, trends within purely textual extraction have shifted toward full-document understanding rather than considering sentences as independent units. With this in mind, it is worth considering the information extraction problem as a whole to motivate solutions that harness textual semantics along with visual and semi-structured layout information. We will discuss these approaches and suggest avenues for future work.

'''T3: Reviewing Natural Language Processing Research''' (introductory) 
Kevin Cohen, Karën Fort, Margot Mieskes and Aurélie Névéol 
As the demand for reviewing grows, so must the pool of reviewers. As the [http://www.livecongress.it/aol/indexSA.php?id=E2EAED7D&ticket= survey] presented by Graham Neubig at the 2019 ACL showed, a considerable number of reviewers are junior researchers, who might lack the experience and expertise necessary for high-quality reviews. Some of them might not have the environment or lack opportunities that allow them to learn the skills necessary. A tutorial on reviewing for the NLP community might increase reviewers’ confidence, as well as the quality of the reviews. This introductory tutorial will cover the goals, processes, and evaluation of reviewing research papers in natural language processing.

'''T4: Stylized Text Generation: Approaches and Applications''' (cutting-edge) 
Lili Mou and Olga Vechtomova 
Text generation has played an important role in various applications of natural language processing (NLP), and kn recent studies, researchers are paying increasing attention to modeling and manipulating the style of the generation text, which we call stylized text generation. In this tutorial, we will provide a comprehensive literature review in this direction. We start from the definition of style and different settings of stylized text generation, illustrated with various applications. Then, we present different settings of stylized generation, such as parallel supervised, style label-supervised, and unsupervised. In each setting, we delve deep into machine learning methods, including embedding learning techniques to represent style}, adversarial learning and reinforcement learning with cycle consistency to match content but to distinguish different styles. We also introduce current approaches of evaluating stylized text generation systems. We conclude our tutorial by presenting the challenges of stylized text generation and discussing future directions, such as small-data training, non-categorical style modeling, and a generalized scope of style transfer (e.g., controlling the syntax as a style).

'''T5: Achieving Common Ground in Multi-modal Dialogue''' (cutting-edge) 
Malihe Alikhani and Matthew Stone 
All communication aims at achieving common ground (grounding): interlocutors can work together effectively only with mutual beliefs about what the state of the world is, about what their goals are, and about how they plan to make their goals a reality. Computational dialogue research offers some classic results on grouding, which unfortunately offer scant guidance to the design of grounding modules and behaviors in cutting-edge systems. In this tutorial, we focus on three main topic areas: 1) grounding in human-human communication; 2) grounding in dialogue systems; and 3) grounding in multi-modal interactive systems, including image-oriented conversations and human-robot interactions. We highlight a number of achievements of recent computational research in coordinating complex content, show how these results lead to rich and challenging opportunities for doing grounding in more flexible and powerful ways, and canvass relevant insights from the literature on human--human conversation. We expect that the tutorial will be of interest to researchers in dialogue systems, computational semantics and cognitive modeling, and hope that it will catalyze research and system building that more directly explores the creative, strategic ways conversational agents might be able to seek and offer evidence about their understanding of their interlocutors.

'''T6: Commonsense Reasoning for Natural Language Processing''' (introductory) 
Maarten Sap, Vered Shwartz, Antoine Bosselut, Dan Roth and Yejin Choi 
In our tutorial, we (1) outline the various types of commonsense (e.g., physical, social), and (2) discuss techniques to gather and represent commonsense knowledge, while highlighting the challenges specific to this type of knowledge (e.g., reporting bias). We will then (3) discuss the types of commonsense knowledge captured by modern NLP systems (e.g., large pretrained language models), and (4) present ways to measure systems' commonsense reasoning abilities. We finish with (5) a discussion of various ways in which commonsense reasoning can be used to improve performance on NLP tasks, exemplified by an (6) interactive session on integrating commonsense into a downstream task.

'''T7: Integrating Ethics into the NLP Curriculum''' (introductory) 
Emily M. Bender, Dirk Hovy and Alexandra Schofield 
Our goal in this tutorial is to empower NLP researchers and practitioners with tools and resources to teach others about how to ethically apply NLP techniques. Our tutorial will present both high-level strategies for developing an ethics-oriented curriculum, based on experience and best practices, as well as specific sample exercises that can be brought to a classroom. We plan to make this a highly interactive work session culminating in a shared online resource page that pools lesson plans, assignments, exercise ideas, reading suggestions, and ideas from the attendees. We consider three primary topics with our session that frequently underlie ethical issues in NLP research: Dual use, bias and privacy. 
In this setting, a key lesson is that there is no single approach to ethical NLP: each project requires thoughtful consideration about what steps can be taken to best support people affected by that project. However, we can learn (and teach) what kinds of issues to be aware of and what kinds of strategies are available for mitigating harm. To teach this process, we apply and promote interactive exercises that provide an opportunity to ideate, discuss, and reflect. We plan to facilitate this in a way that encourages positive discussion, emphasizing the creation of ideas for the future instead of negative opinions of previous work.

'''T8: Recent Advances in Open-Domain Question Answering''' (cutting-edge) 
Danqi Chen and Scott Wen-tau Yih 
Open-domain (textual) question answering (QA), the task of finding answers to open-domain questions by searching a large collection of documents, has been a long-standing problem in NLP, information retrieval (IR) and related fields (Voorhees et al., 1999; Moldovan et al., 2000; Brill et al.,2002; Ferrucci et al., 2010). Traditional QA systems were usually constructed as a pipeline, consisting of many different components such as question processing, document/passage retrieval and answer processing. With the rapid development of neural reading comprehension (Chen, 2018), modern open-domain QA systems have been restructured by combining traditional IR techniques and neural reading comprehension models (Chen et al., 2017; Yang et al., 2019) or even implemented in a fully end-to-end fashion (Lee et al., 2019; Seo et al., 2019). While the system architecture has been drastically simplified, two technical challenges remain critical:(1) “Retriever”: finding documents that (might)contain an answer from a large collection of documents; (2) “Reader”: finding the answer in a given paragraph or a document. 
In this tutorial, we aim to provide a comprehensive and coherent overview of recent advances in this line of research. We will start by first giving a brief historical background of open-domain question answering, discussing the basic setup and core technical challenges of the research problem.The focus will then shift to modern techniques and resources proposed for open-domain QA, including the basics of latest neural reading comprehension systems, new datasets and models. The scope will also be broadened to cover the information retrieval component on how to effectively identify passages relevant to the questions. Moreover, in-depth discussions will be given on the use of traditional / neural IR modules, as well as the trade-offs between modular design and end-to-end training. If time permits, we also plan to discuss some hybrid approaches for answering questions using both text and large knowledge bases (e.g. (Sun et al., 2018)) and give a critical review on how structured data complements the information from unstructured text. 
At the end of our tutorial, we will discuss some important questions, including (1) How much progress have we made compared to the QA systems developed in the last decade?(2) What are the main challenges and limitations of cur-rent approaches? (3) How to trade off the efficiency (computational time and memory requirements) and accuracy in the deep learning era? We hope that our tutorial will not only serve as a useful resource for the audience to efficiently acquire the up-to-date knowledge, but also provide new perspectives to stimulate the advances of open-domain QA research in the next phase.

===Virtual Conference Organization===

In April the whole conference was decided to be held in virtual. We carried on the preparations jointly with the tutorial instructors, the virtual infrastructure chairs, the website chairs and the SlidesLive team to set up the virtual conference infrastructure, and to publicize it for each tutorial.

We first made a decision on the time slots for tutorial presentations. Five major time zones were considered, including US west coast, US east coast, central Europe, China/Asia and Australia (India was also considered). Our aim was to offer at least two convenient time slots (i.e. within regular working hours) for each of these zones.

The final time slots include (Jul 5 Pacific Time):

* slot 1 – 7:00 PM (-1D) to 10:30 PM (-1D)
* slot 2 – 12:00 AM to 3:30 AM
* slot 3 – 6:00 AM to 9:30 AM
* slot 4 – 10:30 AM to 2:00 PM
* slot 5 – 3:00 PM to 6:30 PM

We asked the tutorial teachers to indicate their preferences among the time slots, and – if possible – to agree to give their tutorial twice, so as to increase accessibility. Our final decisions on slot assignments traded off the teacher preferences and slot availability. In the end, the assignments were the following (see also the [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ blog post] and [https://acl2020.org/program/tutorials/ online schedule]):

* T1 – twice on slots 3 and 4
* T2 – once on slot 5
* T3 – once on slot 4
* T4 – once on slot 4
* T5 – twice on slots 3 and 5
* T6 – once on slot 5
* T7 – twice on slots 3 and 4
* T8 – once on slot 5

Unfortunately, the tutorial teachers proved hardly available for slots 1 and 2.

In addition to time slot arrangements, we gave the tutorial teacher two options on presentation, namely a pre-recorded and a fully interactive form of tutorials. The former allows the teachers to pre-record the main content of their tutorials, and use the time slots mainly for question answering and detailed discussions. The latter is more interactive, the teachers use the online sessions for both giving the lecture and other activities such as active discussions. All these arrangements were made for taking advantage of the virtual conference. In the end the choices were the following:

* T1 – pre-recorded
* T2 – interactive (the initial decision was pre-recorded but a change was made in June)
* T3 – pre-recorded
* T4 – pre-recorded
* T5 – pre-recorded
* T6 – interactive
* T7 – interactive
* T8 – interactive

After making decisions on time slots and teaching forms, we started to work on the technical details of the virtual conference infrastructure jointly with the infrastructure chairs, the website chairs, the SlidesLive support team and individual tutorial teachers. We collected needs from each tutorial from time to time, inquired information from the SlidesLive team, and tried to accommodate specific requirement for each tutorial. Since this is the first time ACL was virtual, it typically took several rounds of information exchange for making a certain technical feature work. Major things discussed included:

* Sorting out how to pre-record a tutorial using the system
* Recording of interactive sessions
* Additional RocketChat channel
* Virtual tutorial webpage setup
* Coordinators for tutorial sessions and SlidesLive team participation
* Rehearsal of interactive sessions
* Captioning of pre-recorded and interactive tutorials
* Privacy issues
* Special needs from particular tutorials
** Integrating [https://www.dory.app/ Dory] for question answering (in T1)
** Fully interactive sessions, with occasional split into break-out rooms (in T7)

To ensure smooth running of the sessions, we sent two rounds of reminders to each tutorial to ensure that each teacher is registered to the conference. We collected rough timelines of each tutorial so that the SlidesLive team can work closely during the live sessions. To further help the teachers prepare their live session content we additionally collected the lists of registered attendees twice before the conference, and shared them with the teachers.

In the end, the numbers of registrations for each tutorial were:
* T1 -- 1762
* T2 -- 793
* T3 -- 503
* T4 -- 571
* T5 -- 641
* T6 -- 1288
* T7 -- 378
* T8 – 1087
This sums up to 7023 registrations.

===Archiving and Publicity===

There are three main channels for publicity, namely the static [http://www.acl2020.org/program/tutorials/ website], the website of the [https://virtual.acl2020.org/tutorials.html virtual conference] (restricted to the registered participants) and the social media (including [https://twitter.com/aclmeeting/ Twitter]). For the former, we collected photos from each instructor, which are shown with the title and abstract of their tutorials. Following NAACL 2019, we also displayed the time slots as a right-hand-side bar on the tutorial website. Due to special arrangements of the virtual conference, we marked the time zones explicitly. In addition, we asked the tutorial teachers to provide URL links to teaching materials (e.g., at GitHub) during their final submission, and included these links to their tutorial information. For social media, we drafted two blog posts (i.e., [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ this post] and [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]) for advertising the tutorials.

The archiving consists of three main items, namely the tutorial proceedings, the slides for each tutorial and the videos. For the tutorial proceedings, we collected final abstracts from the authors using the START system. We asked the instructors to sign the CC BY v4 agreement when submitting their final materials on SoftConf. For the slides, we asked the tutorial teachers to provide a first version of the slides by June 17 regardless whether they are pre-recorded or interactive. Some tutorials made further updates to their slides before ACL. For the video recordings, we worked with the ACL Anthology team discussing the technical details since this is the first time the tutorial videos are shared in the website. [Note: this is currently undergoing at the time of writing of this summary] One important aspect of for inclusiveness and accessibility are tutorial captions. With the help of SlidesLive, automatic captions generated for pre-recording were collected and sent to the teachers for edition. They should be included in the tutorial archives.

===Lessons Learned===

While organizing a virtual conference it is important to stress that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]).

It is also important to announce the individual modalities with the attendees early enough. For instance, watching pre-recordings in advance was necessary for some tutorials (since their live sessions only included question answering) and such constrains should have been known more in advance.

If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to virtual meeting tools like Zoom only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.

One thing to remember which we missed is to publish the reading list for each tutorial (which the teachers submit with the proposal) on the tutorial website.

2020Q3 Reports: Tutorial Chairs

2020-07-23T11:36:02Z

AgataSavary: /* Archiving and Publicity */

Tutorial Chairs:

<blockquote>Agata Savary, University of Tours, France

Yue Zhang, Westlake University, Hangzhou, China
</blockquote>

=== Preparatory Work, Call, Reviewing and Decision-Making ===
The call, submission, reviewing and selection of tutorials was coordinated jointly for 4 conferences: ACL, AACL-IJCNLP, COLING and EMNLP.

Before drafting the call, we collected lists of tutorials offered within the past 4 years. We analysed previous calls for tutorials and reports from tutorial chairs (from [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs 2016], [https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs 2017], [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs 2018] and [http://aclweb.org/adminwiki/index.php?title=2019Q1_Reports:_ACL_2019 2019]). We consulted previous tutorial chairs with a questionnaire including questions about: the number of submissions, encouraging submissions on specific topics or from specific lecturers, the review procedure, the evaluation criteria, the post-tutorial availability of the slides/codes, and lessons learned from tutorial coordination. We also discussed the publication of slides and video recordings from future tutorials with the persons in charge of the ACL Anthology. As a result of these steps, we created two new sections for the [https://www.aclweb.org/adminwiki/index.php?title=Conference_Handbook ACL Conference Handbook] (future chairs might consider updating these documents yearly):

* the list of [https://www.aclweb.org/adminwiki/index.php?title=Past_tutorials past tutorials] at ACL, COLING, EACL, EMNLP, and NAACL in 2016-2019
* a [https://www.aclweb.org/adminwiki/index.php?title=Tutorial_chair_handbook tutorial chair handbook]

The final [https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020 call] differs from previous calls in several aspects: (i) the expectations about tutorial proposals were made clearer, (ii) following the central ACL decision, the teachers' payment policy was replaced by a fee-waiving policy, (iii) the required submission details include two new items: diversity considerations and agreement for open access publication of slides, codes, data and video recordings, (iv) the evaluation criteria (see below) are announced.

We recruited a review committee of 19 members, including the 8 tutorial chairs and 11 external members selected for their large understanding of the NLP domain and a good experience in reviewing and/or tutorial teaching:

'''Review Committee'''

* Timothy Baldwin (University of Melbourne, Australia) - AACL-IJCNLP 2020 tutorial chair
* Daniel Beck (University of Melbourne, Australia) - COLING 2020 tutorial chair
* Emily M. Bender (University of Washington, WA, USA)
* Erik Cambria (Nanyang Technological University, Singapore)
* Gaël Dias (University of Caen Normandie, France)
* Stefan Evert (Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany)
* Yang Liu (Tsinghua University, Beijing, China)
* Agata Savary (University of Tours, France) - ACL 2020 tutorial chair
* João Sedoc (Johns Hopkins University, Baltimore, MD, USA)
* Lucia Specia (Sheffield University, UK) - COLING 2020 tutorial chair
* Xu Sun (Peking University, China)
* Yulia Tsvetkov (Carnegie Mellon University, Pittsburgh, PA, USA)
* Benjamin Van Durme (Johns Hopkins University, Baltimore, MD, USA) - EMNLP 2020 tutorial chair
* Aline Villavicencio (University of Sheffield, UK and Federal University of Rio Grande do Sul, Brazil) - EMNLP 2020 tutorial chair
* Taro Watanabe (Google, Inc., Tokyo, Japan)
* Aaron Steven White (University of Rochester, NY, USA)
* Fei Xia (University of Washington, WA, USA) - AACL-IJCNLP 2020 tutorial chair
* Yue Zhang (Westlake University, Hangzhou, China) - ACL 2020 tutorial chair
* Meishan Zhang (Tianjin University, China)

In total, we received 43 submissions for the 4 conferences. Each reviewer was assigned 6-7 proposals and each proposal received 3 reviews. The selection criteria included: clarity and preparedness, novelty or timely character of the topic, lecturers' experience, likely audience interest, open access of the teaching material, diversity aspects (multilingualism, gender, age and country of the lecturers), and compatibility with the preferred venues. We accepted 31 proposals.

The decision making was handled via an online meeting of the 8 tutorial chairs. In particular, the selection of tutorials for each conference was done via the expression of interest of the tutorial chairs on a round-robin basis. Some slight adjustments were also performed after the meeting to better fit the authors' preferences. In total, 8, 8, 8 and 7 proposals were selected for ACL, AACL-IJCNLP, COLING and EMNLP, respectively. Upon the announcement the results, 2 of the proposals accepted for AACL-IJCNLP were withdrawn.

The submission, review, selection and collection of final descriptions for all tutorials was handled via a dedicated SoftConf space, shared by the 4 coordinating conferences. After the selection of proposals, a separate track was created on SoftConf for each conference. The final submission page (one per conference) was set up so as to collect all the necessary data including notably: the tutorial slides, URLs for course material (if any), printable material (if any) and agreement for open access publication.

===Tutorials Selected for ACL 2020===

The final selection for ACL 2020 consists of the following 8 tutorials of 3 hours each (each of them had ACL as the preferred or the second preferred venue):

'''T1: Interpretability and Analysis in Neural NLP''' (cutting-edge) 
Yonatan Belinkov, Sebastian Gehrmann and Ellie Pavlick 
While deep learning has transformed the NLP field and impacted the larger computational linguistics community, the rise of neural networks is stained by their opaque nature: It is challenging to interpret the inner workings of neural network models, and explicate their behavior. Therefore, in the last few years, an increasingly large body of work has been devoted to the analysis and interpretation of neural network models in NLP. 
This body of work is so far lacking a common framework and methodology. Moreover, approaching the analysis of modern neural networks can be difficult for newcomers to the field. This tutorial aims to fill this gap and introduce the nascent field of interpretability and analysis of neural networks in NLP. 
The tutorial covers the main lines of analysis work, such as probing classifier, behavior studies and test suites, psycholinguistic methods, visualizations, adversarial examples, and other methods. We highlight not only the most commonly applied analysis methods, but also the specific limitations and shortcomings of current approaches, in order to inform participants where to focus future efforts.

'''T2: Multi-modal Information Extraction from Text, Semi-structured, and Tabular Data on the Web''' (cutting-edge) 
Xin Luna Dong, Hannaneh Hajishirzi, Colin Lockard and Prashant Shiralkar 
The World Wide Web contains vast quantities of textual information in several forms: unstructured text, template-based semi-structured webpages (which present data in key-value pairs and lists), and tables. Methods for extracting information from these sources and converting it to a structured form have been a target of research from the natural language processing (NLP), data mining, and database communities. While these researchers have largely separated extraction from web data into different problems based on the modality of the data, they have faced similar problems such as learning with limited labeled data, defining (or avoiding defining) ontologies, making use of prior knowledge, and scaling solutions to deal with the size of the Web. 
In this tutorial we take a holistic view toward information extraction, exploring the commonalities in the challenges and solutions developed to address these different forms of text. We will explore the approaches targeted at unstructured text that largely rely on learning syntactic or semantic textual patterns, approaches targeted at semi-structured documents that learn to identify structural patterns in the template, and approaches targeting web tables which rely heavily on entity linking and type information. 
While these different data modalities have largely been considered separately in the past, recent research has started taking a more inclusive approach toward textual extraction, in which the multiple signals offered by textual, layout, and visual clues are combined into a single extraction model made possible by new deep learning approaches. At the same time, trends within purely textual extraction have shifted toward full-document understanding rather than considering sentences as independent units. With this in mind, it is worth considering the information extraction problem as a whole to motivate solutions that harness textual semantics along with visual and semi-structured layout information. We will discuss these approaches and suggest avenues for future work.

'''T3: Reviewing Natural Language Processing Research''' (introductory) 
Kevin Cohen, Karën Fort, Margot Mieskes and Aurélie Névéol 
As the demand for reviewing grows, so must the pool of reviewers. As the [http://www.livecongress.it/aol/indexSA.php?id=E2EAED7D&ticket= survey] presented by Graham Neubig at the 2019 ACL showed, a considerable number of reviewers are junior researchers, who might lack the experience and expertise necessary for high-quality reviews. Some of them might not have the environment or lack opportunities that allow them to learn the skills necessary. A tutorial on reviewing for the NLP community might increase reviewers’ confidence, as well as the quality of the reviews. This introductory tutorial will cover the goals, processes, and evaluation of reviewing research papers in natural language processing.

'''T4: Stylized Text Generation: Approaches and Applications''' (cutting-edge) 
Lili Mou and Olga Vechtomova 
Text generation has played an important role in various applications of natural language processing (NLP), and kn recent studies, researchers are paying increasing attention to modeling and manipulating the style of the generation text, which we call stylized text generation. In this tutorial, we will provide a comprehensive literature review in this direction. We start from the definition of style and different settings of stylized text generation, illustrated with various applications. Then, we present different settings of stylized generation, such as parallel supervised, style label-supervised, and unsupervised. In each setting, we delve deep into machine learning methods, including embedding learning techniques to represent style}, adversarial learning and reinforcement learning with cycle consistency to match content but to distinguish different styles. We also introduce current approaches of evaluating stylized text generation systems. We conclude our tutorial by presenting the challenges of stylized text generation and discussing future directions, such as small-data training, non-categorical style modeling, and a generalized scope of style transfer (e.g., controlling the syntax as a style).

'''T5: Achieving Common Ground in Multi-modal Dialogue''' (cutting-edge) 
Malihe Alikhani and Matthew Stone 
All communication aims at achieving common ground (grounding): interlocutors can work together effectively only with mutual beliefs about what the state of the world is, about what their goals are, and about how they plan to make their goals a reality. Computational dialogue research offers some classic results on grouding, which unfortunately offer scant guidance to the design of grounding modules and behaviors in cutting-edge systems. In this tutorial, we focus on three main topic areas: 1) grounding in human-human communication; 2) grounding in dialogue systems; and 3) grounding in multi-modal interactive systems, including image-oriented conversations and human-robot interactions. We highlight a number of achievements of recent computational research in coordinating complex content, show how these results lead to rich and challenging opportunities for doing grounding in more flexible and powerful ways, and canvass relevant insights from the literature on human--human conversation. We expect that the tutorial will be of interest to researchers in dialogue systems, computational semantics and cognitive modeling, and hope that it will catalyze research and system building that more directly explores the creative, strategic ways conversational agents might be able to seek and offer evidence about their understanding of their interlocutors.

'''T6: Commonsense Reasoning for Natural Language Processing''' (introductory) 
Maarten Sap, Vered Shwartz, Antoine Bosselut, Dan Roth and Yejin Choi 
In our tutorial, we (1) outline the various types of commonsense (e.g., physical, social), and (2) discuss techniques to gather and represent commonsense knowledge, while highlighting the challenges specific to this type of knowledge (e.g., reporting bias). We will then (3) discuss the types of commonsense knowledge captured by modern NLP systems (e.g., large pretrained language models), and (4) present ways to measure systems' commonsense reasoning abilities. We finish with (5) a discussion of various ways in which commonsense reasoning can be used to improve performance on NLP tasks, exemplified by an (6) interactive session on integrating commonsense into a downstream task.

'''T7: Integrating Ethics into the NLP Curriculum''' (introductory) 
Emily M. Bender, Dirk Hovy and Alexandra Schofield 
Our goal in this tutorial is to empower NLP researchers and practitioners with tools and resources to teach others about how to ethically apply NLP techniques. Our tutorial will present both high-level strategies for developing an ethics-oriented curriculum, based on experience and best practices, as well as specific sample exercises that can be brought to a classroom. We plan to make this a highly interactive work session culminating in a shared online resource page that pools lesson plans, assignments, exercise ideas, reading suggestions, and ideas from the attendees. We consider three primary topics with our session that frequently underlie ethical issues in NLP research: Dual use, bias and privacy. 
In this setting, a key lesson is that there is no single approach to ethical NLP: each project requires thoughtful consideration about what steps can be taken to best support people affected by that project. However, we can learn (and teach) what kinds of issues to be aware of and what kinds of strategies are available for mitigating harm. To teach this process, we apply and promote interactive exercises that provide an opportunity to ideate, discuss, and reflect. We plan to facilitate this in a way that encourages positive discussion, emphasizing the creation of ideas for the future instead of negative opinions of previous work.

'''T8: Recent Advances in Open-Domain Question Answering''' (cutting-edge) 
Danqi Chen and Scott Wen-tau Yih 
Open-domain (textual) question answering (QA), the task of finding answers to open-domain questions by searching a large collection of documents, has been a long-standing problem in NLP, information retrieval (IR) and related fields (Voorhees et al., 1999; Moldovan et al., 2000; Brill et al.,2002; Ferrucci et al., 2010). Traditional QA systems were usually constructed as a pipeline, consisting of many different components such as question processing, document/passage retrieval and answer processing. With the rapid development of neural reading comprehension (Chen, 2018), modern open-domain QA systems have been restructured by combining traditional IR techniques and neural reading comprehension models (Chen et al., 2017; Yang et al., 2019) or even implemented in a fully end-to-end fashion (Lee et al., 2019; Seo et al., 2019). While the system architecture has been drastically simplified, two technical challenges remain critical:(1) “Retriever”: finding documents that (might)contain an answer from a large collection of documents; (2) “Reader”: finding the answer in a given paragraph or a document. 
In this tutorial, we aim to provide a comprehensive and coherent overview of recent advances in this line of research. We will start by first giving a brief historical background of open-domain question answering, discussing the basic setup and core technical challenges of the research problem.The focus will then shift to modern techniques and resources proposed for open-domain QA, including the basics of latest neural reading comprehension systems, new datasets and models. The scope will also be broadened to cover the information retrieval component on how to effectively identify passages relevant to the questions. Moreover, in-depth discussions will be given on the use of traditional / neural IR modules, as well as the trade-offs between modular design and end-to-end training. If time permits, we also plan to discuss some hybrid approaches for answering questions using both text and large knowledge bases (e.g. (Sun et al., 2018)) and give a critical review on how structured data complements the information from unstructured text. 
At the end of our tutorial, we will discuss some important questions, including (1) How much progress have we made compared to the QA systems developed in the last decade?(2) What are the main challenges and limitations of cur-rent approaches? (3) How to trade off the efficiency (computational time and memory requirements) and accuracy in the deep learning era? We hope that our tutorial will not only serve as a useful resource for the audience to efficiently acquire the up-to-date knowledge, but also provide new perspectives to stimulate the advances of open-domain QA research in the next phase.

===Virtual Conference Organization===

In April the whole conference was decided to be held in virtual. We carried on the preparations jointly with the tutorial instructors, the virtual infrastructure chairs, the website chairs and the SlidesLive team to set up the virtual conference infrastructure, and to publicize it for each tutorial.

We first made a decision on the time slots for tutorial presentations. Five major time zones were considered, including US west coast, US east coast, central Europe, China/Asia and Australia (India was also considered). Our aim was to offer at least two convenient time slots (i.e. within regular working hours) for each of these zones.

The final time slots include (Jul 5 Pacific Time):

* slot 1 – 7:00 PM (-1D) to 10:30 PM (-1D)
* slot 2 – 12:00 AM to 3:30 AM
* slot 3 – 6:00 AM to 9:30 AM
* slot 4 – 10:30 AM to 2:00 PM
* slot 5 – 3:00 PM to 6:30 PM

We asked the tutorial teachers to indicate their preferences among the time slots, and – if possible – to agree to give their tutorial twice, so as to increase accessibility. Our final decisions on slot assignments traded off the teacher preferences and slot availability. In the end, the assignments were the following (see also the [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ blog post] and [https://acl2020.org/program/tutorials/ online schedule]):

* T1 – twice on slots 3 and 4
* T2 – once on slot 5
* T3 – once on slot 4
* T4 – once on slot 4
* T5 – twice on slots 3 and 5
* T6 – once on slot 5
* T7 – twice on slots 3 and 4
* T8 – once on slot 5

Unfortunately, the tutorial teachers proved hardly available for slots 1 and 2.

In addition to time slot arrangements, we gave the tutorial teacher two options on presentation, namely a pre-recorded and a fully interactive form of tutorials. The former allows the teachers to pre-record the main content of their tutorials, and use the time slots mainly for question answering and detailed discussions. The latter is more interactive, the teachers use the online sessions for both giving the lecture and other activities such as active discussions. All these arrangements were made for taking advantage of the virtual conference. In the end the choices were the following:

* T1 – pre-recorded
* T2 – interactive (the initial decision was pre-recorded but a change was made in June)
* T3 – pre-recorded
* T4 – pre-recorded
* T5 – pre-recorded
* T6 – interactive
* T7 – interactive
* T8 – interactive

After making decisions on time slots and teaching forms, we started to work on the technical details of the virtual conference infrastructure jointly with the infrastructure chairs, the website chairs, the SlidesLive support team and individual tutorial teachers. We collected needs from each tutorial from time to time, inquired information from the SlidesLive team, and tried to accommodate specific requirement for each tutorial. Since this is the first time ACL was virtual, it typically took several rounds of information exchange for making a certain technical feature work. Major things discussed included:

* Sorting out how to pre-record a tutorial using the system
* Recording of interactive sessions
* Additional RocketChat channel
* Virtual tutorial webpage setup
* Coordinators for tutorial sessions and SlidesLive team participation
* Rehearsal of interactive sessions
* Captioning of pre-recorded and interactive tutorials
* Privacy issues
* Special needs from particular tutorials
** Integrating [https://www.dory.app/ Dory] for question answering (in T1)
** Fully interactive sessions, with occasional split into break-out rooms (in T7)

To ensure smooth running of the sessions, we sent two rounds of reminders to each tutorial to ensure that each teacher is registered to the conference. We collected rough timelines of each tutorial so that the SlidesLive team can work closely during the live sessions. To further help the teachers prepare their live session content we additionally collected the lists of registered attendees twice before the conference, and shared them with the teachers.

In the end, the numbers of registrations for each tutorial were:
* T1 -- 1762
* T2 -- 793
* T3 -- 503
* T4 -- 571
* T5 -- 641
* T6 -- 1288
* T7 -- 378
* T8 – 1087
This sums up to 7023 registrations.

===Archiving and Publicity===

There are three main channels for publicity, namely the static [http://www.acl2020.org/program/tutorials/ website], the website of the [https://virtual.acl2020.org/tutorials.html virtual conference] (restricted to the registered participants) and the social media (including [https://twitter.com/aclmeeting/ Twitter]). For the former, we collected photos from each instructor, which are shown with the title and abstract of their tutorials. Following NAACL 2019, we also displayed the time slots as a right-hand-side bar on the tutorial website. Due to special arrangements of the virtual conference, we marked the time zones explicitly. In addition, we asked the tutorial teachers to provide URL links to teaching materials (e.g., at GitHub) during their final submission, and included these links to their tutorial information. For social media, we drafted two blog posts (i.e., [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ this post] and [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]) for advertising the tutorials.

The archiving consists of three main items, namely the tutorial proceedings, the slides for each tutorial and the videos. For the tutorial proceedings, we collected final abstracts from the authors using the START system. We asked the instructors to sign the CC BY v4 agreement when submitting their final materials on SoftConf. For the slides, we asked the tutorial teachers to provide a first version of the slides by June 17 regardless whether they are pre-recorded or interactive. Some tutorials made further updates to their slides before ACL. For the video recordings, we worked with the ACL Anthology team discussing the technical details since this is the first time the tutorial videos are shared in the website. [Note: this is currently undergoing at the time of writing of this summary] One important aspect of for inclusiveness and accessibility are tutorial captions. With the help of SlidesLive, automatic captions produced for pre-recording were collected and sent to the teachers for edition. They should be included in the tutorial archives.

===Lessons Learned===

While organizing a virtual conference it is important to stress that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]).

It is also important to announce the individual modalities with the attendees early enough. For instance, watching pre-recordings in advance was necessary for some tutorials (since their live sessions only included question answering) and such constrains should have been known more in advance.

If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to virtual meeting tools like Zoom only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.

One thing to remember which we missed is to publish the reading list for each tutorial (which the teachers submit with the proposal) on the tutorial website.

2020Q3 Reports: Tutorial Chairs

2020-07-23T10:29:16Z

AgataSavary: /* Lessons Learned */

Tutorial Chairs:

<blockquote>Agata Savary, University of Tours, France

Yue Zhang, Westlake University, Hangzhou, China
</blockquote>

=== Preparatory Work, Call, Reviewing and Decision-Making ===
The call, submission, reviewing and selection of tutorials was coordinated jointly for 4 conferences: ACL, AACL-IJCNLP, COLING and EMNLP.

Before drafting the call, we collected lists of tutorials offered within the past 4 years. We analysed previous calls for tutorials and reports from tutorial chairs (from [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs 2016], [https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs 2017], [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs 2018] and [http://aclweb.org/adminwiki/index.php?title=2019Q1_Reports:_ACL_2019 2019]). We consulted previous tutorial chairs with a questionnaire including questions about: the number of submissions, encouraging submissions on specific topics or from specific lecturers, the review procedure, the evaluation criteria, the post-tutorial availability of the slides/codes, and lessons learned from tutorial coordination. We also discussed the publication of slides and video recordings from future tutorials with the persons in charge of the ACL Anthology. As a result of these steps, we created two new sections for the [https://www.aclweb.org/adminwiki/index.php?title=Conference_Handbook ACL Conference Handbook] (future chairs might consider updating these documents yearly):

* the list of [https://www.aclweb.org/adminwiki/index.php?title=Past_tutorials past tutorials] at ACL, COLING, EACL, EMNLP, and NAACL in 2016-2019
* a [https://www.aclweb.org/adminwiki/index.php?title=Tutorial_chair_handbook tutorial chair handbook]

The final [https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020 call] differs from previous calls in several aspects: (i) the expectations about tutorial proposals were made clearer, (ii) following the central ACL decision, the teachers' payment policy was replaced by a fee-waiving policy, (iii) the required submission details include two new items: diversity considerations and agreement for open access publication of slides, codes, data and video recordings, (iv) the evaluation criteria (see below) are announced.

We recruited a review committee of 19 members, including the 8 tutorial chairs and 11 external members selected for their large understanding of the NLP domain and a good experience in reviewing and/or tutorial teaching:

'''Review Committee'''

* Timothy Baldwin (University of Melbourne, Australia) - AACL-IJCNLP 2020 tutorial chair
* Daniel Beck (University of Melbourne, Australia) - COLING 2020 tutorial chair
* Emily M. Bender (University of Washington, WA, USA)
* Erik Cambria (Nanyang Technological University, Singapore)
* Gaël Dias (University of Caen Normandie, France)
* Stefan Evert (Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany)
* Yang Liu (Tsinghua University, Beijing, China)
* Agata Savary (University of Tours, France) - ACL 2020 tutorial chair
* João Sedoc (Johns Hopkins University, Baltimore, MD, USA)
* Lucia Specia (Sheffield University, UK) - COLING 2020 tutorial chair
* Xu Sun (Peking University, China)
* Yulia Tsvetkov (Carnegie Mellon University, Pittsburgh, PA, USA)
* Benjamin Van Durme (Johns Hopkins University, Baltimore, MD, USA) - EMNLP 2020 tutorial chair
* Aline Villavicencio (University of Sheffield, UK and Federal University of Rio Grande do Sul, Brazil) - EMNLP 2020 tutorial chair
* Taro Watanabe (Google, Inc., Tokyo, Japan)
* Aaron Steven White (University of Rochester, NY, USA)
* Fei Xia (University of Washington, WA, USA) - AACL-IJCNLP 2020 tutorial chair
* Yue Zhang (Westlake University, Hangzhou, China) - ACL 2020 tutorial chair
* Meishan Zhang (Tianjin University, China)

In total, we received 43 submissions for the 4 conferences. Each reviewer was assigned 6-7 proposals and each proposal received 3 reviews. The selection criteria included: clarity and preparedness, novelty or timely character of the topic, lecturers' experience, likely audience interest, open access of the teaching material, diversity aspects (multilingualism, gender, age and country of the lecturers), and compatibility with the preferred venues. We accepted 31 proposals.

The decision making was handled via an online meeting of the 8 tutorial chairs. In particular, the selection of tutorials for each conference was done via the expression of interest of the tutorial chairs on a round-robin basis. Some slight adjustments were also performed after the meeting to better fit the authors' preferences. In total, 8, 8, 8 and 7 proposals were selected for ACL, AACL-IJCNLP, COLING and EMNLP, respectively. Upon the announcement the results, 2 of the proposals accepted for AACL-IJCNLP were withdrawn.

The submission, review, selection and collection of final descriptions for all tutorials was handled via a dedicated SoftConf space, shared by the 4 coordinating conferences. After the selection of proposals, a separate track was created on SoftConf for each conference. The final submission page (one per conference) was set up so as to collect all the necessary data including notably: the tutorial slides, URLs for course material (if any), printable material (if any) and agreement for open access publication.

===Tutorials Selected for ACL 2020===

The final selection for ACL 2020 consists of the following 8 tutorials of 3 hours each (each of them had ACL as the preferred or the second preferred venue):

'''T1: Interpretability and Analysis in Neural NLP''' (cutting-edge) 
Yonatan Belinkov, Sebastian Gehrmann and Ellie Pavlick 
While deep learning has transformed the NLP field and impacted the larger computational linguistics community, the rise of neural networks is stained by their opaque nature: It is challenging to interpret the inner workings of neural network models, and explicate their behavior. Therefore, in the last few years, an increasingly large body of work has been devoted to the analysis and interpretation of neural network models in NLP. 
This body of work is so far lacking a common framework and methodology. Moreover, approaching the analysis of modern neural networks can be difficult for newcomers to the field. This tutorial aims to fill this gap and introduce the nascent field of interpretability and analysis of neural networks in NLP. 
The tutorial covers the main lines of analysis work, such as probing classifier, behavior studies and test suites, psycholinguistic methods, visualizations, adversarial examples, and other methods. We highlight not only the most commonly applied analysis methods, but also the specific limitations and shortcomings of current approaches, in order to inform participants where to focus future efforts.

'''T2: Multi-modal Information Extraction from Text, Semi-structured, and Tabular Data on the Web''' (cutting-edge) 
Xin Luna Dong, Hannaneh Hajishirzi, Colin Lockard and Prashant Shiralkar 
The World Wide Web contains vast quantities of textual information in several forms: unstructured text, template-based semi-structured webpages (which present data in key-value pairs and lists), and tables. Methods for extracting information from these sources and converting it to a structured form have been a target of research from the natural language processing (NLP), data mining, and database communities. While these researchers have largely separated extraction from web data into different problems based on the modality of the data, they have faced similar problems such as learning with limited labeled data, defining (or avoiding defining) ontologies, making use of prior knowledge, and scaling solutions to deal with the size of the Web. 
In this tutorial we take a holistic view toward information extraction, exploring the commonalities in the challenges and solutions developed to address these different forms of text. We will explore the approaches targeted at unstructured text that largely rely on learning syntactic or semantic textual patterns, approaches targeted at semi-structured documents that learn to identify structural patterns in the template, and approaches targeting web tables which rely heavily on entity linking and type information. 
While these different data modalities have largely been considered separately in the past, recent research has started taking a more inclusive approach toward textual extraction, in which the multiple signals offered by textual, layout, and visual clues are combined into a single extraction model made possible by new deep learning approaches. At the same time, trends within purely textual extraction have shifted toward full-document understanding rather than considering sentences as independent units. With this in mind, it is worth considering the information extraction problem as a whole to motivate solutions that harness textual semantics along with visual and semi-structured layout information. We will discuss these approaches and suggest avenues for future work.

'''T3: Reviewing Natural Language Processing Research''' (introductory) 
Kevin Cohen, Karën Fort, Margot Mieskes and Aurélie Névéol 
As the demand for reviewing grows, so must the pool of reviewers. As the [http://www.livecongress.it/aol/indexSA.php?id=E2EAED7D&ticket= survey] presented by Graham Neubig at the 2019 ACL showed, a considerable number of reviewers are junior researchers, who might lack the experience and expertise necessary for high-quality reviews. Some of them might not have the environment or lack opportunities that allow them to learn the skills necessary. A tutorial on reviewing for the NLP community might increase reviewers’ confidence, as well as the quality of the reviews. This introductory tutorial will cover the goals, processes, and evaluation of reviewing research papers in natural language processing.

'''T4: Stylized Text Generation: Approaches and Applications''' (cutting-edge) 
Lili Mou and Olga Vechtomova 
Text generation has played an important role in various applications of natural language processing (NLP), and kn recent studies, researchers are paying increasing attention to modeling and manipulating the style of the generation text, which we call stylized text generation. In this tutorial, we will provide a comprehensive literature review in this direction. We start from the definition of style and different settings of stylized text generation, illustrated with various applications. Then, we present different settings of stylized generation, such as parallel supervised, style label-supervised, and unsupervised. In each setting, we delve deep into machine learning methods, including embedding learning techniques to represent style}, adversarial learning and reinforcement learning with cycle consistency to match content but to distinguish different styles. We also introduce current approaches of evaluating stylized text generation systems. We conclude our tutorial by presenting the challenges of stylized text generation and discussing future directions, such as small-data training, non-categorical style modeling, and a generalized scope of style transfer (e.g., controlling the syntax as a style).

'''T5: Achieving Common Ground in Multi-modal Dialogue''' (cutting-edge) 
Malihe Alikhani and Matthew Stone 
All communication aims at achieving common ground (grounding): interlocutors can work together effectively only with mutual beliefs about what the state of the world is, about what their goals are, and about how they plan to make their goals a reality. Computational dialogue research offers some classic results on grouding, which unfortunately offer scant guidance to the design of grounding modules and behaviors in cutting-edge systems. In this tutorial, we focus on three main topic areas: 1) grounding in human-human communication; 2) grounding in dialogue systems; and 3) grounding in multi-modal interactive systems, including image-oriented conversations and human-robot interactions. We highlight a number of achievements of recent computational research in coordinating complex content, show how these results lead to rich and challenging opportunities for doing grounding in more flexible and powerful ways, and canvass relevant insights from the literature on human--human conversation. We expect that the tutorial will be of interest to researchers in dialogue systems, computational semantics and cognitive modeling, and hope that it will catalyze research and system building that more directly explores the creative, strategic ways conversational agents might be able to seek and offer evidence about their understanding of their interlocutors.

'''T6: Commonsense Reasoning for Natural Language Processing''' (introductory) 
Maarten Sap, Vered Shwartz, Antoine Bosselut, Dan Roth and Yejin Choi 
In our tutorial, we (1) outline the various types of commonsense (e.g., physical, social), and (2) discuss techniques to gather and represent commonsense knowledge, while highlighting the challenges specific to this type of knowledge (e.g., reporting bias). We will then (3) discuss the types of commonsense knowledge captured by modern NLP systems (e.g., large pretrained language models), and (4) present ways to measure systems' commonsense reasoning abilities. We finish with (5) a discussion of various ways in which commonsense reasoning can be used to improve performance on NLP tasks, exemplified by an (6) interactive session on integrating commonsense into a downstream task.

'''T7: Integrating Ethics into the NLP Curriculum''' (introductory) 
Emily M. Bender, Dirk Hovy and Alexandra Schofield 
Our goal in this tutorial is to empower NLP researchers and practitioners with tools and resources to teach others about how to ethically apply NLP techniques. Our tutorial will present both high-level strategies for developing an ethics-oriented curriculum, based on experience and best practices, as well as specific sample exercises that can be brought to a classroom. We plan to make this a highly interactive work session culminating in a shared online resource page that pools lesson plans, assignments, exercise ideas, reading suggestions, and ideas from the attendees. We consider three primary topics with our session that frequently underlie ethical issues in NLP research: Dual use, bias and privacy. 
In this setting, a key lesson is that there is no single approach to ethical NLP: each project requires thoughtful consideration about what steps can be taken to best support people affected by that project. However, we can learn (and teach) what kinds of issues to be aware of and what kinds of strategies are available for mitigating harm. To teach this process, we apply and promote interactive exercises that provide an opportunity to ideate, discuss, and reflect. We plan to facilitate this in a way that encourages positive discussion, emphasizing the creation of ideas for the future instead of negative opinions of previous work.

'''T8: Recent Advances in Open-Domain Question Answering''' (cutting-edge) 
Danqi Chen and Scott Wen-tau Yih 
Open-domain (textual) question answering (QA), the task of finding answers to open-domain questions by searching a large collection of documents, has been a long-standing problem in NLP, information retrieval (IR) and related fields (Voorhees et al., 1999; Moldovan et al., 2000; Brill et al.,2002; Ferrucci et al., 2010). Traditional QA systems were usually constructed as a pipeline, consisting of many different components such as question processing, document/passage retrieval and answer processing. With the rapid development of neural reading comprehension (Chen, 2018), modern open-domain QA systems have been restructured by combining traditional IR techniques and neural reading comprehension models (Chen et al., 2017; Yang et al., 2019) or even implemented in a fully end-to-end fashion (Lee et al., 2019; Seo et al., 2019). While the system architecture has been drastically simplified, two technical challenges remain critical:(1) “Retriever”: finding documents that (might)contain an answer from a large collection of documents; (2) “Reader”: finding the answer in a given paragraph or a document. 
In this tutorial, we aim to provide a comprehensive and coherent overview of recent advances in this line of research. We will start by first giving a brief historical background of open-domain question answering, discussing the basic setup and core technical challenges of the research problem.The focus will then shift to modern techniques and resources proposed for open-domain QA, including the basics of latest neural reading comprehension systems, new datasets and models. The scope will also be broadened to cover the information retrieval component on how to effectively identify passages relevant to the questions. Moreover, in-depth discussions will be given on the use of traditional / neural IR modules, as well as the trade-offs between modular design and end-to-end training. If time permits, we also plan to discuss some hybrid approaches for answering questions using both text and large knowledge bases (e.g. (Sun et al., 2018)) and give a critical review on how structured data complements the information from unstructured text. 
At the end of our tutorial, we will discuss some important questions, including (1) How much progress have we made compared to the QA systems developed in the last decade?(2) What are the main challenges and limitations of cur-rent approaches? (3) How to trade off the efficiency (computational time and memory requirements) and accuracy in the deep learning era? We hope that our tutorial will not only serve as a useful resource for the audience to efficiently acquire the up-to-date knowledge, but also provide new perspectives to stimulate the advances of open-domain QA research in the next phase.

===Virtual Conference Organization===

In April the whole conference was decided to be held in virtual. We carried on the preparations jointly with the tutorial instructors, the virtual infrastructure chairs, the website chairs and the SlidesLive team to set up the virtual conference infrastructure, and to publicize it for each tutorial.

We first made a decision on the time slots for tutorial presentations. Five major time zones were considered, including US west coast, US east coast, central Europe, China/Asia and Australia (India was also considered). Our aim was to offer at least two convenient time slots (i.e. within regular working hours) for each of these zones.

The final time slots include (Jul 5 Pacific Time):

* slot 1 – 7:00 PM (-1D) to 10:30 PM (-1D)
* slot 2 – 12:00 AM to 3:30 AM
* slot 3 – 6:00 AM to 9:30 AM
* slot 4 – 10:30 AM to 2:00 PM
* slot 5 – 3:00 PM to 6:30 PM

We asked the tutorial teachers to indicate their preferences among the time slots, and – if possible – to agree to give their tutorial twice, so as to increase accessibility. Our final decisions on slot assignments traded off the teacher preferences and slot availability. In the end, the assignments were the following (see also the [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ blog post] and [https://acl2020.org/program/tutorials/ online schedule]):

* T1 – twice on slots 3 and 4
* T2 – once on slot 5
* T3 – once on slot 4
* T4 – once on slot 4
* T5 – twice on slots 3 and 5
* T6 – once on slot 5
* T7 – twice on slots 3 and 4
* T8 – once on slot 5

Unfortunately, the tutorial teachers proved hardly available for slots 1 and 2.

In addition to time slot arrangements, we gave the tutorial teacher two options on presentation, namely a pre-recorded and a fully interactive form of tutorials. The former allows the teachers to pre-record the main content of their tutorials, and use the time slots mainly for question answering and detailed discussions. The latter is more interactive, the teachers use the online sessions for both giving the lecture and other activities such as active discussions. All these arrangements were made for taking advantage of the virtual conference. In the end the choices were the following:

* T1 – pre-recorded
* T2 – interactive (the initial decision was pre-recorded but a change was made in June)
* T3 – pre-recorded
* T4 – pre-recorded
* T5 – pre-recorded
* T6 – interactive
* T7 – interactive
* T8 – interactive

After making decisions on time slots and teaching forms, we started to work on the technical details of the virtual conference infrastructure jointly with the infrastructure chairs, the website chairs, the SlidesLive support team and individual tutorial teachers. We collected needs from each tutorial from time to time, inquired information from the SlidesLive team, and tried to accommodate specific requirement for each tutorial. Since this is the first time ACL was virtual, it typically took several rounds of information exchange for making a certain technical feature work. Major things discussed included:

* Sorting out how to pre-record a tutorial using the system
* Recording of interactive sessions
* Additional RocketChat channel
* Virtual tutorial webpage setup
* Coordinators for tutorial sessions and SlidesLive team participation
* Rehearsal of interactive sessions
* Captioning of pre-recorded and interactive tutorials
* Privacy issues
* Special needs from particular tutorials
** Integrating [https://www.dory.app/ Dory] for question answering (in T1)
** Fully interactive sessions, with occasional split into break-out rooms (in T7)

To ensure smooth running of the sessions, we sent two rounds of reminders to each tutorial to ensure that each teacher is registered to the conference. We collected rough timelines of each tutorial so that the SlidesLive team can work closely during the live sessions. To further help the teachers prepare their live session content we additionally collected the lists of registered attendees twice before the conference, and shared them with the teachers.

In the end, the numbers of registrations for each tutorial were:
* T1 -- 1762
* T2 -- 793
* T3 -- 503
* T4 -- 571
* T5 -- 641
* T6 -- 1288
* T7 -- 378
* T8 – 1087
This sums up to 7023 registrations.

===Archiving and Publicity===

There are three main channels for publicity, namely the static [http://www.acl2020.org/program/tutorials/ website], the website of the [https://virtual.acl2020.org/tutorials.html virtual conference] (restricted to the registered participants) and the social media (including [https://twitter.com/aclmeeting/ Twitter]). For the former, we collected photos from each instructor, which are shown with the title and abstract of their tutorials. Following NAACL 2019, we also displayed the time slots as a right-hand-side bar on the tutorial website. Due to special arrangements of the virtual conference, we marked the time zones explicitly. In addition, we asked the tutorial teachers to provide URL links to teaching materials (e.g., at GitHub) during their final submission, and included these links to their tutorial information. For social media, we drafted two blog posts (i.e., [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ this post] and [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]) for advertising the tutorials.

The archiving consists of three main items, namely the tutorial proceedings, the slides for each tutorial and the videos. For the tutorial proceedings, we collected final abstracts from the authors using the START system. We asked the instructors to sign the CC BY v4 agreement when submitting their final materials on SoftConf. For the slides, we asked the tutorial teachers to provide a first version of the slides by June 17 regardless whether they are pre-recorded or interactive. Some tutorials made further updates to their slides before ACL. For the video recordings, we worked with the ACL Anthology team discussing the technical details since this is the first time the tutorial videos are shared in the website. [Note: this is currently undergoing at the time of writing of this summary]

===Lessons Learned===

While organizing a virtual conference it is important to stress that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]).

It is also important to announce the individual modalities with the attendees early enough. For instance, watching pre-recordings in advance was necessary for some tutorials (since their live sessions only included question answering) and such constrains should have been known more in advance.

If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to virtual meeting tools like Zoom only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.

One thing to remember which we missed is to publish the reading list for each tutorial (which the teachers submit with the proposal) on the tutorial website.

2020Q3 Reports: Tutorial Chairs

2020-07-23T10:13:13Z

AgataSavary: /* Archiving and Publicity */

Tutorial Chairs:

<blockquote>Agata Savary, University of Tours, France

Yue Zhang, Westlake University, Hangzhou, China
</blockquote>

=== Preparatory Work, Call, Reviewing and Decision-Making ===
The call, submission, reviewing and selection of tutorials was coordinated jointly for 4 conferences: ACL, AACL-IJCNLP, COLING and EMNLP.

Before drafting the call, we collected lists of tutorials offered within the past 4 years. We analysed previous calls for tutorials and reports from tutorial chairs (from [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs 2016], [https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs 2017], [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs 2018] and [http://aclweb.org/adminwiki/index.php?title=2019Q1_Reports:_ACL_2019 2019]). We consulted previous tutorial chairs with a questionnaire including questions about: the number of submissions, encouraging submissions on specific topics or from specific lecturers, the review procedure, the evaluation criteria, the post-tutorial availability of the slides/codes, and lessons learned from tutorial coordination. We also discussed the publication of slides and video recordings from future tutorials with the persons in charge of the ACL Anthology. As a result of these steps, we created two new sections for the [https://www.aclweb.org/adminwiki/index.php?title=Conference_Handbook ACL Conference Handbook] (future chairs might consider updating these documents yearly):

* the list of [https://www.aclweb.org/adminwiki/index.php?title=Past_tutorials past tutorials] at ACL, COLING, EACL, EMNLP, and NAACL in 2016-2019
* a [https://www.aclweb.org/adminwiki/index.php?title=Tutorial_chair_handbook tutorial chair handbook]

The final [https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020 call] differs from previous calls in several aspects: (i) the expectations about tutorial proposals were made clearer, (ii) following the central ACL decision, the teachers' payment policy was replaced by a fee-waiving policy, (iii) the required submission details include two new items: diversity considerations and agreement for open access publication of slides, codes, data and video recordings, (iv) the evaluation criteria (see below) are announced.

We recruited a review committee of 19 members, including the 8 tutorial chairs and 11 external members selected for their large understanding of the NLP domain and a good experience in reviewing and/or tutorial teaching:

'''Review Committee'''

* Timothy Baldwin (University of Melbourne, Australia) - AACL-IJCNLP 2020 tutorial chair
* Daniel Beck (University of Melbourne, Australia) - COLING 2020 tutorial chair
* Emily M. Bender (University of Washington, WA, USA)
* Erik Cambria (Nanyang Technological University, Singapore)
* Gaël Dias (University of Caen Normandie, France)
* Stefan Evert (Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany)
* Yang Liu (Tsinghua University, Beijing, China)
* Agata Savary (University of Tours, France) - ACL 2020 tutorial chair
* João Sedoc (Johns Hopkins University, Baltimore, MD, USA)
* Lucia Specia (Sheffield University, UK) - COLING 2020 tutorial chair
* Xu Sun (Peking University, China)
* Yulia Tsvetkov (Carnegie Mellon University, Pittsburgh, PA, USA)
* Benjamin Van Durme (Johns Hopkins University, Baltimore, MD, USA) - EMNLP 2020 tutorial chair
* Aline Villavicencio (University of Sheffield, UK and Federal University of Rio Grande do Sul, Brazil) - EMNLP 2020 tutorial chair
* Taro Watanabe (Google, Inc., Tokyo, Japan)
* Aaron Steven White (University of Rochester, NY, USA)
* Fei Xia (University of Washington, WA, USA) - AACL-IJCNLP 2020 tutorial chair
* Yue Zhang (Westlake University, Hangzhou, China) - ACL 2020 tutorial chair
* Meishan Zhang (Tianjin University, China)

In total, we received 43 submissions for the 4 conferences. Each reviewer was assigned 6-7 proposals and each proposal received 3 reviews. The selection criteria included: clarity and preparedness, novelty or timely character of the topic, lecturers' experience, likely audience interest, open access of the teaching material, diversity aspects (multilingualism, gender, age and country of the lecturers), and compatibility with the preferred venues. We accepted 31 proposals.

The decision making was handled via an online meeting of the 8 tutorial chairs. In particular, the selection of tutorials for each conference was done via the expression of interest of the tutorial chairs on a round-robin basis. Some slight adjustments were also performed after the meeting to better fit the authors' preferences. In total, 8, 8, 8 and 7 proposals were selected for ACL, AACL-IJCNLP, COLING and EMNLP, respectively. Upon the announcement the results, 2 of the proposals accepted for AACL-IJCNLP were withdrawn.

The submission, review, selection and collection of final descriptions for all tutorials was handled via a dedicated SoftConf space, shared by the 4 coordinating conferences. After the selection of proposals, a separate track was created on SoftConf for each conference. The final submission page (one per conference) was set up so as to collect all the necessary data including notably: the tutorial slides, URLs for course material (if any), printable material (if any) and agreement for open access publication.

===Tutorials Selected for ACL 2020===

The final selection for ACL 2020 consists of the following 8 tutorials of 3 hours each (each of them had ACL as the preferred or the second preferred venue):

'''T1: Interpretability and Analysis in Neural NLP''' (cutting-edge) 
Yonatan Belinkov, Sebastian Gehrmann and Ellie Pavlick 
While deep learning has transformed the NLP field and impacted the larger computational linguistics community, the rise of neural networks is stained by their opaque nature: It is challenging to interpret the inner workings of neural network models, and explicate their behavior. Therefore, in the last few years, an increasingly large body of work has been devoted to the analysis and interpretation of neural network models in NLP. 
This body of work is so far lacking a common framework and methodology. Moreover, approaching the analysis of modern neural networks can be difficult for newcomers to the field. This tutorial aims to fill this gap and introduce the nascent field of interpretability and analysis of neural networks in NLP. 
The tutorial covers the main lines of analysis work, such as probing classifier, behavior studies and test suites, psycholinguistic methods, visualizations, adversarial examples, and other methods. We highlight not only the most commonly applied analysis methods, but also the specific limitations and shortcomings of current approaches, in order to inform participants where to focus future efforts.

'''T2: Multi-modal Information Extraction from Text, Semi-structured, and Tabular Data on the Web''' (cutting-edge) 
Xin Luna Dong, Hannaneh Hajishirzi, Colin Lockard and Prashant Shiralkar 
The World Wide Web contains vast quantities of textual information in several forms: unstructured text, template-based semi-structured webpages (which present data in key-value pairs and lists), and tables. Methods for extracting information from these sources and converting it to a structured form have been a target of research from the natural language processing (NLP), data mining, and database communities. While these researchers have largely separated extraction from web data into different problems based on the modality of the data, they have faced similar problems such as learning with limited labeled data, defining (or avoiding defining) ontologies, making use of prior knowledge, and scaling solutions to deal with the size of the Web. 
In this tutorial we take a holistic view toward information extraction, exploring the commonalities in the challenges and solutions developed to address these different forms of text. We will explore the approaches targeted at unstructured text that largely rely on learning syntactic or semantic textual patterns, approaches targeted at semi-structured documents that learn to identify structural patterns in the template, and approaches targeting web tables which rely heavily on entity linking and type information. 
While these different data modalities have largely been considered separately in the past, recent research has started taking a more inclusive approach toward textual extraction, in which the multiple signals offered by textual, layout, and visual clues are combined into a single extraction model made possible by new deep learning approaches. At the same time, trends within purely textual extraction have shifted toward full-document understanding rather than considering sentences as independent units. With this in mind, it is worth considering the information extraction problem as a whole to motivate solutions that harness textual semantics along with visual and semi-structured layout information. We will discuss these approaches and suggest avenues for future work.

'''T3: Reviewing Natural Language Processing Research''' (introductory) 
Kevin Cohen, Karën Fort, Margot Mieskes and Aurélie Névéol 
As the demand for reviewing grows, so must the pool of reviewers. As the [http://www.livecongress.it/aol/indexSA.php?id=E2EAED7D&ticket= survey] presented by Graham Neubig at the 2019 ACL showed, a considerable number of reviewers are junior researchers, who might lack the experience and expertise necessary for high-quality reviews. Some of them might not have the environment or lack opportunities that allow them to learn the skills necessary. A tutorial on reviewing for the NLP community might increase reviewers’ confidence, as well as the quality of the reviews. This introductory tutorial will cover the goals, processes, and evaluation of reviewing research papers in natural language processing.

'''T4: Stylized Text Generation: Approaches and Applications''' (cutting-edge) 
Lili Mou and Olga Vechtomova 
Text generation has played an important role in various applications of natural language processing (NLP), and kn recent studies, researchers are paying increasing attention to modeling and manipulating the style of the generation text, which we call stylized text generation. In this tutorial, we will provide a comprehensive literature review in this direction. We start from the definition of style and different settings of stylized text generation, illustrated with various applications. Then, we present different settings of stylized generation, such as parallel supervised, style label-supervised, and unsupervised. In each setting, we delve deep into machine learning methods, including embedding learning techniques to represent style}, adversarial learning and reinforcement learning with cycle consistency to match content but to distinguish different styles. We also introduce current approaches of evaluating stylized text generation systems. We conclude our tutorial by presenting the challenges of stylized text generation and discussing future directions, such as small-data training, non-categorical style modeling, and a generalized scope of style transfer (e.g., controlling the syntax as a style).

'''T5: Achieving Common Ground in Multi-modal Dialogue''' (cutting-edge) 
Malihe Alikhani and Matthew Stone 
All communication aims at achieving common ground (grounding): interlocutors can work together effectively only with mutual beliefs about what the state of the world is, about what their goals are, and about how they plan to make their goals a reality. Computational dialogue research offers some classic results on grouding, which unfortunately offer scant guidance to the design of grounding modules and behaviors in cutting-edge systems. In this tutorial, we focus on three main topic areas: 1) grounding in human-human communication; 2) grounding in dialogue systems; and 3) grounding in multi-modal interactive systems, including image-oriented conversations and human-robot interactions. We highlight a number of achievements of recent computational research in coordinating complex content, show how these results lead to rich and challenging opportunities for doing grounding in more flexible and powerful ways, and canvass relevant insights from the literature on human--human conversation. We expect that the tutorial will be of interest to researchers in dialogue systems, computational semantics and cognitive modeling, and hope that it will catalyze research and system building that more directly explores the creative, strategic ways conversational agents might be able to seek and offer evidence about their understanding of their interlocutors.

'''T6: Commonsense Reasoning for Natural Language Processing''' (introductory) 
Maarten Sap, Vered Shwartz, Antoine Bosselut, Dan Roth and Yejin Choi 
In our tutorial, we (1) outline the various types of commonsense (e.g., physical, social), and (2) discuss techniques to gather and represent commonsense knowledge, while highlighting the challenges specific to this type of knowledge (e.g., reporting bias). We will then (3) discuss the types of commonsense knowledge captured by modern NLP systems (e.g., large pretrained language models), and (4) present ways to measure systems' commonsense reasoning abilities. We finish with (5) a discussion of various ways in which commonsense reasoning can be used to improve performance on NLP tasks, exemplified by an (6) interactive session on integrating commonsense into a downstream task.

'''T7: Integrating Ethics into the NLP Curriculum''' (introductory) 
Emily M. Bender, Dirk Hovy and Alexandra Schofield 
Our goal in this tutorial is to empower NLP researchers and practitioners with tools and resources to teach others about how to ethically apply NLP techniques. Our tutorial will present both high-level strategies for developing an ethics-oriented curriculum, based on experience and best practices, as well as specific sample exercises that can be brought to a classroom. We plan to make this a highly interactive work session culminating in a shared online resource page that pools lesson plans, assignments, exercise ideas, reading suggestions, and ideas from the attendees. We consider three primary topics with our session that frequently underlie ethical issues in NLP research: Dual use, bias and privacy. 
In this setting, a key lesson is that there is no single approach to ethical NLP: each project requires thoughtful consideration about what steps can be taken to best support people affected by that project. However, we can learn (and teach) what kinds of issues to be aware of and what kinds of strategies are available for mitigating harm. To teach this process, we apply and promote interactive exercises that provide an opportunity to ideate, discuss, and reflect. We plan to facilitate this in a way that encourages positive discussion, emphasizing the creation of ideas for the future instead of negative opinions of previous work.

'''T8: Recent Advances in Open-Domain Question Answering''' (cutting-edge) 
Danqi Chen and Scott Wen-tau Yih 
Open-domain (textual) question answering (QA), the task of finding answers to open-domain questions by searching a large collection of documents, has been a long-standing problem in NLP, information retrieval (IR) and related fields (Voorhees et al., 1999; Moldovan et al., 2000; Brill et al.,2002; Ferrucci et al., 2010). Traditional QA systems were usually constructed as a pipeline, consisting of many different components such as question processing, document/passage retrieval and answer processing. With the rapid development of neural reading comprehension (Chen, 2018), modern open-domain QA systems have been restructured by combining traditional IR techniques and neural reading comprehension models (Chen et al., 2017; Yang et al., 2019) or even implemented in a fully end-to-end fashion (Lee et al., 2019; Seo et al., 2019). While the system architecture has been drastically simplified, two technical challenges remain critical:(1) “Retriever”: finding documents that (might)contain an answer from a large collection of documents; (2) “Reader”: finding the answer in a given paragraph or a document. 
In this tutorial, we aim to provide a comprehensive and coherent overview of recent advances in this line of research. We will start by first giving a brief historical background of open-domain question answering, discussing the basic setup and core technical challenges of the research problem.The focus will then shift to modern techniques and resources proposed for open-domain QA, including the basics of latest neural reading comprehension systems, new datasets and models. The scope will also be broadened to cover the information retrieval component on how to effectively identify passages relevant to the questions. Moreover, in-depth discussions will be given on the use of traditional / neural IR modules, as well as the trade-offs between modular design and end-to-end training. If time permits, we also plan to discuss some hybrid approaches for answering questions using both text and large knowledge bases (e.g. (Sun et al., 2018)) and give a critical review on how structured data complements the information from unstructured text. 
At the end of our tutorial, we will discuss some important questions, including (1) How much progress have we made compared to the QA systems developed in the last decade?(2) What are the main challenges and limitations of cur-rent approaches? (3) How to trade off the efficiency (computational time and memory requirements) and accuracy in the deep learning era? We hope that our tutorial will not only serve as a useful resource for the audience to efficiently acquire the up-to-date knowledge, but also provide new perspectives to stimulate the advances of open-domain QA research in the next phase.

===Virtual Conference Organization===

In April the whole conference was decided to be held in virtual. We carried on the preparations jointly with the tutorial instructors, the virtual infrastructure chairs, the website chairs and the SlidesLive team to set up the virtual conference infrastructure, and to publicize it for each tutorial.

We first made a decision on the time slots for tutorial presentations. Five major time zones were considered, including US west coast, US east coast, central Europe, China/Asia and Australia (India was also considered). Our aim was to offer at least two convenient time slots (i.e. within regular working hours) for each of these zones.

The final time slots include (Jul 5 Pacific Time):

* slot 1 – 7:00 PM (-1D) to 10:30 PM (-1D)
* slot 2 – 12:00 AM to 3:30 AM
* slot 3 – 6:00 AM to 9:30 AM
* slot 4 – 10:30 AM to 2:00 PM
* slot 5 – 3:00 PM to 6:30 PM

We asked the tutorial teachers to indicate their preferences among the time slots, and – if possible – to agree to give their tutorial twice, so as to increase accessibility. Our final decisions on slot assignments traded off the teacher preferences and slot availability. In the end, the assignments were the following (see also the [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ blog post] and [https://acl2020.org/program/tutorials/ online schedule]):

* T1 – twice on slots 3 and 4
* T2 – once on slot 5
* T3 – once on slot 4
* T4 – once on slot 4
* T5 – twice on slots 3 and 5
* T6 – once on slot 5
* T7 – twice on slots 3 and 4
* T8 – once on slot 5

Unfortunately, the tutorial teachers proved hardly available for slots 1 and 2.

In addition to time slot arrangements, we gave the tutorial teacher two options on presentation, namely a pre-recorded and a fully interactive form of tutorials. The former allows the teachers to pre-record the main content of their tutorials, and use the time slots mainly for question answering and detailed discussions. The latter is more interactive, the teachers use the online sessions for both giving the lecture and other activities such as active discussions. All these arrangements were made for taking advantage of the virtual conference. In the end the choices were the following:

* T1 – pre-recorded
* T2 – interactive (the initial decision was pre-recorded but a change was made in June)
* T3 – pre-recorded
* T4 – pre-recorded
* T5 – pre-recorded
* T6 – interactive
* T7 – interactive
* T8 – interactive

After making decisions on time slots and teaching forms, we started to work on the technical details of the virtual conference infrastructure jointly with the infrastructure chairs, the website chairs, the SlidesLive support team and individual tutorial teachers. We collected needs from each tutorial from time to time, inquired information from the SlidesLive team, and tried to accommodate specific requirement for each tutorial. Since this is the first time ACL was virtual, it typically took several rounds of information exchange for making a certain technical feature work. Major things discussed included:

* Sorting out how to pre-record a tutorial using the system
* Recording of interactive sessions
* Additional RocketChat channel
* Virtual tutorial webpage setup
* Coordinators for tutorial sessions and SlidesLive team participation
* Rehearsal of interactive sessions
* Captioning of pre-recorded and interactive tutorials
* Privacy issues
* Special needs from particular tutorials
** Integrating [https://www.dory.app/ Dory] for question answering (in T1)
** Fully interactive sessions, with occasional split into break-out rooms (in T7)

To ensure smooth running of the sessions, we sent two rounds of reminders to each tutorial to ensure that each teacher is registered to the conference. We collected rough timelines of each tutorial so that the SlidesLive team can work closely during the live sessions. To further help the teachers prepare their live session content we additionally collected the lists of registered attendees twice before the conference, and shared them with the teachers.

In the end, the numbers of registrations for each tutorial were:
* T1 -- 1762
* T2 -- 793
* T3 -- 503
* T4 -- 571
* T5 -- 641
* T6 -- 1288
* T7 -- 378
* T8 – 1087
This sums up to 7023 registrations.

===Archiving and Publicity===

There are three main channels for publicity, namely the static [http://www.acl2020.org/program/tutorials/ website], the website of the [https://virtual.acl2020.org/tutorials.html virtual conference] (restricted to the registered participants) and the social media (including [https://twitter.com/aclmeeting/ Twitter]). For the former, we collected photos from each instructor, which are shown with the title and abstract of their tutorials. Following NAACL 2019, we also displayed the time slots as a right-hand-side bar on the tutorial website. Due to special arrangements of the virtual conference, we marked the time zones explicitly. In addition, we asked the tutorial teachers to provide URL links to teaching materials (e.g., at GitHub) during their final submission, and included these links to their tutorial information. For social media, we drafted two blog posts (i.e., [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ this post] and [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]) for advertising the tutorials.

The archiving consists of three main items, namely the tutorial proceedings, the slides for each tutorial and the videos. For the tutorial proceedings, we collected final abstracts from the authors using the START system. We asked the instructors to sign the CC BY v4 agreement when submitting their final materials on SoftConf. For the slides, we asked the tutorial teachers to provide a first version of the slides by June 17 regardless whether they are pre-recorded or interactive. Some tutorials made further updates to their slides before ACL. For the video recordings, we worked with the ACL Anthology team discussing the technical details since this is the first time the tutorial videos are shared in the website. [Note: this is currently undergoing at the time of writing of this summary]

===Lessons Learned===

While organizing a virtual conference it is important to stress that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]).

It is also important to announce the individual modalities with the attendees early enough. For instance, watching pre-recordings in advance was necessary for some tutorials (since their live sessions only included question answering) and such constrains should have been known more in advance.

If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to Zoom rooms only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.

One thing to remember which we missed is to publish the reading list for each tutorial (which the teachers submit with the proposal) on the tutorial website.

2020Q3 Reports: Tutorial Chairs

2020-07-23T10:12:55Z

AgataSavary: /* Archiving and Publicity */

Tutorial Chairs:

<blockquote>Agata Savary, University of Tours, France

Yue Zhang, Westlake University, Hangzhou, China
</blockquote>

=== Preparatory Work, Call, Reviewing and Decision-Making ===
The call, submission, reviewing and selection of tutorials was coordinated jointly for 4 conferences: ACL, AACL-IJCNLP, COLING and EMNLP.

Before drafting the call, we collected lists of tutorials offered within the past 4 years. We analysed previous calls for tutorials and reports from tutorial chairs (from [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs 2016], [https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs 2017], [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs 2018] and [http://aclweb.org/adminwiki/index.php?title=2019Q1_Reports:_ACL_2019 2019]). We consulted previous tutorial chairs with a questionnaire including questions about: the number of submissions, encouraging submissions on specific topics or from specific lecturers, the review procedure, the evaluation criteria, the post-tutorial availability of the slides/codes, and lessons learned from tutorial coordination. We also discussed the publication of slides and video recordings from future tutorials with the persons in charge of the ACL Anthology. As a result of these steps, we created two new sections for the [https://www.aclweb.org/adminwiki/index.php?title=Conference_Handbook ACL Conference Handbook] (future chairs might consider updating these documents yearly):

* the list of [https://www.aclweb.org/adminwiki/index.php?title=Past_tutorials past tutorials] at ACL, COLING, EACL, EMNLP, and NAACL in 2016-2019
* a [https://www.aclweb.org/adminwiki/index.php?title=Tutorial_chair_handbook tutorial chair handbook]

The final [https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020 call] differs from previous calls in several aspects: (i) the expectations about tutorial proposals were made clearer, (ii) following the central ACL decision, the teachers' payment policy was replaced by a fee-waiving policy, (iii) the required submission details include two new items: diversity considerations and agreement for open access publication of slides, codes, data and video recordings, (iv) the evaluation criteria (see below) are announced.

We recruited a review committee of 19 members, including the 8 tutorial chairs and 11 external members selected for their large understanding of the NLP domain and a good experience in reviewing and/or tutorial teaching:

'''Review Committee'''

* Timothy Baldwin (University of Melbourne, Australia) - AACL-IJCNLP 2020 tutorial chair
* Daniel Beck (University of Melbourne, Australia) - COLING 2020 tutorial chair
* Emily M. Bender (University of Washington, WA, USA)
* Erik Cambria (Nanyang Technological University, Singapore)
* Gaël Dias (University of Caen Normandie, France)
* Stefan Evert (Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany)
* Yang Liu (Tsinghua University, Beijing, China)
* Agata Savary (University of Tours, France) - ACL 2020 tutorial chair
* João Sedoc (Johns Hopkins University, Baltimore, MD, USA)
* Lucia Specia (Sheffield University, UK) - COLING 2020 tutorial chair
* Xu Sun (Peking University, China)
* Yulia Tsvetkov (Carnegie Mellon University, Pittsburgh, PA, USA)
* Benjamin Van Durme (Johns Hopkins University, Baltimore, MD, USA) - EMNLP 2020 tutorial chair
* Aline Villavicencio (University of Sheffield, UK and Federal University of Rio Grande do Sul, Brazil) - EMNLP 2020 tutorial chair
* Taro Watanabe (Google, Inc., Tokyo, Japan)
* Aaron Steven White (University of Rochester, NY, USA)
* Fei Xia (University of Washington, WA, USA) - AACL-IJCNLP 2020 tutorial chair
* Yue Zhang (Westlake University, Hangzhou, China) - ACL 2020 tutorial chair
* Meishan Zhang (Tianjin University, China)

In total, we received 43 submissions for the 4 conferences. Each reviewer was assigned 6-7 proposals and each proposal received 3 reviews. The selection criteria included: clarity and preparedness, novelty or timely character of the topic, lecturers' experience, likely audience interest, open access of the teaching material, diversity aspects (multilingualism, gender, age and country of the lecturers), and compatibility with the preferred venues. We accepted 31 proposals.

The decision making was handled via an online meeting of the 8 tutorial chairs. In particular, the selection of tutorials for each conference was done via the expression of interest of the tutorial chairs on a round-robin basis. Some slight adjustments were also performed after the meeting to better fit the authors' preferences. In total, 8, 8, 8 and 7 proposals were selected for ACL, AACL-IJCNLP, COLING and EMNLP, respectively. Upon the announcement the results, 2 of the proposals accepted for AACL-IJCNLP were withdrawn.

The submission, review, selection and collection of final descriptions for all tutorials was handled via a dedicated SoftConf space, shared by the 4 coordinating conferences. After the selection of proposals, a separate track was created on SoftConf for each conference. The final submission page (one per conference) was set up so as to collect all the necessary data including notably: the tutorial slides, URLs for course material (if any), printable material (if any) and agreement for open access publication.

===Tutorials Selected for ACL 2020===

The final selection for ACL 2020 consists of the following 8 tutorials of 3 hours each (each of them had ACL as the preferred or the second preferred venue):

'''T1: Interpretability and Analysis in Neural NLP''' (cutting-edge) 
Yonatan Belinkov, Sebastian Gehrmann and Ellie Pavlick 
While deep learning has transformed the NLP field and impacted the larger computational linguistics community, the rise of neural networks is stained by their opaque nature: It is challenging to interpret the inner workings of neural network models, and explicate their behavior. Therefore, in the last few years, an increasingly large body of work has been devoted to the analysis and interpretation of neural network models in NLP. 
This body of work is so far lacking a common framework and methodology. Moreover, approaching the analysis of modern neural networks can be difficult for newcomers to the field. This tutorial aims to fill this gap and introduce the nascent field of interpretability and analysis of neural networks in NLP. 
The tutorial covers the main lines of analysis work, such as probing classifier, behavior studies and test suites, psycholinguistic methods, visualizations, adversarial examples, and other methods. We highlight not only the most commonly applied analysis methods, but also the specific limitations and shortcomings of current approaches, in order to inform participants where to focus future efforts.

'''T2: Multi-modal Information Extraction from Text, Semi-structured, and Tabular Data on the Web''' (cutting-edge) 
Xin Luna Dong, Hannaneh Hajishirzi, Colin Lockard and Prashant Shiralkar 
The World Wide Web contains vast quantities of textual information in several forms: unstructured text, template-based semi-structured webpages (which present data in key-value pairs and lists), and tables. Methods for extracting information from these sources and converting it to a structured form have been a target of research from the natural language processing (NLP), data mining, and database communities. While these researchers have largely separated extraction from web data into different problems based on the modality of the data, they have faced similar problems such as learning with limited labeled data, defining (or avoiding defining) ontologies, making use of prior knowledge, and scaling solutions to deal with the size of the Web. 
In this tutorial we take a holistic view toward information extraction, exploring the commonalities in the challenges and solutions developed to address these different forms of text. We will explore the approaches targeted at unstructured text that largely rely on learning syntactic or semantic textual patterns, approaches targeted at semi-structured documents that learn to identify structural patterns in the template, and approaches targeting web tables which rely heavily on entity linking and type information. 
While these different data modalities have largely been considered separately in the past, recent research has started taking a more inclusive approach toward textual extraction, in which the multiple signals offered by textual, layout, and visual clues are combined into a single extraction model made possible by new deep learning approaches. At the same time, trends within purely textual extraction have shifted toward full-document understanding rather than considering sentences as independent units. With this in mind, it is worth considering the information extraction problem as a whole to motivate solutions that harness textual semantics along with visual and semi-structured layout information. We will discuss these approaches and suggest avenues for future work.

'''T3: Reviewing Natural Language Processing Research''' (introductory) 
Kevin Cohen, Karën Fort, Margot Mieskes and Aurélie Névéol 
As the demand for reviewing grows, so must the pool of reviewers. As the [http://www.livecongress.it/aol/indexSA.php?id=E2EAED7D&ticket= survey] presented by Graham Neubig at the 2019 ACL showed, a considerable number of reviewers are junior researchers, who might lack the experience and expertise necessary for high-quality reviews. Some of them might not have the environment or lack opportunities that allow them to learn the skills necessary. A tutorial on reviewing for the NLP community might increase reviewers’ confidence, as well as the quality of the reviews. This introductory tutorial will cover the goals, processes, and evaluation of reviewing research papers in natural language processing.

'''T4: Stylized Text Generation: Approaches and Applications''' (cutting-edge) 
Lili Mou and Olga Vechtomova 
Text generation has played an important role in various applications of natural language processing (NLP), and kn recent studies, researchers are paying increasing attention to modeling and manipulating the style of the generation text, which we call stylized text generation. In this tutorial, we will provide a comprehensive literature review in this direction. We start from the definition of style and different settings of stylized text generation, illustrated with various applications. Then, we present different settings of stylized generation, such as parallel supervised, style label-supervised, and unsupervised. In each setting, we delve deep into machine learning methods, including embedding learning techniques to represent style}, adversarial learning and reinforcement learning with cycle consistency to match content but to distinguish different styles. We also introduce current approaches of evaluating stylized text generation systems. We conclude our tutorial by presenting the challenges of stylized text generation and discussing future directions, such as small-data training, non-categorical style modeling, and a generalized scope of style transfer (e.g., controlling the syntax as a style).

'''T5: Achieving Common Ground in Multi-modal Dialogue''' (cutting-edge) 
Malihe Alikhani and Matthew Stone 
All communication aims at achieving common ground (grounding): interlocutors can work together effectively only with mutual beliefs about what the state of the world is, about what their goals are, and about how they plan to make their goals a reality. Computational dialogue research offers some classic results on grouding, which unfortunately offer scant guidance to the design of grounding modules and behaviors in cutting-edge systems. In this tutorial, we focus on three main topic areas: 1) grounding in human-human communication; 2) grounding in dialogue systems; and 3) grounding in multi-modal interactive systems, including image-oriented conversations and human-robot interactions. We highlight a number of achievements of recent computational research in coordinating complex content, show how these results lead to rich and challenging opportunities for doing grounding in more flexible and powerful ways, and canvass relevant insights from the literature on human--human conversation. We expect that the tutorial will be of interest to researchers in dialogue systems, computational semantics and cognitive modeling, and hope that it will catalyze research and system building that more directly explores the creative, strategic ways conversational agents might be able to seek and offer evidence about their understanding of their interlocutors.

'''T6: Commonsense Reasoning for Natural Language Processing''' (introductory) 
Maarten Sap, Vered Shwartz, Antoine Bosselut, Dan Roth and Yejin Choi 
In our tutorial, we (1) outline the various types of commonsense (e.g., physical, social), and (2) discuss techniques to gather and represent commonsense knowledge, while highlighting the challenges specific to this type of knowledge (e.g., reporting bias). We will then (3) discuss the types of commonsense knowledge captured by modern NLP systems (e.g., large pretrained language models), and (4) present ways to measure systems' commonsense reasoning abilities. We finish with (5) a discussion of various ways in which commonsense reasoning can be used to improve performance on NLP tasks, exemplified by an (6) interactive session on integrating commonsense into a downstream task.

'''T7: Integrating Ethics into the NLP Curriculum''' (introductory) 
Emily M. Bender, Dirk Hovy and Alexandra Schofield 
Our goal in this tutorial is to empower NLP researchers and practitioners with tools and resources to teach others about how to ethically apply NLP techniques. Our tutorial will present both high-level strategies for developing an ethics-oriented curriculum, based on experience and best practices, as well as specific sample exercises that can be brought to a classroom. We plan to make this a highly interactive work session culminating in a shared online resource page that pools lesson plans, assignments, exercise ideas, reading suggestions, and ideas from the attendees. We consider three primary topics with our session that frequently underlie ethical issues in NLP research: Dual use, bias and privacy. 
In this setting, a key lesson is that there is no single approach to ethical NLP: each project requires thoughtful consideration about what steps can be taken to best support people affected by that project. However, we can learn (and teach) what kinds of issues to be aware of and what kinds of strategies are available for mitigating harm. To teach this process, we apply and promote interactive exercises that provide an opportunity to ideate, discuss, and reflect. We plan to facilitate this in a way that encourages positive discussion, emphasizing the creation of ideas for the future instead of negative opinions of previous work.

'''T8: Recent Advances in Open-Domain Question Answering''' (cutting-edge) 
Danqi Chen and Scott Wen-tau Yih 
Open-domain (textual) question answering (QA), the task of finding answers to open-domain questions by searching a large collection of documents, has been a long-standing problem in NLP, information retrieval (IR) and related fields (Voorhees et al., 1999; Moldovan et al., 2000; Brill et al.,2002; Ferrucci et al., 2010). Traditional QA systems were usually constructed as a pipeline, consisting of many different components such as question processing, document/passage retrieval and answer processing. With the rapid development of neural reading comprehension (Chen, 2018), modern open-domain QA systems have been restructured by combining traditional IR techniques and neural reading comprehension models (Chen et al., 2017; Yang et al., 2019) or even implemented in a fully end-to-end fashion (Lee et al., 2019; Seo et al., 2019). While the system architecture has been drastically simplified, two technical challenges remain critical:(1) “Retriever”: finding documents that (might)contain an answer from a large collection of documents; (2) “Reader”: finding the answer in a given paragraph or a document. 
In this tutorial, we aim to provide a comprehensive and coherent overview of recent advances in this line of research. We will start by first giving a brief historical background of open-domain question answering, discussing the basic setup and core technical challenges of the research problem.The focus will then shift to modern techniques and resources proposed for open-domain QA, including the basics of latest neural reading comprehension systems, new datasets and models. The scope will also be broadened to cover the information retrieval component on how to effectively identify passages relevant to the questions. Moreover, in-depth discussions will be given on the use of traditional / neural IR modules, as well as the trade-offs between modular design and end-to-end training. If time permits, we also plan to discuss some hybrid approaches for answering questions using both text and large knowledge bases (e.g. (Sun et al., 2018)) and give a critical review on how structured data complements the information from unstructured text. 
At the end of our tutorial, we will discuss some important questions, including (1) How much progress have we made compared to the QA systems developed in the last decade?(2) What are the main challenges and limitations of cur-rent approaches? (3) How to trade off the efficiency (computational time and memory requirements) and accuracy in the deep learning era? We hope that our tutorial will not only serve as a useful resource for the audience to efficiently acquire the up-to-date knowledge, but also provide new perspectives to stimulate the advances of open-domain QA research in the next phase.

===Virtual Conference Organization===

In April the whole conference was decided to be held in virtual. We carried on the preparations jointly with the tutorial instructors, the virtual infrastructure chairs, the website chairs and the SlidesLive team to set up the virtual conference infrastructure, and to publicize it for each tutorial.

We first made a decision on the time slots for tutorial presentations. Five major time zones were considered, including US west coast, US east coast, central Europe, China/Asia and Australia (India was also considered). Our aim was to offer at least two convenient time slots (i.e. within regular working hours) for each of these zones.

The final time slots include (Jul 5 Pacific Time):

* slot 1 – 7:00 PM (-1D) to 10:30 PM (-1D)
* slot 2 – 12:00 AM to 3:30 AM
* slot 3 – 6:00 AM to 9:30 AM
* slot 4 – 10:30 AM to 2:00 PM
* slot 5 – 3:00 PM to 6:30 PM

We asked the tutorial teachers to indicate their preferences among the time slots, and – if possible – to agree to give their tutorial twice, so as to increase accessibility. Our final decisions on slot assignments traded off the teacher preferences and slot availability. In the end, the assignments were the following (see also the [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ blog post] and [https://acl2020.org/program/tutorials/ online schedule]):

* T1 – twice on slots 3 and 4
* T2 – once on slot 5
* T3 – once on slot 4
* T4 – once on slot 4
* T5 – twice on slots 3 and 5
* T6 – once on slot 5
* T7 – twice on slots 3 and 4
* T8 – once on slot 5

Unfortunately, the tutorial teachers proved hardly available for slots 1 and 2.

In addition to time slot arrangements, we gave the tutorial teacher two options on presentation, namely a pre-recorded and a fully interactive form of tutorials. The former allows the teachers to pre-record the main content of their tutorials, and use the time slots mainly for question answering and detailed discussions. The latter is more interactive, the teachers use the online sessions for both giving the lecture and other activities such as active discussions. All these arrangements were made for taking advantage of the virtual conference. In the end the choices were the following:

* T1 – pre-recorded
* T2 – interactive (the initial decision was pre-recorded but a change was made in June)
* T3 – pre-recorded
* T4 – pre-recorded
* T5 – pre-recorded
* T6 – interactive
* T7 – interactive
* T8 – interactive

After making decisions on time slots and teaching forms, we started to work on the technical details of the virtual conference infrastructure jointly with the infrastructure chairs, the website chairs, the SlidesLive support team and individual tutorial teachers. We collected needs from each tutorial from time to time, inquired information from the SlidesLive team, and tried to accommodate specific requirement for each tutorial. Since this is the first time ACL was virtual, it typically took several rounds of information exchange for making a certain technical feature work. Major things discussed included:

* Sorting out how to pre-record a tutorial using the system
* Recording of interactive sessions
* Additional RocketChat channel
* Virtual tutorial webpage setup
* Coordinators for tutorial sessions and SlidesLive team participation
* Rehearsal of interactive sessions
* Captioning of pre-recorded and interactive tutorials
* Privacy issues
* Special needs from particular tutorials
** Integrating [https://www.dory.app/ Dory] for question answering (in T1)
** Fully interactive sessions, with occasional split into break-out rooms (in T7)

To ensure smooth running of the sessions, we sent two rounds of reminders to each tutorial to ensure that each teacher is registered to the conference. We collected rough timelines of each tutorial so that the SlidesLive team can work closely during the live sessions. To further help the teachers prepare their live session content we additionally collected the lists of registered attendees twice before the conference, and shared them with the teachers.

In the end, the numbers of registrations for each tutorial were:
* T1 -- 1762
* T2 -- 793
* T3 -- 503
* T4 -- 571
* T5 -- 641
* T6 -- 1288
* T7 -- 378
* T8 – 1087
This sums up to 7023 registrations.

===Archiving and Publicity===

There are three main channels for publicity, namely the static [http://www.acl2020.org/program/tutorials/ website], the website of the [https://virtual.acl2020.org/tutorials.html virtual conference] (restricted to the registered participants) and the social media (including [https://twitter.com/aclmeeting/ Twitter]. For the former, we collected photos from each instructor, which are shown with the title and abstract of their tutorials. Following NAACL 2019, we also displayed the time slots as a right-hand-side bar on the tutorial website. Due to special arrangements of the virtual conference, we marked the time zones explicitly. In addition, we asked the tutorial teachers to provide URL links to teaching materials (e.g., at GitHub) during their final submission, and included these links to their tutorial information. For social media, we drafted two blog posts (i.e., [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ this post] and [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]) for advertising the tutorials.

The archiving consists of three main items, namely the tutorial proceedings, the slides for each tutorial and the videos. For the tutorial proceedings, we collected final abstracts from the authors using the START system. We asked the instructors to sign the CC BY v4 agreement when submitting their final materials on SoftConf. For the slides, we asked the tutorial teachers to provide a first version of the slides by June 17 regardless whether they are pre-recorded or interactive. Some tutorials made further updates to their slides before ACL. For the video recordings, we worked with the ACL Anthology team discussing the technical details since this is the first time the tutorial videos are shared in the website. [Note: this is currently undergoing at the time of writing of this summary]

===Lessons Learned===

While organizing a virtual conference it is important to stress that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]).

It is also important to announce the individual modalities with the attendees early enough. For instance, watching pre-recordings in advance was necessary for some tutorials (since their live sessions only included question answering) and such constrains should have been known more in advance.

If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to Zoom rooms only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.

One thing to remember which we missed is to publish the reading list for each tutorial (which the teachers submit with the proposal) on the tutorial website.

2020Q3 Reports: Tutorial Chairs

2020-07-23T10:11:12Z

AgataSavary: /* Archiving and Publicity */

Tutorial Chairs:

<blockquote>Agata Savary, University of Tours, France

Yue Zhang, Westlake University, Hangzhou, China
</blockquote>

=== Preparatory Work, Call, Reviewing and Decision-Making ===
The call, submission, reviewing and selection of tutorials was coordinated jointly for 4 conferences: ACL, AACL-IJCNLP, COLING and EMNLP.

Before drafting the call, we collected lists of tutorials offered within the past 4 years. We analysed previous calls for tutorials and reports from tutorial chairs (from [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs 2016], [https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs 2017], [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs 2018] and [http://aclweb.org/adminwiki/index.php?title=2019Q1_Reports:_ACL_2019 2019]). We consulted previous tutorial chairs with a questionnaire including questions about: the number of submissions, encouraging submissions on specific topics or from specific lecturers, the review procedure, the evaluation criteria, the post-tutorial availability of the slides/codes, and lessons learned from tutorial coordination. We also discussed the publication of slides and video recordings from future tutorials with the persons in charge of the ACL Anthology. As a result of these steps, we created two new sections for the [https://www.aclweb.org/adminwiki/index.php?title=Conference_Handbook ACL Conference Handbook] (future chairs might consider updating these documents yearly):

* the list of [https://www.aclweb.org/adminwiki/index.php?title=Past_tutorials past tutorials] at ACL, COLING, EACL, EMNLP, and NAACL in 2016-2019
* a [https://www.aclweb.org/adminwiki/index.php?title=Tutorial_chair_handbook tutorial chair handbook]

The final [https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020 call] differs from previous calls in several aspects: (i) the expectations about tutorial proposals were made clearer, (ii) following the central ACL decision, the teachers' payment policy was replaced by a fee-waiving policy, (iii) the required submission details include two new items: diversity considerations and agreement for open access publication of slides, codes, data and video recordings, (iv) the evaluation criteria (see below) are announced.

We recruited a review committee of 19 members, including the 8 tutorial chairs and 11 external members selected for their large understanding of the NLP domain and a good experience in reviewing and/or tutorial teaching:

'''Review Committee'''

* Timothy Baldwin (University of Melbourne, Australia) - AACL-IJCNLP 2020 tutorial chair
* Daniel Beck (University of Melbourne, Australia) - COLING 2020 tutorial chair
* Emily M. Bender (University of Washington, WA, USA)
* Erik Cambria (Nanyang Technological University, Singapore)
* Gaël Dias (University of Caen Normandie, France)
* Stefan Evert (Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany)
* Yang Liu (Tsinghua University, Beijing, China)
* Agata Savary (University of Tours, France) - ACL 2020 tutorial chair
* João Sedoc (Johns Hopkins University, Baltimore, MD, USA)
* Lucia Specia (Sheffield University, UK) - COLING 2020 tutorial chair
* Xu Sun (Peking University, China)
* Yulia Tsvetkov (Carnegie Mellon University, Pittsburgh, PA, USA)
* Benjamin Van Durme (Johns Hopkins University, Baltimore, MD, USA) - EMNLP 2020 tutorial chair
* Aline Villavicencio (University of Sheffield, UK and Federal University of Rio Grande do Sul, Brazil) - EMNLP 2020 tutorial chair
* Taro Watanabe (Google, Inc., Tokyo, Japan)
* Aaron Steven White (University of Rochester, NY, USA)
* Fei Xia (University of Washington, WA, USA) - AACL-IJCNLP 2020 tutorial chair
* Yue Zhang (Westlake University, Hangzhou, China) - ACL 2020 tutorial chair
* Meishan Zhang (Tianjin University, China)

In total, we received 43 submissions for the 4 conferences. Each reviewer was assigned 6-7 proposals and each proposal received 3 reviews. The selection criteria included: clarity and preparedness, novelty or timely character of the topic, lecturers' experience, likely audience interest, open access of the teaching material, diversity aspects (multilingualism, gender, age and country of the lecturers), and compatibility with the preferred venues. We accepted 31 proposals.

The decision making was handled via an online meeting of the 8 tutorial chairs. In particular, the selection of tutorials for each conference was done via the expression of interest of the tutorial chairs on a round-robin basis. Some slight adjustments were also performed after the meeting to better fit the authors' preferences. In total, 8, 8, 8 and 7 proposals were selected for ACL, AACL-IJCNLP, COLING and EMNLP, respectively. Upon the announcement the results, 2 of the proposals accepted for AACL-IJCNLP were withdrawn.

The submission, review, selection and collection of final descriptions for all tutorials was handled via a dedicated SoftConf space, shared by the 4 coordinating conferences. After the selection of proposals, a separate track was created on SoftConf for each conference. The final submission page (one per conference) was set up so as to collect all the necessary data including notably: the tutorial slides, URLs for course material (if any), printable material (if any) and agreement for open access publication.

===Tutorials Selected for ACL 2020===

The final selection for ACL 2020 consists of the following 8 tutorials of 3 hours each (each of them had ACL as the preferred or the second preferred venue):

'''T1: Interpretability and Analysis in Neural NLP''' (cutting-edge) 
Yonatan Belinkov, Sebastian Gehrmann and Ellie Pavlick 
While deep learning has transformed the NLP field and impacted the larger computational linguistics community, the rise of neural networks is stained by their opaque nature: It is challenging to interpret the inner workings of neural network models, and explicate their behavior. Therefore, in the last few years, an increasingly large body of work has been devoted to the analysis and interpretation of neural network models in NLP. 
This body of work is so far lacking a common framework and methodology. Moreover, approaching the analysis of modern neural networks can be difficult for newcomers to the field. This tutorial aims to fill this gap and introduce the nascent field of interpretability and analysis of neural networks in NLP. 
The tutorial covers the main lines of analysis work, such as probing classifier, behavior studies and test suites, psycholinguistic methods, visualizations, adversarial examples, and other methods. We highlight not only the most commonly applied analysis methods, but also the specific limitations and shortcomings of current approaches, in order to inform participants where to focus future efforts.

'''T2: Multi-modal Information Extraction from Text, Semi-structured, and Tabular Data on the Web''' (cutting-edge) 
Xin Luna Dong, Hannaneh Hajishirzi, Colin Lockard and Prashant Shiralkar 
The World Wide Web contains vast quantities of textual information in several forms: unstructured text, template-based semi-structured webpages (which present data in key-value pairs and lists), and tables. Methods for extracting information from these sources and converting it to a structured form have been a target of research from the natural language processing (NLP), data mining, and database communities. While these researchers have largely separated extraction from web data into different problems based on the modality of the data, they have faced similar problems such as learning with limited labeled data, defining (or avoiding defining) ontologies, making use of prior knowledge, and scaling solutions to deal with the size of the Web. 
In this tutorial we take a holistic view toward information extraction, exploring the commonalities in the challenges and solutions developed to address these different forms of text. We will explore the approaches targeted at unstructured text that largely rely on learning syntactic or semantic textual patterns, approaches targeted at semi-structured documents that learn to identify structural patterns in the template, and approaches targeting web tables which rely heavily on entity linking and type information. 
While these different data modalities have largely been considered separately in the past, recent research has started taking a more inclusive approach toward textual extraction, in which the multiple signals offered by textual, layout, and visual clues are combined into a single extraction model made possible by new deep learning approaches. At the same time, trends within purely textual extraction have shifted toward full-document understanding rather than considering sentences as independent units. With this in mind, it is worth considering the information extraction problem as a whole to motivate solutions that harness textual semantics along with visual and semi-structured layout information. We will discuss these approaches and suggest avenues for future work.

'''T3: Reviewing Natural Language Processing Research''' (introductory) 
Kevin Cohen, Karën Fort, Margot Mieskes and Aurélie Névéol 
As the demand for reviewing grows, so must the pool of reviewers. As the [http://www.livecongress.it/aol/indexSA.php?id=E2EAED7D&ticket= survey] presented by Graham Neubig at the 2019 ACL showed, a considerable number of reviewers are junior researchers, who might lack the experience and expertise necessary for high-quality reviews. Some of them might not have the environment or lack opportunities that allow them to learn the skills necessary. A tutorial on reviewing for the NLP community might increase reviewers’ confidence, as well as the quality of the reviews. This introductory tutorial will cover the goals, processes, and evaluation of reviewing research papers in natural language processing.

'''T4: Stylized Text Generation: Approaches and Applications''' (cutting-edge) 
Lili Mou and Olga Vechtomova 
Text generation has played an important role in various applications of natural language processing (NLP), and kn recent studies, researchers are paying increasing attention to modeling and manipulating the style of the generation text, which we call stylized text generation. In this tutorial, we will provide a comprehensive literature review in this direction. We start from the definition of style and different settings of stylized text generation, illustrated with various applications. Then, we present different settings of stylized generation, such as parallel supervised, style label-supervised, and unsupervised. In each setting, we delve deep into machine learning methods, including embedding learning techniques to represent style}, adversarial learning and reinforcement learning with cycle consistency to match content but to distinguish different styles. We also introduce current approaches of evaluating stylized text generation systems. We conclude our tutorial by presenting the challenges of stylized text generation and discussing future directions, such as small-data training, non-categorical style modeling, and a generalized scope of style transfer (e.g., controlling the syntax as a style).

'''T5: Achieving Common Ground in Multi-modal Dialogue''' (cutting-edge) 
Malihe Alikhani and Matthew Stone 
All communication aims at achieving common ground (grounding): interlocutors can work together effectively only with mutual beliefs about what the state of the world is, about what their goals are, and about how they plan to make their goals a reality. Computational dialogue research offers some classic results on grouding, which unfortunately offer scant guidance to the design of grounding modules and behaviors in cutting-edge systems. In this tutorial, we focus on three main topic areas: 1) grounding in human-human communication; 2) grounding in dialogue systems; and 3) grounding in multi-modal interactive systems, including image-oriented conversations and human-robot interactions. We highlight a number of achievements of recent computational research in coordinating complex content, show how these results lead to rich and challenging opportunities for doing grounding in more flexible and powerful ways, and canvass relevant insights from the literature on human--human conversation. We expect that the tutorial will be of interest to researchers in dialogue systems, computational semantics and cognitive modeling, and hope that it will catalyze research and system building that more directly explores the creative, strategic ways conversational agents might be able to seek and offer evidence about their understanding of their interlocutors.

'''T6: Commonsense Reasoning for Natural Language Processing''' (introductory) 
Maarten Sap, Vered Shwartz, Antoine Bosselut, Dan Roth and Yejin Choi 
In our tutorial, we (1) outline the various types of commonsense (e.g., physical, social), and (2) discuss techniques to gather and represent commonsense knowledge, while highlighting the challenges specific to this type of knowledge (e.g., reporting bias). We will then (3) discuss the types of commonsense knowledge captured by modern NLP systems (e.g., large pretrained language models), and (4) present ways to measure systems' commonsense reasoning abilities. We finish with (5) a discussion of various ways in which commonsense reasoning can be used to improve performance on NLP tasks, exemplified by an (6) interactive session on integrating commonsense into a downstream task.

'''T7: Integrating Ethics into the NLP Curriculum''' (introductory) 
Emily M. Bender, Dirk Hovy and Alexandra Schofield 
Our goal in this tutorial is to empower NLP researchers and practitioners with tools and resources to teach others about how to ethically apply NLP techniques. Our tutorial will present both high-level strategies for developing an ethics-oriented curriculum, based on experience and best practices, as well as specific sample exercises that can be brought to a classroom. We plan to make this a highly interactive work session culminating in a shared online resource page that pools lesson plans, assignments, exercise ideas, reading suggestions, and ideas from the attendees. We consider three primary topics with our session that frequently underlie ethical issues in NLP research: Dual use, bias and privacy. 
In this setting, a key lesson is that there is no single approach to ethical NLP: each project requires thoughtful consideration about what steps can be taken to best support people affected by that project. However, we can learn (and teach) what kinds of issues to be aware of and what kinds of strategies are available for mitigating harm. To teach this process, we apply and promote interactive exercises that provide an opportunity to ideate, discuss, and reflect. We plan to facilitate this in a way that encourages positive discussion, emphasizing the creation of ideas for the future instead of negative opinions of previous work.

'''T8: Recent Advances in Open-Domain Question Answering''' (cutting-edge) 
Danqi Chen and Scott Wen-tau Yih 
Open-domain (textual) question answering (QA), the task of finding answers to open-domain questions by searching a large collection of documents, has been a long-standing problem in NLP, information retrieval (IR) and related fields (Voorhees et al., 1999; Moldovan et al., 2000; Brill et al.,2002; Ferrucci et al., 2010). Traditional QA systems were usually constructed as a pipeline, consisting of many different components such as question processing, document/passage retrieval and answer processing. With the rapid development of neural reading comprehension (Chen, 2018), modern open-domain QA systems have been restructured by combining traditional IR techniques and neural reading comprehension models (Chen et al., 2017; Yang et al., 2019) or even implemented in a fully end-to-end fashion (Lee et al., 2019; Seo et al., 2019). While the system architecture has been drastically simplified, two technical challenges remain critical:(1) “Retriever”: finding documents that (might)contain an answer from a large collection of documents; (2) “Reader”: finding the answer in a given paragraph or a document. 
In this tutorial, we aim to provide a comprehensive and coherent overview of recent advances in this line of research. We will start by first giving a brief historical background of open-domain question answering, discussing the basic setup and core technical challenges of the research problem.The focus will then shift to modern techniques and resources proposed for open-domain QA, including the basics of latest neural reading comprehension systems, new datasets and models. The scope will also be broadened to cover the information retrieval component on how to effectively identify passages relevant to the questions. Moreover, in-depth discussions will be given on the use of traditional / neural IR modules, as well as the trade-offs between modular design and end-to-end training. If time permits, we also plan to discuss some hybrid approaches for answering questions using both text and large knowledge bases (e.g. (Sun et al., 2018)) and give a critical review on how structured data complements the information from unstructured text. 
At the end of our tutorial, we will discuss some important questions, including (1) How much progress have we made compared to the QA systems developed in the last decade?(2) What are the main challenges and limitations of cur-rent approaches? (3) How to trade off the efficiency (computational time and memory requirements) and accuracy in the deep learning era? We hope that our tutorial will not only serve as a useful resource for the audience to efficiently acquire the up-to-date knowledge, but also provide new perspectives to stimulate the advances of open-domain QA research in the next phase.

===Virtual Conference Organization===

In April the whole conference was decided to be held in virtual. We carried on the preparations jointly with the tutorial instructors, the virtual infrastructure chairs, the website chairs and the SlidesLive team to set up the virtual conference infrastructure, and to publicize it for each tutorial.

We first made a decision on the time slots for tutorial presentations. Five major time zones were considered, including US west coast, US east coast, central Europe, China/Asia and Australia (India was also considered). Our aim was to offer at least two convenient time slots (i.e. within regular working hours) for each of these zones.

The final time slots include (Jul 5 Pacific Time):

* slot 1 – 7:00 PM (-1D) to 10:30 PM (-1D)
* slot 2 – 12:00 AM to 3:30 AM
* slot 3 – 6:00 AM to 9:30 AM
* slot 4 – 10:30 AM to 2:00 PM
* slot 5 – 3:00 PM to 6:30 PM

We asked the tutorial teachers to indicate their preferences among the time slots, and – if possible – to agree to give their tutorial twice, so as to increase accessibility. Our final decisions on slot assignments traded off the teacher preferences and slot availability. In the end, the assignments were the following (see also the [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ blog post] and [https://acl2020.org/program/tutorials/ online schedule]):

* T1 – twice on slots 3 and 4
* T2 – once on slot 5
* T3 – once on slot 4
* T4 – once on slot 4
* T5 – twice on slots 3 and 5
* T6 – once on slot 5
* T7 – twice on slots 3 and 4
* T8 – once on slot 5

Unfortunately, the tutorial teachers proved hardly available for slots 1 and 2.

In addition to time slot arrangements, we gave the tutorial teacher two options on presentation, namely a pre-recorded and a fully interactive form of tutorials. The former allows the teachers to pre-record the main content of their tutorials, and use the time slots mainly for question answering and detailed discussions. The latter is more interactive, the teachers use the online sessions for both giving the lecture and other activities such as active discussions. All these arrangements were made for taking advantage of the virtual conference. In the end the choices were the following:

* T1 – pre-recorded
* T2 – interactive (the initial decision was pre-recorded but a change was made in June)
* T3 – pre-recorded
* T4 – pre-recorded
* T5 – pre-recorded
* T6 – interactive
* T7 – interactive
* T8 – interactive

After making decisions on time slots and teaching forms, we started to work on the technical details of the virtual conference infrastructure jointly with the infrastructure chairs, the website chairs, the SlidesLive support team and individual tutorial teachers. We collected needs from each tutorial from time to time, inquired information from the SlidesLive team, and tried to accommodate specific requirement for each tutorial. Since this is the first time ACL was virtual, it typically took several rounds of information exchange for making a certain technical feature work. Major things discussed included:

* Sorting out how to pre-record a tutorial using the system
* Recording of interactive sessions
* Additional RocketChat channel
* Virtual tutorial webpage setup
* Coordinators for tutorial sessions and SlidesLive team participation
* Rehearsal of interactive sessions
* Captioning of pre-recorded and interactive tutorials
* Privacy issues
* Special needs from particular tutorials
** Integrating [https://www.dory.app/ Dory] for question answering (in T1)
** Fully interactive sessions, with occasional split into break-out rooms (in T7)

To ensure smooth running of the sessions, we sent two rounds of reminders to each tutorial to ensure that each teacher is registered to the conference. We collected rough timelines of each tutorial so that the SlidesLive team can work closely during the live sessions. To further help the teachers prepare their live session content we additionally collected the lists of registered attendees twice before the conference, and shared them with the teachers.

In the end, the numbers of registrations for each tutorial were:
* T1 -- 1762
* T2 -- 793
* T3 -- 503
* T4 -- 571
* T5 -- 641
* T6 -- 1288
* T7 -- 378
* T8 – 1087
This sums up to 7023 registrations.

===Archiving and Publicity===

There are three main channels for publicity, namely the static [http://www.acl2020.org/program/tutorials/ website], the website of the [https://virtual.acl2020.org/tutorials.html virtual conference] (restricted to the registered participants) and the social media. For the former, we collected photos from each instructor, which are shown with the title and abstract of their tutorials. Following NAACL 2019, we also displayed the time slots as a right-hand-side bar on the tutorial website. Due to special arrangements of the virtual conference, we marked the time zones explicitly. In addition, we asked the tutorial teachers to provide URL links to teaching materials (e.g., at GitHub) during their final submission, and included these links to their tutorial information. For social media, we drafted two blog posts (i.e., [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ this post] and [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]) for advertising the tutorials.

The archiving consists of three main items, namely the tutorial proceedings, the slides for each tutorial and the videos. For the tutorial proceedings, we collected final abstracts from the authors using the START system. We asked the instructors to sign the CC BY v4 agreement when submitting their final materials on SoftConf. For the slides, we asked the tutorial teachers to provide a first version of the slides by June 17 regardless whether they are pre-recorded or interactive. Some tutorials made further updates to their slides before ACL. For the video recordings, we worked with the ACL Anthology team discussing the technical details since this is the first time the tutorial videos are shared in the website. [Note: this is currently undergoing at the time of writing of this summary]

===Lessons Learned===

While organizing a virtual conference it is important to stress that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]).

It is also important to announce the individual modalities with the attendees early enough. For instance, watching pre-recordings in advance was necessary for some tutorials (since their live sessions only included question answering) and such constrains should have been known more in advance.

If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to Zoom rooms only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.

One thing to remember which we missed is to publish the reading list for each tutorial (which the teachers submit with the proposal) on the tutorial website.

2020Q3 Reports: Tutorial Chairs

2020-07-23T10:10:51Z

AgataSavary: /* Archiving and Publicity */

Tutorial Chairs:

<blockquote>Agata Savary, University of Tours, France

Yue Zhang, Westlake University, Hangzhou, China
</blockquote>

=== Preparatory Work, Call, Reviewing and Decision-Making ===
The call, submission, reviewing and selection of tutorials was coordinated jointly for 4 conferences: ACL, AACL-IJCNLP, COLING and EMNLP.

Before drafting the call, we collected lists of tutorials offered within the past 4 years. We analysed previous calls for tutorials and reports from tutorial chairs (from [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs 2016], [https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs 2017], [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs 2018] and [http://aclweb.org/adminwiki/index.php?title=2019Q1_Reports:_ACL_2019 2019]). We consulted previous tutorial chairs with a questionnaire including questions about: the number of submissions, encouraging submissions on specific topics or from specific lecturers, the review procedure, the evaluation criteria, the post-tutorial availability of the slides/codes, and lessons learned from tutorial coordination. We also discussed the publication of slides and video recordings from future tutorials with the persons in charge of the ACL Anthology. As a result of these steps, we created two new sections for the [https://www.aclweb.org/adminwiki/index.php?title=Conference_Handbook ACL Conference Handbook] (future chairs might consider updating these documents yearly):

* the list of [https://www.aclweb.org/adminwiki/index.php?title=Past_tutorials past tutorials] at ACL, COLING, EACL, EMNLP, and NAACL in 2016-2019
* a [https://www.aclweb.org/adminwiki/index.php?title=Tutorial_chair_handbook tutorial chair handbook]

The final [https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020 call] differs from previous calls in several aspects: (i) the expectations about tutorial proposals were made clearer, (ii) following the central ACL decision, the teachers' payment policy was replaced by a fee-waiving policy, (iii) the required submission details include two new items: diversity considerations and agreement for open access publication of slides, codes, data and video recordings, (iv) the evaluation criteria (see below) are announced.

We recruited a review committee of 19 members, including the 8 tutorial chairs and 11 external members selected for their large understanding of the NLP domain and a good experience in reviewing and/or tutorial teaching:

'''Review Committee'''

* Timothy Baldwin (University of Melbourne, Australia) - AACL-IJCNLP 2020 tutorial chair
* Daniel Beck (University of Melbourne, Australia) - COLING 2020 tutorial chair
* Emily M. Bender (University of Washington, WA, USA)
* Erik Cambria (Nanyang Technological University, Singapore)
* Gaël Dias (University of Caen Normandie, France)
* Stefan Evert (Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany)
* Yang Liu (Tsinghua University, Beijing, China)
* Agata Savary (University of Tours, France) - ACL 2020 tutorial chair
* João Sedoc (Johns Hopkins University, Baltimore, MD, USA)
* Lucia Specia (Sheffield University, UK) - COLING 2020 tutorial chair
* Xu Sun (Peking University, China)
* Yulia Tsvetkov (Carnegie Mellon University, Pittsburgh, PA, USA)
* Benjamin Van Durme (Johns Hopkins University, Baltimore, MD, USA) - EMNLP 2020 tutorial chair
* Aline Villavicencio (University of Sheffield, UK and Federal University of Rio Grande do Sul, Brazil) - EMNLP 2020 tutorial chair
* Taro Watanabe (Google, Inc., Tokyo, Japan)
* Aaron Steven White (University of Rochester, NY, USA)
* Fei Xia (University of Washington, WA, USA) - AACL-IJCNLP 2020 tutorial chair
* Yue Zhang (Westlake University, Hangzhou, China) - ACL 2020 tutorial chair
* Meishan Zhang (Tianjin University, China)

In total, we received 43 submissions for the 4 conferences. Each reviewer was assigned 6-7 proposals and each proposal received 3 reviews. The selection criteria included: clarity and preparedness, novelty or timely character of the topic, lecturers' experience, likely audience interest, open access of the teaching material, diversity aspects (multilingualism, gender, age and country of the lecturers), and compatibility with the preferred venues. We accepted 31 proposals.

The decision making was handled via an online meeting of the 8 tutorial chairs. In particular, the selection of tutorials for each conference was done via the expression of interest of the tutorial chairs on a round-robin basis. Some slight adjustments were also performed after the meeting to better fit the authors' preferences. In total, 8, 8, 8 and 7 proposals were selected for ACL, AACL-IJCNLP, COLING and EMNLP, respectively. Upon the announcement the results, 2 of the proposals accepted for AACL-IJCNLP were withdrawn.

The submission, review, selection and collection of final descriptions for all tutorials was handled via a dedicated SoftConf space, shared by the 4 coordinating conferences. After the selection of proposals, a separate track was created on SoftConf for each conference. The final submission page (one per conference) was set up so as to collect all the necessary data including notably: the tutorial slides, URLs for course material (if any), printable material (if any) and agreement for open access publication.

===Tutorials Selected for ACL 2020===

The final selection for ACL 2020 consists of the following 8 tutorials of 3 hours each (each of them had ACL as the preferred or the second preferred venue):

'''T1: Interpretability and Analysis in Neural NLP''' (cutting-edge) 
Yonatan Belinkov, Sebastian Gehrmann and Ellie Pavlick 
While deep learning has transformed the NLP field and impacted the larger computational linguistics community, the rise of neural networks is stained by their opaque nature: It is challenging to interpret the inner workings of neural network models, and explicate their behavior. Therefore, in the last few years, an increasingly large body of work has been devoted to the analysis and interpretation of neural network models in NLP. 
This body of work is so far lacking a common framework and methodology. Moreover, approaching the analysis of modern neural networks can be difficult for newcomers to the field. This tutorial aims to fill this gap and introduce the nascent field of interpretability and analysis of neural networks in NLP. 
The tutorial covers the main lines of analysis work, such as probing classifier, behavior studies and test suites, psycholinguistic methods, visualizations, adversarial examples, and other methods. We highlight not only the most commonly applied analysis methods, but also the specific limitations and shortcomings of current approaches, in order to inform participants where to focus future efforts.

'''T2: Multi-modal Information Extraction from Text, Semi-structured, and Tabular Data on the Web''' (cutting-edge) 
Xin Luna Dong, Hannaneh Hajishirzi, Colin Lockard and Prashant Shiralkar 
The World Wide Web contains vast quantities of textual information in several forms: unstructured text, template-based semi-structured webpages (which present data in key-value pairs and lists), and tables. Methods for extracting information from these sources and converting it to a structured form have been a target of research from the natural language processing (NLP), data mining, and database communities. While these researchers have largely separated extraction from web data into different problems based on the modality of the data, they have faced similar problems such as learning with limited labeled data, defining (or avoiding defining) ontologies, making use of prior knowledge, and scaling solutions to deal with the size of the Web. 
In this tutorial we take a holistic view toward information extraction, exploring the commonalities in the challenges and solutions developed to address these different forms of text. We will explore the approaches targeted at unstructured text that largely rely on learning syntactic or semantic textual patterns, approaches targeted at semi-structured documents that learn to identify structural patterns in the template, and approaches targeting web tables which rely heavily on entity linking and type information. 
While these different data modalities have largely been considered separately in the past, recent research has started taking a more inclusive approach toward textual extraction, in which the multiple signals offered by textual, layout, and visual clues are combined into a single extraction model made possible by new deep learning approaches. At the same time, trends within purely textual extraction have shifted toward full-document understanding rather than considering sentences as independent units. With this in mind, it is worth considering the information extraction problem as a whole to motivate solutions that harness textual semantics along with visual and semi-structured layout information. We will discuss these approaches and suggest avenues for future work.

'''T3: Reviewing Natural Language Processing Research''' (introductory) 
Kevin Cohen, Karën Fort, Margot Mieskes and Aurélie Névéol 
As the demand for reviewing grows, so must the pool of reviewers. As the [http://www.livecongress.it/aol/indexSA.php?id=E2EAED7D&ticket= survey] presented by Graham Neubig at the 2019 ACL showed, a considerable number of reviewers are junior researchers, who might lack the experience and expertise necessary for high-quality reviews. Some of them might not have the environment or lack opportunities that allow them to learn the skills necessary. A tutorial on reviewing for the NLP community might increase reviewers’ confidence, as well as the quality of the reviews. This introductory tutorial will cover the goals, processes, and evaluation of reviewing research papers in natural language processing.

'''T4: Stylized Text Generation: Approaches and Applications''' (cutting-edge) 
Lili Mou and Olga Vechtomova 
Text generation has played an important role in various applications of natural language processing (NLP), and kn recent studies, researchers are paying increasing attention to modeling and manipulating the style of the generation text, which we call stylized text generation. In this tutorial, we will provide a comprehensive literature review in this direction. We start from the definition of style and different settings of stylized text generation, illustrated with various applications. Then, we present different settings of stylized generation, such as parallel supervised, style label-supervised, and unsupervised. In each setting, we delve deep into machine learning methods, including embedding learning techniques to represent style}, adversarial learning and reinforcement learning with cycle consistency to match content but to distinguish different styles. We also introduce current approaches of evaluating stylized text generation systems. We conclude our tutorial by presenting the challenges of stylized text generation and discussing future directions, such as small-data training, non-categorical style modeling, and a generalized scope of style transfer (e.g., controlling the syntax as a style).

'''T5: Achieving Common Ground in Multi-modal Dialogue''' (cutting-edge) 
Malihe Alikhani and Matthew Stone 
All communication aims at achieving common ground (grounding): interlocutors can work together effectively only with mutual beliefs about what the state of the world is, about what their goals are, and about how they plan to make their goals a reality. Computational dialogue research offers some classic results on grouding, which unfortunately offer scant guidance to the design of grounding modules and behaviors in cutting-edge systems. In this tutorial, we focus on three main topic areas: 1) grounding in human-human communication; 2) grounding in dialogue systems; and 3) grounding in multi-modal interactive systems, including image-oriented conversations and human-robot interactions. We highlight a number of achievements of recent computational research in coordinating complex content, show how these results lead to rich and challenging opportunities for doing grounding in more flexible and powerful ways, and canvass relevant insights from the literature on human--human conversation. We expect that the tutorial will be of interest to researchers in dialogue systems, computational semantics and cognitive modeling, and hope that it will catalyze research and system building that more directly explores the creative, strategic ways conversational agents might be able to seek and offer evidence about their understanding of their interlocutors.

'''T6: Commonsense Reasoning for Natural Language Processing''' (introductory) 
Maarten Sap, Vered Shwartz, Antoine Bosselut, Dan Roth and Yejin Choi 
In our tutorial, we (1) outline the various types of commonsense (e.g., physical, social), and (2) discuss techniques to gather and represent commonsense knowledge, while highlighting the challenges specific to this type of knowledge (e.g., reporting bias). We will then (3) discuss the types of commonsense knowledge captured by modern NLP systems (e.g., large pretrained language models), and (4) present ways to measure systems' commonsense reasoning abilities. We finish with (5) a discussion of various ways in which commonsense reasoning can be used to improve performance on NLP tasks, exemplified by an (6) interactive session on integrating commonsense into a downstream task.

'''T7: Integrating Ethics into the NLP Curriculum''' (introductory) 
Emily M. Bender, Dirk Hovy and Alexandra Schofield 
Our goal in this tutorial is to empower NLP researchers and practitioners with tools and resources to teach others about how to ethically apply NLP techniques. Our tutorial will present both high-level strategies for developing an ethics-oriented curriculum, based on experience and best practices, as well as specific sample exercises that can be brought to a classroom. We plan to make this a highly interactive work session culminating in a shared online resource page that pools lesson plans, assignments, exercise ideas, reading suggestions, and ideas from the attendees. We consider three primary topics with our session that frequently underlie ethical issues in NLP research: Dual use, bias and privacy. 
In this setting, a key lesson is that there is no single approach to ethical NLP: each project requires thoughtful consideration about what steps can be taken to best support people affected by that project. However, we can learn (and teach) what kinds of issues to be aware of and what kinds of strategies are available for mitigating harm. To teach this process, we apply and promote interactive exercises that provide an opportunity to ideate, discuss, and reflect. We plan to facilitate this in a way that encourages positive discussion, emphasizing the creation of ideas for the future instead of negative opinions of previous work.

'''T8: Recent Advances in Open-Domain Question Answering''' (cutting-edge) 
Danqi Chen and Scott Wen-tau Yih 
Open-domain (textual) question answering (QA), the task of finding answers to open-domain questions by searching a large collection of documents, has been a long-standing problem in NLP, information retrieval (IR) and related fields (Voorhees et al., 1999; Moldovan et al., 2000; Brill et al.,2002; Ferrucci et al., 2010). Traditional QA systems were usually constructed as a pipeline, consisting of many different components such as question processing, document/passage retrieval and answer processing. With the rapid development of neural reading comprehension (Chen, 2018), modern open-domain QA systems have been restructured by combining traditional IR techniques and neural reading comprehension models (Chen et al., 2017; Yang et al., 2019) or even implemented in a fully end-to-end fashion (Lee et al., 2019; Seo et al., 2019). While the system architecture has been drastically simplified, two technical challenges remain critical:(1) “Retriever”: finding documents that (might)contain an answer from a large collection of documents; (2) “Reader”: finding the answer in a given paragraph or a document. 
In this tutorial, we aim to provide a comprehensive and coherent overview of recent advances in this line of research. We will start by first giving a brief historical background of open-domain question answering, discussing the basic setup and core technical challenges of the research problem.The focus will then shift to modern techniques and resources proposed for open-domain QA, including the basics of latest neural reading comprehension systems, new datasets and models. The scope will also be broadened to cover the information retrieval component on how to effectively identify passages relevant to the questions. Moreover, in-depth discussions will be given on the use of traditional / neural IR modules, as well as the trade-offs between modular design and end-to-end training. If time permits, we also plan to discuss some hybrid approaches for answering questions using both text and large knowledge bases (e.g. (Sun et al., 2018)) and give a critical review on how structured data complements the information from unstructured text. 
At the end of our tutorial, we will discuss some important questions, including (1) How much progress have we made compared to the QA systems developed in the last decade?(2) What are the main challenges and limitations of cur-rent approaches? (3) How to trade off the efficiency (computational time and memory requirements) and accuracy in the deep learning era? We hope that our tutorial will not only serve as a useful resource for the audience to efficiently acquire the up-to-date knowledge, but also provide new perspectives to stimulate the advances of open-domain QA research in the next phase.

===Virtual Conference Organization===

In April the whole conference was decided to be held in virtual. We carried on the preparations jointly with the tutorial instructors, the virtual infrastructure chairs, the website chairs and the SlidesLive team to set up the virtual conference infrastructure, and to publicize it for each tutorial.

We first made a decision on the time slots for tutorial presentations. Five major time zones were considered, including US west coast, US east coast, central Europe, China/Asia and Australia (India was also considered). Our aim was to offer at least two convenient time slots (i.e. within regular working hours) for each of these zones.

The final time slots include (Jul 5 Pacific Time):

* slot 1 – 7:00 PM (-1D) to 10:30 PM (-1D)
* slot 2 – 12:00 AM to 3:30 AM
* slot 3 – 6:00 AM to 9:30 AM
* slot 4 – 10:30 AM to 2:00 PM
* slot 5 – 3:00 PM to 6:30 PM

We asked the tutorial teachers to indicate their preferences among the time slots, and – if possible – to agree to give their tutorial twice, so as to increase accessibility. Our final decisions on slot assignments traded off the teacher preferences and slot availability. In the end, the assignments were the following (see also the [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ blog post] and [https://acl2020.org/program/tutorials/ online schedule]):

* T1 – twice on slots 3 and 4
* T2 – once on slot 5
* T3 – once on slot 4
* T4 – once on slot 4
* T5 – twice on slots 3 and 5
* T6 – once on slot 5
* T7 – twice on slots 3 and 4
* T8 – once on slot 5

Unfortunately, the tutorial teachers proved hardly available for slots 1 and 2.

In addition to time slot arrangements, we gave the tutorial teacher two options on presentation, namely a pre-recorded and a fully interactive form of tutorials. The former allows the teachers to pre-record the main content of their tutorials, and use the time slots mainly for question answering and detailed discussions. The latter is more interactive, the teachers use the online sessions for both giving the lecture and other activities such as active discussions. All these arrangements were made for taking advantage of the virtual conference. In the end the choices were the following:

* T1 – pre-recorded
* T2 – interactive (the initial decision was pre-recorded but a change was made in June)
* T3 – pre-recorded
* T4 – pre-recorded
* T5 – pre-recorded
* T6 – interactive
* T7 – interactive
* T8 – interactive

After making decisions on time slots and teaching forms, we started to work on the technical details of the virtual conference infrastructure jointly with the infrastructure chairs, the website chairs, the SlidesLive support team and individual tutorial teachers. We collected needs from each tutorial from time to time, inquired information from the SlidesLive team, and tried to accommodate specific requirement for each tutorial. Since this is the first time ACL was virtual, it typically took several rounds of information exchange for making a certain technical feature work. Major things discussed included:

* Sorting out how to pre-record a tutorial using the system
* Recording of interactive sessions
* Additional RocketChat channel
* Virtual tutorial webpage setup
* Coordinators for tutorial sessions and SlidesLive team participation
* Rehearsal of interactive sessions
* Captioning of pre-recorded and interactive tutorials
* Privacy issues
* Special needs from particular tutorials
** Integrating [https://www.dory.app/ Dory] for question answering (in T1)
** Fully interactive sessions, with occasional split into break-out rooms (in T7)

To ensure smooth running of the sessions, we sent two rounds of reminders to each tutorial to ensure that each teacher is registered to the conference. We collected rough timelines of each tutorial so that the SlidesLive team can work closely during the live sessions. To further help the teachers prepare their live session content we additionally collected the lists of registered attendees twice before the conference, and shared them with the teachers.

In the end, the numbers of registrations for each tutorial were:
* T1 -- 1762
* T2 -- 793
* T3 -- 503
* T4 -- 571
* T5 -- 641
* T6 -- 1288
* T7 -- 378
* T8 – 1087
This sums up to 7023 registrations.

===Archiving and Publicity===

There are three main channels for publicity, namely the static [http://www.acl2020.org/program/tutorials/ website], the website of the [https://virtual.acl2020.org/tutorials.html virtual conference] (restricted to the registered participant) and the social media. For the former, we collected photos from each instructor, which are shown with the title and abstract of their tutorials. Following NAACL 2019, we also displayed the time slots as a right-hand-side bar on the tutorial website. Due to special arrangements of the virtual conference, we marked the time zones explicitly. In addition, we asked the tutorial teachers to provide URL links to teaching materials (e.g., at GitHub) during their final submission, and included these links to their tutorial information. For social media, we drafted two blog posts (i.e., [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ this post] and [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]) for advertising the tutorials.

The archiving consists of three main items, namely the tutorial proceedings, the slides for each tutorial and the videos. For the tutorial proceedings, we collected final abstracts from the authors using the START system. We asked the instructors to sign the CC BY v4 agreement when submitting their final materials on SoftConf. For the slides, we asked the tutorial teachers to provide a first version of the slides by June 17 regardless whether they are pre-recorded or interactive. Some tutorials made further updates to their slides before ACL. For the video recordings, we worked with the ACL Anthology team discussing the technical details since this is the first time the tutorial videos are shared in the website. [Note: this is currently undergoing at the time of writing of this summary]

===Lessons Learned===

While organizing a virtual conference it is important to stress that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]).

It is also important to announce the individual modalities with the attendees early enough. For instance, watching pre-recordings in advance was necessary for some tutorials (since their live sessions only included question answering) and such constrains should have been known more in advance.

If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to Zoom rooms only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.

One thing to remember which we missed is to publish the reading list for each tutorial (which the teachers submit with the proposal) on the tutorial website.

2020Q3 Reports: Tutorial Chairs

2020-07-23T10:09:26Z

AgataSavary: /* Virtual Conference Organization */

Tutorial Chairs:

<blockquote>Agata Savary, University of Tours, France

Yue Zhang, Westlake University, Hangzhou, China
</blockquote>

=== Preparatory Work, Call, Reviewing and Decision-Making ===
The call, submission, reviewing and selection of tutorials was coordinated jointly for 4 conferences: ACL, AACL-IJCNLP, COLING and EMNLP.

Before drafting the call, we collected lists of tutorials offered within the past 4 years. We analysed previous calls for tutorials and reports from tutorial chairs (from [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs 2016], [https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs 2017], [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs 2018] and [http://aclweb.org/adminwiki/index.php?title=2019Q1_Reports:_ACL_2019 2019]). We consulted previous tutorial chairs with a questionnaire including questions about: the number of submissions, encouraging submissions on specific topics or from specific lecturers, the review procedure, the evaluation criteria, the post-tutorial availability of the slides/codes, and lessons learned from tutorial coordination. We also discussed the publication of slides and video recordings from future tutorials with the persons in charge of the ACL Anthology. As a result of these steps, we created two new sections for the [https://www.aclweb.org/adminwiki/index.php?title=Conference_Handbook ACL Conference Handbook] (future chairs might consider updating these documents yearly):

* the list of [https://www.aclweb.org/adminwiki/index.php?title=Past_tutorials past tutorials] at ACL, COLING, EACL, EMNLP, and NAACL in 2016-2019
* a [https://www.aclweb.org/adminwiki/index.php?title=Tutorial_chair_handbook tutorial chair handbook]

The final [https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020 call] differs from previous calls in several aspects: (i) the expectations about tutorial proposals were made clearer, (ii) following the central ACL decision, the teachers' payment policy was replaced by a fee-waiving policy, (iii) the required submission details include two new items: diversity considerations and agreement for open access publication of slides, codes, data and video recordings, (iv) the evaluation criteria (see below) are announced.

We recruited a review committee of 19 members, including the 8 tutorial chairs and 11 external members selected for their large understanding of the NLP domain and a good experience in reviewing and/or tutorial teaching:

'''Review Committee'''

* Timothy Baldwin (University of Melbourne, Australia) - AACL-IJCNLP 2020 tutorial chair
* Daniel Beck (University of Melbourne, Australia) - COLING 2020 tutorial chair
* Emily M. Bender (University of Washington, WA, USA)
* Erik Cambria (Nanyang Technological University, Singapore)
* Gaël Dias (University of Caen Normandie, France)
* Stefan Evert (Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany)
* Yang Liu (Tsinghua University, Beijing, China)
* Agata Savary (University of Tours, France) - ACL 2020 tutorial chair
* João Sedoc (Johns Hopkins University, Baltimore, MD, USA)
* Lucia Specia (Sheffield University, UK) - COLING 2020 tutorial chair
* Xu Sun (Peking University, China)
* Yulia Tsvetkov (Carnegie Mellon University, Pittsburgh, PA, USA)
* Benjamin Van Durme (Johns Hopkins University, Baltimore, MD, USA) - EMNLP 2020 tutorial chair
* Aline Villavicencio (University of Sheffield, UK and Federal University of Rio Grande do Sul, Brazil) - EMNLP 2020 tutorial chair
* Taro Watanabe (Google, Inc., Tokyo, Japan)
* Aaron Steven White (University of Rochester, NY, USA)
* Fei Xia (University of Washington, WA, USA) - AACL-IJCNLP 2020 tutorial chair
* Yue Zhang (Westlake University, Hangzhou, China) - ACL 2020 tutorial chair
* Meishan Zhang (Tianjin University, China)

In total, we received 43 submissions for the 4 conferences. Each reviewer was assigned 6-7 proposals and each proposal received 3 reviews. The selection criteria included: clarity and preparedness, novelty or timely character of the topic, lecturers' experience, likely audience interest, open access of the teaching material, diversity aspects (multilingualism, gender, age and country of the lecturers), and compatibility with the preferred venues. We accepted 31 proposals.

The decision making was handled via an online meeting of the 8 tutorial chairs. In particular, the selection of tutorials for each conference was done via the expression of interest of the tutorial chairs on a round-robin basis. Some slight adjustments were also performed after the meeting to better fit the authors' preferences. In total, 8, 8, 8 and 7 proposals were selected for ACL, AACL-IJCNLP, COLING and EMNLP, respectively. Upon the announcement the results, 2 of the proposals accepted for AACL-IJCNLP were withdrawn.

The submission, review, selection and collection of final descriptions for all tutorials was handled via a dedicated SoftConf space, shared by the 4 coordinating conferences. After the selection of proposals, a separate track was created on SoftConf for each conference. The final submission page (one per conference) was set up so as to collect all the necessary data including notably: the tutorial slides, URLs for course material (if any), printable material (if any) and agreement for open access publication.

===Tutorials Selected for ACL 2020===

The final selection for ACL 2020 consists of the following 8 tutorials of 3 hours each (each of them had ACL as the preferred or the second preferred venue):

'''T1: Interpretability and Analysis in Neural NLP''' (cutting-edge) 
Yonatan Belinkov, Sebastian Gehrmann and Ellie Pavlick 
While deep learning has transformed the NLP field and impacted the larger computational linguistics community, the rise of neural networks is stained by their opaque nature: It is challenging to interpret the inner workings of neural network models, and explicate their behavior. Therefore, in the last few years, an increasingly large body of work has been devoted to the analysis and interpretation of neural network models in NLP. 
This body of work is so far lacking a common framework and methodology. Moreover, approaching the analysis of modern neural networks can be difficult for newcomers to the field. This tutorial aims to fill this gap and introduce the nascent field of interpretability and analysis of neural networks in NLP. 
The tutorial covers the main lines of analysis work, such as probing classifier, behavior studies and test suites, psycholinguistic methods, visualizations, adversarial examples, and other methods. We highlight not only the most commonly applied analysis methods, but also the specific limitations and shortcomings of current approaches, in order to inform participants where to focus future efforts.

'''T2: Multi-modal Information Extraction from Text, Semi-structured, and Tabular Data on the Web''' (cutting-edge) 
Xin Luna Dong, Hannaneh Hajishirzi, Colin Lockard and Prashant Shiralkar 
The World Wide Web contains vast quantities of textual information in several forms: unstructured text, template-based semi-structured webpages (which present data in key-value pairs and lists), and tables. Methods for extracting information from these sources and converting it to a structured form have been a target of research from the natural language processing (NLP), data mining, and database communities. While these researchers have largely separated extraction from web data into different problems based on the modality of the data, they have faced similar problems such as learning with limited labeled data, defining (or avoiding defining) ontologies, making use of prior knowledge, and scaling solutions to deal with the size of the Web. 
In this tutorial we take a holistic view toward information extraction, exploring the commonalities in the challenges and solutions developed to address these different forms of text. We will explore the approaches targeted at unstructured text that largely rely on learning syntactic or semantic textual patterns, approaches targeted at semi-structured documents that learn to identify structural patterns in the template, and approaches targeting web tables which rely heavily on entity linking and type information. 
While these different data modalities have largely been considered separately in the past, recent research has started taking a more inclusive approach toward textual extraction, in which the multiple signals offered by textual, layout, and visual clues are combined into a single extraction model made possible by new deep learning approaches. At the same time, trends within purely textual extraction have shifted toward full-document understanding rather than considering sentences as independent units. With this in mind, it is worth considering the information extraction problem as a whole to motivate solutions that harness textual semantics along with visual and semi-structured layout information. We will discuss these approaches and suggest avenues for future work.

'''T3: Reviewing Natural Language Processing Research''' (introductory) 
Kevin Cohen, Karën Fort, Margot Mieskes and Aurélie Névéol 
As the demand for reviewing grows, so must the pool of reviewers. As the [http://www.livecongress.it/aol/indexSA.php?id=E2EAED7D&ticket= survey] presented by Graham Neubig at the 2019 ACL showed, a considerable number of reviewers are junior researchers, who might lack the experience and expertise necessary for high-quality reviews. Some of them might not have the environment or lack opportunities that allow them to learn the skills necessary. A tutorial on reviewing for the NLP community might increase reviewers’ confidence, as well as the quality of the reviews. This introductory tutorial will cover the goals, processes, and evaluation of reviewing research papers in natural language processing.

'''T4: Stylized Text Generation: Approaches and Applications''' (cutting-edge) 
Lili Mou and Olga Vechtomova 
Text generation has played an important role in various applications of natural language processing (NLP), and kn recent studies, researchers are paying increasing attention to modeling and manipulating the style of the generation text, which we call stylized text generation. In this tutorial, we will provide a comprehensive literature review in this direction. We start from the definition of style and different settings of stylized text generation, illustrated with various applications. Then, we present different settings of stylized generation, such as parallel supervised, style label-supervised, and unsupervised. In each setting, we delve deep into machine learning methods, including embedding learning techniques to represent style}, adversarial learning and reinforcement learning with cycle consistency to match content but to distinguish different styles. We also introduce current approaches of evaluating stylized text generation systems. We conclude our tutorial by presenting the challenges of stylized text generation and discussing future directions, such as small-data training, non-categorical style modeling, and a generalized scope of style transfer (e.g., controlling the syntax as a style).

'''T5: Achieving Common Ground in Multi-modal Dialogue''' (cutting-edge) 
Malihe Alikhani and Matthew Stone 
All communication aims at achieving common ground (grounding): interlocutors can work together effectively only with mutual beliefs about what the state of the world is, about what their goals are, and about how they plan to make their goals a reality. Computational dialogue research offers some classic results on grouding, which unfortunately offer scant guidance to the design of grounding modules and behaviors in cutting-edge systems. In this tutorial, we focus on three main topic areas: 1) grounding in human-human communication; 2) grounding in dialogue systems; and 3) grounding in multi-modal interactive systems, including image-oriented conversations and human-robot interactions. We highlight a number of achievements of recent computational research in coordinating complex content, show how these results lead to rich and challenging opportunities for doing grounding in more flexible and powerful ways, and canvass relevant insights from the literature on human--human conversation. We expect that the tutorial will be of interest to researchers in dialogue systems, computational semantics and cognitive modeling, and hope that it will catalyze research and system building that more directly explores the creative, strategic ways conversational agents might be able to seek and offer evidence about their understanding of their interlocutors.

'''T6: Commonsense Reasoning for Natural Language Processing''' (introductory) 
Maarten Sap, Vered Shwartz, Antoine Bosselut, Dan Roth and Yejin Choi 
In our tutorial, we (1) outline the various types of commonsense (e.g., physical, social), and (2) discuss techniques to gather and represent commonsense knowledge, while highlighting the challenges specific to this type of knowledge (e.g., reporting bias). We will then (3) discuss the types of commonsense knowledge captured by modern NLP systems (e.g., large pretrained language models), and (4) present ways to measure systems' commonsense reasoning abilities. We finish with (5) a discussion of various ways in which commonsense reasoning can be used to improve performance on NLP tasks, exemplified by an (6) interactive session on integrating commonsense into a downstream task.

'''T7: Integrating Ethics into the NLP Curriculum''' (introductory) 
Emily M. Bender, Dirk Hovy and Alexandra Schofield 
Our goal in this tutorial is to empower NLP researchers and practitioners with tools and resources to teach others about how to ethically apply NLP techniques. Our tutorial will present both high-level strategies for developing an ethics-oriented curriculum, based on experience and best practices, as well as specific sample exercises that can be brought to a classroom. We plan to make this a highly interactive work session culminating in a shared online resource page that pools lesson plans, assignments, exercise ideas, reading suggestions, and ideas from the attendees. We consider three primary topics with our session that frequently underlie ethical issues in NLP research: Dual use, bias and privacy. 
In this setting, a key lesson is that there is no single approach to ethical NLP: each project requires thoughtful consideration about what steps can be taken to best support people affected by that project. However, we can learn (and teach) what kinds of issues to be aware of and what kinds of strategies are available for mitigating harm. To teach this process, we apply and promote interactive exercises that provide an opportunity to ideate, discuss, and reflect. We plan to facilitate this in a way that encourages positive discussion, emphasizing the creation of ideas for the future instead of negative opinions of previous work.

'''T8: Recent Advances in Open-Domain Question Answering''' (cutting-edge) 
Danqi Chen and Scott Wen-tau Yih 
Open-domain (textual) question answering (QA), the task of finding answers to open-domain questions by searching a large collection of documents, has been a long-standing problem in NLP, information retrieval (IR) and related fields (Voorhees et al., 1999; Moldovan et al., 2000; Brill et al.,2002; Ferrucci et al., 2010). Traditional QA systems were usually constructed as a pipeline, consisting of many different components such as question processing, document/passage retrieval and answer processing. With the rapid development of neural reading comprehension (Chen, 2018), modern open-domain QA systems have been restructured by combining traditional IR techniques and neural reading comprehension models (Chen et al., 2017; Yang et al., 2019) or even implemented in a fully end-to-end fashion (Lee et al., 2019; Seo et al., 2019). While the system architecture has been drastically simplified, two technical challenges remain critical:(1) “Retriever”: finding documents that (might)contain an answer from a large collection of documents; (2) “Reader”: finding the answer in a given paragraph or a document. 
In this tutorial, we aim to provide a comprehensive and coherent overview of recent advances in this line of research. We will start by first giving a brief historical background of open-domain question answering, discussing the basic setup and core technical challenges of the research problem.The focus will then shift to modern techniques and resources proposed for open-domain QA, including the basics of latest neural reading comprehension systems, new datasets and models. The scope will also be broadened to cover the information retrieval component on how to effectively identify passages relevant to the questions. Moreover, in-depth discussions will be given on the use of traditional / neural IR modules, as well as the trade-offs between modular design and end-to-end training. If time permits, we also plan to discuss some hybrid approaches for answering questions using both text and large knowledge bases (e.g. (Sun et al., 2018)) and give a critical review on how structured data complements the information from unstructured text. 
At the end of our tutorial, we will discuss some important questions, including (1) How much progress have we made compared to the QA systems developed in the last decade?(2) What are the main challenges and limitations of cur-rent approaches? (3) How to trade off the efficiency (computational time and memory requirements) and accuracy in the deep learning era? We hope that our tutorial will not only serve as a useful resource for the audience to efficiently acquire the up-to-date knowledge, but also provide new perspectives to stimulate the advances of open-domain QA research in the next phase.

===Virtual Conference Organization===

In April the whole conference was decided to be held in virtual. We carried on the preparations jointly with the tutorial instructors, the virtual infrastructure chairs, the website chairs and the SlidesLive team to set up the virtual conference infrastructure, and to publicize it for each tutorial.

We first made a decision on the time slots for tutorial presentations. Five major time zones were considered, including US west coast, US east coast, central Europe, China/Asia and Australia (India was also considered). Our aim was to offer at least two convenient time slots (i.e. within regular working hours) for each of these zones.

The final time slots include (Jul 5 Pacific Time):

* slot 1 – 7:00 PM (-1D) to 10:30 PM (-1D)
* slot 2 – 12:00 AM to 3:30 AM
* slot 3 – 6:00 AM to 9:30 AM
* slot 4 – 10:30 AM to 2:00 PM
* slot 5 – 3:00 PM to 6:30 PM

We asked the tutorial teachers to indicate their preferences among the time slots, and – if possible – to agree to give their tutorial twice, so as to increase accessibility. Our final decisions on slot assignments traded off the teacher preferences and slot availability. In the end, the assignments were the following (see also the [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ blog post] and [https://acl2020.org/program/tutorials/ online schedule]):

* T1 – twice on slots 3 and 4
* T2 – once on slot 5
* T3 – once on slot 4
* T4 – once on slot 4
* T5 – twice on slots 3 and 5
* T6 – once on slot 5
* T7 – twice on slots 3 and 4
* T8 – once on slot 5

Unfortunately, the tutorial teachers proved hardly available for slots 1 and 2.

In addition to time slot arrangements, we gave the tutorial teacher two options on presentation, namely a pre-recorded and a fully interactive form of tutorials. The former allows the teachers to pre-record the main content of their tutorials, and use the time slots mainly for question answering and detailed discussions. The latter is more interactive, the teachers use the online sessions for both giving the lecture and other activities such as active discussions. All these arrangements were made for taking advantage of the virtual conference. In the end the choices were the following:

* T1 – pre-recorded
* T2 – interactive (the initial decision was pre-recorded but a change was made in June)
* T3 – pre-recorded
* T4 – pre-recorded
* T5 – pre-recorded
* T6 – interactive
* T7 – interactive
* T8 – interactive

After making decisions on time slots and teaching forms, we started to work on the technical details of the virtual conference infrastructure jointly with the infrastructure chairs, the website chairs, the SlidesLive support team and individual tutorial teachers. We collected needs from each tutorial from time to time, inquired information from the SlidesLive team, and tried to accommodate specific requirement for each tutorial. Since this is the first time ACL was virtual, it typically took several rounds of information exchange for making a certain technical feature work. Major things discussed included:

* Sorting out how to pre-record a tutorial using the system
* Recording of interactive sessions
* Additional RocketChat channel
* Virtual tutorial webpage setup
* Coordinators for tutorial sessions and SlidesLive team participation
* Rehearsal of interactive sessions
* Captioning of pre-recorded and interactive tutorials
* Privacy issues
* Special needs from particular tutorials
** Integrating [https://www.dory.app/ Dory] for question answering (in T1)
** Fully interactive sessions, with occasional split into break-out rooms (in T7)

To ensure smooth running of the sessions, we sent two rounds of reminders to each tutorial to ensure that each teacher is registered to the conference. We collected rough timelines of each tutorial so that the SlidesLive team can work closely during the live sessions. To further help the teachers prepare their live session content we additionally collected the lists of registered attendees twice before the conference, and shared them with the teachers.

In the end, the numbers of registrations for each tutorial were:
* T1 -- 1762
* T2 -- 793
* T3 -- 503
* T4 -- 571
* T5 -- 641
* T6 -- 1288
* T7 -- 378
* T8 – 1087
This sums up to 7023 registrations.

===Archiving and Publicity===

There are three main channels for publicity, namely the static [http://www.acl2020.org/program/tutorials/ website], the website of the [https://virtual.acl2020.org/tutorials.html virtual conference] and the social media. For the former, we collected photos from each instructor, which are shown with the title and abstract of their tutorials. Following NAACL 2019, we also displayed the time slots as a right-hand-side bar on the tutorial website. Due to special arrangements of the virtual conference, we marked the time zones explicitly. In addition, we asked the tutorial teachers to provide URL links to teaching materials (e.g., at GitHub) during their final submission, and included these links to their tutorial information. For social media, we drafted two blog posts (i.e., [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ this post] and [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]) for advertising the tutorials.

The archiving consists of three main items, namely the tutorial proceedings, the slides for each tutorial and the videos. For the tutorial proceedings, we collected final abstracts from the authors using the START system. We asked the instructors to sign the CC BY v4 agreement when submitting their final materials on SoftConf. For the slides, we asked the tutorial teachers to provide a first version of the slides by June 17 regardless whether they are pre-recorded or interactive. Some tutorials made further updates to their slides before ACL. For the video recordings, we worked with the ACL Anthology team discussing the technical details since this is the first time the tutorial videos are shared in the website. [Note: this is currently undergoing at the time of writing of this summary]

===Lessons Learned===

While organizing a virtual conference it is important to stress that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]).

It is also important to announce the individual modalities with the attendees early enough. For instance, watching pre-recordings in advance was necessary for some tutorials (since their live sessions only included question answering) and such constrains should have been known more in advance.

If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to Zoom rooms only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.

One thing to remember which we missed is to publish the reading list for each tutorial (which the teachers submit with the proposal) on the tutorial website.

2020Q3 Reports: Tutorial Chairs

2020-07-23T10:05:35Z

AgataSavary: /* Virtual Conference Organization */

Tutorial Chairs:

<blockquote>Agata Savary, University of Tours, France

Yue Zhang, Westlake University, Hangzhou, China
</blockquote>

=== Preparatory Work, Call, Reviewing and Decision-Making ===
The call, submission, reviewing and selection of tutorials was coordinated jointly for 4 conferences: ACL, AACL-IJCNLP, COLING and EMNLP.

Before drafting the call, we collected lists of tutorials offered within the past 4 years. We analysed previous calls for tutorials and reports from tutorial chairs (from [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs 2016], [https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs 2017], [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs 2018] and [http://aclweb.org/adminwiki/index.php?title=2019Q1_Reports:_ACL_2019 2019]). We consulted previous tutorial chairs with a questionnaire including questions about: the number of submissions, encouraging submissions on specific topics or from specific lecturers, the review procedure, the evaluation criteria, the post-tutorial availability of the slides/codes, and lessons learned from tutorial coordination. We also discussed the publication of slides and video recordings from future tutorials with the persons in charge of the ACL Anthology. As a result of these steps, we created two new sections for the [https://www.aclweb.org/adminwiki/index.php?title=Conference_Handbook ACL Conference Handbook] (future chairs might consider updating these documents yearly):

* the list of [https://www.aclweb.org/adminwiki/index.php?title=Past_tutorials past tutorials] at ACL, COLING, EACL, EMNLP, and NAACL in 2016-2019
* a [https://www.aclweb.org/adminwiki/index.php?title=Tutorial_chair_handbook tutorial chair handbook]

The final [https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020 call] differs from previous calls in several aspects: (i) the expectations about tutorial proposals were made clearer, (ii) following the central ACL decision, the teachers' payment policy was replaced by a fee-waiving policy, (iii) the required submission details include two new items: diversity considerations and agreement for open access publication of slides, codes, data and video recordings, (iv) the evaluation criteria (see below) are announced.

We recruited a review committee of 19 members, including the 8 tutorial chairs and 11 external members selected for their large understanding of the NLP domain and a good experience in reviewing and/or tutorial teaching:

'''Review Committee'''

* Timothy Baldwin (University of Melbourne, Australia) - AACL-IJCNLP 2020 tutorial chair
* Daniel Beck (University of Melbourne, Australia) - COLING 2020 tutorial chair
* Emily M. Bender (University of Washington, WA, USA)
* Erik Cambria (Nanyang Technological University, Singapore)
* Gaël Dias (University of Caen Normandie, France)
* Stefan Evert (Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany)
* Yang Liu (Tsinghua University, Beijing, China)
* Agata Savary (University of Tours, France) - ACL 2020 tutorial chair
* João Sedoc (Johns Hopkins University, Baltimore, MD, USA)
* Lucia Specia (Sheffield University, UK) - COLING 2020 tutorial chair
* Xu Sun (Peking University, China)
* Yulia Tsvetkov (Carnegie Mellon University, Pittsburgh, PA, USA)
* Benjamin Van Durme (Johns Hopkins University, Baltimore, MD, USA) - EMNLP 2020 tutorial chair
* Aline Villavicencio (University of Sheffield, UK and Federal University of Rio Grande do Sul, Brazil) - EMNLP 2020 tutorial chair
* Taro Watanabe (Google, Inc., Tokyo, Japan)
* Aaron Steven White (University of Rochester, NY, USA)
* Fei Xia (University of Washington, WA, USA) - AACL-IJCNLP 2020 tutorial chair
* Yue Zhang (Westlake University, Hangzhou, China) - ACL 2020 tutorial chair
* Meishan Zhang (Tianjin University, China)

In total, we received 43 submissions for the 4 conferences. Each reviewer was assigned 6-7 proposals and each proposal received 3 reviews. The selection criteria included: clarity and preparedness, novelty or timely character of the topic, lecturers' experience, likely audience interest, open access of the teaching material, diversity aspects (multilingualism, gender, age and country of the lecturers), and compatibility with the preferred venues. We accepted 31 proposals.

The decision making was handled via an online meeting of the 8 tutorial chairs. In particular, the selection of tutorials for each conference was done via the expression of interest of the tutorial chairs on a round-robin basis. Some slight adjustments were also performed after the meeting to better fit the authors' preferences. In total, 8, 8, 8 and 7 proposals were selected for ACL, AACL-IJCNLP, COLING and EMNLP, respectively. Upon the announcement the results, 2 of the proposals accepted for AACL-IJCNLP were withdrawn.

The submission, review, selection and collection of final descriptions for all tutorials was handled via a dedicated SoftConf space, shared by the 4 coordinating conferences. After the selection of proposals, a separate track was created on SoftConf for each conference. The final submission page (one per conference) was set up so as to collect all the necessary data including notably: the tutorial slides, URLs for course material (if any), printable material (if any) and agreement for open access publication.

===Tutorials Selected for ACL 2020===

The final selection for ACL 2020 consists of the following 8 tutorials of 3 hours each (each of them had ACL as the preferred or the second preferred venue):

'''T1: Interpretability and Analysis in Neural NLP''' (cutting-edge) 
Yonatan Belinkov, Sebastian Gehrmann and Ellie Pavlick 
While deep learning has transformed the NLP field and impacted the larger computational linguistics community, the rise of neural networks is stained by their opaque nature: It is challenging to interpret the inner workings of neural network models, and explicate their behavior. Therefore, in the last few years, an increasingly large body of work has been devoted to the analysis and interpretation of neural network models in NLP. 
This body of work is so far lacking a common framework and methodology. Moreover, approaching the analysis of modern neural networks can be difficult for newcomers to the field. This tutorial aims to fill this gap and introduce the nascent field of interpretability and analysis of neural networks in NLP. 
The tutorial covers the main lines of analysis work, such as probing classifier, behavior studies and test suites, psycholinguistic methods, visualizations, adversarial examples, and other methods. We highlight not only the most commonly applied analysis methods, but also the specific limitations and shortcomings of current approaches, in order to inform participants where to focus future efforts.

'''T2: Multi-modal Information Extraction from Text, Semi-structured, and Tabular Data on the Web''' (cutting-edge) 
Xin Luna Dong, Hannaneh Hajishirzi, Colin Lockard and Prashant Shiralkar 
The World Wide Web contains vast quantities of textual information in several forms: unstructured text, template-based semi-structured webpages (which present data in key-value pairs and lists), and tables. Methods for extracting information from these sources and converting it to a structured form have been a target of research from the natural language processing (NLP), data mining, and database communities. While these researchers have largely separated extraction from web data into different problems based on the modality of the data, they have faced similar problems such as learning with limited labeled data, defining (or avoiding defining) ontologies, making use of prior knowledge, and scaling solutions to deal with the size of the Web. 
In this tutorial we take a holistic view toward information extraction, exploring the commonalities in the challenges and solutions developed to address these different forms of text. We will explore the approaches targeted at unstructured text that largely rely on learning syntactic or semantic textual patterns, approaches targeted at semi-structured documents that learn to identify structural patterns in the template, and approaches targeting web tables which rely heavily on entity linking and type information. 
While these different data modalities have largely been considered separately in the past, recent research has started taking a more inclusive approach toward textual extraction, in which the multiple signals offered by textual, layout, and visual clues are combined into a single extraction model made possible by new deep learning approaches. At the same time, trends within purely textual extraction have shifted toward full-document understanding rather than considering sentences as independent units. With this in mind, it is worth considering the information extraction problem as a whole to motivate solutions that harness textual semantics along with visual and semi-structured layout information. We will discuss these approaches and suggest avenues for future work.

'''T3: Reviewing Natural Language Processing Research''' (introductory) 
Kevin Cohen, Karën Fort, Margot Mieskes and Aurélie Névéol 
As the demand for reviewing grows, so must the pool of reviewers. As the [http://www.livecongress.it/aol/indexSA.php?id=E2EAED7D&ticket= survey] presented by Graham Neubig at the 2019 ACL showed, a considerable number of reviewers are junior researchers, who might lack the experience and expertise necessary for high-quality reviews. Some of them might not have the environment or lack opportunities that allow them to learn the skills necessary. A tutorial on reviewing for the NLP community might increase reviewers’ confidence, as well as the quality of the reviews. This introductory tutorial will cover the goals, processes, and evaluation of reviewing research papers in natural language processing.

'''T4: Stylized Text Generation: Approaches and Applications''' (cutting-edge) 
Lili Mou and Olga Vechtomova 
Text generation has played an important role in various applications of natural language processing (NLP), and kn recent studies, researchers are paying increasing attention to modeling and manipulating the style of the generation text, which we call stylized text generation. In this tutorial, we will provide a comprehensive literature review in this direction. We start from the definition of style and different settings of stylized text generation, illustrated with various applications. Then, we present different settings of stylized generation, such as parallel supervised, style label-supervised, and unsupervised. In each setting, we delve deep into machine learning methods, including embedding learning techniques to represent style}, adversarial learning and reinforcement learning with cycle consistency to match content but to distinguish different styles. We also introduce current approaches of evaluating stylized text generation systems. We conclude our tutorial by presenting the challenges of stylized text generation and discussing future directions, such as small-data training, non-categorical style modeling, and a generalized scope of style transfer (e.g., controlling the syntax as a style).

'''T5: Achieving Common Ground in Multi-modal Dialogue''' (cutting-edge) 
Malihe Alikhani and Matthew Stone 
All communication aims at achieving common ground (grounding): interlocutors can work together effectively only with mutual beliefs about what the state of the world is, about what their goals are, and about how they plan to make their goals a reality. Computational dialogue research offers some classic results on grouding, which unfortunately offer scant guidance to the design of grounding modules and behaviors in cutting-edge systems. In this tutorial, we focus on three main topic areas: 1) grounding in human-human communication; 2) grounding in dialogue systems; and 3) grounding in multi-modal interactive systems, including image-oriented conversations and human-robot interactions. We highlight a number of achievements of recent computational research in coordinating complex content, show how these results lead to rich and challenging opportunities for doing grounding in more flexible and powerful ways, and canvass relevant insights from the literature on human--human conversation. We expect that the tutorial will be of interest to researchers in dialogue systems, computational semantics and cognitive modeling, and hope that it will catalyze research and system building that more directly explores the creative, strategic ways conversational agents might be able to seek and offer evidence about their understanding of their interlocutors.

'''T6: Commonsense Reasoning for Natural Language Processing''' (introductory) 
Maarten Sap, Vered Shwartz, Antoine Bosselut, Dan Roth and Yejin Choi 
In our tutorial, we (1) outline the various types of commonsense (e.g., physical, social), and (2) discuss techniques to gather and represent commonsense knowledge, while highlighting the challenges specific to this type of knowledge (e.g., reporting bias). We will then (3) discuss the types of commonsense knowledge captured by modern NLP systems (e.g., large pretrained language models), and (4) present ways to measure systems' commonsense reasoning abilities. We finish with (5) a discussion of various ways in which commonsense reasoning can be used to improve performance on NLP tasks, exemplified by an (6) interactive session on integrating commonsense into a downstream task.

'''T7: Integrating Ethics into the NLP Curriculum''' (introductory) 
Emily M. Bender, Dirk Hovy and Alexandra Schofield 
Our goal in this tutorial is to empower NLP researchers and practitioners with tools and resources to teach others about how to ethically apply NLP techniques. Our tutorial will present both high-level strategies for developing an ethics-oriented curriculum, based on experience and best practices, as well as specific sample exercises that can be brought to a classroom. We plan to make this a highly interactive work session culminating in a shared online resource page that pools lesson plans, assignments, exercise ideas, reading suggestions, and ideas from the attendees. We consider three primary topics with our session that frequently underlie ethical issues in NLP research: Dual use, bias and privacy. 
In this setting, a key lesson is that there is no single approach to ethical NLP: each project requires thoughtful consideration about what steps can be taken to best support people affected by that project. However, we can learn (and teach) what kinds of issues to be aware of and what kinds of strategies are available for mitigating harm. To teach this process, we apply and promote interactive exercises that provide an opportunity to ideate, discuss, and reflect. We plan to facilitate this in a way that encourages positive discussion, emphasizing the creation of ideas for the future instead of negative opinions of previous work.

'''T8: Recent Advances in Open-Domain Question Answering''' (cutting-edge) 
Danqi Chen and Scott Wen-tau Yih 
Open-domain (textual) question answering (QA), the task of finding answers to open-domain questions by searching a large collection of documents, has been a long-standing problem in NLP, information retrieval (IR) and related fields (Voorhees et al., 1999; Moldovan et al., 2000; Brill et al.,2002; Ferrucci et al., 2010). Traditional QA systems were usually constructed as a pipeline, consisting of many different components such as question processing, document/passage retrieval and answer processing. With the rapid development of neural reading comprehension (Chen, 2018), modern open-domain QA systems have been restructured by combining traditional IR techniques and neural reading comprehension models (Chen et al., 2017; Yang et al., 2019) or even implemented in a fully end-to-end fashion (Lee et al., 2019; Seo et al., 2019). While the system architecture has been drastically simplified, two technical challenges remain critical:(1) “Retriever”: finding documents that (might)contain an answer from a large collection of documents; (2) “Reader”: finding the answer in a given paragraph or a document. 
In this tutorial, we aim to provide a comprehensive and coherent overview of recent advances in this line of research. We will start by first giving a brief historical background of open-domain question answering, discussing the basic setup and core technical challenges of the research problem.The focus will then shift to modern techniques and resources proposed for open-domain QA, including the basics of latest neural reading comprehension systems, new datasets and models. The scope will also be broadened to cover the information retrieval component on how to effectively identify passages relevant to the questions. Moreover, in-depth discussions will be given on the use of traditional / neural IR modules, as well as the trade-offs between modular design and end-to-end training. If time permits, we also plan to discuss some hybrid approaches for answering questions using both text and large knowledge bases (e.g. (Sun et al., 2018)) and give a critical review on how structured data complements the information from unstructured text. 
At the end of our tutorial, we will discuss some important questions, including (1) How much progress have we made compared to the QA systems developed in the last decade?(2) What are the main challenges and limitations of cur-rent approaches? (3) How to trade off the efficiency (computational time and memory requirements) and accuracy in the deep learning era? We hope that our tutorial will not only serve as a useful resource for the audience to efficiently acquire the up-to-date knowledge, but also provide new perspectives to stimulate the advances of open-domain QA research in the next phase.

===Virtual Conference Organization===

In April the whole conference was decided to be held in virtual. We carried on the preparations jointly with the tutorial instructors, the virtual infrastructure chairs, the website chairs and the SlidesLive team to set up the virtual conference infrastructure, and to publicize it for each tutorial.

We first made a decision on the time slots for tutorial presentations. Five major time zones were considered, including US west coast, US east coast, central Europe, China/Asia and Australia (India was also considered). Our aim was to offer at least two convenient time slots (i.e. within regular working hours) for each of these zones.

The final time slots include (Jul 5 Pacific Time):

* slot 1 – 7:00 PM (-1D) to 10:30 PM (-1D)
* slot 2 – 12:00 AM to 3:30 AM
* slot 3 – 6:00 AM to 9:30 AM
* slot 4 – 10:30 AM to 2:00 PM
* slot 5 – 3:00 PM to 6:30 PM

We asked the tutorial teachers to indicate their preferences among the time slots, and – if possible – to agree to give their tutorial twice, so as to increase accessibility. Our final decisions on slot assignments traded off the teacher preferences and slot availability. In the end, the assignments were the following (see also the [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ blog post] and [https://acl2020.org/program/tutorials/ online schedule]):

* T1 – twice on slots 3 and 4
* T2 – once on slot 5
* T3 – once on slot 4
* T4 – once on slot 4
* T5 – twice on slots 3 and 5
* T6 – once on slot 5
* T7 – twice on slots 3 and 4
* T8 – once on slot 5

Unfortunately, the tutorial teachers proved hardly available for slots 1 and 2.

In addition to time slot arrangements, we gave the tutorial teacher two options on presentation, namely a pre-recorded and a fully interactive form of tutorials. The former allows the teachers to pre-record the main content of their tutorials, and use the time slots mainly for question answering and detailed discussions. The latter is more interactive, the teachers use the online sessions for both giving the lecture and other activities such as active discussions. All these arrangements were made for taking advantage of the virtual conference. In the end the choices were the following:

* T1 – pre-recorded
* T2 – interactive (the initial decision was pre-recorded but a change was made in June)
* T3 – pre-recorded
* T4 – pre-recorded
* T5 – pre-recorded
* T6 – interactive
* T7 – interactive
* T8 – interactive

After making decisions on time slots and teaching forms, we started to work on the technical details of the virtual conference infrastructure jointly with the infrastructure chairs, the website chairs, the SlidesLive support team and individual tutorial teachers. We collected needs from each tutorial from time to time, inquired information from the SlidesLive team, and tried to accommodate specific requirement for each tutorial. Since this is the first time ACL was virtual, it typically took several rounds of information exchange for making a certain technical feature work. Major things discussed included:

* Sorting out how to pre-record a tutorial using the system
* Recording of interactive sessions
* Additional RocketChat channel
* Virtual tutorial webpage setup
* Coordinators for tutorial sessions and SlidesLive team participation
* Rehearsal of interactive sessions
* Captioning of pre-recorded and interactive tutorials
* Privacy issues
* Special needs from particular tutorials
** Integrating [https://www.dory.app/ Dory] for question answering (in T1)
** Fully interactive sessions, with occasional split into break-out rooms (in T7)

To ensure smooth running of the sessions, we sent two rounds of reminders to each tutorial to ensure that each teacher is registered to the conference. We collected rough timelines of each tutorial so that the SlidesLive team can work closely during the live sessions. To further help the teachers prepare their live session content we additionally collected the lists of registered attendees twice before the conference, and shared them with the teachers.

In the end, the numbers of registrations for each tutorial were:
* T1 -- 1762
* T2 -- 793
* T3 -- 503
* T4 -- 571
* T5 -- 641
* T6 -- 1288
* T7 -- 378
* T8 – 1087

===Archiving and Publicity===

There are three main channels for publicity, namely the static [http://www.acl2020.org/program/tutorials/ website], the website of the [https://virtual.acl2020.org/tutorials.html virtual conference] and the social media. For the former, we collected photos from each instructor, which are shown with the title and abstract of their tutorials. Following NAACL 2019, we also displayed the time slots as a right-hand-side bar on the tutorial website. Due to special arrangements of the virtual conference, we marked the time zones explicitly. In addition, we asked the tutorial teachers to provide URL links to teaching materials (e.g., at GitHub) during their final submission, and included these links to their tutorial information. For social media, we drafted two blog posts (i.e., [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ this post] and [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]) for advertising the tutorials.

The archiving consists of three main items, namely the tutorial proceedings, the slides for each tutorial and the videos. For the tutorial proceedings, we collected final abstracts from the authors using the START system. We asked the instructors to sign the CC BY v4 agreement when submitting their final materials on SoftConf. For the slides, we asked the tutorial teachers to provide a first version of the slides by June 17 regardless whether they are pre-recorded or interactive. Some tutorials made further updates to their slides before ACL. For the video recordings, we worked with the ACL Anthology team discussing the technical details since this is the first time the tutorial videos are shared in the website. [Note: this is currently undergoing at the time of writing of this summary]

===Lessons Learned===

While organizing a virtual conference it is important to stress that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]).

It is also important to announce the individual modalities with the attendees early enough. For instance, watching pre-recordings in advance was necessary for some tutorials (since their live sessions only included question answering) and such constrains should have been known more in advance.

If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to Zoom rooms only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.

One thing to remember which we missed is to publish the reading list for each tutorial (which the teachers submit with the proposal) on the tutorial website.

2020Q3 Reports: Tutorial Chairs

2020-07-23T10:04:11Z

AgataSavary: /* Virtual Conference Organization */

Tutorial Chairs:

<blockquote>Agata Savary, University of Tours, France

Yue Zhang, Westlake University, Hangzhou, China
</blockquote>

=== Preparatory Work, Call, Reviewing and Decision-Making ===
The call, submission, reviewing and selection of tutorials was coordinated jointly for 4 conferences: ACL, AACL-IJCNLP, COLING and EMNLP.

Before drafting the call, we collected lists of tutorials offered within the past 4 years. We analysed previous calls for tutorials and reports from tutorial chairs (from [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs 2016], [https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs 2017], [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs 2018] and [http://aclweb.org/adminwiki/index.php?title=2019Q1_Reports:_ACL_2019 2019]). We consulted previous tutorial chairs with a questionnaire including questions about: the number of submissions, encouraging submissions on specific topics or from specific lecturers, the review procedure, the evaluation criteria, the post-tutorial availability of the slides/codes, and lessons learned from tutorial coordination. We also discussed the publication of slides and video recordings from future tutorials with the persons in charge of the ACL Anthology. As a result of these steps, we created two new sections for the [https://www.aclweb.org/adminwiki/index.php?title=Conference_Handbook ACL Conference Handbook] (future chairs might consider updating these documents yearly):

* the list of [https://www.aclweb.org/adminwiki/index.php?title=Past_tutorials past tutorials] at ACL, COLING, EACL, EMNLP, and NAACL in 2016-2019
* a [https://www.aclweb.org/adminwiki/index.php?title=Tutorial_chair_handbook tutorial chair handbook]

The final [https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020 call] differs from previous calls in several aspects: (i) the expectations about tutorial proposals were made clearer, (ii) following the central ACL decision, the teachers' payment policy was replaced by a fee-waiving policy, (iii) the required submission details include two new items: diversity considerations and agreement for open access publication of slides, codes, data and video recordings, (iv) the evaluation criteria (see below) are announced.

We recruited a review committee of 19 members, including the 8 tutorial chairs and 11 external members selected for their large understanding of the NLP domain and a good experience in reviewing and/or tutorial teaching:

'''Review Committee'''

* Timothy Baldwin (University of Melbourne, Australia) - AACL-IJCNLP 2020 tutorial chair
* Daniel Beck (University of Melbourne, Australia) - COLING 2020 tutorial chair
* Emily M. Bender (University of Washington, WA, USA)
* Erik Cambria (Nanyang Technological University, Singapore)
* Gaël Dias (University of Caen Normandie, France)
* Stefan Evert (Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany)
* Yang Liu (Tsinghua University, Beijing, China)
* Agata Savary (University of Tours, France) - ACL 2020 tutorial chair
* João Sedoc (Johns Hopkins University, Baltimore, MD, USA)
* Lucia Specia (Sheffield University, UK) - COLING 2020 tutorial chair
* Xu Sun (Peking University, China)
* Yulia Tsvetkov (Carnegie Mellon University, Pittsburgh, PA, USA)
* Benjamin Van Durme (Johns Hopkins University, Baltimore, MD, USA) - EMNLP 2020 tutorial chair
* Aline Villavicencio (University of Sheffield, UK and Federal University of Rio Grande do Sul, Brazil) - EMNLP 2020 tutorial chair
* Taro Watanabe (Google, Inc., Tokyo, Japan)
* Aaron Steven White (University of Rochester, NY, USA)
* Fei Xia (University of Washington, WA, USA) - AACL-IJCNLP 2020 tutorial chair
* Yue Zhang (Westlake University, Hangzhou, China) - ACL 2020 tutorial chair
* Meishan Zhang (Tianjin University, China)

In total, we received 43 submissions for the 4 conferences. Each reviewer was assigned 6-7 proposals and each proposal received 3 reviews. The selection criteria included: clarity and preparedness, novelty or timely character of the topic, lecturers' experience, likely audience interest, open access of the teaching material, diversity aspects (multilingualism, gender, age and country of the lecturers), and compatibility with the preferred venues. We accepted 31 proposals.

The decision making was handled via an online meeting of the 8 tutorial chairs. In particular, the selection of tutorials for each conference was done via the expression of interest of the tutorial chairs on a round-robin basis. Some slight adjustments were also performed after the meeting to better fit the authors' preferences. In total, 8, 8, 8 and 7 proposals were selected for ACL, AACL-IJCNLP, COLING and EMNLP, respectively. Upon the announcement the results, 2 of the proposals accepted for AACL-IJCNLP were withdrawn.

The submission, review, selection and collection of final descriptions for all tutorials was handled via a dedicated SoftConf space, shared by the 4 coordinating conferences. After the selection of proposals, a separate track was created on SoftConf for each conference. The final submission page (one per conference) was set up so as to collect all the necessary data including notably: the tutorial slides, URLs for course material (if any), printable material (if any) and agreement for open access publication.

===Tutorials Selected for ACL 2020===

The final selection for ACL 2020 consists of the following 8 tutorials of 3 hours each (each of them had ACL as the preferred or the second preferred venue):

'''T1: Interpretability and Analysis in Neural NLP''' (cutting-edge) 
Yonatan Belinkov, Sebastian Gehrmann and Ellie Pavlick 
While deep learning has transformed the NLP field and impacted the larger computational linguistics community, the rise of neural networks is stained by their opaque nature: It is challenging to interpret the inner workings of neural network models, and explicate their behavior. Therefore, in the last few years, an increasingly large body of work has been devoted to the analysis and interpretation of neural network models in NLP. 
This body of work is so far lacking a common framework and methodology. Moreover, approaching the analysis of modern neural networks can be difficult for newcomers to the field. This tutorial aims to fill this gap and introduce the nascent field of interpretability and analysis of neural networks in NLP. 
The tutorial covers the main lines of analysis work, such as probing classifier, behavior studies and test suites, psycholinguistic methods, visualizations, adversarial examples, and other methods. We highlight not only the most commonly applied analysis methods, but also the specific limitations and shortcomings of current approaches, in order to inform participants where to focus future efforts.

'''T2: Multi-modal Information Extraction from Text, Semi-structured, and Tabular Data on the Web''' (cutting-edge) 
Xin Luna Dong, Hannaneh Hajishirzi, Colin Lockard and Prashant Shiralkar 
The World Wide Web contains vast quantities of textual information in several forms: unstructured text, template-based semi-structured webpages (which present data in key-value pairs and lists), and tables. Methods for extracting information from these sources and converting it to a structured form have been a target of research from the natural language processing (NLP), data mining, and database communities. While these researchers have largely separated extraction from web data into different problems based on the modality of the data, they have faced similar problems such as learning with limited labeled data, defining (or avoiding defining) ontologies, making use of prior knowledge, and scaling solutions to deal with the size of the Web. 
In this tutorial we take a holistic view toward information extraction, exploring the commonalities in the challenges and solutions developed to address these different forms of text. We will explore the approaches targeted at unstructured text that largely rely on learning syntactic or semantic textual patterns, approaches targeted at semi-structured documents that learn to identify structural patterns in the template, and approaches targeting web tables which rely heavily on entity linking and type information. 
While these different data modalities have largely been considered separately in the past, recent research has started taking a more inclusive approach toward textual extraction, in which the multiple signals offered by textual, layout, and visual clues are combined into a single extraction model made possible by new deep learning approaches. At the same time, trends within purely textual extraction have shifted toward full-document understanding rather than considering sentences as independent units. With this in mind, it is worth considering the information extraction problem as a whole to motivate solutions that harness textual semantics along with visual and semi-structured layout information. We will discuss these approaches and suggest avenues for future work.

'''T3: Reviewing Natural Language Processing Research''' (introductory) 
Kevin Cohen, Karën Fort, Margot Mieskes and Aurélie Névéol 
As the demand for reviewing grows, so must the pool of reviewers. As the [http://www.livecongress.it/aol/indexSA.php?id=E2EAED7D&ticket= survey] presented by Graham Neubig at the 2019 ACL showed, a considerable number of reviewers are junior researchers, who might lack the experience and expertise necessary for high-quality reviews. Some of them might not have the environment or lack opportunities that allow them to learn the skills necessary. A tutorial on reviewing for the NLP community might increase reviewers’ confidence, as well as the quality of the reviews. This introductory tutorial will cover the goals, processes, and evaluation of reviewing research papers in natural language processing.

'''T4: Stylized Text Generation: Approaches and Applications''' (cutting-edge) 
Lili Mou and Olga Vechtomova 
Text generation has played an important role in various applications of natural language processing (NLP), and kn recent studies, researchers are paying increasing attention to modeling and manipulating the style of the generation text, which we call stylized text generation. In this tutorial, we will provide a comprehensive literature review in this direction. We start from the definition of style and different settings of stylized text generation, illustrated with various applications. Then, we present different settings of stylized generation, such as parallel supervised, style label-supervised, and unsupervised. In each setting, we delve deep into machine learning methods, including embedding learning techniques to represent style}, adversarial learning and reinforcement learning with cycle consistency to match content but to distinguish different styles. We also introduce current approaches of evaluating stylized text generation systems. We conclude our tutorial by presenting the challenges of stylized text generation and discussing future directions, such as small-data training, non-categorical style modeling, and a generalized scope of style transfer (e.g., controlling the syntax as a style).

'''T5: Achieving Common Ground in Multi-modal Dialogue''' (cutting-edge) 
Malihe Alikhani and Matthew Stone 
All communication aims at achieving common ground (grounding): interlocutors can work together effectively only with mutual beliefs about what the state of the world is, about what their goals are, and about how they plan to make their goals a reality. Computational dialogue research offers some classic results on grouding, which unfortunately offer scant guidance to the design of grounding modules and behaviors in cutting-edge systems. In this tutorial, we focus on three main topic areas: 1) grounding in human-human communication; 2) grounding in dialogue systems; and 3) grounding in multi-modal interactive systems, including image-oriented conversations and human-robot interactions. We highlight a number of achievements of recent computational research in coordinating complex content, show how these results lead to rich and challenging opportunities for doing grounding in more flexible and powerful ways, and canvass relevant insights from the literature on human--human conversation. We expect that the tutorial will be of interest to researchers in dialogue systems, computational semantics and cognitive modeling, and hope that it will catalyze research and system building that more directly explores the creative, strategic ways conversational agents might be able to seek and offer evidence about their understanding of their interlocutors.

'''T6: Commonsense Reasoning for Natural Language Processing''' (introductory) 
Maarten Sap, Vered Shwartz, Antoine Bosselut, Dan Roth and Yejin Choi 
In our tutorial, we (1) outline the various types of commonsense (e.g., physical, social), and (2) discuss techniques to gather and represent commonsense knowledge, while highlighting the challenges specific to this type of knowledge (e.g., reporting bias). We will then (3) discuss the types of commonsense knowledge captured by modern NLP systems (e.g., large pretrained language models), and (4) present ways to measure systems' commonsense reasoning abilities. We finish with (5) a discussion of various ways in which commonsense reasoning can be used to improve performance on NLP tasks, exemplified by an (6) interactive session on integrating commonsense into a downstream task.

'''T7: Integrating Ethics into the NLP Curriculum''' (introductory) 
Emily M. Bender, Dirk Hovy and Alexandra Schofield 
Our goal in this tutorial is to empower NLP researchers and practitioners with tools and resources to teach others about how to ethically apply NLP techniques. Our tutorial will present both high-level strategies for developing an ethics-oriented curriculum, based on experience and best practices, as well as specific sample exercises that can be brought to a classroom. We plan to make this a highly interactive work session culminating in a shared online resource page that pools lesson plans, assignments, exercise ideas, reading suggestions, and ideas from the attendees. We consider three primary topics with our session that frequently underlie ethical issues in NLP research: Dual use, bias and privacy. 
In this setting, a key lesson is that there is no single approach to ethical NLP: each project requires thoughtful consideration about what steps can be taken to best support people affected by that project. However, we can learn (and teach) what kinds of issues to be aware of and what kinds of strategies are available for mitigating harm. To teach this process, we apply and promote interactive exercises that provide an opportunity to ideate, discuss, and reflect. We plan to facilitate this in a way that encourages positive discussion, emphasizing the creation of ideas for the future instead of negative opinions of previous work.

'''T8: Recent Advances in Open-Domain Question Answering''' (cutting-edge) 
Danqi Chen and Scott Wen-tau Yih 
Open-domain (textual) question answering (QA), the task of finding answers to open-domain questions by searching a large collection of documents, has been a long-standing problem in NLP, information retrieval (IR) and related fields (Voorhees et al., 1999; Moldovan et al., 2000; Brill et al.,2002; Ferrucci et al., 2010). Traditional QA systems were usually constructed as a pipeline, consisting of many different components such as question processing, document/passage retrieval and answer processing. With the rapid development of neural reading comprehension (Chen, 2018), modern open-domain QA systems have been restructured by combining traditional IR techniques and neural reading comprehension models (Chen et al., 2017; Yang et al., 2019) or even implemented in a fully end-to-end fashion (Lee et al., 2019; Seo et al., 2019). While the system architecture has been drastically simplified, two technical challenges remain critical:(1) “Retriever”: finding documents that (might)contain an answer from a large collection of documents; (2) “Reader”: finding the answer in a given paragraph or a document. 
In this tutorial, we aim to provide a comprehensive and coherent overview of recent advances in this line of research. We will start by first giving a brief historical background of open-domain question answering, discussing the basic setup and core technical challenges of the research problem.The focus will then shift to modern techniques and resources proposed for open-domain QA, including the basics of latest neural reading comprehension systems, new datasets and models. The scope will also be broadened to cover the information retrieval component on how to effectively identify passages relevant to the questions. Moreover, in-depth discussions will be given on the use of traditional / neural IR modules, as well as the trade-offs between modular design and end-to-end training. If time permits, we also plan to discuss some hybrid approaches for answering questions using both text and large knowledge bases (e.g. (Sun et al., 2018)) and give a critical review on how structured data complements the information from unstructured text. 
At the end of our tutorial, we will discuss some important questions, including (1) How much progress have we made compared to the QA systems developed in the last decade?(2) What are the main challenges and limitations of cur-rent approaches? (3) How to trade off the efficiency (computational time and memory requirements) and accuracy in the deep learning era? We hope that our tutorial will not only serve as a useful resource for the audience to efficiently acquire the up-to-date knowledge, but also provide new perspectives to stimulate the advances of open-domain QA research in the next phase.

===Virtual Conference Organization===

In April the whole conference was decided to be held in virtual. We carried on the preparations jointly with the tutorial instructors, the virtual infrastructure chairs, the website chairs and the SlidesLive team to set up the virtual conference infrastructure, and to publicize it for each tutorial.

We first made a decision on the time slots for tutorial presentations. Five major time zones were considered, including US west coast, US east coast, central Europe, China/Asia and Australia (India was also considered). Our aim was to offer at least two convenient time slots (i.e. within regular working hours) for each of these zones.

The final time slots include (Jul 5 Pacific Time):

* slot 1 – 7:00 PM (-1D) to 10:30 PM (-1D)
* slot 2 – 12:00 AM to 3:30 AM
* slot 3 – 6:00 AM to 9:30 AM
* slot 4 – 10:30 AM to 2:00 PM
* slot 5 – 3:00 PM to 6:30 PM

We asked the tutorial teachers to indicate their preferences among the time slots, and – if possible – to agree to give their tutorial twice, so as to increase accessibility. Our final decisions on slot assignments traded off the teacher preferences and slot availability. In the end, the assignments were the following (see also the [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ blog post] and [https://acl2020.org/program/tutorials/ online schedule]):

* T1 – twice on slots 3 and 4
* T2 – once on slot 5
* T3 – once on slot 4
* T4 – once on slot 4
* T5 – twice on slots 3 and 5
* T6 – once on slot 5
* T7 – twice on slots 3 and 4
* T8 – once on slot 5

Unfortunately, the tutorial teachers proved hardly available for slots 1 and 2.

In addition to time slot arrangements, we gave the tutorial teacher two options on presentation, namely a pre-recorded and a fully interactive form of tutorials. The former allows the teachers to pre-record the main content of their tutorials, and use the time slots mainly for question answering and detailed discussions. The latter is more interactive, where the teachers use the online sessions for both giving the lecture and other activities such as active discussions. All these arrangements are made for taking advantage of the virtual conference. In the end the choices were the following:

* T1 – pre-recorded
* T2 – interactive (the initial decision was pre-recorded but a change was made in June)
* T3 – pre-recorded
* T4 – pre-recorded
* T5 – pre-recorded
* T6 – interactive
* T7 – interactive
* T8 – interactive

After making decisions on time slots and teaching forms, we started to work on the technical details of the virtual conference infrastructure jointly with the infrastructure chairs, the website chairs, the SlidesLive support team and individual tutorial teachers. We collected needs from each tutorial from time to time, inquired information from the SlidesLive team, and tried to accommodate specific requirement for each tutorial. Since this is the first time ACL was virtual, it typically took several rounds of information exchange for making a certain technical feature work. Major things discussed included:

* Sorting out how to pre-record a tutorial using the system
* Recording of interactive sessions
* Additional RocketChat channel
* Virtual tutorial webpage setup
* Coordinators for tutorial sessions and SlidesLive team participation
* Rehearsal of interactive sessions
* Captioning of pre-recorded and interactive tutorials
* Privacy issues
* Special needs from particular tutorials
** Integrating [https://www.dory.app/ Dory] for question answering (in T1)
** Fully interactive sessions, with occasional split into break-out rooms (in T7)

To ensure smooth running of the sessions, we sent two rounds of reminders to each tutorial to ensure that each teacher is registered to the conference. We collected rough timelines of each tutorial so that the SlidesLive team can work closely during the live sessions. To further help the teachers prepare their live session content we additionally collected the lists of registered attendees twice before the conference, and shared them with the teachers.

In the end, the numbers of registrations for each tutorial were:
* T1 -- 1762
* T2 -- 793
* T3 -- 503
* T4 -- 571
* T5 -- 641
* T6 -- 1288
* T7 -- 378
* T8 – 1087

===Archiving and Publicity===

There are three main channels for publicity, namely the static [http://www.acl2020.org/program/tutorials/ website], the website of the [https://virtual.acl2020.org/tutorials.html virtual conference] and the social media. For the former, we collected photos from each instructor, which are shown with the title and abstract of their tutorials. Following NAACL 2019, we also displayed the time slots as a right-hand-side bar on the tutorial website. Due to special arrangements of the virtual conference, we marked the time zones explicitly. In addition, we asked the tutorial teachers to provide URL links to teaching materials (e.g., at GitHub) during their final submission, and included these links to their tutorial information. For social media, we drafted two blog posts (i.e., [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ this post] and [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]) for advertising the tutorials.

The archiving consists of three main items, namely the tutorial proceedings, the slides for each tutorial and the videos. For the tutorial proceedings, we collected final abstracts from the authors using the START system. We asked the instructors to sign the CC BY v4 agreement when submitting their final materials on SoftConf. For the slides, we asked the tutorial teachers to provide a first version of the slides by June 17 regardless whether they are pre-recorded or interactive. Some tutorials made further updates to their slides before ACL. For the video recordings, we worked with the ACL Anthology team discussing the technical details since this is the first time the tutorial videos are shared in the website. [Note: this is currently undergoing at the time of writing of this summary]

===Lessons Learned===

While organizing a virtual conference it is important to stress that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]).

It is also important to announce the individual modalities with the attendees early enough. For instance, watching pre-recordings in advance was necessary for some tutorials (since their live sessions only included question answering) and such constrains should have been known more in advance.

If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to Zoom rooms only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.

One thing to remember which we missed is to publish the reading list for each tutorial (which the teachers submit with the proposal) on the tutorial website.

2020Q3 Reports: Tutorial Chairs

2020-07-23T10:03:41Z

AgataSavary: /* Virtual Conference Organization */

Tutorial Chairs:

<blockquote>Agata Savary, University of Tours, France

Yue Zhang, Westlake University, Hangzhou, China
</blockquote>

=== Preparatory Work, Call, Reviewing and Decision-Making ===
The call, submission, reviewing and selection of tutorials was coordinated jointly for 4 conferences: ACL, AACL-IJCNLP, COLING and EMNLP.

Before drafting the call, we collected lists of tutorials offered within the past 4 years. We analysed previous calls for tutorials and reports from tutorial chairs (from [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs 2016], [https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs 2017], [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs 2018] and [http://aclweb.org/adminwiki/index.php?title=2019Q1_Reports:_ACL_2019 2019]). We consulted previous tutorial chairs with a questionnaire including questions about: the number of submissions, encouraging submissions on specific topics or from specific lecturers, the review procedure, the evaluation criteria, the post-tutorial availability of the slides/codes, and lessons learned from tutorial coordination. We also discussed the publication of slides and video recordings from future tutorials with the persons in charge of the ACL Anthology. As a result of these steps, we created two new sections for the [https://www.aclweb.org/adminwiki/index.php?title=Conference_Handbook ACL Conference Handbook] (future chairs might consider updating these documents yearly):

* the list of [https://www.aclweb.org/adminwiki/index.php?title=Past_tutorials past tutorials] at ACL, COLING, EACL, EMNLP, and NAACL in 2016-2019
* a [https://www.aclweb.org/adminwiki/index.php?title=Tutorial_chair_handbook tutorial chair handbook]

The final [https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020 call] differs from previous calls in several aspects: (i) the expectations about tutorial proposals were made clearer, (ii) following the central ACL decision, the teachers' payment policy was replaced by a fee-waiving policy, (iii) the required submission details include two new items: diversity considerations and agreement for open access publication of slides, codes, data and video recordings, (iv) the evaluation criteria (see below) are announced.

We recruited a review committee of 19 members, including the 8 tutorial chairs and 11 external members selected for their large understanding of the NLP domain and a good experience in reviewing and/or tutorial teaching:

'''Review Committee'''

* Timothy Baldwin (University of Melbourne, Australia) - AACL-IJCNLP 2020 tutorial chair
* Daniel Beck (University of Melbourne, Australia) - COLING 2020 tutorial chair
* Emily M. Bender (University of Washington, WA, USA)
* Erik Cambria (Nanyang Technological University, Singapore)
* Gaël Dias (University of Caen Normandie, France)
* Stefan Evert (Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany)
* Yang Liu (Tsinghua University, Beijing, China)
* Agata Savary (University of Tours, France) - ACL 2020 tutorial chair
* João Sedoc (Johns Hopkins University, Baltimore, MD, USA)
* Lucia Specia (Sheffield University, UK) - COLING 2020 tutorial chair
* Xu Sun (Peking University, China)
* Yulia Tsvetkov (Carnegie Mellon University, Pittsburgh, PA, USA)
* Benjamin Van Durme (Johns Hopkins University, Baltimore, MD, USA) - EMNLP 2020 tutorial chair
* Aline Villavicencio (University of Sheffield, UK and Federal University of Rio Grande do Sul, Brazil) - EMNLP 2020 tutorial chair
* Taro Watanabe (Google, Inc., Tokyo, Japan)
* Aaron Steven White (University of Rochester, NY, USA)
* Fei Xia (University of Washington, WA, USA) - AACL-IJCNLP 2020 tutorial chair
* Yue Zhang (Westlake University, Hangzhou, China) - ACL 2020 tutorial chair
* Meishan Zhang (Tianjin University, China)

In total, we received 43 submissions for the 4 conferences. Each reviewer was assigned 6-7 proposals and each proposal received 3 reviews. The selection criteria included: clarity and preparedness, novelty or timely character of the topic, lecturers' experience, likely audience interest, open access of the teaching material, diversity aspects (multilingualism, gender, age and country of the lecturers), and compatibility with the preferred venues. We accepted 31 proposals.

The decision making was handled via an online meeting of the 8 tutorial chairs. In particular, the selection of tutorials for each conference was done via the expression of interest of the tutorial chairs on a round-robin basis. Some slight adjustments were also performed after the meeting to better fit the authors' preferences. In total, 8, 8, 8 and 7 proposals were selected for ACL, AACL-IJCNLP, COLING and EMNLP, respectively. Upon the announcement the results, 2 of the proposals accepted for AACL-IJCNLP were withdrawn.

The submission, review, selection and collection of final descriptions for all tutorials was handled via a dedicated SoftConf space, shared by the 4 coordinating conferences. After the selection of proposals, a separate track was created on SoftConf for each conference. The final submission page (one per conference) was set up so as to collect all the necessary data including notably: the tutorial slides, URLs for course material (if any), printable material (if any) and agreement for open access publication.

===Tutorials Selected for ACL 2020===

The final selection for ACL 2020 consists of the following 8 tutorials of 3 hours each (each of them had ACL as the preferred or the second preferred venue):

'''T1: Interpretability and Analysis in Neural NLP''' (cutting-edge) 
Yonatan Belinkov, Sebastian Gehrmann and Ellie Pavlick 
While deep learning has transformed the NLP field and impacted the larger computational linguistics community, the rise of neural networks is stained by their opaque nature: It is challenging to interpret the inner workings of neural network models, and explicate their behavior. Therefore, in the last few years, an increasingly large body of work has been devoted to the analysis and interpretation of neural network models in NLP. 
This body of work is so far lacking a common framework and methodology. Moreover, approaching the analysis of modern neural networks can be difficult for newcomers to the field. This tutorial aims to fill this gap and introduce the nascent field of interpretability and analysis of neural networks in NLP. 
The tutorial covers the main lines of analysis work, such as probing classifier, behavior studies and test suites, psycholinguistic methods, visualizations, adversarial examples, and other methods. We highlight not only the most commonly applied analysis methods, but also the specific limitations and shortcomings of current approaches, in order to inform participants where to focus future efforts.

'''T2: Multi-modal Information Extraction from Text, Semi-structured, and Tabular Data on the Web''' (cutting-edge) 
Xin Luna Dong, Hannaneh Hajishirzi, Colin Lockard and Prashant Shiralkar 
The World Wide Web contains vast quantities of textual information in several forms: unstructured text, template-based semi-structured webpages (which present data in key-value pairs and lists), and tables. Methods for extracting information from these sources and converting it to a structured form have been a target of research from the natural language processing (NLP), data mining, and database communities. While these researchers have largely separated extraction from web data into different problems based on the modality of the data, they have faced similar problems such as learning with limited labeled data, defining (or avoiding defining) ontologies, making use of prior knowledge, and scaling solutions to deal with the size of the Web. 
In this tutorial we take a holistic view toward information extraction, exploring the commonalities in the challenges and solutions developed to address these different forms of text. We will explore the approaches targeted at unstructured text that largely rely on learning syntactic or semantic textual patterns, approaches targeted at semi-structured documents that learn to identify structural patterns in the template, and approaches targeting web tables which rely heavily on entity linking and type information. 
While these different data modalities have largely been considered separately in the past, recent research has started taking a more inclusive approach toward textual extraction, in which the multiple signals offered by textual, layout, and visual clues are combined into a single extraction model made possible by new deep learning approaches. At the same time, trends within purely textual extraction have shifted toward full-document understanding rather than considering sentences as independent units. With this in mind, it is worth considering the information extraction problem as a whole to motivate solutions that harness textual semantics along with visual and semi-structured layout information. We will discuss these approaches and suggest avenues for future work.

'''T3: Reviewing Natural Language Processing Research''' (introductory) 
Kevin Cohen, Karën Fort, Margot Mieskes and Aurélie Névéol 
As the demand for reviewing grows, so must the pool of reviewers. As the [http://www.livecongress.it/aol/indexSA.php?id=E2EAED7D&ticket= survey] presented by Graham Neubig at the 2019 ACL showed, a considerable number of reviewers are junior researchers, who might lack the experience and expertise necessary for high-quality reviews. Some of them might not have the environment or lack opportunities that allow them to learn the skills necessary. A tutorial on reviewing for the NLP community might increase reviewers’ confidence, as well as the quality of the reviews. This introductory tutorial will cover the goals, processes, and evaluation of reviewing research papers in natural language processing.

'''T4: Stylized Text Generation: Approaches and Applications''' (cutting-edge) 
Lili Mou and Olga Vechtomova 
Text generation has played an important role in various applications of natural language processing (NLP), and kn recent studies, researchers are paying increasing attention to modeling and manipulating the style of the generation text, which we call stylized text generation. In this tutorial, we will provide a comprehensive literature review in this direction. We start from the definition of style and different settings of stylized text generation, illustrated with various applications. Then, we present different settings of stylized generation, such as parallel supervised, style label-supervised, and unsupervised. In each setting, we delve deep into machine learning methods, including embedding learning techniques to represent style}, adversarial learning and reinforcement learning with cycle consistency to match content but to distinguish different styles. We also introduce current approaches of evaluating stylized text generation systems. We conclude our tutorial by presenting the challenges of stylized text generation and discussing future directions, such as small-data training, non-categorical style modeling, and a generalized scope of style transfer (e.g., controlling the syntax as a style).

'''T5: Achieving Common Ground in Multi-modal Dialogue''' (cutting-edge) 
Malihe Alikhani and Matthew Stone 
All communication aims at achieving common ground (grounding): interlocutors can work together effectively only with mutual beliefs about what the state of the world is, about what their goals are, and about how they plan to make their goals a reality. Computational dialogue research offers some classic results on grouding, which unfortunately offer scant guidance to the design of grounding modules and behaviors in cutting-edge systems. In this tutorial, we focus on three main topic areas: 1) grounding in human-human communication; 2) grounding in dialogue systems; and 3) grounding in multi-modal interactive systems, including image-oriented conversations and human-robot interactions. We highlight a number of achievements of recent computational research in coordinating complex content, show how these results lead to rich and challenging opportunities for doing grounding in more flexible and powerful ways, and canvass relevant insights from the literature on human--human conversation. We expect that the tutorial will be of interest to researchers in dialogue systems, computational semantics and cognitive modeling, and hope that it will catalyze research and system building that more directly explores the creative, strategic ways conversational agents might be able to seek and offer evidence about their understanding of their interlocutors.

'''T6: Commonsense Reasoning for Natural Language Processing''' (introductory) 
Maarten Sap, Vered Shwartz, Antoine Bosselut, Dan Roth and Yejin Choi 
In our tutorial, we (1) outline the various types of commonsense (e.g., physical, social), and (2) discuss techniques to gather and represent commonsense knowledge, while highlighting the challenges specific to this type of knowledge (e.g., reporting bias). We will then (3) discuss the types of commonsense knowledge captured by modern NLP systems (e.g., large pretrained language models), and (4) present ways to measure systems' commonsense reasoning abilities. We finish with (5) a discussion of various ways in which commonsense reasoning can be used to improve performance on NLP tasks, exemplified by an (6) interactive session on integrating commonsense into a downstream task.

'''T7: Integrating Ethics into the NLP Curriculum''' (introductory) 
Emily M. Bender, Dirk Hovy and Alexandra Schofield 
Our goal in this tutorial is to empower NLP researchers and practitioners with tools and resources to teach others about how to ethically apply NLP techniques. Our tutorial will present both high-level strategies for developing an ethics-oriented curriculum, based on experience and best practices, as well as specific sample exercises that can be brought to a classroom. We plan to make this a highly interactive work session culminating in a shared online resource page that pools lesson plans, assignments, exercise ideas, reading suggestions, and ideas from the attendees. We consider three primary topics with our session that frequently underlie ethical issues in NLP research: Dual use, bias and privacy. 
In this setting, a key lesson is that there is no single approach to ethical NLP: each project requires thoughtful consideration about what steps can be taken to best support people affected by that project. However, we can learn (and teach) what kinds of issues to be aware of and what kinds of strategies are available for mitigating harm. To teach this process, we apply and promote interactive exercises that provide an opportunity to ideate, discuss, and reflect. We plan to facilitate this in a way that encourages positive discussion, emphasizing the creation of ideas for the future instead of negative opinions of previous work.

'''T8: Recent Advances in Open-Domain Question Answering''' (cutting-edge) 
Danqi Chen and Scott Wen-tau Yih 
Open-domain (textual) question answering (QA), the task of finding answers to open-domain questions by searching a large collection of documents, has been a long-standing problem in NLP, information retrieval (IR) and related fields (Voorhees et al., 1999; Moldovan et al., 2000; Brill et al.,2002; Ferrucci et al., 2010). Traditional QA systems were usually constructed as a pipeline, consisting of many different components such as question processing, document/passage retrieval and answer processing. With the rapid development of neural reading comprehension (Chen, 2018), modern open-domain QA systems have been restructured by combining traditional IR techniques and neural reading comprehension models (Chen et al., 2017; Yang et al., 2019) or even implemented in a fully end-to-end fashion (Lee et al., 2019; Seo et al., 2019). While the system architecture has been drastically simplified, two technical challenges remain critical:(1) “Retriever”: finding documents that (might)contain an answer from a large collection of documents; (2) “Reader”: finding the answer in a given paragraph or a document. 
In this tutorial, we aim to provide a comprehensive and coherent overview of recent advances in this line of research. We will start by first giving a brief historical background of open-domain question answering, discussing the basic setup and core technical challenges of the research problem.The focus will then shift to modern techniques and resources proposed for open-domain QA, including the basics of latest neural reading comprehension systems, new datasets and models. The scope will also be broadened to cover the information retrieval component on how to effectively identify passages relevant to the questions. Moreover, in-depth discussions will be given on the use of traditional / neural IR modules, as well as the trade-offs between modular design and end-to-end training. If time permits, we also plan to discuss some hybrid approaches for answering questions using both text and large knowledge bases (e.g. (Sun et al., 2018)) and give a critical review on how structured data complements the information from unstructured text. 
At the end of our tutorial, we will discuss some important questions, including (1) How much progress have we made compared to the QA systems developed in the last decade?(2) What are the main challenges and limitations of cur-rent approaches? (3) How to trade off the efficiency (computational time and memory requirements) and accuracy in the deep learning era? We hope that our tutorial will not only serve as a useful resource for the audience to efficiently acquire the up-to-date knowledge, but also provide new perspectives to stimulate the advances of open-domain QA research in the next phase.

===Virtual Conference Organization===

In April the whole conference was decided to be held in virtual. We carried on the preparations jointly with the tutorial instructors, the virtual infrastructure chairs, the website chairs and the SlidesLive team to set up the virtual conference infrastructure, and to publicize it for each tutorial.

We first made a decision on the time slots for tutorial presentations. Five major time zones were considered, including US west coast, US east coast, central Europe, China/Asia and Australia (India was also considered). Our aim was to offer at least two convenient time slots (i.e. within regular working hours) for each of these zones.

The final time slots include (Jul 5 Pacific Time):

* slot 1 – 7:00 PM (-1D) to 10:30 PM (-1D)
* slot 2 – 12:00 AM to 3:30 AM
* slot 3 – 6:00 AM to 9:30 AM
* slot 4 – 10:30 AM to 2:00 PM
* slot 5 – 3:00 PM to 6:30 PM

We asked the tutorial teachers to indicate their preferences among the time slots, and – if possible – to agree to give their tutorial twice, so as to increase accessibility. Our final decisions on slot assignments traded off the teacher preferences and slot availability. In the end, the assignments were the following (see also the [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ blog post] and [https://acl2020.org/program/tutorials/ online schedule]):

* twice on slots 3 and 4
* T2 – once on slot 5
* T3 – once on slot 4
* T4 – once on slot 4
* T5 – twice on slots 3 and 5
* T6 – once on slot 5
* T7 – twice on slots 3 and 4
* T8 – once on slot 5

Unfortunately, the tutorial teachers proved hardly available for slots 1 and 2.

In addition to time slot arrangements, we gave the tutorial teacher two options on presentation, namely a pre-recorded and a fully interactive form of tutorials. The former allows the teachers to pre-record the main content of their tutorials, and use the time slots mainly for question answering and detailed discussions. The latter is more interactive, where the teachers use the online sessions for both giving the lecture and other activities such as active discussions. All these arrangements are made for taking advantage of the virtual conference. In the end the choices were the following:

* T1 – pre-recorded
* T2 – interactive (the initial decision was pre-recorded but a change was made in June)
* T3 – pre-recorded
* T4 – pre-recorded
* T5 – pre-recorded
* T6 – interactive
* T7 – interactive
* T8 – interactive

After making decisions on time slots and teaching forms, we started to work on the technical details of the virtual conference infrastructure jointly with the infrastructure chairs, the website chairs, the SlidesLive support team and individual tutorial teachers. We collected needs from each tutorial from time to time, inquired information from the SlidesLive team, and tried to accommodate specific requirement for each tutorial. Since this is the first time ACL was virtual, it typically took several rounds of information exchange for making a certain technical feature work. Major things discussed included:

* Sorting out how to pre-record a tutorial using the system
* Recording of interactive sessions
* Additional RocketChat channel
* Virtual tutorial webpage setup
* Coordinators for tutorial sessions and SlidesLive team participation
* Rehearsal of interactive sessions
* Captioning of pre-recorded and interactive tutorials
* Privacy issues
* Special needs from particular tutorials
** Integrating [https://www.dory.app/ Dory] for question answering (in T1)
** Fully interactive sessions, with occasional split into break-out rooms (in T7)

To ensure smooth running of the sessions, we sent two rounds of reminders to each tutorial to ensure that each teacher is registered to the conference. We collected rough timelines of each tutorial so that the SlidesLive team can work closely during the live sessions. To further help the teachers prepare their live session content we additionally collected the lists of registered attendees twice before the conference, and shared them with the teachers.

In the end, the numbers of registrations for each tutorial were:
* T1 -- 1762
* T2 -- 793
* T3 -- 503
* T4 -- 571
* T5 -- 641
* T6 -- 1288
* T7 -- 378
* T8 – 1087

===Archiving and Publicity===

There are three main channels for publicity, namely the static [http://www.acl2020.org/program/tutorials/ website], the website of the [https://virtual.acl2020.org/tutorials.html virtual conference] and the social media. For the former, we collected photos from each instructor, which are shown with the title and abstract of their tutorials. Following NAACL 2019, we also displayed the time slots as a right-hand-side bar on the tutorial website. Due to special arrangements of the virtual conference, we marked the time zones explicitly. In addition, we asked the tutorial teachers to provide URL links to teaching materials (e.g., at GitHub) during their final submission, and included these links to their tutorial information. For social media, we drafted two blog posts (i.e., [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ this post] and [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]) for advertising the tutorials.

The archiving consists of three main items, namely the tutorial proceedings, the slides for each tutorial and the videos. For the tutorial proceedings, we collected final abstracts from the authors using the START system. We asked the instructors to sign the CC BY v4 agreement when submitting their final materials on SoftConf. For the slides, we asked the tutorial teachers to provide a first version of the slides by June 17 regardless whether they are pre-recorded or interactive. Some tutorials made further updates to their slides before ACL. For the video recordings, we worked with the ACL Anthology team discussing the technical details since this is the first time the tutorial videos are shared in the website. [Note: this is currently undergoing at the time of writing of this summary]

===Lessons Learned===

While organizing a virtual conference it is important to stress that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]).

It is also important to announce the individual modalities with the attendees early enough. For instance, watching pre-recordings in advance was necessary for some tutorials (since their live sessions only included question answering) and such constrains should have been known more in advance.

If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to Zoom rooms only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.

One thing to remember which we missed is to publish the reading list for each tutorial (which the teachers submit with the proposal) on the tutorial website.

2020Q3 Reports: Tutorial Chairs

2020-07-23T10:00:59Z

AgataSavary: /* Virtual Conference Organization */

Tutorial Chairs:

<blockquote>Agata Savary, University of Tours, France

Yue Zhang, Westlake University, Hangzhou, China
</blockquote>

=== Preparatory Work, Call, Reviewing and Decision-Making ===
The call, submission, reviewing and selection of tutorials was coordinated jointly for 4 conferences: ACL, AACL-IJCNLP, COLING and EMNLP.

Before drafting the call, we collected lists of tutorials offered within the past 4 years. We analysed previous calls for tutorials and reports from tutorial chairs (from [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs 2016], [https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs 2017], [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs 2018] and [http://aclweb.org/adminwiki/index.php?title=2019Q1_Reports:_ACL_2019 2019]). We consulted previous tutorial chairs with a questionnaire including questions about: the number of submissions, encouraging submissions on specific topics or from specific lecturers, the review procedure, the evaluation criteria, the post-tutorial availability of the slides/codes, and lessons learned from tutorial coordination. We also discussed the publication of slides and video recordings from future tutorials with the persons in charge of the ACL Anthology. As a result of these steps, we created two new sections for the [https://www.aclweb.org/adminwiki/index.php?title=Conference_Handbook ACL Conference Handbook] (future chairs might consider updating these documents yearly):

* the list of [https://www.aclweb.org/adminwiki/index.php?title=Past_tutorials past tutorials] at ACL, COLING, EACL, EMNLP, and NAACL in 2016-2019
* a [https://www.aclweb.org/adminwiki/index.php?title=Tutorial_chair_handbook tutorial chair handbook]

The final [https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020 call] differs from previous calls in several aspects: (i) the expectations about tutorial proposals were made clearer, (ii) following the central ACL decision, the teachers' payment policy was replaced by a fee-waiving policy, (iii) the required submission details include two new items: diversity considerations and agreement for open access publication of slides, codes, data and video recordings, (iv) the evaluation criteria (see below) are announced.

We recruited a review committee of 19 members, including the 8 tutorial chairs and 11 external members selected for their large understanding of the NLP domain and a good experience in reviewing and/or tutorial teaching:

'''Review Committee'''

* Timothy Baldwin (University of Melbourne, Australia) - AACL-IJCNLP 2020 tutorial chair
* Daniel Beck (University of Melbourne, Australia) - COLING 2020 tutorial chair
* Emily M. Bender (University of Washington, WA, USA)
* Erik Cambria (Nanyang Technological University, Singapore)
* Gaël Dias (University of Caen Normandie, France)
* Stefan Evert (Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany)
* Yang Liu (Tsinghua University, Beijing, China)
* Agata Savary (University of Tours, France) - ACL 2020 tutorial chair
* João Sedoc (Johns Hopkins University, Baltimore, MD, USA)
* Lucia Specia (Sheffield University, UK) - COLING 2020 tutorial chair
* Xu Sun (Peking University, China)
* Yulia Tsvetkov (Carnegie Mellon University, Pittsburgh, PA, USA)
* Benjamin Van Durme (Johns Hopkins University, Baltimore, MD, USA) - EMNLP 2020 tutorial chair
* Aline Villavicencio (University of Sheffield, UK and Federal University of Rio Grande do Sul, Brazil) - EMNLP 2020 tutorial chair
* Taro Watanabe (Google, Inc., Tokyo, Japan)
* Aaron Steven White (University of Rochester, NY, USA)
* Fei Xia (University of Washington, WA, USA) - AACL-IJCNLP 2020 tutorial chair
* Yue Zhang (Westlake University, Hangzhou, China) - ACL 2020 tutorial chair
* Meishan Zhang (Tianjin University, China)

In total, we received 43 submissions for the 4 conferences. Each reviewer was assigned 6-7 proposals and each proposal received 3 reviews. The selection criteria included: clarity and preparedness, novelty or timely character of the topic, lecturers' experience, likely audience interest, open access of the teaching material, diversity aspects (multilingualism, gender, age and country of the lecturers), and compatibility with the preferred venues. We accepted 31 proposals.

The decision making was handled via an online meeting of the 8 tutorial chairs. In particular, the selection of tutorials for each conference was done via the expression of interest of the tutorial chairs on a round-robin basis. Some slight adjustments were also performed after the meeting to better fit the authors' preferences. In total, 8, 8, 8 and 7 proposals were selected for ACL, AACL-IJCNLP, COLING and EMNLP, respectively. Upon the announcement the results, 2 of the proposals accepted for AACL-IJCNLP were withdrawn.

The submission, review, selection and collection of final descriptions for all tutorials was handled via a dedicated SoftConf space, shared by the 4 coordinating conferences. After the selection of proposals, a separate track was created on SoftConf for each conference. The final submission page (one per conference) was set up so as to collect all the necessary data including notably: the tutorial slides, URLs for course material (if any), printable material (if any) and agreement for open access publication.

===Tutorials Selected for ACL 2020===

The final selection for ACL 2020 consists of the following 8 tutorials of 3 hours each (each of them had ACL as the preferred or the second preferred venue):

'''T1: Interpretability and Analysis in Neural NLP''' (cutting-edge) 
Yonatan Belinkov, Sebastian Gehrmann and Ellie Pavlick 
While deep learning has transformed the NLP field and impacted the larger computational linguistics community, the rise of neural networks is stained by their opaque nature: It is challenging to interpret the inner workings of neural network models, and explicate their behavior. Therefore, in the last few years, an increasingly large body of work has been devoted to the analysis and interpretation of neural network models in NLP. 
This body of work is so far lacking a common framework and methodology. Moreover, approaching the analysis of modern neural networks can be difficult for newcomers to the field. This tutorial aims to fill this gap and introduce the nascent field of interpretability and analysis of neural networks in NLP. 
The tutorial covers the main lines of analysis work, such as probing classifier, behavior studies and test suites, psycholinguistic methods, visualizations, adversarial examples, and other methods. We highlight not only the most commonly applied analysis methods, but also the specific limitations and shortcomings of current approaches, in order to inform participants where to focus future efforts.

'''T2: Multi-modal Information Extraction from Text, Semi-structured, and Tabular Data on the Web''' (cutting-edge) 
Xin Luna Dong, Hannaneh Hajishirzi, Colin Lockard and Prashant Shiralkar 
The World Wide Web contains vast quantities of textual information in several forms: unstructured text, template-based semi-structured webpages (which present data in key-value pairs and lists), and tables. Methods for extracting information from these sources and converting it to a structured form have been a target of research from the natural language processing (NLP), data mining, and database communities. While these researchers have largely separated extraction from web data into different problems based on the modality of the data, they have faced similar problems such as learning with limited labeled data, defining (or avoiding defining) ontologies, making use of prior knowledge, and scaling solutions to deal with the size of the Web. 
In this tutorial we take a holistic view toward information extraction, exploring the commonalities in the challenges and solutions developed to address these different forms of text. We will explore the approaches targeted at unstructured text that largely rely on learning syntactic or semantic textual patterns, approaches targeted at semi-structured documents that learn to identify structural patterns in the template, and approaches targeting web tables which rely heavily on entity linking and type information. 
While these different data modalities have largely been considered separately in the past, recent research has started taking a more inclusive approach toward textual extraction, in which the multiple signals offered by textual, layout, and visual clues are combined into a single extraction model made possible by new deep learning approaches. At the same time, trends within purely textual extraction have shifted toward full-document understanding rather than considering sentences as independent units. With this in mind, it is worth considering the information extraction problem as a whole to motivate solutions that harness textual semantics along with visual and semi-structured layout information. We will discuss these approaches and suggest avenues for future work.

'''T3: Reviewing Natural Language Processing Research''' (introductory) 
Kevin Cohen, Karën Fort, Margot Mieskes and Aurélie Névéol 
As the demand for reviewing grows, so must the pool of reviewers. As the [http://www.livecongress.it/aol/indexSA.php?id=E2EAED7D&ticket= survey] presented by Graham Neubig at the 2019 ACL showed, a considerable number of reviewers are junior researchers, who might lack the experience and expertise necessary for high-quality reviews. Some of them might not have the environment or lack opportunities that allow them to learn the skills necessary. A tutorial on reviewing for the NLP community might increase reviewers’ confidence, as well as the quality of the reviews. This introductory tutorial will cover the goals, processes, and evaluation of reviewing research papers in natural language processing.

'''T4: Stylized Text Generation: Approaches and Applications''' (cutting-edge) 
Lili Mou and Olga Vechtomova 
Text generation has played an important role in various applications of natural language processing (NLP), and kn recent studies, researchers are paying increasing attention to modeling and manipulating the style of the generation text, which we call stylized text generation. In this tutorial, we will provide a comprehensive literature review in this direction. We start from the definition of style and different settings of stylized text generation, illustrated with various applications. Then, we present different settings of stylized generation, such as parallel supervised, style label-supervised, and unsupervised. In each setting, we delve deep into machine learning methods, including embedding learning techniques to represent style}, adversarial learning and reinforcement learning with cycle consistency to match content but to distinguish different styles. We also introduce current approaches of evaluating stylized text generation systems. We conclude our tutorial by presenting the challenges of stylized text generation and discussing future directions, such as small-data training, non-categorical style modeling, and a generalized scope of style transfer (e.g., controlling the syntax as a style).

'''T5: Achieving Common Ground in Multi-modal Dialogue''' (cutting-edge) 
Malihe Alikhani and Matthew Stone 
All communication aims at achieving common ground (grounding): interlocutors can work together effectively only with mutual beliefs about what the state of the world is, about what their goals are, and about how they plan to make their goals a reality. Computational dialogue research offers some classic results on grouding, which unfortunately offer scant guidance to the design of grounding modules and behaviors in cutting-edge systems. In this tutorial, we focus on three main topic areas: 1) grounding in human-human communication; 2) grounding in dialogue systems; and 3) grounding in multi-modal interactive systems, including image-oriented conversations and human-robot interactions. We highlight a number of achievements of recent computational research in coordinating complex content, show how these results lead to rich and challenging opportunities for doing grounding in more flexible and powerful ways, and canvass relevant insights from the literature on human--human conversation. We expect that the tutorial will be of interest to researchers in dialogue systems, computational semantics and cognitive modeling, and hope that it will catalyze research and system building that more directly explores the creative, strategic ways conversational agents might be able to seek and offer evidence about their understanding of their interlocutors.

'''T6: Commonsense Reasoning for Natural Language Processing''' (introductory) 
Maarten Sap, Vered Shwartz, Antoine Bosselut, Dan Roth and Yejin Choi 
In our tutorial, we (1) outline the various types of commonsense (e.g., physical, social), and (2) discuss techniques to gather and represent commonsense knowledge, while highlighting the challenges specific to this type of knowledge (e.g., reporting bias). We will then (3) discuss the types of commonsense knowledge captured by modern NLP systems (e.g., large pretrained language models), and (4) present ways to measure systems' commonsense reasoning abilities. We finish with (5) a discussion of various ways in which commonsense reasoning can be used to improve performance on NLP tasks, exemplified by an (6) interactive session on integrating commonsense into a downstream task.

'''T7: Integrating Ethics into the NLP Curriculum''' (introductory) 
Emily M. Bender, Dirk Hovy and Alexandra Schofield 
Our goal in this tutorial is to empower NLP researchers and practitioners with tools and resources to teach others about how to ethically apply NLP techniques. Our tutorial will present both high-level strategies for developing an ethics-oriented curriculum, based on experience and best practices, as well as specific sample exercises that can be brought to a classroom. We plan to make this a highly interactive work session culminating in a shared online resource page that pools lesson plans, assignments, exercise ideas, reading suggestions, and ideas from the attendees. We consider three primary topics with our session that frequently underlie ethical issues in NLP research: Dual use, bias and privacy. 
In this setting, a key lesson is that there is no single approach to ethical NLP: each project requires thoughtful consideration about what steps can be taken to best support people affected by that project. However, we can learn (and teach) what kinds of issues to be aware of and what kinds of strategies are available for mitigating harm. To teach this process, we apply and promote interactive exercises that provide an opportunity to ideate, discuss, and reflect. We plan to facilitate this in a way that encourages positive discussion, emphasizing the creation of ideas for the future instead of negative opinions of previous work.

'''T8: Recent Advances in Open-Domain Question Answering''' (cutting-edge) 
Danqi Chen and Scott Wen-tau Yih 
Open-domain (textual) question answering (QA), the task of finding answers to open-domain questions by searching a large collection of documents, has been a long-standing problem in NLP, information retrieval (IR) and related fields (Voorhees et al., 1999; Moldovan et al., 2000; Brill et al.,2002; Ferrucci et al., 2010). Traditional QA systems were usually constructed as a pipeline, consisting of many different components such as question processing, document/passage retrieval and answer processing. With the rapid development of neural reading comprehension (Chen, 2018), modern open-domain QA systems have been restructured by combining traditional IR techniques and neural reading comprehension models (Chen et al., 2017; Yang et al., 2019) or even implemented in a fully end-to-end fashion (Lee et al., 2019; Seo et al., 2019). While the system architecture has been drastically simplified, two technical challenges remain critical:(1) “Retriever”: finding documents that (might)contain an answer from a large collection of documents; (2) “Reader”: finding the answer in a given paragraph or a document. 
In this tutorial, we aim to provide a comprehensive and coherent overview of recent advances in this line of research. We will start by first giving a brief historical background of open-domain question answering, discussing the basic setup and core technical challenges of the research problem.The focus will then shift to modern techniques and resources proposed for open-domain QA, including the basics of latest neural reading comprehension systems, new datasets and models. The scope will also be broadened to cover the information retrieval component on how to effectively identify passages relevant to the questions. Moreover, in-depth discussions will be given on the use of traditional / neural IR modules, as well as the trade-offs between modular design and end-to-end training. If time permits, we also plan to discuss some hybrid approaches for answering questions using both text and large knowledge bases (e.g. (Sun et al., 2018)) and give a critical review on how structured data complements the information from unstructured text. 
At the end of our tutorial, we will discuss some important questions, including (1) How much progress have we made compared to the QA systems developed in the last decade?(2) What are the main challenges and limitations of cur-rent approaches? (3) How to trade off the efficiency (computational time and memory requirements) and accuracy in the deep learning era? We hope that our tutorial will not only serve as a useful resource for the audience to efficiently acquire the up-to-date knowledge, but also provide new perspectives to stimulate the advances of open-domain QA research in the next phase.

===Virtual Conference Organization===

In April the whole conference was decided to be held in virtual. We carried on the preparations jointly with the tutorial instructors, the virtual infrastructure chairs, the website chairs and the SlidesLive team to set up the virtual conference infrastructure, and to publicize it for each tutorial.

We first made a decision on the time slots for tutorial presentations. Five major time zones were considered, including US west coast, US east coast, central Europe, China/Asia and Australia (India was also considered). Our aim was to offer at least two convenient time slots (i.e. within regular working hours) for each of these zones.

The final time slots include (Jul 5 Pacific Time):

* slot 1 – 7:00 PM (-1D) to 10:30 PM (-1D)
* slot 2 – 12:00 AM to 3:30 AM
* slot 3 – 6:00 AM to 9:30 AM
* slot 4 – 10:30 AM to 2:00 PM
* slot 5 – 3:00 PM to 6:30 PM

We asked the tutorial teachers to indicate their preferences among the time slots, and – if possible – to agree to give their tutorial twice, so as to increase accessibility. Our final decisions on slot assignments traded off the teacher preferences and slot availability. In the end, the assignments were the following (see also the [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ blog post] and [https://acl2020.org/program/tutorials/ online schedule]):

<blockquote>T1 – twice on slots 3 and 4

T2 – once on slot 5

T3 – once on slot 4

T4 – once on slot 4

T5 – twice on slots 3 and 5

T6 – once on slot 5

T7 – twice on slots 3 and 4

T8 – once on slot 5
</blockquote>
Unfortunately, the tutorial teachers proved hardly available for slots 1 and 2.

In addition to time slot arrangements, we gave the tutorial teacher two options on presentation, namely a pre-recorded and a fully interactive form of tutorials. The former allows the teachers to pre-record the main content of their tutorials, and use the time slots mainly for question answering and detailed discussions. The latter is more interactive, where the teachers use the online sessions for both giving the lecture and other activities such as active discussions. All these arrangements are made for taking advantage of the virtual conference. In the end the choices were the following:

<blockquote>T1 – pre-recorded

T2 – interactive (the initial decision was pre-recorded but a change was made in June)

T3 – pre-recorded

T4 – pre-recorded

T5 – pre-recorded

T6 – interactive

T7 – interactive

T8 – interactive
</blockquote>
After making decisions on time slots and teaching forms, we started to work on the technical details of the virtual conference infrastructure jointly with the infrastructure chairs, the website chairs, the SlidesLive support team and individual tutorial teachers. We collected needs from each tutorial from time to time, inquired information from the SlidesLive team, and tried to accommodate specific requirement for each tutorial. Since this is the first time ACL was virtual, it typically took several rounds of information exchange for making a certain technical feature work. Major things discussed included:

* Sorting out how to pre-record a tutorial using the system
* Recording of interactive sessions
* Additional RocketChat channel
* Virtual tutorial webpage setup
* Coordinators for tutorial sessions and SlidesLive team participation
* Rehearsal of interactive sessions
* Captioning of pre-recorded and interactive tutorials
* Privacy issues
* Special needs from particular tutorials
** Integrating [https://www.dory.app/ Dory] for question answering (in T1)
** Fully interactive sessions, with occasional split into break-out rooms (in T7)

To ensure smooth running of the sessions, we sent two rounds of reminders to each tutorial to ensure that each teacher is registered to the conference. We collected rough timelines of each tutorial so that the SlidesLive team can work closely during the live sessions. To further help the teachers prepare their live session content we additionally collected the lists of registered attendees twice before the conference, and shared them with the teachers.

In the end, the numbers of registrations for each tutorial were:

<blockquote>T1 -- 1762

T2 -- 793

T3 -- 503

T4 -- 571

T5 -- 641

T6 -- 1288

T7 -- 378

T8 – 1087

===Archiving and Publicity===

There are three main channels for publicity, namely the static [http://www.acl2020.org/program/tutorials/ website], the website of the [https://virtual.acl2020.org/tutorials.html virtual conference] and the social media. For the former, we collected photos from each instructor, which are shown with the title and abstract of their tutorials. Following NAACL 2019, we also displayed the time slots as a right-hand-side bar on the tutorial website. Due to special arrangements of the virtual conference, we marked the time zones explicitly. In addition, we asked the tutorial teachers to provide URL links to teaching materials (e.g., at GitHub) during their final submission, and included these links to their tutorial information. For social media, we drafted two blog posts (i.e., [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ this post] and [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]) for advertising the tutorials.

The archiving consists of three main items, namely the tutorial proceedings, the slides for each tutorial and the videos. For the tutorial proceedings, we collected final abstracts from the authors using the START system. We asked the instructors to sign the CC BY v4 agreement when submitting their final materials on SoftConf. For the slides, we asked the tutorial teachers to provide a first version of the slides by June 17 regardless whether they are pre-recorded or interactive. Some tutorials made further updates to their slides before ACL. For the video recordings, we worked with the ACL Anthology team discussing the technical details since this is the first time the tutorial videos are shared in the website. [Note: this is currently undergoing at the time of writing of this summary]

===Lessons Learned===

While organizing a virtual conference it is important to stress that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]).

It is also important to announce the individual modalities with the attendees early enough. For instance, watching pre-recordings in advance was necessary for some tutorials (since their live sessions only included question answering) and such constrains should have been known more in advance.

If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to Zoom rooms only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.

One thing to remember which we missed is to publish the reading list for each tutorial (which the teachers submit with the proposal) on the tutorial website.

2020Q3 Reports: Tutorial Chairs

2020-07-23T10:00:39Z

AgataSavary: /* Virtual Conference Organization */

Tutorial Chairs:

<blockquote>Agata Savary, University of Tours, France

Yue Zhang, Westlake University, Hangzhou, China
</blockquote>

=== Preparatory Work, Call, Reviewing and Decision-Making ===
The call, submission, reviewing and selection of tutorials was coordinated jointly for 4 conferences: ACL, AACL-IJCNLP, COLING and EMNLP.

Before drafting the call, we collected lists of tutorials offered within the past 4 years. We analysed previous calls for tutorials and reports from tutorial chairs (from [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs 2016], [https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs 2017], [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs 2018] and [http://aclweb.org/adminwiki/index.php?title=2019Q1_Reports:_ACL_2019 2019]). We consulted previous tutorial chairs with a questionnaire including questions about: the number of submissions, encouraging submissions on specific topics or from specific lecturers, the review procedure, the evaluation criteria, the post-tutorial availability of the slides/codes, and lessons learned from tutorial coordination. We also discussed the publication of slides and video recordings from future tutorials with the persons in charge of the ACL Anthology. As a result of these steps, we created two new sections for the [https://www.aclweb.org/adminwiki/index.php?title=Conference_Handbook ACL Conference Handbook] (future chairs might consider updating these documents yearly):

* the list of [https://www.aclweb.org/adminwiki/index.php?title=Past_tutorials past tutorials] at ACL, COLING, EACL, EMNLP, and NAACL in 2016-2019
* a [https://www.aclweb.org/adminwiki/index.php?title=Tutorial_chair_handbook tutorial chair handbook]

The final [https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020 call] differs from previous calls in several aspects: (i) the expectations about tutorial proposals were made clearer, (ii) following the central ACL decision, the teachers' payment policy was replaced by a fee-waiving policy, (iii) the required submission details include two new items: diversity considerations and agreement for open access publication of slides, codes, data and video recordings, (iv) the evaluation criteria (see below) are announced.

We recruited a review committee of 19 members, including the 8 tutorial chairs and 11 external members selected for their large understanding of the NLP domain and a good experience in reviewing and/or tutorial teaching:

'''Review Committee'''

* Timothy Baldwin (University of Melbourne, Australia) - AACL-IJCNLP 2020 tutorial chair
* Daniel Beck (University of Melbourne, Australia) - COLING 2020 tutorial chair
* Emily M. Bender (University of Washington, WA, USA)
* Erik Cambria (Nanyang Technological University, Singapore)
* Gaël Dias (University of Caen Normandie, France)
* Stefan Evert (Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany)
* Yang Liu (Tsinghua University, Beijing, China)
* Agata Savary (University of Tours, France) - ACL 2020 tutorial chair
* João Sedoc (Johns Hopkins University, Baltimore, MD, USA)
* Lucia Specia (Sheffield University, UK) - COLING 2020 tutorial chair
* Xu Sun (Peking University, China)
* Yulia Tsvetkov (Carnegie Mellon University, Pittsburgh, PA, USA)
* Benjamin Van Durme (Johns Hopkins University, Baltimore, MD, USA) - EMNLP 2020 tutorial chair
* Aline Villavicencio (University of Sheffield, UK and Federal University of Rio Grande do Sul, Brazil) - EMNLP 2020 tutorial chair
* Taro Watanabe (Google, Inc., Tokyo, Japan)
* Aaron Steven White (University of Rochester, NY, USA)
* Fei Xia (University of Washington, WA, USA) - AACL-IJCNLP 2020 tutorial chair
* Yue Zhang (Westlake University, Hangzhou, China) - ACL 2020 tutorial chair
* Meishan Zhang (Tianjin University, China)

In total, we received 43 submissions for the 4 conferences. Each reviewer was assigned 6-7 proposals and each proposal received 3 reviews. The selection criteria included: clarity and preparedness, novelty or timely character of the topic, lecturers' experience, likely audience interest, open access of the teaching material, diversity aspects (multilingualism, gender, age and country of the lecturers), and compatibility with the preferred venues. We accepted 31 proposals.

The decision making was handled via an online meeting of the 8 tutorial chairs. In particular, the selection of tutorials for each conference was done via the expression of interest of the tutorial chairs on a round-robin basis. Some slight adjustments were also performed after the meeting to better fit the authors' preferences. In total, 8, 8, 8 and 7 proposals were selected for ACL, AACL-IJCNLP, COLING and EMNLP, respectively. Upon the announcement the results, 2 of the proposals accepted for AACL-IJCNLP were withdrawn.

The submission, review, selection and collection of final descriptions for all tutorials was handled via a dedicated SoftConf space, shared by the 4 coordinating conferences. After the selection of proposals, a separate track was created on SoftConf for each conference. The final submission page (one per conference) was set up so as to collect all the necessary data including notably: the tutorial slides, URLs for course material (if any), printable material (if any) and agreement for open access publication.

===Tutorials Selected for ACL 2020===

The final selection for ACL 2020 consists of the following 8 tutorials of 3 hours each (each of them had ACL as the preferred or the second preferred venue):

'''T1: Interpretability and Analysis in Neural NLP''' (cutting-edge) 
Yonatan Belinkov, Sebastian Gehrmann and Ellie Pavlick 
While deep learning has transformed the NLP field and impacted the larger computational linguistics community, the rise of neural networks is stained by their opaque nature: It is challenging to interpret the inner workings of neural network models, and explicate their behavior. Therefore, in the last few years, an increasingly large body of work has been devoted to the analysis and interpretation of neural network models in NLP. 
This body of work is so far lacking a common framework and methodology. Moreover, approaching the analysis of modern neural networks can be difficult for newcomers to the field. This tutorial aims to fill this gap and introduce the nascent field of interpretability and analysis of neural networks in NLP. 
The tutorial covers the main lines of analysis work, such as probing classifier, behavior studies and test suites, psycholinguistic methods, visualizations, adversarial examples, and other methods. We highlight not only the most commonly applied analysis methods, but also the specific limitations and shortcomings of current approaches, in order to inform participants where to focus future efforts.

'''T2: Multi-modal Information Extraction from Text, Semi-structured, and Tabular Data on the Web''' (cutting-edge) 
Xin Luna Dong, Hannaneh Hajishirzi, Colin Lockard and Prashant Shiralkar 
The World Wide Web contains vast quantities of textual information in several forms: unstructured text, template-based semi-structured webpages (which present data in key-value pairs and lists), and tables. Methods for extracting information from these sources and converting it to a structured form have been a target of research from the natural language processing (NLP), data mining, and database communities. While these researchers have largely separated extraction from web data into different problems based on the modality of the data, they have faced similar problems such as learning with limited labeled data, defining (or avoiding defining) ontologies, making use of prior knowledge, and scaling solutions to deal with the size of the Web. 
In this tutorial we take a holistic view toward information extraction, exploring the commonalities in the challenges and solutions developed to address these different forms of text. We will explore the approaches targeted at unstructured text that largely rely on learning syntactic or semantic textual patterns, approaches targeted at semi-structured documents that learn to identify structural patterns in the template, and approaches targeting web tables which rely heavily on entity linking and type information. 
While these different data modalities have largely been considered separately in the past, recent research has started taking a more inclusive approach toward textual extraction, in which the multiple signals offered by textual, layout, and visual clues are combined into a single extraction model made possible by new deep learning approaches. At the same time, trends within purely textual extraction have shifted toward full-document understanding rather than considering sentences as independent units. With this in mind, it is worth considering the information extraction problem as a whole to motivate solutions that harness textual semantics along with visual and semi-structured layout information. We will discuss these approaches and suggest avenues for future work.

'''T3: Reviewing Natural Language Processing Research''' (introductory) 
Kevin Cohen, Karën Fort, Margot Mieskes and Aurélie Névéol 
As the demand for reviewing grows, so must the pool of reviewers. As the [http://www.livecongress.it/aol/indexSA.php?id=E2EAED7D&ticket= survey] presented by Graham Neubig at the 2019 ACL showed, a considerable number of reviewers are junior researchers, who might lack the experience and expertise necessary for high-quality reviews. Some of them might not have the environment or lack opportunities that allow them to learn the skills necessary. A tutorial on reviewing for the NLP community might increase reviewers’ confidence, as well as the quality of the reviews. This introductory tutorial will cover the goals, processes, and evaluation of reviewing research papers in natural language processing.

'''T4: Stylized Text Generation: Approaches and Applications''' (cutting-edge) 
Lili Mou and Olga Vechtomova 
Text generation has played an important role in various applications of natural language processing (NLP), and kn recent studies, researchers are paying increasing attention to modeling and manipulating the style of the generation text, which we call stylized text generation. In this tutorial, we will provide a comprehensive literature review in this direction. We start from the definition of style and different settings of stylized text generation, illustrated with various applications. Then, we present different settings of stylized generation, such as parallel supervised, style label-supervised, and unsupervised. In each setting, we delve deep into machine learning methods, including embedding learning techniques to represent style}, adversarial learning and reinforcement learning with cycle consistency to match content but to distinguish different styles. We also introduce current approaches of evaluating stylized text generation systems. We conclude our tutorial by presenting the challenges of stylized text generation and discussing future directions, such as small-data training, non-categorical style modeling, and a generalized scope of style transfer (e.g., controlling the syntax as a style).

'''T5: Achieving Common Ground in Multi-modal Dialogue''' (cutting-edge) 
Malihe Alikhani and Matthew Stone 
All communication aims at achieving common ground (grounding): interlocutors can work together effectively only with mutual beliefs about what the state of the world is, about what their goals are, and about how they plan to make their goals a reality. Computational dialogue research offers some classic results on grouding, which unfortunately offer scant guidance to the design of grounding modules and behaviors in cutting-edge systems. In this tutorial, we focus on three main topic areas: 1) grounding in human-human communication; 2) grounding in dialogue systems; and 3) grounding in multi-modal interactive systems, including image-oriented conversations and human-robot interactions. We highlight a number of achievements of recent computational research in coordinating complex content, show how these results lead to rich and challenging opportunities for doing grounding in more flexible and powerful ways, and canvass relevant insights from the literature on human--human conversation. We expect that the tutorial will be of interest to researchers in dialogue systems, computational semantics and cognitive modeling, and hope that it will catalyze research and system building that more directly explores the creative, strategic ways conversational agents might be able to seek and offer evidence about their understanding of their interlocutors.

'''T6: Commonsense Reasoning for Natural Language Processing''' (introductory) 
Maarten Sap, Vered Shwartz, Antoine Bosselut, Dan Roth and Yejin Choi 
In our tutorial, we (1) outline the various types of commonsense (e.g., physical, social), and (2) discuss techniques to gather and represent commonsense knowledge, while highlighting the challenges specific to this type of knowledge (e.g., reporting bias). We will then (3) discuss the types of commonsense knowledge captured by modern NLP systems (e.g., large pretrained language models), and (4) present ways to measure systems' commonsense reasoning abilities. We finish with (5) a discussion of various ways in which commonsense reasoning can be used to improve performance on NLP tasks, exemplified by an (6) interactive session on integrating commonsense into a downstream task.

'''T7: Integrating Ethics into the NLP Curriculum''' (introductory) 
Emily M. Bender, Dirk Hovy and Alexandra Schofield 
Our goal in this tutorial is to empower NLP researchers and practitioners with tools and resources to teach others about how to ethically apply NLP techniques. Our tutorial will present both high-level strategies for developing an ethics-oriented curriculum, based on experience and best practices, as well as specific sample exercises that can be brought to a classroom. We plan to make this a highly interactive work session culminating in a shared online resource page that pools lesson plans, assignments, exercise ideas, reading suggestions, and ideas from the attendees. We consider three primary topics with our session that frequently underlie ethical issues in NLP research: Dual use, bias and privacy. 
In this setting, a key lesson is that there is no single approach to ethical NLP: each project requires thoughtful consideration about what steps can be taken to best support people affected by that project. However, we can learn (and teach) what kinds of issues to be aware of and what kinds of strategies are available for mitigating harm. To teach this process, we apply and promote interactive exercises that provide an opportunity to ideate, discuss, and reflect. We plan to facilitate this in a way that encourages positive discussion, emphasizing the creation of ideas for the future instead of negative opinions of previous work.

'''T8: Recent Advances in Open-Domain Question Answering''' (cutting-edge) 
Danqi Chen and Scott Wen-tau Yih 
Open-domain (textual) question answering (QA), the task of finding answers to open-domain questions by searching a large collection of documents, has been a long-standing problem in NLP, information retrieval (IR) and related fields (Voorhees et al., 1999; Moldovan et al., 2000; Brill et al.,2002; Ferrucci et al., 2010). Traditional QA systems were usually constructed as a pipeline, consisting of many different components such as question processing, document/passage retrieval and answer processing. With the rapid development of neural reading comprehension (Chen, 2018), modern open-domain QA systems have been restructured by combining traditional IR techniques and neural reading comprehension models (Chen et al., 2017; Yang et al., 2019) or even implemented in a fully end-to-end fashion (Lee et al., 2019; Seo et al., 2019). While the system architecture has been drastically simplified, two technical challenges remain critical:(1) “Retriever”: finding documents that (might)contain an answer from a large collection of documents; (2) “Reader”: finding the answer in a given paragraph or a document. 
In this tutorial, we aim to provide a comprehensive and coherent overview of recent advances in this line of research. We will start by first giving a brief historical background of open-domain question answering, discussing the basic setup and core technical challenges of the research problem.The focus will then shift to modern techniques and resources proposed for open-domain QA, including the basics of latest neural reading comprehension systems, new datasets and models. The scope will also be broadened to cover the information retrieval component on how to effectively identify passages relevant to the questions. Moreover, in-depth discussions will be given on the use of traditional / neural IR modules, as well as the trade-offs between modular design and end-to-end training. If time permits, we also plan to discuss some hybrid approaches for answering questions using both text and large knowledge bases (e.g. (Sun et al., 2018)) and give a critical review on how structured data complements the information from unstructured text. 
At the end of our tutorial, we will discuss some important questions, including (1) How much progress have we made compared to the QA systems developed in the last decade?(2) What are the main challenges and limitations of cur-rent approaches? (3) How to trade off the efficiency (computational time and memory requirements) and accuracy in the deep learning era? We hope that our tutorial will not only serve as a useful resource for the audience to efficiently acquire the up-to-date knowledge, but also provide new perspectives to stimulate the advances of open-domain QA research in the next phase.

===Virtual Conference Organization===

In April the whole conference was decided to be held in virtual. We carried on the preparations jointly with the tutorial instructors, the virtual infrastructure chairs, the website chairs and the SlidesLive team to set up the virtual conference infrastructure, and to publicize it for each tutorial.

We first made a decision on the time slots for tutorial presentations. Five major time zones were considered, including US west coast, US east coast, central Europe, China/Asia and Australia (India was also considered). Our aim was to offer at least two convenient time slots (i.e. within regular working hours) for each of these zones.

The final time slots include (Jul 5 Pacific Time):

* slot 1 – 7:00 PM (-1D) to 10:30 PM (-1D)
* slot 2 – 12:00 AM to 3:30 AM
* slot 3 – 6:00 AM to 9:30 AM
* slot 4 – 10:30 AM to 2:00 PM

slot 5 – 3:00 PM to 6:30 PM
</blockquote>
We asked the tutorial teachers to indicate their preferences among the time slots, and – if possible – to agree to give their tutorial twice, so as to increase accessibility. Our final decisions on slot assignments traded off the teacher preferences and slot availability. In the end, the assignments were the following (see also the [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ blog post] and [https://acl2020.org/program/tutorials/ online schedule]):

<blockquote>T1 – twice on slots 3 and 4

T2 – once on slot 5

T3 – once on slot 4

T4 – once on slot 4

T5 – twice on slots 3 and 5

T6 – once on slot 5

T7 – twice on slots 3 and 4

T8 – once on slot 5
</blockquote>
Unfortunately, the tutorial teachers proved hardly available for slots 1 and 2.

In addition to time slot arrangements, we gave the tutorial teacher two options on presentation, namely a pre-recorded and a fully interactive form of tutorials. The former allows the teachers to pre-record the main content of their tutorials, and use the time slots mainly for question answering and detailed discussions. The latter is more interactive, where the teachers use the online sessions for both giving the lecture and other activities such as active discussions. All these arrangements are made for taking advantage of the virtual conference. In the end the choices were the following:

<blockquote>T1 – pre-recorded

T2 – interactive (the initial decision was pre-recorded but a change was made in June)

T3 – pre-recorded

T4 – pre-recorded

T5 – pre-recorded

T6 – interactive

T7 – interactive

T8 – interactive
</blockquote>
After making decisions on time slots and teaching forms, we started to work on the technical details of the virtual conference infrastructure jointly with the infrastructure chairs, the website chairs, the SlidesLive support team and individual tutorial teachers. We collected needs from each tutorial from time to time, inquired information from the SlidesLive team, and tried to accommodate specific requirement for each tutorial. Since this is the first time ACL was virtual, it typically took several rounds of information exchange for making a certain technical feature work. Major things discussed included:

* Sorting out how to pre-record a tutorial using the system
* Recording of interactive sessions
* Additional RocketChat channel
* Virtual tutorial webpage setup
* Coordinators for tutorial sessions and SlidesLive team participation
* Rehearsal of interactive sessions
* Captioning of pre-recorded and interactive tutorials
* Privacy issues
* Special needs from particular tutorials
** Integrating [https://www.dory.app/ Dory] for question answering (in T1)
** Fully interactive sessions, with occasional split into break-out rooms (in T7)

To ensure smooth running of the sessions, we sent two rounds of reminders to each tutorial to ensure that each teacher is registered to the conference. We collected rough timelines of each tutorial so that the SlidesLive team can work closely during the live sessions. To further help the teachers prepare their live session content we additionally collected the lists of registered attendees twice before the conference, and shared them with the teachers.

In the end, the numbers of registrations for each tutorial were:

<blockquote>T1 -- 1762

T2 -- 793

T3 -- 503

T4 -- 571

T5 -- 641

T6 -- 1288

T7 -- 378

T8 – 1087

===Archiving and Publicity===

There are three main channels for publicity, namely the static [http://www.acl2020.org/program/tutorials/ website], the website of the [https://virtual.acl2020.org/tutorials.html virtual conference] and the social media. For the former, we collected photos from each instructor, which are shown with the title and abstract of their tutorials. Following NAACL 2019, we also displayed the time slots as a right-hand-side bar on the tutorial website. Due to special arrangements of the virtual conference, we marked the time zones explicitly. In addition, we asked the tutorial teachers to provide URL links to teaching materials (e.g., at GitHub) during their final submission, and included these links to their tutorial information. For social media, we drafted two blog posts (i.e., [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ this post] and [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]) for advertising the tutorials.

The archiving consists of three main items, namely the tutorial proceedings, the slides for each tutorial and the videos. For the tutorial proceedings, we collected final abstracts from the authors using the START system. We asked the instructors to sign the CC BY v4 agreement when submitting their final materials on SoftConf. For the slides, we asked the tutorial teachers to provide a first version of the slides by June 17 regardless whether they are pre-recorded or interactive. Some tutorials made further updates to their slides before ACL. For the video recordings, we worked with the ACL Anthology team discussing the technical details since this is the first time the tutorial videos are shared in the website. [Note: this is currently undergoing at the time of writing of this summary]

===Lessons Learned===

While organizing a virtual conference it is important to stress that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]).

It is also important to announce the individual modalities with the attendees early enough. For instance, watching pre-recordings in advance was necessary for some tutorials (since their live sessions only included question answering) and such constrains should have been known more in advance.

If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to Zoom rooms only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.

One thing to remember which we missed is to publish the reading list for each tutorial (which the teachers submit with the proposal) on the tutorial website.

Tutorial chair handbook

2020-07-23T09:58:10Z

AgataSavary:

== Duties ==
Tutorial chairs are nominated by the general chair of the main conference. Their duties are described in [[Tutorial chair duties|here]].

== Timeline ==
The timeline for the organisation of tutorials should follow the general [[conference planning schedule]]. Below, we complete this schedule with more detailed tasks, as defined for the ACL 2020 tutorials. The dates from the [[conference planning schedule]] are highlighted in '''bold'''. 
In recent years, calls and selection of tutorial proposals have been coordinated among some major CL conferences (usually a subset of ACL, NAACL, EMNLP, AACL and COLING). Note that if your conference is the first in the year to come, you will need to be the major driving force in the initial steps described below. 
Tutorial proposals are usually reviewed by the tutorial chairs of the coordinating conferences, but it may be a good idea to also rectruit a small group of external reviewers. They should have a good experience in reviewing and a large understanding of the recent progress in the CL domain.

* [T-12]
** Learn from the general chair which conferences will have their tutorials coordinated with yours.
** Establish contact with the tutorial chairs of these other conferences, and discuss
*** Recruitment of external reviewers
*** Proposal evaluation criteria
*** Soliciting tutorial proposals from particular potential lecturers on particular topics
*** Organizing a public poll about interest in the submitted tutorial
** Discuss general strategies of the main conference with the general chair and the program chairs
***promotion of some topics
***mailing lists and conference alert portals to be used for all announcements
** Set up a SoftConf space for proposal submission
*** Solicit the creation of a SoftConf space from [mailto:support@softconf.com support] (currently Rich Gerber). Note that submissions will be joint for several conferences but the proceedings will be edited for each conference separately. Therefore, a new SoftConf space will have to be created later for the selected proposals of each conference.
*** If you plan to have external tutorial reviewers, invite them via SoftConf.
*** Set up a submission page.
** Set up a joint email for tutorial chairs.
** Confirm the number (usually 6-8) and length (usually 3 hours) of tutorial slots with the ACL Business Manager (currently Priscilla Rasmussen).
** Draft the joint call for tutorials (based on [[#Past calls for tutorials|past calls]]).

* '''[T-11]''' ''(in the future, it would be good to extend the time slot between the call and the deadline to 2 months at least, especially if July/August is included)''
** Send out the call for tutorials.
** For posting the call on the ACL website, contact the ACL Business Manager (currently Priscilla Rasmussen).
** If you planned to encourage submission of proposals from particular potential lecturers, contact them now.
** Set up a SoftConf page for proposal reviews.

* [T-10]
** Deadline for tutorial proposal submission.
** If you decided to run a public poll about interest in the submitted tutorial, set it up and send out a call for expression of interest.
** Assign proposals to reviewers, send out invitations to review (via SoftConf).

* '''[T-9]'''
** Select tutorials jointly with other conference chairs.
** Notify the authors (via SoftConf).
** Liaise with local chair for space needs and scheduling.
** Send instructions to tutorial teachers about printed material, agenda/time management, payments, fees, etc.
** Liaise the conference website chair for updates about tutorials and authors' guidelines.
** Liaise with the ACL Office for the registration form.

* '''[T-4]'''
** Collect short descriptions for proceedings from tutorial teachers.
** Provide them to the webmaster and the general and program chairs .

* '''[T-3]'''
** Confirm the tutorials with authors.
** Authors submit the final tutorial descriptions for the proceedings.
** Arrange for printing of tutorial materials.

* [T-2]
** Arrange with the [mailto:support@softconf.com SoftConf support] the creation of a separate SoftConf space for the selected tutorials of your conference only. Indicate which tutorial submissions should be copied there.
** Set up the final submission form in the new SoftConf space. Include the submissions of slides, which will be published on the ACL Anthology. Include signing the CC-BY-4.0 license agreement.

* '''[T-1]'''
** Collect and check final tutorial materials.
** Create the tutorial proceedings, including slides.

* '''[T-0]''' Conference and tutorials

* [T+1]
** Invite the lecturers to update the slides at the ACL Anthology via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form] (under CC-BY-4.0).
** Update this tutorial page and the list of [[Past tutorials|past tutorials]] with the data from your conference. For access to the ACL [[Conference Handbook]], contact the ACL Secretary (currently Shiqi Zhou).

== Tutorials in a virtual conference ==
ACL 2020 was transformed into a fully virtual conference due to the COVID-19 pandemic. This decision was taken only 3 months before the conference and added a bunch of new tasks on top of the usual organization. This new organization was largely to be invented from scratch, additionally to the previous experience from [https://icml.cc/ ICML]. SlideLive was the subcontractor for the virtual infrastructure.

The tasks for tutorials included:
* collecting slides and pre-recordings from the tutorial teachers (recorded with the SlideLive online software)
* defining [https://acl2020.org/_pages/docs/ACL_2020_Tutorial_Timeslots.pdf time slots] which would increase participation across various time zones
* collecting teachers' preferences for various time slots and encouraging them to give their tutorials twice
* setting up the [https://acl2020.org/program/tutorials/ schedule]
* informing the participants about [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ virtual participation] and about the [https://acl2020.org/blog/detailed-modalities-of-tutorials/ detailed modalities] for each tutorial
* defining the contents of live sessions (a mixture of pre-recordings, question-answering, break-out rooms for one tutorial, etc.)
* creating automatic captioning, sending the captions to the teachers and uploading the edited versions
* dry runs with teachers and SlidesLive people
* archiving pre-recordings or recordings of the live sessions additionally to slides
More details are given in the [https://www.aclweb.org/adminwiki/index.php?title=2020Q3_Reports:_Tutorial_Chairs final report].

== Publication of tutorials ==
For NAACL 2019, ACL Anthology stores the tutorial descriptions, slides and video recordings. The same should go for ACL 2020 (which went fully virtual due to the COVID pandemic), except that recordings of live sessions are sometimes replaced with pre-recordings.

In tutorials organized until 2019, there was no homogeneous policy as to the publication of tutorial material. Data (abstracts, slides, code, etc.) from selected tutorials were usually scattered in 2 or 3 places:
* conference website
* ACL Anthology
* private repositories of the authors
The sustainability of these varies from venue to venue. The main websites of some conferences are not maintained, therefore ACL collects [https://www.aclweb.org/portal/node/2481 mirrors] of some of them (but not all).
The tutorial proceedings, published in the ACL Anthology, until recently included only abstracts but no tutorial slides.

As a result, for many [[past tutorials]], no slides are available online, so the community cannot benefit from these assets. It is, therefore, important to promote '''Open Access''' of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation '''slides in the ACL Anthology'''. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form].

It is recommended for tutorial chairs to:
* announce the publication of the tutorial slides under the CC-BY-4.0 license (used in the ACL Anthology)
* encourage the authors to also publish, under open licenses, their code and data used in the tutorials
* include the initial versions of tutorial slides, together with abstract, in the proceedings
* encourage the tutorial authors to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]

== Organization of past tutorials ==
For the last years, the call and selection of tutorials has been coordinated among several major CL conferences. The table below summarizes some data collected from existing reports and from direct contacts with past tutorial chairs.

{| border="1" cellspacing="0" cellpadding="5"
|'''Year'''
|'''Coordinated conferences'''
|'''Tutorial chairs'''
|'''Submission platform'''
|'''# Submissions'''
|'''Reviewers'''
|'''# Reviews per proposal'''
|'''# Selected tutorials'''
|'''Selection criteria'''
|'''Useful links'''
|-
|2020
|ACL EMNLP AACL-IJCNLP COLING
|ACL: Agata Savary (France), Yue Zhang (China) EMNLP: Benjamin Van Durme (USA), Aline Villavicencio (UK & Brazil) AACL-IJCNLP: Tim Baldwin (Australia), Fei Xia (USA) COLING: Daniel Beck (Australia), Lucia Specia (UK)
|SoftConf
|43
|Tutorial chairs + external experts
|3
|ACL: 8 EMNLP: 7 ACCL-IJCNLP: 6 COLING: 7
|Clarity and preparedness Novelty or timely character of the topic Lecturers' experience Likely audience interest Open access of the teaching material Diversity aspects (multilingualism, gender, age and country of the lecturers) Compatibility with the required venues
|[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 Call] [https://acl2020.org/program/tutorials/ ACL tutorials] [https://2020.emnlp.org/tutorials EMNLP tutorials] [http://aacl2020.org/ ACCL-IJCNLP tutorials] [https://coling2020.org/pages/tutorials COLING tutorials]
|
|-
|2019
|NAACL ACL EMNLP
|ACL: Preslav Nakov (Qatar), Alexis Palmer (USA) NAACL: Anoop Sarkar (Canda), Michael Strube (Germany) EMNLP-IJCNLP: Marine Carpuat (USA), Tim Baldwin (Australia)
|SoftConf
|45
|Tutorial chairs
|2-3
|ACL: 9 NAACL: 6 EMNLP: 7
|Presenters (1-5) Description (1-5) Interest Level (1-5) Preparedness (1-5)
|[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials Call] [http://www.acl2019.org/EN/tutorials.xhtml ACL tutorials] [https://naacl2019.org/program/tutorials/ NAACL tutorials]
|-
|2018
|ACL NAACL EMNLP COLING
|ACL: Yoav Artzi (USA), Jacob Eisenstein (USA) COLING: Pascale Fung (Hong Kong), Donia Scott (UK), Marilyn Walker (USA) EMNLP: Mausam (India), Lu Wang (USA) NAACL: Mohit Bansal (USA), Rebecca Passonneau (USA)
|SoftConf
|49
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6 COLING: 6
|Quality of plan Diversity of topics Qualifications of presenter likely audience interest
|[https://acl2018.org/2017/09/11/joint-call-for-tutorials/ Call] 
[https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs Report] 
[https://acl2018.org/tutorials/ ACL tutorials] 
[https://emnlp2018.org/program/tutorials/ EMNLP tutorials] 
[https://coling2018.org/tutorial-list/ COLING tutorials]
|-
|2017
|ACL EACL EMNLP
|ACL: Maja Popović (Germany), Jordan Boyd-Graber (USA) EACL: Alex Klementiev (Germany), Lucia Specia (UK) EMNLP: Nathan Schneider (USA), Alexandra Birch (UK)
|SoftConf
|26
|Tutorial chairs
|3
|ACL: 6 EACL: 6 EMNLP: 7
|Clarity (description and outline) Topic
|[https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs Report] [http://acl2017.org/tutorials/ ACL tutorials] [http://eacl2017.org/index.php/tutorials EACL tutorials]
|-
|2016
|ACL NAACL EMNLP
|ACL: Alexandra Birch (UK), Willem Zuidema (Netherlands) NAACL: Alexander Rush and Bishan Yang EMNLP: Rebecca Hwa and Mohit Bansal
|SoftConf
|32
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6
|Relevance to ACL community (15) Quality of instructor (15) Outline and depth/breadth of the proposal (15) 
Potential attendance Newly emerging area not previously covered in an ACL-related tutorial (yes or no) Introduction into related fields (yes or no) 
Overall score (15) 1st preference for venue as indicated in proposal comments
|[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 Call] [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs Report] [http://naacl.org/naacl-hlt-2016/tutorials.html NAACL tutorials] [https://www.aclweb.org/mirror/emnlp2016/tutorials.html EMNLP tutorials]
|}

== Past calls for tutorials ==
*[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 ACL/AACL-IJCNLP/EMNLP/COLING 2020 joint call for tutorials]
*[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials NAACL/ACL/EMNLP 2019 joint call for tutorials]
*[https://www.aclweb.org/portal/content/final-joint-call-tutorial-proposals-aclcolingemnlpnaacl-2018 ACL/COLING/EMNLP/NAACL 2018 joint call for tutorials]
*[https://www.aclweb.org/portal/content/eacl-2017-call-tutorial-proposals EACL 2017 call for tutorials]
*[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 NAACL/ACL/EMNLP 2016 joint call for tutorials]
*[https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-naacl-2015-acl-2015-emnlp-2015-0 NAACL/ACL/EMNLP 2015 joint call for tutorials]

== Reports from past tutorial chairs ==
*[[2020Q3_Reports:_Tutorial_Chairs|ACL 2020 report]] (by Agata Savary and Yue Zhang)
*[[2019Q1_Reports:_ACL_2019#Tutorial_Co-Chairs|ACL 2019 report]] (by Preslav Nakov and Alexis Palmer)
*[[2018Q3 Reports: Tutorial Chairs|ACL 2018 report]] (by Yoav Artzi and Jacob Eisenstein)
*[[2017Q3 Reports: Tutorial Chairs|ACL 2017 report]] (by Maja Popović and Jordan Boyd-Graber)
*[[2016Q3 Reports: Tutorial Chairs|ACL 2016 report]] (by Alexandra Birch and Willem Zuidema)

== Lessons learned by past tutorial chairs ==
* [Agata Savary, 2020] If the conference is virtual:
** Remember that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]).
** Announce the individual modalities with the attendees early enough (for instance, if watching pre-recordings in advance is necessary for some tutorials because their live sessions only included question answering)
** If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to Zoom rooms only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.
* [Agata Savary, 2020] Use a common SoftConf space for joint tutorial submissions for all coordinated conference. Once the proposals have been selected and assigned to conferences, use one track per conference in the same SoftConf space.
* [Agata Savary, 2020] Publish the list of recommended readings on each tutorial page
* [Agata Savary, 2020] A good way to proceed while selecting tutorials for several venues is to have a common online meeting, in which
** Tutorials are clustered per themes (to distribute similar tutorials over different conferences)
** The chairs for each venue select their best choices in a round robin manner.
** Generate csv files with reviews and scores: Manager -> Other tools -> Spreadsheet maker -> Reports
* [Agata Savary, 2020] While setting up a review form on SoftConf, include:
** All evaluation criteria, including diversity, in the review form
** Cutting edge vs. introductory category
** Duration (3 or 6 hours)
* [Agata Savary, 2020] Currently, the generic conference schedule assumes only 2 months between the call for tutorials and notification to proposers. If a solid 1-month review procedure is to take place, this means, that proposers only have 1 month between the call and the submission deadline. It would be good to shift the call to [T-12].
* [Agata Savary, 2020] For many past tutorials, no slides are available online. It is, therefore, important to promote Open Access of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation slides on the ACL Anthology. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]. As a tutorial chair, make sure that
** initial versions of tutorial slides are submitted to the proceedings together with the tutorial abstracts
** tutorial authors are encouraged to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]
* [Marine Capuat, 2019] Continue soliciting both cutting edge and introductory tutorials
* [Marine Capuat, 2019] Ask proposers to express their preferred location/conference in one place (i.e. either in proposal pdf, or in softconf form, not both) to avoid inconsistencies
* [ACL 2018 chairs] Issues with proceedings edition, when one joint SoftConf space is used for many conferences (see the [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs report])
* [ACL 2016 chairs] Avoid confusion about the submission procedures for the final tutorial descriptions, those were submitted by email in many different formats

Author: Agata Savary, July 2019-July 2020

Updates are welcome

[[Category:Conference Handbook]]

2020Q3 Reports: Tutorial Chairs

2020-07-23T09:56:28Z

AgataSavary:

Tutorial Chairs:

<blockquote>Agata Savary, University of Tours, France

Yue Zhang, Westlake University, Hangzhou, China
</blockquote>

=== Preparatory Work, Call, Reviewing and Decision-Making ===
The call, submission, reviewing and selection of tutorials was coordinated jointly for 4 conferences: ACL, AACL-IJCNLP, COLING and EMNLP.

Before drafting the call, we collected lists of tutorials offered within the past 4 years. We analysed previous calls for tutorials and reports from tutorial chairs (from [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs 2016], [https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs 2017], [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs 2018] and [http://aclweb.org/adminwiki/index.php?title=2019Q1_Reports:_ACL_2019 2019]). We consulted previous tutorial chairs with a questionnaire including questions about: the number of submissions, encouraging submissions on specific topics or from specific lecturers, the review procedure, the evaluation criteria, the post-tutorial availability of the slides/codes, and lessons learned from tutorial coordination. We also discussed the publication of slides and video recordings from future tutorials with the persons in charge of the ACL Anthology. As a result of these steps, we created two new sections for the [https://www.aclweb.org/adminwiki/index.php?title=Conference_Handbook ACL Conference Handbook] (future chairs might consider updating these documents yearly):

* the list of [https://www.aclweb.org/adminwiki/index.php?title=Past_tutorials past tutorials] at ACL, COLING, EACL, EMNLP, and NAACL in 2016-2019
* a [https://www.aclweb.org/adminwiki/index.php?title=Tutorial_chair_handbook tutorial chair handbook]

The final [https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020 call] differs from previous calls in several aspects: (i) the expectations about tutorial proposals were made clearer, (ii) following the central ACL decision, the teachers' payment policy was replaced by a fee-waiving policy, (iii) the required submission details include two new items: diversity considerations and agreement for open access publication of slides, codes, data and video recordings, (iv) the evaluation criteria (see below) are announced.

We recruited a review committee of 19 members, including the 8 tutorial chairs and 11 external members selected for their large understanding of the NLP domain and a good experience in reviewing and/or tutorial teaching:

'''Review Committee'''

* Timothy Baldwin (University of Melbourne, Australia) - AACL-IJCNLP 2020 tutorial chair
* Daniel Beck (University of Melbourne, Australia) - COLING 2020 tutorial chair
* Emily M. Bender (University of Washington, WA, USA)
* Erik Cambria (Nanyang Technological University, Singapore)
* Gaël Dias (University of Caen Normandie, France)
* Stefan Evert (Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany)
* Yang Liu (Tsinghua University, Beijing, China)
* Agata Savary (University of Tours, France) - ACL 2020 tutorial chair
* João Sedoc (Johns Hopkins University, Baltimore, MD, USA)
* Lucia Specia (Sheffield University, UK) - COLING 2020 tutorial chair
* Xu Sun (Peking University, China)
* Yulia Tsvetkov (Carnegie Mellon University, Pittsburgh, PA, USA)
* Benjamin Van Durme (Johns Hopkins University, Baltimore, MD, USA) - EMNLP 2020 tutorial chair
* Aline Villavicencio (University of Sheffield, UK and Federal University of Rio Grande do Sul, Brazil) - EMNLP 2020 tutorial chair
* Taro Watanabe (Google, Inc., Tokyo, Japan)
* Aaron Steven White (University of Rochester, NY, USA)
* Fei Xia (University of Washington, WA, USA) - AACL-IJCNLP 2020 tutorial chair
* Yue Zhang (Westlake University, Hangzhou, China) - ACL 2020 tutorial chair
* Meishan Zhang (Tianjin University, China)

In total, we received 43 submissions for the 4 conferences. Each reviewer was assigned 6-7 proposals and each proposal received 3 reviews. The selection criteria included: clarity and preparedness, novelty or timely character of the topic, lecturers' experience, likely audience interest, open access of the teaching material, diversity aspects (multilingualism, gender, age and country of the lecturers), and compatibility with the preferred venues. We accepted 31 proposals.

The decision making was handled via an online meeting of the 8 tutorial chairs. In particular, the selection of tutorials for each conference was done via the expression of interest of the tutorial chairs on a round-robin basis. Some slight adjustments were also performed after the meeting to better fit the authors' preferences. In total, 8, 8, 8 and 7 proposals were selected for ACL, AACL-IJCNLP, COLING and EMNLP, respectively. Upon the announcement the results, 2 of the proposals accepted for AACL-IJCNLP were withdrawn.

The submission, review, selection and collection of final descriptions for all tutorials was handled via a dedicated SoftConf space, shared by the 4 coordinating conferences. After the selection of proposals, a separate track was created on SoftConf for each conference. The final submission page (one per conference) was set up so as to collect all the necessary data including notably: the tutorial slides, URLs for course material (if any), printable material (if any) and agreement for open access publication.

===Tutorials Selected for ACL 2020===

The final selection for ACL 2020 consists of the following 8 tutorials of 3 hours each (each of them had ACL as the preferred or the second preferred venue):

'''T1: Interpretability and Analysis in Neural NLP''' (cutting-edge) 
Yonatan Belinkov, Sebastian Gehrmann and Ellie Pavlick 
While deep learning has transformed the NLP field and impacted the larger computational linguistics community, the rise of neural networks is stained by their opaque nature: It is challenging to interpret the inner workings of neural network models, and explicate their behavior. Therefore, in the last few years, an increasingly large body of work has been devoted to the analysis and interpretation of neural network models in NLP. 
This body of work is so far lacking a common framework and methodology. Moreover, approaching the analysis of modern neural networks can be difficult for newcomers to the field. This tutorial aims to fill this gap and introduce the nascent field of interpretability and analysis of neural networks in NLP. 
The tutorial covers the main lines of analysis work, such as probing classifier, behavior studies and test suites, psycholinguistic methods, visualizations, adversarial examples, and other methods. We highlight not only the most commonly applied analysis methods, but also the specific limitations and shortcomings of current approaches, in order to inform participants where to focus future efforts.

'''T2: Multi-modal Information Extraction from Text, Semi-structured, and Tabular Data on the Web''' (cutting-edge) 
Xin Luna Dong, Hannaneh Hajishirzi, Colin Lockard and Prashant Shiralkar 
The World Wide Web contains vast quantities of textual information in several forms: unstructured text, template-based semi-structured webpages (which present data in key-value pairs and lists), and tables. Methods for extracting information from these sources and converting it to a structured form have been a target of research from the natural language processing (NLP), data mining, and database communities. While these researchers have largely separated extraction from web data into different problems based on the modality of the data, they have faced similar problems such as learning with limited labeled data, defining (or avoiding defining) ontologies, making use of prior knowledge, and scaling solutions to deal with the size of the Web. 
In this tutorial we take a holistic view toward information extraction, exploring the commonalities in the challenges and solutions developed to address these different forms of text. We will explore the approaches targeted at unstructured text that largely rely on learning syntactic or semantic textual patterns, approaches targeted at semi-structured documents that learn to identify structural patterns in the template, and approaches targeting web tables which rely heavily on entity linking and type information. 
While these different data modalities have largely been considered separately in the past, recent research has started taking a more inclusive approach toward textual extraction, in which the multiple signals offered by textual, layout, and visual clues are combined into a single extraction model made possible by new deep learning approaches. At the same time, trends within purely textual extraction have shifted toward full-document understanding rather than considering sentences as independent units. With this in mind, it is worth considering the information extraction problem as a whole to motivate solutions that harness textual semantics along with visual and semi-structured layout information. We will discuss these approaches and suggest avenues for future work.

'''T3: Reviewing Natural Language Processing Research''' (introductory) 
Kevin Cohen, Karën Fort, Margot Mieskes and Aurélie Névéol 
As the demand for reviewing grows, so must the pool of reviewers. As the [http://www.livecongress.it/aol/indexSA.php?id=E2EAED7D&ticket= survey] presented by Graham Neubig at the 2019 ACL showed, a considerable number of reviewers are junior researchers, who might lack the experience and expertise necessary for high-quality reviews. Some of them might not have the environment or lack opportunities that allow them to learn the skills necessary. A tutorial on reviewing for the NLP community might increase reviewers’ confidence, as well as the quality of the reviews. This introductory tutorial will cover the goals, processes, and evaluation of reviewing research papers in natural language processing.

'''T4: Stylized Text Generation: Approaches and Applications''' (cutting-edge) 
Lili Mou and Olga Vechtomova 
Text generation has played an important role in various applications of natural language processing (NLP), and kn recent studies, researchers are paying increasing attention to modeling and manipulating the style of the generation text, which we call stylized text generation. In this tutorial, we will provide a comprehensive literature review in this direction. We start from the definition of style and different settings of stylized text generation, illustrated with various applications. Then, we present different settings of stylized generation, such as parallel supervised, style label-supervised, and unsupervised. In each setting, we delve deep into machine learning methods, including embedding learning techniques to represent style}, adversarial learning and reinforcement learning with cycle consistency to match content but to distinguish different styles. We also introduce current approaches of evaluating stylized text generation systems. We conclude our tutorial by presenting the challenges of stylized text generation and discussing future directions, such as small-data training, non-categorical style modeling, and a generalized scope of style transfer (e.g., controlling the syntax as a style).

'''T5: Achieving Common Ground in Multi-modal Dialogue''' (cutting-edge) 
Malihe Alikhani and Matthew Stone 
All communication aims at achieving common ground (grounding): interlocutors can work together effectively only with mutual beliefs about what the state of the world is, about what their goals are, and about how they plan to make their goals a reality. Computational dialogue research offers some classic results on grouding, which unfortunately offer scant guidance to the design of grounding modules and behaviors in cutting-edge systems. In this tutorial, we focus on three main topic areas: 1) grounding in human-human communication; 2) grounding in dialogue systems; and 3) grounding in multi-modal interactive systems, including image-oriented conversations and human-robot interactions. We highlight a number of achievements of recent computational research in coordinating complex content, show how these results lead to rich and challenging opportunities for doing grounding in more flexible and powerful ways, and canvass relevant insights from the literature on human--human conversation. We expect that the tutorial will be of interest to researchers in dialogue systems, computational semantics and cognitive modeling, and hope that it will catalyze research and system building that more directly explores the creative, strategic ways conversational agents might be able to seek and offer evidence about their understanding of their interlocutors.

'''T6: Commonsense Reasoning for Natural Language Processing''' (introductory) 
Maarten Sap, Vered Shwartz, Antoine Bosselut, Dan Roth and Yejin Choi 
In our tutorial, we (1) outline the various types of commonsense (e.g., physical, social), and (2) discuss techniques to gather and represent commonsense knowledge, while highlighting the challenges specific to this type of knowledge (e.g., reporting bias). We will then (3) discuss the types of commonsense knowledge captured by modern NLP systems (e.g., large pretrained language models), and (4) present ways to measure systems' commonsense reasoning abilities. We finish with (5) a discussion of various ways in which commonsense reasoning can be used to improve performance on NLP tasks, exemplified by an (6) interactive session on integrating commonsense into a downstream task.

'''T7: Integrating Ethics into the NLP Curriculum''' (introductory) 
Emily M. Bender, Dirk Hovy and Alexandra Schofield 
Our goal in this tutorial is to empower NLP researchers and practitioners with tools and resources to teach others about how to ethically apply NLP techniques. Our tutorial will present both high-level strategies for developing an ethics-oriented curriculum, based on experience and best practices, as well as specific sample exercises that can be brought to a classroom. We plan to make this a highly interactive work session culminating in a shared online resource page that pools lesson plans, assignments, exercise ideas, reading suggestions, and ideas from the attendees. We consider three primary topics with our session that frequently underlie ethical issues in NLP research: Dual use, bias and privacy. 
In this setting, a key lesson is that there is no single approach to ethical NLP: each project requires thoughtful consideration about what steps can be taken to best support people affected by that project. However, we can learn (and teach) what kinds of issues to be aware of and what kinds of strategies are available for mitigating harm. To teach this process, we apply and promote interactive exercises that provide an opportunity to ideate, discuss, and reflect. We plan to facilitate this in a way that encourages positive discussion, emphasizing the creation of ideas for the future instead of negative opinions of previous work.

'''T8: Recent Advances in Open-Domain Question Answering''' (cutting-edge) 
Danqi Chen and Scott Wen-tau Yih 
Open-domain (textual) question answering (QA), the task of finding answers to open-domain questions by searching a large collection of documents, has been a long-standing problem in NLP, information retrieval (IR) and related fields (Voorhees et al., 1999; Moldovan et al., 2000; Brill et al.,2002; Ferrucci et al., 2010). Traditional QA systems were usually constructed as a pipeline, consisting of many different components such as question processing, document/passage retrieval and answer processing. With the rapid development of neural reading comprehension (Chen, 2018), modern open-domain QA systems have been restructured by combining traditional IR techniques and neural reading comprehension models (Chen et al., 2017; Yang et al., 2019) or even implemented in a fully end-to-end fashion (Lee et al., 2019; Seo et al., 2019). While the system architecture has been drastically simplified, two technical challenges remain critical:(1) “Retriever”: finding documents that (might)contain an answer from a large collection of documents; (2) “Reader”: finding the answer in a given paragraph or a document. 
In this tutorial, we aim to provide a comprehensive and coherent overview of recent advances in this line of research. We will start by first giving a brief historical background of open-domain question answering, discussing the basic setup and core technical challenges of the research problem.The focus will then shift to modern techniques and resources proposed for open-domain QA, including the basics of latest neural reading comprehension systems, new datasets and models. The scope will also be broadened to cover the information retrieval component on how to effectively identify passages relevant to the questions. Moreover, in-depth discussions will be given on the use of traditional / neural IR modules, as well as the trade-offs between modular design and end-to-end training. If time permits, we also plan to discuss some hybrid approaches for answering questions using both text and large knowledge bases (e.g. (Sun et al., 2018)) and give a critical review on how structured data complements the information from unstructured text. 
At the end of our tutorial, we will discuss some important questions, including (1) How much progress have we made compared to the QA systems developed in the last decade?(2) What are the main challenges and limitations of cur-rent approaches? (3) How to trade off the efficiency (computational time and memory requirements) and accuracy in the deep learning era? We hope that our tutorial will not only serve as a useful resource for the audience to efficiently acquire the up-to-date knowledge, but also provide new perspectives to stimulate the advances of open-domain QA research in the next phase.

===Virtual Conference Organization===

In April the whole conference was decided to be held in virtual. We carried on the preparations jointly with the tutorial instructors, the virtual infrastructure chairs, the website chairs and the SlidesLive team to set up the virtual conference infrastructure, and to publicize it for each tutorial.

We first made a decision on the time slots for tutorial presentations. Five major time zones were considered, including US west coast, US east coast, central Europe, China/Asia and Australia (India was also considered). Our aim was to offer at least two convenient time slots (i.e. within regular working hours) for each of these zones.

The final time slots include (Jul 5 Pacific Time):

<blockquote>slot 1 – 7:00 PM (-1D) to 10:30 PM (-1D)

slot 2 – 12:00 AM to 3:30 AM

slot 3 – 6:00 AM to 9:30 AM

slot 4 – 10:30 AM to 2:00 PM

slot 5 – 3:00 PM to 6:30 PM
</blockquote>
We asked the tutorial teachers to indicate their preferences among the time slots, and – if possible – to agree to give their tutorial twice, so as to increase accessibility. Our final decisions on slot assignments traded off the teacher preferences and slot availability. In the end, the assignments were the following (see also the [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ blog post] and [https://acl2020.org/program/tutorials/ online schedule]):

<blockquote>T1 – twice on slots 3 and 4

T2 – once on slot 5

T3 – once on slot 4

T4 – once on slot 4

T5 – twice on slots 3 and 5

T6 – once on slot 5

T7 – twice on slots 3 and 4

T8 – once on slot 5
</blockquote>
Unfortunately, the tutorial teachers proved hardly available for slots 1 and 2.

In addition to time slot arrangements, we gave the tutorial teacher two options on presentation, namely a pre-recorded and a fully interactive form of tutorials. The former allows the teachers to pre-record the main content of their tutorials, and use the time slots mainly for question answering and detailed discussions. The latter is more interactive, where the teachers use the online sessions for both giving the lecture and other activities such as active discussions. All these arrangements are made for taking advantage of the virtual conference. In the end the choices were the following:

<blockquote>T1 – pre-recorded

T2 – interactive (the initial decision was pre-recorded but a change was made in June)

T3 – pre-recorded

T4 – pre-recorded

T5 – pre-recorded

T6 – interactive

T7 – interactive

T8 – interactive
</blockquote>
After making decisions on time slots and teaching forms, we started to work on the technical details of the virtual conference infrastructure jointly with the infrastructure chairs, the website chairs, the SlidesLive support team and individual tutorial teachers. We collected needs from each tutorial from time to time, inquired information from the SlidesLive team, and tried to accommodate specific requirement for each tutorial. Since this is the first time ACL was virtual, it typically took several rounds of information exchange for making a certain technical feature work. Major things discussed included:

* Sorting out how to pre-record a tutorial using the system
* Recording of interactive sessions
* Additional RocketChat channel
* Virtual tutorial webpage setup
* Coordinators for tutorial sessions and SlidesLive team participation
* Rehearsal of interactive sessions
* Captioning of pre-recorded and interactive tutorials
* Privacy issues
* Special needs from particular tutorials
** Integrating [https://www.dory.app/ Dory] for question answering (in T1)
** Fully interactive sessions, with occasional split into break-out rooms (in T7)

To ensure smooth running of the sessions, we sent two rounds of reminders to each tutorial to ensure that each teacher is registered to the conference. We collected rough timelines of each tutorial so that the SlidesLive team can work closely during the live sessions. To further help the teachers prepare their live session content we additionally collected the lists of registered attendees twice before the conference, and shared them with the teachers.

In the end, the numbers of registrations for each tutorial were:

<blockquote>T1 -- 1762

T2 -- 793

T3 -- 503

T4 -- 571

T5 -- 641

T6 -- 1288

T7 -- 378

T8 – 1087

===Archiving and Publicity===

There are three main channels for publicity, namely the static [http://www.acl2020.org/program/tutorials/ website], the website of the [https://virtual.acl2020.org/tutorials.html virtual conference] and the social media. For the former, we collected photos from each instructor, which are shown with the title and abstract of their tutorials. Following NAACL 2019, we also displayed the time slots as a right-hand-side bar on the tutorial website. Due to special arrangements of the virtual conference, we marked the time zones explicitly. In addition, we asked the tutorial teachers to provide URL links to teaching materials (e.g., at GitHub) during their final submission, and included these links to their tutorial information. For social media, we drafted two blog posts (i.e., [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ this post] and [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]) for advertising the tutorials.

The archiving consists of three main items, namely the tutorial proceedings, the slides for each tutorial and the videos. For the tutorial proceedings, we collected final abstracts from the authors using the START system. We asked the instructors to sign the CC BY v4 agreement when submitting their final materials on SoftConf. For the slides, we asked the tutorial teachers to provide a first version of the slides by June 17 regardless whether they are pre-recorded or interactive. Some tutorials made further updates to their slides before ACL. For the video recordings, we worked with the ACL Anthology team discussing the technical details since this is the first time the tutorial videos are shared in the website. [Note: this is currently undergoing at the time of writing of this summary]

===Lessons Learned===

While organizing a virtual conference it is important to stress that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]).

It is also important to announce the individual modalities with the attendees early enough. For instance, watching pre-recordings in advance was necessary for some tutorials (since their live sessions only included question answering) and such constrains should have been known more in advance.

If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to Zoom rooms only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.

One thing to remember which we missed is to publish the reading list for each tutorial (which the teachers submit with the proposal) on the tutorial website.

2020Q3 Reports: Tutorial Chairs

2020-07-23T09:54:18Z

AgataSavary:

Tutorial Chairs:

<blockquote>Agata Savary, University of Tours, France

Yue Zhang, Westlake University, Hangzhou, China
</blockquote>

=== Preparatory Work, Call, Reviewing and Decision-Making ===
The call, submission, reviewing and selection of tutorials was coordinated jointly for 4 conferences: ACL, AACL-IJCNLP, COLING and EMNLP.

Before drafting the call, we collected lists of tutorials offered within the past 4 years. We analysed previous calls for tutorials and reports from tutorial chairs (from [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs 2016], [https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs 2017], [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs 2018] and [http://aclweb.org/adminwiki/index.php?title=2019Q1_Reports:_ACL_2019 2019]). We consulted previous tutorial chairs with a questionnaire including questions about: the number of submissions, encouraging submissions on specific topics or from specific lecturers, the review procedure, the evaluation criteria, the post-tutorial availability of the slides/codes, and lessons learned from tutorial coordination. We also discussed the publication of slides and video recordings from future tutorials with the persons in charge of the ACL Anthology. As a result of these steps, we created two new sections for the [https://www.aclweb.org/adminwiki/index.php?title=Conference_Handbook ACL Conference Handbook] (future chairs might consider updating these documents yearly):

<ul>
<li><blockquote>the list of [https://www.aclweb.org/adminwiki/index.php?title=Past_tutorials past tutorials] at ACL, COLING, EACL, EMNLP, and NAACL in 2016-2019</blockquote></li>
<li><blockquote>a [https://www.aclweb.org/adminwiki/index.php?title=Tutorial_chair_handbook tutorial chair handbook]</blockquote></li></ul>

* the list of [https://www.aclweb.org/adminwiki/index.php?title=Past_tutorials past tutorials] at ACL, COLING, EACL, EMNLP, and NAACL in 2016-2019
* a [https://www.aclweb.org/adminwiki/index.php?title=Tutorial_chair_handbook tutorial chair handbook]

The final [https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020 call] differs from previous calls in several aspects: (i) the expectations about tutorial proposals were made clearer, (ii) following the central ACL decision, the teachers' payment policy was replaced by a fee-waiving policy, (iii) the required submission details include two new items: diversity considerations and agreement for open access publication of slides, codes, data and video recordings, (iv) the evaluation criteria (see below) are announced.

We recruited a review committee of 19 members, including the 8 tutorial chairs and 11 external members selected for their large understanding of the NLP domain and a good experience in reviewing and/or tutorial teaching:

'''Review Committee'''

<ul>
<li><blockquote>Timothy Baldwin (University of Melbourne, Australia) - AACL-IJCNLP 2020 tutorial chair</blockquote></li>
<li><blockquote>Daniel Beck (University of Melbourne, Australia) - COLING 2020 tutorial chair</blockquote></li>
<li><blockquote>Emily M. Bender (University of Washington, WA, USA)</blockquote></li>
<li><blockquote>Erik Cambria (Nanyang Technological University, Singapore)</blockquote></li>
<li><blockquote>Gaël Dias (University of Caen Normandie, France)</blockquote></li>
<li><blockquote>Stefan Evert (Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany)</blockquote></li>
<li><blockquote>Yang Liu (Tsinghua University, Beijing, China)</blockquote></li>
<li><blockquote>Agata Savary (University of Tours, France) - ACL 2020 tutorial chair</blockquote></li>
<li><blockquote>João Sedoc (Johns Hopkins University, Baltimore, MD, USA)</blockquote></li>
<li><blockquote>Lucia Specia (Sheffield University, UK) - COLING 2020 tutorial chair </blockquote></li>
<li><blockquote>Xu Sun (Peking University, China)</blockquote></li>
<li><blockquote>Yulia Tsvetkov (Carnegie Mellon University, Pittsburgh, PA, USA)</blockquote></li>
<li><blockquote>Benjamin Van Durme (Johns Hopkins University, Baltimore, MD, USA) - EMNLP 2020 tutorial chair</blockquote></li>
<li><blockquote>Aline Villavicencio (University of Sheffield, UK and Federal University of Rio Grande do Sul, Brazil) - EMNLP 2020 tutorial chair</blockquote></li>
<li><blockquote>Taro Watanabe (Google, Inc., Tokyo, Japan)</blockquote></li>
<li><blockquote>Aaron Steven White (University of Rochester, NY, USA)</blockquote></li>
<li><blockquote>Fei Xia (University of Washington, WA, USA) - AACL-IJCNLP 2020 tutorial chair</blockquote></li>
<li><blockquote>Yue Zhang (Westlake University, Hangzhou, China) - ACL 2020 tutorial chair</blockquote></li>
<li><blockquote>Meishan Zhang (Tianjin University, China)</blockquote></li></ul>

In total, we received 43 submissions for the 4 conferences. Each reviewer was assigned 6-7 proposals and each proposal received 3 reviews. The selection criteria included: clarity and preparedness, novelty or timely character of the topic, lecturers' experience, likely audience interest, open access of the teaching material, diversity aspects (multilingualism, gender, age and country of the lecturers), and compatibility with the preferred venues. We accepted 31 proposals.

The decision making was handled via an online meeting of the 8 tutorial chairs. In particular, the selection of tutorials for each conference was done via the expression of interest of the tutorial chairs on a round-robin basis. Some slight adjustments were also performed after the meeting to better fit the authors' preferences. In total, 8, 8, 8 and 7 proposals were selected for ACL, AACL-IJCNLP, COLING and EMNLP, respectively. Upon the announcement the results, 2 of the proposals accepted for AACL-IJCNLP were withdrawn.

The submission, review, selection and collection of final descriptions for all tutorials was handled via a dedicated SoftConf space, shared by the 4 coordinating conferences. After the selection of proposals, a separate track was created on SoftConf for each conference. The final submission page (one per conference) was set up so as to collect all the necessary data including notably: the tutorial slides, URLs for course material (if any), printable material (if any) and agreement for open access publication.

===Tutorials Selected for ACL 2020===

The final selection for ACL 2020 consists of the following 8 tutorials of 3 hours each (each of them had ACL as the preferred or the second preferred venue):

'''T1: Interpretability and Analysis in Neural NLP''' (cutting-edge) 
Yonatan Belinkov, Sebastian Gehrmann and Ellie Pavlick 
While deep learning has transformed the NLP field and impacted the larger computational linguistics community, the rise of neural networks is stained by their opaque nature: It is challenging to interpret the inner workings of neural network models, and explicate their behavior. Therefore, in the last few years, an increasingly large body of work has been devoted to the analysis and interpretation of neural network models in NLP. 
This body of work is so far lacking a common framework and methodology. Moreover, approaching the analysis of modern neural networks can be difficult for newcomers to the field. This tutorial aims to fill this gap and introduce the nascent field of interpretability and analysis of neural networks in NLP. 
The tutorial covers the main lines of analysis work, such as probing classifier, behavior studies and test suites, psycholinguistic methods, visualizations, adversarial examples, and other methods. We highlight not only the most commonly applied analysis methods, but also the specific limitations and shortcomings of current approaches, in order to inform participants where to focus future efforts.

'''T2: Multi-modal Information Extraction from Text, Semi-structured, and Tabular Data on the Web''' (cutting-edge) 
Xin Luna Dong, Hannaneh Hajishirzi, Colin Lockard and Prashant Shiralkar 
The World Wide Web contains vast quantities of textual information in several forms: unstructured text, template-based semi-structured webpages (which present data in key-value pairs and lists), and tables. Methods for extracting information from these sources and converting it to a structured form have been a target of research from the natural language processing (NLP), data mining, and database communities. While these researchers have largely separated extraction from web data into different problems based on the modality of the data, they have faced similar problems such as learning with limited labeled data, defining (or avoiding defining) ontologies, making use of prior knowledge, and scaling solutions to deal with the size of the Web. 
In this tutorial we take a holistic view toward information extraction, exploring the commonalities in the challenges and solutions developed to address these different forms of text. We will explore the approaches targeted at unstructured text that largely rely on learning syntactic or semantic textual patterns, approaches targeted at semi-structured documents that learn to identify structural patterns in the template, and approaches targeting web tables which rely heavily on entity linking and type information. 
While these different data modalities have largely been considered separately in the past, recent research has started taking a more inclusive approach toward textual extraction, in which the multiple signals offered by textual, layout, and visual clues are combined into a single extraction model made possible by new deep learning approaches. At the same time, trends within purely textual extraction have shifted toward full-document understanding rather than considering sentences as independent units. With this in mind, it is worth considering the information extraction problem as a whole to motivate solutions that harness textual semantics along with visual and semi-structured layout information. We will discuss these approaches and suggest avenues for future work.

'''T3: Reviewing Natural Language Processing Research''' (introductory) 
Kevin Cohen, Karën Fort, Margot Mieskes and Aurélie Névéol 
As the demand for reviewing grows, so must the pool of reviewers. As the [http://www.livecongress.it/aol/indexSA.php?id=E2EAED7D&ticket= survey] presented by Graham Neubig at the 2019 ACL showed, a considerable number of reviewers are junior researchers, who might lack the experience and expertise necessary for high-quality reviews. Some of them might not have the environment or lack opportunities that allow them to learn the skills necessary. A tutorial on reviewing for the NLP community might increase reviewers’ confidence, as well as the quality of the reviews. This introductory tutorial will cover the goals, processes, and evaluation of reviewing research papers in natural language processing.

'''T4: Stylized Text Generation: Approaches and Applications''' (cutting-edge) 
Lili Mou and Olga Vechtomova 
Text generation has played an important role in various applications of natural language processing (NLP), and kn recent studies, researchers are paying increasing attention to modeling and manipulating the style of the generation text, which we call stylized text generation. In this tutorial, we will provide a comprehensive literature review in this direction. We start from the definition of style and different settings of stylized text generation, illustrated with various applications. Then, we present different settings of stylized generation, such as parallel supervised, style label-supervised, and unsupervised. In each setting, we delve deep into machine learning methods, including embedding learning techniques to represent style}, adversarial learning and reinforcement learning with cycle consistency to match content but to distinguish different styles. We also introduce current approaches of evaluating stylized text generation systems. We conclude our tutorial by presenting the challenges of stylized text generation and discussing future directions, such as small-data training, non-categorical style modeling, and a generalized scope of style transfer (e.g., controlling the syntax as a style).

'''T5: Achieving Common Ground in Multi-modal Dialogue''' (cutting-edge) 
Malihe Alikhani and Matthew Stone 
All communication aims at achieving common ground (grounding): interlocutors can work together effectively only with mutual beliefs about what the state of the world is, about what their goals are, and about how they plan to make their goals a reality. Computational dialogue research offers some classic results on grouding, which unfortunately offer scant guidance to the design of grounding modules and behaviors in cutting-edge systems. In this tutorial, we focus on three main topic areas: 1) grounding in human-human communication; 2) grounding in dialogue systems; and 3) grounding in multi-modal interactive systems, including image-oriented conversations and human-robot interactions. We highlight a number of achievements of recent computational research in coordinating complex content, show how these results lead to rich and challenging opportunities for doing grounding in more flexible and powerful ways, and canvass relevant insights from the literature on human--human conversation. We expect that the tutorial will be of interest to researchers in dialogue systems, computational semantics and cognitive modeling, and hope that it will catalyze research and system building that more directly explores the creative, strategic ways conversational agents might be able to seek and offer evidence about their understanding of their interlocutors.

'''T6: Commonsense Reasoning for Natural Language Processing''' (introductory) 
Maarten Sap, Vered Shwartz, Antoine Bosselut, Dan Roth and Yejin Choi 
In our tutorial, we (1) outline the various types of commonsense (e.g., physical, social), and (2) discuss techniques to gather and represent commonsense knowledge, while highlighting the challenges specific to this type of knowledge (e.g., reporting bias). We will then (3) discuss the types of commonsense knowledge captured by modern NLP systems (e.g., large pretrained language models), and (4) present ways to measure systems' commonsense reasoning abilities. We finish with (5) a discussion of various ways in which commonsense reasoning can be used to improve performance on NLP tasks, exemplified by an (6) interactive session on integrating commonsense into a downstream task.

'''T7: Integrating Ethics into the NLP Curriculum''' (introductory) 
Emily M. Bender, Dirk Hovy and Alexandra Schofield 
Our goal in this tutorial is to empower NLP researchers and practitioners with tools and resources to teach others about how to ethically apply NLP techniques. Our tutorial will present both high-level strategies for developing an ethics-oriented curriculum, based on experience and best practices, as well as specific sample exercises that can be brought to a classroom. We plan to make this a highly interactive work session culminating in a shared online resource page that pools lesson plans, assignments, exercise ideas, reading suggestions, and ideas from the attendees. We consider three primary topics with our session that frequently underlie ethical issues in NLP research: Dual use, bias and privacy. 
In this setting, a key lesson is that there is no single approach to ethical NLP: each project requires thoughtful consideration about what steps can be taken to best support people affected by that project. However, we can learn (and teach) what kinds of issues to be aware of and what kinds of strategies are available for mitigating harm. To teach this process, we apply and promote interactive exercises that provide an opportunity to ideate, discuss, and reflect. We plan to facilitate this in a way that encourages positive discussion, emphasizing the creation of ideas for the future instead of negative opinions of previous work.

'''T8: Recent Advances in Open-Domain Question Answering''' (cutting-edge) 
Danqi Chen and Scott Wen-tau Yih 
Open-domain (textual) question answering (QA), the task of finding answers to open-domain questions by searching a large collection of documents, has been a long-standing problem in NLP, information retrieval (IR) and related fields (Voorhees et al., 1999; Moldovan et al., 2000; Brill et al.,2002; Ferrucci et al., 2010). Traditional QA systems were usually constructed as a pipeline, consisting of many different components such as question processing, document/passage retrieval and answer processing. With the rapid development of neural reading comprehension (Chen, 2018), modern open-domain QA systems have been restructured by combining traditional IR techniques and neural reading comprehension models (Chen et al., 2017; Yang et al., 2019) or even implemented in a fully end-to-end fashion (Lee et al., 2019; Seo et al., 2019). While the system architecture has been drastically simplified, two technical challenges remain critical:(1) “Retriever”: finding documents that (might)contain an answer from a large collection of documents; (2) “Reader”: finding the answer in a given paragraph or a document. 
In this tutorial, we aim to provide a comprehensive and coherent overview of recent advances in this line of research. We will start by first giving a brief historical background of open-domain question answering, discussing the basic setup and core technical challenges of the research problem.The focus will then shift to modern techniques and resources proposed for open-domain QA, including the basics of latest neural reading comprehension systems, new datasets and models. The scope will also be broadened to cover the information retrieval component on how to effectively identify passages relevant to the questions. Moreover, in-depth discussions will be given on the use of traditional / neural IR modules, as well as the trade-offs between modular design and end-to-end training. If time permits, we also plan to discuss some hybrid approaches for answering questions using both text and large knowledge bases (e.g. (Sun et al., 2018)) and give a critical review on how structured data complements the information from unstructured text. 
At the end of our tutorial, we will discuss some important questions, including (1) How much progress have we made compared to the QA systems developed in the last decade?(2) What are the main challenges and limitations of cur-rent approaches? (3) How to trade off the efficiency (computational time and memory requirements) and accuracy in the deep learning era? We hope that our tutorial will not only serve as a useful resource for the audience to efficiently acquire the up-to-date knowledge, but also provide new perspectives to stimulate the advances of open-domain QA research in the next phase.

===Virtual Conference Organization===

In April the whole conference was decided to be held in virtual. We carried on the preparations jointly with the tutorial instructors, the virtual infrastructure chairs, the website chairs and the SlidesLive team to set up the virtual conference infrastructure, and to publicize it for each tutorial.

We first made a decision on the time slots for tutorial presentations. Five major time zones were considered, including US west coast, US east coast, central Europe, China/Asia and Australia (India was also considered). Our aim was to offer at least two convenient time slots (i.e. within regular working hours) for each of these zones.

The final time slots include (Jul 5 Pacific Time):

<blockquote>slot 1 – 7:00 PM (-1D) to 10:30 PM (-1D)

slot 2 – 12:00 AM to 3:30 AM

slot 3 – 6:00 AM to 9:30 AM

slot 4 – 10:30 AM to 2:00 PM

slot 5 – 3:00 PM to 6:30 PM
</blockquote>
We asked the tutorial teachers to indicate their preferences among the time slots, and – if possible – to agree to give their tutorial twice, so as to increase accessibility. Our final decisions on slot assignments traded off the teacher preferences and slot availability. In the end, the assignments were the following (see also the [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ blog post] and [https://acl2020.org/program/tutorials/ online schedule]):

<blockquote>T1 – twice on slots 3 and 4

T2 – once on slot 5

T3 – once on slot 4

T4 – once on slot 4

T5 – twice on slots 3 and 5

T6 – once on slot 5

T7 – twice on slots 3 and 4

T8 – once on slot 5
</blockquote>
Unfortunately, the tutorial teachers proved hardly available for slots 1 and 2.

In addition to time slot arrangements, we gave the tutorial teacher two options on presentation, namely a pre-recorded and a fully interactive form of tutorials. The former allows the teachers to pre-record the main content of their tutorials, and use the time slots mainly for question answering and detailed discussions. The latter is more interactive, where the teachers use the online sessions for both giving the lecture and other activities such as active discussions. All these arrangements are made for taking advantage of the virtual conference. In the end the choices were the following:

<blockquote>T1 – pre-recorded

T2 – interactive (the initial decision was pre-recorded but a change was made in June)

T3 – pre-recorded

T4 – pre-recorded

T5 – pre-recorded

T6 – interactive

T7 – interactive

T8 – interactive
</blockquote>
After making decisions on time slots and teaching forms, we started to work on the technical details of the virtual conference infrastructure jointly with the infrastructure chairs, the website chairs, the SlidesLive support team and individual tutorial teachers. We collected needs from each tutorial from time to time, inquired information from the SlidesLive team, and tried to accommodate specific requirement for each tutorial. Since this is the first time ACL was virtual, it typically took several rounds of information exchange for making a certain technical feature work. Major things discussed included:

* Sorting out how to pre-record a tutorial using the system
* Recording of interactive sessions
* Additional RocketChat channel
* Virtual tutorial webpage setup
* Coordinators for tutorial sessions and SlidesLive team participation
* Rehearsal of interactive sessions
* Captioning of pre-recorded and interactive tutorials
* Privacy issues
* Special needs from particular tutorials
** Integrating [https://www.dory.app/ Dory] for question answering (in T1)
** Fully interactive sessions, with occasional split into break-out rooms (in T7)

To ensure smooth running of the sessions, we sent two rounds of reminders to each tutorial to ensure that each teacher is registered to the conference. We collected rough timelines of each tutorial so that the SlidesLive team can work closely during the live sessions. To further help the teachers prepare their live session content we additionally collected the lists of registered attendees twice before the conference, and shared them with the teachers.

In the end, the numbers of registrations for each tutorial were:

<blockquote>T1 -- 1762

T2 -- 793

T3 -- 503

T4 -- 571

T5 -- 641

T6 -- 1288

T7 -- 378

T8 – 1087

===Archiving and Publicity===

There are three main channels for publicity, namely the static [http://www.acl2020.org/program/tutorials/ website], the website of the [https://virtual.acl2020.org/tutorials.html virtual conference] and the social media. For the former, we collected photos from each instructor, which are shown with the title and abstract of their tutorials. Following NAACL 2019, we also displayed the time slots as a right-hand-side bar on the tutorial website. Due to special arrangements of the virtual conference, we marked the time zones explicitly. In addition, we asked the tutorial teachers to provide URL links to teaching materials (e.g., at GitHub) during their final submission, and included these links to their tutorial information. For social media, we drafted two blog posts (i.e., [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ this post] and [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]) for advertising the tutorials.

The archiving consists of three main items, namely the tutorial proceedings, the slides for each tutorial and the videos. For the tutorial proceedings, we collected final abstracts from the authors using the START system. We asked the instructors to sign the CC BY v4 agreement when submitting their final materials on SoftConf. For the slides, we asked the tutorial teachers to provide a first version of the slides by June 17 regardless whether they are pre-recorded or interactive. Some tutorials made further updates to their slides before ACL. For the video recordings, we worked with the ACL Anthology team discussing the technical details since this is the first time the tutorial videos are shared in the website. [Note: this is currently undergoing at the time of writing of this summary]

===Lessons Learned===

While organizing a virtual conference it is important to stress that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]).

It is also important to announce the individual modalities with the attendees early enough. For instance, watching pre-recordings in advance was necessary for some tutorials (since their live sessions only included question answering) and such constrains should have been known more in advance.

If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to Zoom rooms only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.

One thing to remember which we missed is to publish the reading list for each tutorial (which the teachers submit with the proposal) on the tutorial website.

Tutorial chair handbook

2020-07-15T18:48:28Z

AgataSavary: /* Tutorials in a virtual conference */

== Duties ==
Tutorial chairs are nominated by the general chair of the main conference. Their duties are described in [[Tutorial chair duties|here]].

== Timeline ==
The timeline for the organisation of tutorials should follow the general [[conference planning schedule]]. Below, we complete this schedule with more detailed tasks, as defined for the ACL 2020 tutorials. The dates from the [[conference planning schedule]] are highlighted in '''bold'''. 
In recent years, calls and selection of tutorial proposals have been coordinated among some major CL conferences (usually a subset of ACL, NAACL, EMNLP, AACL and COLING). Note that if your conference is the first in the year to come, you will need to be the major driving force in the initial steps described below. 
Tutorial proposals are usually reviewed by the tutorial chairs of the coordinating conferences, but it may be a good idea to also rectruit a small group of external reviewers. They should have a good experience in reviewing and a large understanding of the recent progress in the CL domain.

* [T-12]
** Learn from the general chair which conferences will have their tutorials coordinated with yours.
** Establish contact with the tutorial chairs of these other conferences, and discuss
*** Recruitment of external reviewers
*** Proposal evaluation criteria
*** Soliciting tutorial proposals from particular potential lecturers on particular topics
*** Organizing a public poll about interest in the submitted tutorial
** Discuss general strategies of the main conference with the general chair and the program chairs
***promotion of some topics
***mailing lists and conference alert portals to be used for all announcements
** Set up a SoftConf space for proposal submission
*** Solicit the creation of a SoftConf space from [mailto:support@softconf.com support] (currently Rich Gerber). Note that submissions will be joint for several conferences but the proceedings will be edited for each conference separately. Therefore, a new SoftConf space will have to be created later for the selected proposals of each conference.
*** If you plan to have external tutorial reviewers, invite them via SoftConf.
*** Set up a submission page.
** Set up a joint email for tutorial chairs.
** Confirm the number (usually 6-8) and length (usually 3 hours) of tutorial slots with the ACL Business Manager (currently Priscilla Rasmussen).
** Draft the joint call for tutorials (based on [[#Past calls for tutorials|past calls]]).

* '''[T-11]''' ''(in the future, it would be good to extend the time slot between the call and the deadline to 2 months at least, especially if July/August is included)''
** Send out the call for tutorials.
** For posting the call on the ACL website, contact the ACL Business Manager (currently Priscilla Rasmussen).
** If you planned to encourage submission of proposals from particular potential lecturers, contact them now.
** Set up a SoftConf page for proposal reviews.

* [T-10]
** Deadline for tutorial proposal submission.
** If you decided to run a public poll about interest in the submitted tutorial, set it up and send out a call for expression of interest.
** Assign proposals to reviewers, send out invitations to review (via SoftConf).

* '''[T-9]'''
** Select tutorials jointly with other conference chairs.
** Notify the authors (via SoftConf).
** Liaise with local chair for space needs and scheduling.
** Send instructions to tutorial teachers about printed material, agenda/time management, payments, fees, etc.
** Liaise the conference website chair for updates about tutorials and authors' guidelines.
** Liaise with the ACL Office for the registration form.

* '''[T-4]'''
** Collect short descriptions for proceedings from tutorial teachers.
** Provide them to the webmaster and the general and program chairs .

* '''[T-3]'''
** Confirm the tutorials with authors.
** Authors submit the final tutorial descriptions for the proceedings.
** Arrange for printing of tutorial materials.

* [T-2]
** Arrange with the [mailto:support@softconf.com SoftConf support] the creation of a separate SoftConf space for the selected tutorials of your conference only. Indicate which tutorial submissions should be copied there.
** Set up the final submission form in the new SoftConf space. Include the submissions of slides, which will be published on the ACL Anthology. Include signing the CC-BY-4.0 license agreement.

* '''[T-1]'''
** Collect and check final tutorial materials.
** Create the tutorial proceedings, including slides.

* '''[T-0]''' Conference and tutorials

* [T+1]
** Invite the lecturers to update the slides at the ACL Anthology via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form] (under CC-BY-4.0).
** Update this tutorial page and the list of [[Past tutorials|past tutorials]] with the data from your conference. For access to the ACL [[Conference Handbook]], contact the ACL Secretary (currently Shiqi Zhou).

== Tutorials in a virtual conference ==
ACL 2020 was transformed into a fully virtual conference due to the COVID-19 pandemic. This decision was taken only 3 months before the conference and added a bunch of new tasks on top of the usual organization. This new organization was largely to be invented from scratch, additionally to the previous experience from [https://icml.cc/ ICML]. SlideLive was the subcontractor for the virtual infrastructure.

The tasks for tutorials included:
* collecting slides and pre-recordings from the tutorial teachers (recorded with the SlideLive online software)
* defining [https://acl2020.org/_pages/docs/ACL_2020_Tutorial_Timeslots.pdf time slots] which would increase participation across various time zones
* collecting teachers' preferences for various time slots and encouraging them to give their tutorials twice
* setting up the [https://acl2020.org/program/tutorials/ schedule]
* informing the participants about [https://acl2020.org/blog/intro-to-tutorial-infrastructure/ virtual participation] and about the [https://acl2020.org/blog/detailed-modalities-of-tutorials/ detailed modalities] for each tutorial
* defining the contents of live sessions (a mixture of pre-recordings, question-answering, break-out rooms for one tutorial, etc.)
* creating automatic captioning, sending the captions to the teachers and uploading the edited versions
* dry runs with teachers and SlidesLive people
* archiving pre-recordings or recordings of the live sessions additionally to slides

== Publication of tutorials ==
For NAACL 2019, ACL Anthology stores the tutorial descriptions, slides and video recordings. The same should go for ACL 2020 (which went fully virtual due to the COVID pandemic), except that recordings of live sessions are sometimes replaced with pre-recordings.

In tutorials organized until 2019, there was no homogeneous policy as to the publication of tutorial material. Data (abstracts, slides, code, etc.) from selected tutorials were usually scattered in 2 or 3 places:
* conference website
* ACL Anthology
* private repositories of the authors
The sustainability of these varies from venue to venue. The main websites of some conferences are not maintained, therefore ACL collects [https://www.aclweb.org/portal/node/2481 mirrors] of some of them (but not all).
The tutorial proceedings, published in the ACL Anthology, until recently included only abstracts but no tutorial slides.

As a result, for many [[past tutorials]], no slides are available online, so the community cannot benefit from these assets. It is, therefore, important to promote '''Open Access''' of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation '''slides in the ACL Anthology'''. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form].

It is recommended for tutorial chairs to:
* announce the publication of the tutorial slides under the CC-BY-4.0 license (used in the ACL Anthology)
* encourage the authors to also publish, under open licenses, their code and data used in the tutorials
* include the initial versions of tutorial slides, together with abstract, in the proceedings
* encourage the tutorial authors to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]

== Organization of past tutorials ==
For the last years, the call and selection of tutorials has been coordinated among several major CL conferences. The table below summarizes some data collected from existing reports and from direct contacts with past tutorial chairs.

{| border="1" cellspacing="0" cellpadding="5"
|'''Year'''
|'''Coordinated conferences'''
|'''Tutorial chairs'''
|'''Submission platform'''
|'''# Submissions'''
|'''Reviewers'''
|'''# Reviews per proposal'''
|'''# Selected tutorials'''
|'''Selection criteria'''
|'''Useful links'''
|-
|2020
|ACL EMNLP AACL-IJCNLP COLING
|ACL: Agata Savary (France), Yue Zhang (China) EMNLP: Benjamin Van Durme (USA), Aline Villavicencio (UK & Brazil) AACL-IJCNLP: Tim Baldwin (Australia), Fei Xia (USA) COLING: Daniel Beck (Australia), Lucia Specia (UK)
|SoftConf
|43
|Tutorial chairs + external experts
|3
|ACL: 8 EMNLP: 7 ACCL-IJCNLP: 6 COLING: 7
|Clarity and preparedness Novelty or timely character of the topic Lecturers' experience Likely audience interest Open access of the teaching material Diversity aspects (multilingualism, gender, age and country of the lecturers) Compatibility with the required venues
|[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 Call] [https://acl2020.org/program/tutorials/ ACL tutorials] [https://2020.emnlp.org/tutorials EMNLP tutorials] [http://aacl2020.org/ ACCL-IJCNLP tutorials] [https://coling2020.org/pages/tutorials COLING tutorials]
|
|-
|2019
|NAACL ACL EMNLP
|ACL: Preslav Nakov (Qatar), Alexis Palmer (USA) NAACL: Anoop Sarkar (Canda), Michael Strube (Germany) EMNLP-IJCNLP: Marine Carpuat (USA), Tim Baldwin (Australia)
|SoftConf
|45
|Tutorial chairs
|2-3
|ACL: 9 NAACL: 6 EMNLP: 7
|Presenters (1-5) Description (1-5) Interest Level (1-5) Preparedness (1-5)
|[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials Call] [http://www.acl2019.org/EN/tutorials.xhtml ACL tutorials] [https://naacl2019.org/program/tutorials/ NAACL tutorials]
|-
|2018
|ACL NAACL EMNLP COLING
|ACL: Yoav Artzi (USA), Jacob Eisenstein (USA) COLING: Pascale Fung (Hong Kong), Donia Scott (UK), Marilyn Walker (USA) EMNLP: Mausam (India), Lu Wang (USA) NAACL: Mohit Bansal (USA), Rebecca Passonneau (USA)
|SoftConf
|49
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6 COLING: 6
|Quality of plan Diversity of topics Qualifications of presenter likely audience interest
|[https://acl2018.org/2017/09/11/joint-call-for-tutorials/ Call] 
[https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs Report] 
[https://acl2018.org/tutorials/ ACL tutorials] 
[https://emnlp2018.org/program/tutorials/ EMNLP tutorials] 
[https://coling2018.org/tutorial-list/ COLING tutorials]
|-
|2017
|ACL EACL EMNLP
|ACL: Maja Popović (Germany), Jordan Boyd-Graber (USA) EACL: Alex Klementiev (Germany), Lucia Specia (UK) EMNLP: Nathan Schneider (USA), Alexandra Birch (UK)
|SoftConf
|26
|Tutorial chairs
|3
|ACL: 6 EACL: 6 EMNLP: 7
|Clarity (description and outline) Topic
|[https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs Report] [http://acl2017.org/tutorials/ ACL tutorials] [http://eacl2017.org/index.php/tutorials EACL tutorials]
|-
|2016
|ACL NAACL EMNLP
|ACL: Alexandra Birch (UK), Willem Zuidema (Netherlands) NAACL: Alexander Rush and Bishan Yang EMNLP: Rebecca Hwa and Mohit Bansal
|SoftConf
|32
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6
|Relevance to ACL community (15) Quality of instructor (15) Outline and depth/breadth of the proposal (15) 
Potential attendance Newly emerging area not previously covered in an ACL-related tutorial (yes or no) Introduction into related fields (yes or no) 
Overall score (15) 1st preference for venue as indicated in proposal comments
|[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 Call] [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs Report] [http://naacl.org/naacl-hlt-2016/tutorials.html NAACL tutorials] [https://www.aclweb.org/mirror/emnlp2016/tutorials.html EMNLP tutorials]
|}

== Past calls for tutorials ==
*[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 ACL/AACL-IJCNLP/EMNLP/COLING 2020 joint call for tutorials]
*[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials NAACL/ACL/EMNLP 2019 joint call for tutorials]
*[https://www.aclweb.org/portal/content/final-joint-call-tutorial-proposals-aclcolingemnlpnaacl-2018 ACL/COLING/EMNLP/NAACL 2018 joint call for tutorials]
*[https://www.aclweb.org/portal/content/eacl-2017-call-tutorial-proposals EACL 2017 call for tutorials]
*[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 NAACL/ACL/EMNLP 2016 joint call for tutorials]
*[https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-naacl-2015-acl-2015-emnlp-2015-0 NAACL/ACL/EMNLP 2015 joint call for tutorials]

== Reports from past tutorial chairs ==
*[[2020Q3_Reports:_Tutorial_Chairs|ACL 2020 report]] (by Agata Savary and Yue Zhang)
*[[2019Q1_Reports:_ACL_2019#Tutorial_Co-Chairs|ACL 2019 report]] (by Preslav Nakov and Alexis Palmer)
*[[2018Q3 Reports: Tutorial Chairs|ACL 2018 report]] (by Yoav Artzi and Jacob Eisenstein)
*[[2017Q3 Reports: Tutorial Chairs|ACL 2017 report]] (by Maja Popović and Jordan Boyd-Graber)
*[[2016Q3 Reports: Tutorial Chairs|ACL 2016 report]] (by Alexandra Birch and Willem Zuidema)

== Lessons learned by past tutorial chairs ==
* [Agata Savary, 2020] If the conference is virtual:
** Remember that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]).
** Announce the individual modalities with the attendees early enough (for instance, if watching pre-recordings in advance is necessary for some tutorials because their live sessions only included question answering)
** If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to Zoom rooms only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.
* [Agata Savary, 2020] Use a common SoftConf space for joint tutorial submissions for all coordinated conference. Once the proposals have been selected and assigned to conferences, use one track per conference in the same SoftConf space.
* [Agata Savary, 2020] Publish the list of recommended readings on each tutorial page
* [Agata Savary, 2020] A good way to proceed while selecting tutorials for several venues is to have a common online meeting, in which
** Tutorials are clustered per themes (to distribute similar tutorials over different conferences)
** The chairs for each venue select their best choices in a round robin manner.
** Generate csv files with reviews and scores: Manager -> Other tools -> Spreadsheet maker -> Reports
* [Agata Savary, 2020] While setting up a review form on SoftConf, include:
** All evaluation criteria, including diversity, in the review form
** Cutting edge vs. introductory category
** Duration (3 or 6 hours)
* [Agata Savary, 2020] Currently, the generic conference schedule assumes only 2 months between the call for tutorials and notification to proposers. If a solid 1-month review procedure is to take place, this means, that proposers only have 1 month between the call and the submission deadline. It would be good to shift the call to [T-12].
* [Agata Savary, 2020] For many past tutorials, no slides are available online. It is, therefore, important to promote Open Access of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation slides on the ACL Anthology. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]. As a tutorial chair, make sure that
** initial versions of tutorial slides are submitted to the proceedings together with the tutorial abstracts
** tutorial authors are encouraged to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]
* [Marine Capuat, 2019] Continue soliciting both cutting edge and introductory tutorials
* [Marine Capuat, 2019] Ask proposers to express their preferred location/conference in one place (i.e. either in proposal pdf, or in softconf form, not both) to avoid inconsistencies
* [ACL 2018 chairs] Issues with proceedings edition, when one joint SoftConf space is used for many conferences (see the [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs report])
* [ACL 2016 chairs] Avoid confusion about the submission procedures for the final tutorial descriptions, those were submitted by email in many different formats

Author: Agata Savary, July 2019-July 2020

Updates are welcome

[[Category:Conference Handbook]]

Tutorial chair handbook

2020-07-15T18:29:52Z

AgataSavary: /* Tutorials in a virtual conference */

== Duties ==
Tutorial chairs are nominated by the general chair of the main conference. Their duties are described in [[Tutorial chair duties|here]].

== Timeline ==
The timeline for the organisation of tutorials should follow the general [[conference planning schedule]]. Below, we complete this schedule with more detailed tasks, as defined for the ACL 2020 tutorials. The dates from the [[conference planning schedule]] are highlighted in '''bold'''. 
In recent years, calls and selection of tutorial proposals have been coordinated among some major CL conferences (usually a subset of ACL, NAACL, EMNLP, AACL and COLING). Note that if your conference is the first in the year to come, you will need to be the major driving force in the initial steps described below. 
Tutorial proposals are usually reviewed by the tutorial chairs of the coordinating conferences, but it may be a good idea to also rectruit a small group of external reviewers. They should have a good experience in reviewing and a large understanding of the recent progress in the CL domain.

* [T-12]
** Learn from the general chair which conferences will have their tutorials coordinated with yours.
** Establish contact with the tutorial chairs of these other conferences, and discuss
*** Recruitment of external reviewers
*** Proposal evaluation criteria
*** Soliciting tutorial proposals from particular potential lecturers on particular topics
*** Organizing a public poll about interest in the submitted tutorial
** Discuss general strategies of the main conference with the general chair and the program chairs
***promotion of some topics
***mailing lists and conference alert portals to be used for all announcements
** Set up a SoftConf space for proposal submission
*** Solicit the creation of a SoftConf space from [mailto:support@softconf.com support] (currently Rich Gerber). Note that submissions will be joint for several conferences but the proceedings will be edited for each conference separately. Therefore, a new SoftConf space will have to be created later for the selected proposals of each conference.
*** If you plan to have external tutorial reviewers, invite them via SoftConf.
*** Set up a submission page.
** Set up a joint email for tutorial chairs.
** Confirm the number (usually 6-8) and length (usually 3 hours) of tutorial slots with the ACL Business Manager (currently Priscilla Rasmussen).
** Draft the joint call for tutorials (based on [[#Past calls for tutorials|past calls]]).

* '''[T-11]''' ''(in the future, it would be good to extend the time slot between the call and the deadline to 2 months at least, especially if July/August is included)''
** Send out the call for tutorials.
** For posting the call on the ACL website, contact the ACL Business Manager (currently Priscilla Rasmussen).
** If you planned to encourage submission of proposals from particular potential lecturers, contact them now.
** Set up a SoftConf page for proposal reviews.

* [T-10]
** Deadline for tutorial proposal submission.
** If you decided to run a public poll about interest in the submitted tutorial, set it up and send out a call for expression of interest.
** Assign proposals to reviewers, send out invitations to review (via SoftConf).

* '''[T-9]'''
** Select tutorials jointly with other conference chairs.
** Notify the authors (via SoftConf).
** Liaise with local chair for space needs and scheduling.
** Send instructions to tutorial teachers about printed material, agenda/time management, payments, fees, etc.
** Liaise the conference website chair for updates about tutorials and authors' guidelines.
** Liaise with the ACL Office for the registration form.

* '''[T-4]'''
** Collect short descriptions for proceedings from tutorial teachers.
** Provide them to the webmaster and the general and program chairs .

* '''[T-3]'''
** Confirm the tutorials with authors.
** Authors submit the final tutorial descriptions for the proceedings.
** Arrange for printing of tutorial materials.

* [T-2]
** Arrange with the [mailto:support@softconf.com SoftConf support] the creation of a separate SoftConf space for the selected tutorials of your conference only. Indicate which tutorial submissions should be copied there.
** Set up the final submission form in the new SoftConf space. Include the submissions of slides, which will be published on the ACL Anthology. Include signing the CC-BY-4.0 license agreement.

* '''[T-1]'''
** Collect and check final tutorial materials.
** Create the tutorial proceedings, including slides.

* '''[T-0]''' Conference and tutorials

* [T+1]
** Invite the lecturers to update the slides at the ACL Anthology via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form] (under CC-BY-4.0).
** Update this tutorial page and the list of [[Past tutorials|past tutorials]] with the data from your conference. For access to the ACL [[Conference Handbook]], contact the ACL Secretary (currently Shiqi Zhou).

== Tutorials in a virtual conference ==
ACL 2020 was transformed into a fully virtual conference due to the COVID-19 pandemic. This decision was taken only 3 months before the conference and added a bunch of new tasks on top of the usual organization. This new organization was largely to be invented from scratch, additionally to the previous experience from [https://icml.cc/ ICML].

== Publication of tutorials ==
For NAACL 2019, ACL Anthology stores the tutorial descriptions, slides and video recordings. The same should go for ACL 2020 (which went fully virtual due to the COVID pandemic), except that recordings of live sessions are sometimes replaced with pre-recordings.

In tutorials organized until 2019, there was no homogeneous policy as to the publication of tutorial material. Data (abstracts, slides, code, etc.) from selected tutorials were usually scattered in 2 or 3 places:
* conference website
* ACL Anthology
* private repositories of the authors
The sustainability of these varies from venue to venue. The main websites of some conferences are not maintained, therefore ACL collects [https://www.aclweb.org/portal/node/2481 mirrors] of some of them (but not all).
The tutorial proceedings, published in the ACL Anthology, until recently included only abstracts but no tutorial slides.

As a result, for many [[past tutorials]], no slides are available online, so the community cannot benefit from these assets. It is, therefore, important to promote '''Open Access''' of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation '''slides in the ACL Anthology'''. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form].

It is recommended for tutorial chairs to:
* announce the publication of the tutorial slides under the CC-BY-4.0 license (used in the ACL Anthology)
* encourage the authors to also publish, under open licenses, their code and data used in the tutorials
* include the initial versions of tutorial slides, together with abstract, in the proceedings
* encourage the tutorial authors to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]

== Organization of past tutorials ==
For the last years, the call and selection of tutorials has been coordinated among several major CL conferences. The table below summarizes some data collected from existing reports and from direct contacts with past tutorial chairs.

{| border="1" cellspacing="0" cellpadding="5"
|'''Year'''
|'''Coordinated conferences'''
|'''Tutorial chairs'''
|'''Submission platform'''
|'''# Submissions'''
|'''Reviewers'''
|'''# Reviews per proposal'''
|'''# Selected tutorials'''
|'''Selection criteria'''
|'''Useful links'''
|-
|2020
|ACL EMNLP AACL-IJCNLP COLING
|ACL: Agata Savary (France), Yue Zhang (China) EMNLP: Benjamin Van Durme (USA), Aline Villavicencio (UK & Brazil) AACL-IJCNLP: Tim Baldwin (Australia), Fei Xia (USA) COLING: Daniel Beck (Australia), Lucia Specia (UK)
|SoftConf
|43
|Tutorial chairs + external experts
|3
|ACL: 8 EMNLP: 7 ACCL-IJCNLP: 6 COLING: 7
|Clarity and preparedness Novelty or timely character of the topic Lecturers' experience Likely audience interest Open access of the teaching material Diversity aspects (multilingualism, gender, age and country of the lecturers) Compatibility with the required venues
|[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 Call] [https://acl2020.org/program/tutorials/ ACL tutorials] [https://2020.emnlp.org/tutorials EMNLP tutorials] [http://aacl2020.org/ ACCL-IJCNLP tutorials] [https://coling2020.org/pages/tutorials COLING tutorials]
|
|-
|2019
|NAACL ACL EMNLP
|ACL: Preslav Nakov (Qatar), Alexis Palmer (USA) NAACL: Anoop Sarkar (Canda), Michael Strube (Germany) EMNLP-IJCNLP: Marine Carpuat (USA), Tim Baldwin (Australia)
|SoftConf
|45
|Tutorial chairs
|2-3
|ACL: 9 NAACL: 6 EMNLP: 7
|Presenters (1-5) Description (1-5) Interest Level (1-5) Preparedness (1-5)
|[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials Call] [http://www.acl2019.org/EN/tutorials.xhtml ACL tutorials] [https://naacl2019.org/program/tutorials/ NAACL tutorials]
|-
|2018
|ACL NAACL EMNLP COLING
|ACL: Yoav Artzi (USA), Jacob Eisenstein (USA) COLING: Pascale Fung (Hong Kong), Donia Scott (UK), Marilyn Walker (USA) EMNLP: Mausam (India), Lu Wang (USA) NAACL: Mohit Bansal (USA), Rebecca Passonneau (USA)
|SoftConf
|49
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6 COLING: 6
|Quality of plan Diversity of topics Qualifications of presenter likely audience interest
|[https://acl2018.org/2017/09/11/joint-call-for-tutorials/ Call] 
[https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs Report] 
[https://acl2018.org/tutorials/ ACL tutorials] 
[https://emnlp2018.org/program/tutorials/ EMNLP tutorials] 
[https://coling2018.org/tutorial-list/ COLING tutorials]
|-
|2017
|ACL EACL EMNLP
|ACL: Maja Popović (Germany), Jordan Boyd-Graber (USA) EACL: Alex Klementiev (Germany), Lucia Specia (UK) EMNLP: Nathan Schneider (USA), Alexandra Birch (UK)
|SoftConf
|26
|Tutorial chairs
|3
|ACL: 6 EACL: 6 EMNLP: 7
|Clarity (description and outline) Topic
|[https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs Report] [http://acl2017.org/tutorials/ ACL tutorials] [http://eacl2017.org/index.php/tutorials EACL tutorials]
|-
|2016
|ACL NAACL EMNLP
|ACL: Alexandra Birch (UK), Willem Zuidema (Netherlands) NAACL: Alexander Rush and Bishan Yang EMNLP: Rebecca Hwa and Mohit Bansal
|SoftConf
|32
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6
|Relevance to ACL community (15) Quality of instructor (15) Outline and depth/breadth of the proposal (15) 
Potential attendance Newly emerging area not previously covered in an ACL-related tutorial (yes or no) Introduction into related fields (yes or no) 
Overall score (15) 1st preference for venue as indicated in proposal comments
|[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 Call] [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs Report] [http://naacl.org/naacl-hlt-2016/tutorials.html NAACL tutorials] [https://www.aclweb.org/mirror/emnlp2016/tutorials.html EMNLP tutorials]
|}

== Past calls for tutorials ==
*[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 ACL/AACL-IJCNLP/EMNLP/COLING 2020 joint call for tutorials]
*[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials NAACL/ACL/EMNLP 2019 joint call for tutorials]
*[https://www.aclweb.org/portal/content/final-joint-call-tutorial-proposals-aclcolingemnlpnaacl-2018 ACL/COLING/EMNLP/NAACL 2018 joint call for tutorials]
*[https://www.aclweb.org/portal/content/eacl-2017-call-tutorial-proposals EACL 2017 call for tutorials]
*[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 NAACL/ACL/EMNLP 2016 joint call for tutorials]
*[https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-naacl-2015-acl-2015-emnlp-2015-0 NAACL/ACL/EMNLP 2015 joint call for tutorials]

== Reports from past tutorial chairs ==
*[[2020Q3_Reports:_Tutorial_Chairs|ACL 2020 report]] (by Agata Savary and Yue Zhang)
*[[2019Q1_Reports:_ACL_2019#Tutorial_Co-Chairs|ACL 2019 report]] (by Preslav Nakov and Alexis Palmer)
*[[2018Q3 Reports: Tutorial Chairs|ACL 2018 report]] (by Yoav Artzi and Jacob Eisenstein)
*[[2017Q3 Reports: Tutorial Chairs|ACL 2017 report]] (by Maja Popović and Jordan Boyd-Graber)
*[[2016Q3 Reports: Tutorial Chairs|ACL 2016 report]] (by Alexandra Birch and Willem Zuidema)

== Lessons learned by past tutorial chairs ==
* [Agata Savary, 2020] If the conference is virtual:
** Remember that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]).
** Announce the individual modalities with the attendees early enough (for instance, if watching pre-recordings in advance is necessary for some tutorials because their live sessions only included question answering)
** If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to Zoom rooms only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.
* [Agata Savary, 2020] Use a common SoftConf space for joint tutorial submissions for all coordinated conference. Once the proposals have been selected and assigned to conferences, use one track per conference in the same SoftConf space.
* [Agata Savary, 2020] Publish the list of recommended readings on each tutorial page
* [Agata Savary, 2020] A good way to proceed while selecting tutorials for several venues is to have a common online meeting, in which
** Tutorials are clustered per themes (to distribute similar tutorials over different conferences)
** The chairs for each venue select their best choices in a round robin manner.
** Generate csv files with reviews and scores: Manager -> Other tools -> Spreadsheet maker -> Reports
* [Agata Savary, 2020] While setting up a review form on SoftConf, include:
** All evaluation criteria, including diversity, in the review form
** Cutting edge vs. introductory category
** Duration (3 or 6 hours)
* [Agata Savary, 2020] Currently, the generic conference schedule assumes only 2 months between the call for tutorials and notification to proposers. If a solid 1-month review procedure is to take place, this means, that proposers only have 1 month between the call and the submission deadline. It would be good to shift the call to [T-12].
* [Agata Savary, 2020] For many past tutorials, no slides are available online. It is, therefore, important to promote Open Access of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation slides on the ACL Anthology. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]. As a tutorial chair, make sure that
** initial versions of tutorial slides are submitted to the proceedings together with the tutorial abstracts
** tutorial authors are encouraged to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]
* [Marine Capuat, 2019] Continue soliciting both cutting edge and introductory tutorials
* [Marine Capuat, 2019] Ask proposers to express their preferred location/conference in one place (i.e. either in proposal pdf, or in softconf form, not both) to avoid inconsistencies
* [ACL 2018 chairs] Issues with proceedings edition, when one joint SoftConf space is used for many conferences (see the [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs report])
* [ACL 2016 chairs] Avoid confusion about the submission procedures for the final tutorial descriptions, those were submitted by email in many different formats

Author: Agata Savary, July 2019-July 2020

Updates are welcome

[[Category:Conference Handbook]]

Tutorial chair handbook

2020-07-15T18:29:22Z

AgataSavary:

== Duties ==
Tutorial chairs are nominated by the general chair of the main conference. Their duties are described in [[Tutorial chair duties|here]].

== Timeline ==
The timeline for the organisation of tutorials should follow the general [[conference planning schedule]]. Below, we complete this schedule with more detailed tasks, as defined for the ACL 2020 tutorials. The dates from the [[conference planning schedule]] are highlighted in '''bold'''. 
In recent years, calls and selection of tutorial proposals have been coordinated among some major CL conferences (usually a subset of ACL, NAACL, EMNLP, AACL and COLING). Note that if your conference is the first in the year to come, you will need to be the major driving force in the initial steps described below. 
Tutorial proposals are usually reviewed by the tutorial chairs of the coordinating conferences, but it may be a good idea to also rectruit a small group of external reviewers. They should have a good experience in reviewing and a large understanding of the recent progress in the CL domain.

* [T-12]
** Learn from the general chair which conferences will have their tutorials coordinated with yours.
** Establish contact with the tutorial chairs of these other conferences, and discuss
*** Recruitment of external reviewers
*** Proposal evaluation criteria
*** Soliciting tutorial proposals from particular potential lecturers on particular topics
*** Organizing a public poll about interest in the submitted tutorial
** Discuss general strategies of the main conference with the general chair and the program chairs
***promotion of some topics
***mailing lists and conference alert portals to be used for all announcements
** Set up a SoftConf space for proposal submission
*** Solicit the creation of a SoftConf space from [mailto:support@softconf.com support] (currently Rich Gerber). Note that submissions will be joint for several conferences but the proceedings will be edited for each conference separately. Therefore, a new SoftConf space will have to be created later for the selected proposals of each conference.
*** If you plan to have external tutorial reviewers, invite them via SoftConf.
*** Set up a submission page.
** Set up a joint email for tutorial chairs.
** Confirm the number (usually 6-8) and length (usually 3 hours) of tutorial slots with the ACL Business Manager (currently Priscilla Rasmussen).
** Draft the joint call for tutorials (based on [[#Past calls for tutorials|past calls]]).

* '''[T-11]''' ''(in the future, it would be good to extend the time slot between the call and the deadline to 2 months at least, especially if July/August is included)''
** Send out the call for tutorials.
** For posting the call on the ACL website, contact the ACL Business Manager (currently Priscilla Rasmussen).
** If you planned to encourage submission of proposals from particular potential lecturers, contact them now.
** Set up a SoftConf page for proposal reviews.

* [T-10]
** Deadline for tutorial proposal submission.
** If you decided to run a public poll about interest in the submitted tutorial, set it up and send out a call for expression of interest.
** Assign proposals to reviewers, send out invitations to review (via SoftConf).

* '''[T-9]'''
** Select tutorials jointly with other conference chairs.
** Notify the authors (via SoftConf).
** Liaise with local chair for space needs and scheduling.
** Send instructions to tutorial teachers about printed material, agenda/time management, payments, fees, etc.
** Liaise the conference website chair for updates about tutorials and authors' guidelines.
** Liaise with the ACL Office for the registration form.

* '''[T-4]'''
** Collect short descriptions for proceedings from tutorial teachers.
** Provide them to the webmaster and the general and program chairs .

* '''[T-3]'''
** Confirm the tutorials with authors.
** Authors submit the final tutorial descriptions for the proceedings.
** Arrange for printing of tutorial materials.

* [T-2]
** Arrange with the [mailto:support@softconf.com SoftConf support] the creation of a separate SoftConf space for the selected tutorials of your conference only. Indicate which tutorial submissions should be copied there.
** Set up the final submission form in the new SoftConf space. Include the submissions of slides, which will be published on the ACL Anthology. Include signing the CC-BY-4.0 license agreement.

* '''[T-1]'''
** Collect and check final tutorial materials.
** Create the tutorial proceedings, including slides.

* '''[T-0]''' Conference and tutorials

* [T+1]
** Invite the lecturers to update the slides at the ACL Anthology via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form] (under CC-BY-4.0).
** Update this tutorial page and the list of [[Past tutorials|past tutorials]] with the data from your conference. For access to the ACL [[Conference Handbook]], contact the ACL Secretary (currently Shiqi Zhou).

== Tutorials in a virtual conference ==
ACL 2020 was transformed into a fully virtual conference due to the COVID-19 pandemic. This decision was taken only 3 months before the conference and added a bunch of new tasks on top of the usual organization. This new organization was largely to be invented from scratch, additionally to the previous experience from [[https://icml.cc/|ICML]].

== Publication of tutorials ==
For NAACL 2019, ACL Anthology stores the tutorial descriptions, slides and video recordings. The same should go for ACL 2020 (which went fully virtual due to the COVID pandemic), except that recordings of live sessions are sometimes replaced with pre-recordings.

In tutorials organized until 2019, there was no homogeneous policy as to the publication of tutorial material. Data (abstracts, slides, code, etc.) from selected tutorials were usually scattered in 2 or 3 places:
* conference website
* ACL Anthology
* private repositories of the authors
The sustainability of these varies from venue to venue. The main websites of some conferences are not maintained, therefore ACL collects [https://www.aclweb.org/portal/node/2481 mirrors] of some of them (but not all).
The tutorial proceedings, published in the ACL Anthology, until recently included only abstracts but no tutorial slides.

As a result, for many [[past tutorials]], no slides are available online, so the community cannot benefit from these assets. It is, therefore, important to promote '''Open Access''' of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation '''slides in the ACL Anthology'''. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form].

It is recommended for tutorial chairs to:
* announce the publication of the tutorial slides under the CC-BY-4.0 license (used in the ACL Anthology)
* encourage the authors to also publish, under open licenses, their code and data used in the tutorials
* include the initial versions of tutorial slides, together with abstract, in the proceedings
* encourage the tutorial authors to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]

== Organization of past tutorials ==
For the last years, the call and selection of tutorials has been coordinated among several major CL conferences. The table below summarizes some data collected from existing reports and from direct contacts with past tutorial chairs.

{| border="1" cellspacing="0" cellpadding="5"
|'''Year'''
|'''Coordinated conferences'''
|'''Tutorial chairs'''
|'''Submission platform'''
|'''# Submissions'''
|'''Reviewers'''
|'''# Reviews per proposal'''
|'''# Selected tutorials'''
|'''Selection criteria'''
|'''Useful links'''
|-
|2020
|ACL EMNLP AACL-IJCNLP COLING
|ACL: Agata Savary (France), Yue Zhang (China) EMNLP: Benjamin Van Durme (USA), Aline Villavicencio (UK & Brazil) AACL-IJCNLP: Tim Baldwin (Australia), Fei Xia (USA) COLING: Daniel Beck (Australia), Lucia Specia (UK)
|SoftConf
|43
|Tutorial chairs + external experts
|3
|ACL: 8 EMNLP: 7 ACCL-IJCNLP: 6 COLING: 7
|Clarity and preparedness Novelty or timely character of the topic Lecturers' experience Likely audience interest Open access of the teaching material Diversity aspects (multilingualism, gender, age and country of the lecturers) Compatibility with the required venues
|[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 Call] [https://acl2020.org/program/tutorials/ ACL tutorials] [https://2020.emnlp.org/tutorials EMNLP tutorials] [http://aacl2020.org/ ACCL-IJCNLP tutorials] [https://coling2020.org/pages/tutorials COLING tutorials]
|
|-
|2019
|NAACL ACL EMNLP
|ACL: Preslav Nakov (Qatar), Alexis Palmer (USA) NAACL: Anoop Sarkar (Canda), Michael Strube (Germany) EMNLP-IJCNLP: Marine Carpuat (USA), Tim Baldwin (Australia)
|SoftConf
|45
|Tutorial chairs
|2-3
|ACL: 9 NAACL: 6 EMNLP: 7
|Presenters (1-5) Description (1-5) Interest Level (1-5) Preparedness (1-5)
|[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials Call] [http://www.acl2019.org/EN/tutorials.xhtml ACL tutorials] [https://naacl2019.org/program/tutorials/ NAACL tutorials]
|-
|2018
|ACL NAACL EMNLP COLING
|ACL: Yoav Artzi (USA), Jacob Eisenstein (USA) COLING: Pascale Fung (Hong Kong), Donia Scott (UK), Marilyn Walker (USA) EMNLP: Mausam (India), Lu Wang (USA) NAACL: Mohit Bansal (USA), Rebecca Passonneau (USA)
|SoftConf
|49
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6 COLING: 6
|Quality of plan Diversity of topics Qualifications of presenter likely audience interest
|[https://acl2018.org/2017/09/11/joint-call-for-tutorials/ Call] 
[https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs Report] 
[https://acl2018.org/tutorials/ ACL tutorials] 
[https://emnlp2018.org/program/tutorials/ EMNLP tutorials] 
[https://coling2018.org/tutorial-list/ COLING tutorials]
|-
|2017
|ACL EACL EMNLP
|ACL: Maja Popović (Germany), Jordan Boyd-Graber (USA) EACL: Alex Klementiev (Germany), Lucia Specia (UK) EMNLP: Nathan Schneider (USA), Alexandra Birch (UK)
|SoftConf
|26
|Tutorial chairs
|3
|ACL: 6 EACL: 6 EMNLP: 7
|Clarity (description and outline) Topic
|[https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs Report] [http://acl2017.org/tutorials/ ACL tutorials] [http://eacl2017.org/index.php/tutorials EACL tutorials]
|-
|2016
|ACL NAACL EMNLP
|ACL: Alexandra Birch (UK), Willem Zuidema (Netherlands) NAACL: Alexander Rush and Bishan Yang EMNLP: Rebecca Hwa and Mohit Bansal
|SoftConf
|32
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6
|Relevance to ACL community (15) Quality of instructor (15) Outline and depth/breadth of the proposal (15) 
Potential attendance Newly emerging area not previously covered in an ACL-related tutorial (yes or no) Introduction into related fields (yes or no) 
Overall score (15) 1st preference for venue as indicated in proposal comments
|[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 Call] [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs Report] [http://naacl.org/naacl-hlt-2016/tutorials.html NAACL tutorials] [https://www.aclweb.org/mirror/emnlp2016/tutorials.html EMNLP tutorials]
|}

== Past calls for tutorials ==
*[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 ACL/AACL-IJCNLP/EMNLP/COLING 2020 joint call for tutorials]
*[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials NAACL/ACL/EMNLP 2019 joint call for tutorials]
*[https://www.aclweb.org/portal/content/final-joint-call-tutorial-proposals-aclcolingemnlpnaacl-2018 ACL/COLING/EMNLP/NAACL 2018 joint call for tutorials]
*[https://www.aclweb.org/portal/content/eacl-2017-call-tutorial-proposals EACL 2017 call for tutorials]
*[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 NAACL/ACL/EMNLP 2016 joint call for tutorials]
*[https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-naacl-2015-acl-2015-emnlp-2015-0 NAACL/ACL/EMNLP 2015 joint call for tutorials]

== Reports from past tutorial chairs ==
*[[2020Q3_Reports:_Tutorial_Chairs|ACL 2020 report]] (by Agata Savary and Yue Zhang)
*[[2019Q1_Reports:_ACL_2019#Tutorial_Co-Chairs|ACL 2019 report]] (by Preslav Nakov and Alexis Palmer)
*[[2018Q3 Reports: Tutorial Chairs|ACL 2018 report]] (by Yoav Artzi and Jacob Eisenstein)
*[[2017Q3 Reports: Tutorial Chairs|ACL 2017 report]] (by Maja Popović and Jordan Boyd-Graber)
*[[2016Q3 Reports: Tutorial Chairs|ACL 2016 report]] (by Alexandra Birch and Willem Zuidema)

== Lessons learned by past tutorial chairs ==
* [Agata Savary, 2020] If the conference is virtual:
** Remember that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]).
** Announce the individual modalities with the attendees early enough (for instance, if watching pre-recordings in advance is necessary for some tutorials because their live sessions only included question answering)
** If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to Zoom rooms only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.
* [Agata Savary, 2020] Use a common SoftConf space for joint tutorial submissions for all coordinated conference. Once the proposals have been selected and assigned to conferences, use one track per conference in the same SoftConf space.
* [Agata Savary, 2020] Publish the list of recommended readings on each tutorial page
* [Agata Savary, 2020] A good way to proceed while selecting tutorials for several venues is to have a common online meeting, in which
** Tutorials are clustered per themes (to distribute similar tutorials over different conferences)
** The chairs for each venue select their best choices in a round robin manner.
** Generate csv files with reviews and scores: Manager -> Other tools -> Spreadsheet maker -> Reports
* [Agata Savary, 2020] While setting up a review form on SoftConf, include:
** All evaluation criteria, including diversity, in the review form
** Cutting edge vs. introductory category
** Duration (3 or 6 hours)
* [Agata Savary, 2020] Currently, the generic conference schedule assumes only 2 months between the call for tutorials and notification to proposers. If a solid 1-month review procedure is to take place, this means, that proposers only have 1 month between the call and the submission deadline. It would be good to shift the call to [T-12].
* [Agata Savary, 2020] For many past tutorials, no slides are available online. It is, therefore, important to promote Open Access of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation slides on the ACL Anthology. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]. As a tutorial chair, make sure that
** initial versions of tutorial slides are submitted to the proceedings together with the tutorial abstracts
** tutorial authors are encouraged to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]
* [Marine Capuat, 2019] Continue soliciting both cutting edge and introductory tutorials
* [Marine Capuat, 2019] Ask proposers to express their preferred location/conference in one place (i.e. either in proposal pdf, or in softconf form, not both) to avoid inconsistencies
* [ACL 2018 chairs] Issues with proceedings edition, when one joint SoftConf space is used for many conferences (see the [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs report])
* [ACL 2016 chairs] Avoid confusion about the submission procedures for the final tutorial descriptions, those were submitted by email in many different formats

Author: Agata Savary, July 2019-July 2020

Updates are welcome

[[Category:Conference Handbook]]

Tutorial chair handbook

2020-07-15T18:27:45Z

AgataSavary:

== Duties ==
Tutorial chairs are nominated by the general chair of the main conference. Their duties are described in [[Tutorial chair duties|here]].

== Timeline ==
The timeline for the organisation of tutorials should follow the general [[conference planning schedule]]. Below, we complete this schedule with more detailed tasks, as defined for the ACL 2020 tutorials. The dates from the [[conference planning schedule]] are highlighted in '''bold'''. 
In recent years, calls and selection of tutorial proposals have been coordinated among some major CL conferences (usually a subset of ACL, NAACL, EMNLP, AACL and COLING). Note that if your conference is the first in the year to come, you will need to be the major driving force in the initial steps described below. 
Tutorial proposals are usually reviewed by the tutorial chairs of the coordinating conferences, but it may be a good idea to also rectruit a small group of external reviewers. They should have a good experience in reviewing and a large understanding of the recent progress in the CL domain.

* [T-12]
** Learn from the general chair which conferences will have their tutorials coordinated with yours.
** Establish contact with the tutorial chairs of these other conferences, and discuss
*** Recruitment of external reviewers
*** Proposal evaluation criteria
*** Soliciting tutorial proposals from particular potential lecturers on particular topics
*** Organizing a public poll about interest in the submitted tutorial
** Discuss general strategies of the main conference with the general chair and the program chairs
***promotion of some topics
***mailing lists and conference alert portals to be used for all announcements
** Set up a SoftConf space for proposal submission
*** Solicit the creation of a SoftConf space from [mailto:support@softconf.com support] (currently Rich Gerber). Note that submissions will be joint for several conferences but the proceedings will be edited for each conference separately. Therefore, a new SoftConf space will have to be created later for the selected proposals of each conference.
*** If you plan to have external tutorial reviewers, invite them via SoftConf.
*** Set up a submission page.
** Set up a joint email for tutorial chairs.
** Confirm the number (usually 6-8) and length (usually 3 hours) of tutorial slots with the ACL Business Manager (currently Priscilla Rasmussen).
** Draft the joint call for tutorials (based on [[#Past calls for tutorials|past calls]]).

* '''[T-11]''' ''(in the future, it would be good to extend the time slot between the call and the deadline to 2 months at least, especially if July/August is included)''
** Send out the call for tutorials.
** For posting the call on the ACL website, contact the ACL Business Manager (currently Priscilla Rasmussen).
** If you planned to encourage submission of proposals from particular potential lecturers, contact them now.
** Set up a SoftConf page for proposal reviews.

* [T-10]
** Deadline for tutorial proposal submission.
** If you decided to run a public poll about interest in the submitted tutorial, set it up and send out a call for expression of interest.
** Assign proposals to reviewers, send out invitations to review (via SoftConf).

* '''[T-9]'''
** Select tutorials jointly with other conference chairs.
** Notify the authors (via SoftConf).
** Liaise with local chair for space needs and scheduling.
** Send instructions to tutorial teachers about printed material, agenda/time management, payments, fees, etc.
** Liaise the conference website chair for updates about tutorials and authors' guidelines.
** Liaise with the ACL Office for the registration form.

* '''[T-4]'''
** Collect short descriptions for proceedings from tutorial teachers.
** Provide them to the webmaster and the general and program chairs .

* '''[T-3]'''
** Confirm the tutorials with authors.
** Authors submit the final tutorial descriptions for the proceedings.
** Arrange for printing of tutorial materials.

* [T-2]
** Arrange with the [mailto:support@softconf.com SoftConf support] the creation of a separate SoftConf space for the selected tutorials of your conference only. Indicate which tutorial submissions should be copied there.
** Set up the final submission form in the new SoftConf space. Include the submissions of slides, which will be published on the ACL Anthology. Include signing the CC-BY-4.0 license agreement.

* '''[T-1]'''
** Collect and check final tutorial materials.
** Create the tutorial proceedings, including slides.

* '''[T-0]''' Conference and tutorials

* [T+1]
** Invite the lecturers to update the slides at the ACL Anthology via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form] (under CC-BY-4.0).
** Update this tutorial page and the list of [[Past tutorials|past tutorials]] with the data from your conference. For access to the ACL [[Conference Handbook]], contact the ACL Secretary (currently Shiqi Zhou).

== Publication of tutorials ==
For NAACL 2019, ACL Anthology stores the tutorial descriptions, slides and video recordings. The same should go for ACL 2020 (which went fully virtual due to the COVID pandemic), except that recordings of live sessions are sometimes replaced with pre-recordings.

In tutorials organized until 2019, there was no homogeneous policy as to the publication of tutorial material. Data (abstracts, slides, code, etc.) from selected tutorials were usually scattered in 2 or 3 places:
* conference website
* ACL Anthology
* private repositories of the authors
The sustainability of these varies from venue to venue. The main websites of some conferences are not maintained, therefore ACL collects [https://www.aclweb.org/portal/node/2481 mirrors] of some of them (but not all).
The tutorial proceedings, published in the ACL Anthology, until recently included only abstracts but no tutorial slides.

As a result, for many [[past tutorials]], no slides are available online, so the community cannot benefit from these assets. It is, therefore, important to promote '''Open Access''' of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation '''slides in the ACL Anthology'''. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form].

It is recommended for tutorial chairs to:
* announce the publication of the tutorial slides under the CC-BY-4.0 license (used in the ACL Anthology)
* encourage the authors to also publish, under open licenses, their code and data used in the tutorials
* include the initial versions of tutorial slides, together with abstract, in the proceedings
* encourage the tutorial authors to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]

== Tutorials in a virtual conference ==
ACL 2020 was transformed into a fully virtual conference due to the COVID-19 pandemic. This decision was taken only 3 months before the conference and added a bunch of new tasks on top of the usual organization. This new organization was largely to be invented from scratch, additionally to the previous experience from [[https://icml.cc/|ICML]].

== Organization of past tutorials ==
For the last years, the call and selection of tutorials has been coordinated among several major CL conferences. The table below summarizes some data collected from existing reports and from direct contacts with past tutorial chairs.

{| border="1" cellspacing="0" cellpadding="5"
|'''Year'''
|'''Coordinated conferences'''
|'''Tutorial chairs'''
|'''Submission platform'''
|'''# Submissions'''
|'''Reviewers'''
|'''# Reviews per proposal'''
|'''# Selected tutorials'''
|'''Selection criteria'''
|'''Useful links'''
|-
|2020
|ACL EMNLP AACL-IJCNLP COLING
|ACL: Agata Savary (France), Yue Zhang (China) EMNLP: Benjamin Van Durme (USA), Aline Villavicencio (UK & Brazil) AACL-IJCNLP: Tim Baldwin (Australia), Fei Xia (USA) COLING: Daniel Beck (Australia), Lucia Specia (UK)
|SoftConf
|43
|Tutorial chairs + external experts
|3
|ACL: 8 EMNLP: 7 ACCL-IJCNLP: 6 COLING: 7
|Clarity and preparedness Novelty or timely character of the topic Lecturers' experience Likely audience interest Open access of the teaching material Diversity aspects (multilingualism, gender, age and country of the lecturers) Compatibility with the required venues
|[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 Call] [https://acl2020.org/program/tutorials/ ACL tutorials] [https://2020.emnlp.org/tutorials EMNLP tutorials] [http://aacl2020.org/ ACCL-IJCNLP tutorials] [https://coling2020.org/pages/tutorials COLING tutorials]
|
|-
|2019
|NAACL ACL EMNLP
|ACL: Preslav Nakov (Qatar), Alexis Palmer (USA) NAACL: Anoop Sarkar (Canda), Michael Strube (Germany) EMNLP-IJCNLP: Marine Carpuat (USA), Tim Baldwin (Australia)
|SoftConf
|45
|Tutorial chairs
|2-3
|ACL: 9 NAACL: 6 EMNLP: 7
|Presenters (1-5) Description (1-5) Interest Level (1-5) Preparedness (1-5)
|[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials Call] [http://www.acl2019.org/EN/tutorials.xhtml ACL tutorials] [https://naacl2019.org/program/tutorials/ NAACL tutorials]
|-
|2018
|ACL NAACL EMNLP COLING
|ACL: Yoav Artzi (USA), Jacob Eisenstein (USA) COLING: Pascale Fung (Hong Kong), Donia Scott (UK), Marilyn Walker (USA) EMNLP: Mausam (India), Lu Wang (USA) NAACL: Mohit Bansal (USA), Rebecca Passonneau (USA)
|SoftConf
|49
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6 COLING: 6
|Quality of plan Diversity of topics Qualifications of presenter likely audience interest
|[https://acl2018.org/2017/09/11/joint-call-for-tutorials/ Call] 
[https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs Report] 
[https://acl2018.org/tutorials/ ACL tutorials] 
[https://emnlp2018.org/program/tutorials/ EMNLP tutorials] 
[https://coling2018.org/tutorial-list/ COLING tutorials]
|-
|2017
|ACL EACL EMNLP
|ACL: Maja Popović (Germany), Jordan Boyd-Graber (USA) EACL: Alex Klementiev (Germany), Lucia Specia (UK) EMNLP: Nathan Schneider (USA), Alexandra Birch (UK)
|SoftConf
|26
|Tutorial chairs
|3
|ACL: 6 EACL: 6 EMNLP: 7
|Clarity (description and outline) Topic
|[https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs Report] [http://acl2017.org/tutorials/ ACL tutorials] [http://eacl2017.org/index.php/tutorials EACL tutorials]
|-
|2016
|ACL NAACL EMNLP
|ACL: Alexandra Birch (UK), Willem Zuidema (Netherlands) NAACL: Alexander Rush and Bishan Yang EMNLP: Rebecca Hwa and Mohit Bansal
|SoftConf
|32
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6
|Relevance to ACL community (15) Quality of instructor (15) Outline and depth/breadth of the proposal (15) 
Potential attendance Newly emerging area not previously covered in an ACL-related tutorial (yes or no) Introduction into related fields (yes or no) 
Overall score (15) 1st preference for venue as indicated in proposal comments
|[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 Call] [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs Report] [http://naacl.org/naacl-hlt-2016/tutorials.html NAACL tutorials] [https://www.aclweb.org/mirror/emnlp2016/tutorials.html EMNLP tutorials]
|}

== Past calls for tutorials ==
*[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 ACL/AACL-IJCNLP/EMNLP/COLING 2020 joint call for tutorials]
*[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials NAACL/ACL/EMNLP 2019 joint call for tutorials]
*[https://www.aclweb.org/portal/content/final-joint-call-tutorial-proposals-aclcolingemnlpnaacl-2018 ACL/COLING/EMNLP/NAACL 2018 joint call for tutorials]
*[https://www.aclweb.org/portal/content/eacl-2017-call-tutorial-proposals EACL 2017 call for tutorials]
*[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 NAACL/ACL/EMNLP 2016 joint call for tutorials]
*[https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-naacl-2015-acl-2015-emnlp-2015-0 NAACL/ACL/EMNLP 2015 joint call for tutorials]

== Reports from past tutorial chairs ==
*[[2020Q3_Reports:_Tutorial_Chairs|ACL 2020 report]] (by Agata Savary and Yue Zhang)
*[[2019Q1_Reports:_ACL_2019#Tutorial_Co-Chairs|ACL 2019 report]] (by Preslav Nakov and Alexis Palmer)
*[[2018Q3 Reports: Tutorial Chairs|ACL 2018 report]] (by Yoav Artzi and Jacob Eisenstein)
*[[2017Q3 Reports: Tutorial Chairs|ACL 2017 report]] (by Maja Popović and Jordan Boyd-Graber)
*[[2016Q3 Reports: Tutorial Chairs|ACL 2016 report]] (by Alexandra Birch and Willem Zuidema)

== Lessons learned by past tutorial chairs ==
* [Agata Savary, 2020] If the conference is virtual:
** Remember that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]).
** Announce the individual modalities with the attendees early enough (for instance, if watching pre-recordings in advance is necessary for some tutorials because their live sessions only included question answering)
** If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to Zoom rooms only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.
* [Agata Savary, 2020] Use a common SoftConf space for joint tutorial submissions for all coordinated conference. Once the proposals have been selected and assigned to conferences, use one track per conference in the same SoftConf space.
* [Agata Savary, 2020] Publish the list of recommended readings on each tutorial page
* [Agata Savary, 2020] A good way to proceed while selecting tutorials for several venues is to have a common online meeting, in which
** Tutorials are clustered per themes (to distribute similar tutorials over different conferences)
** The chairs for each venue select their best choices in a round robin manner.
** Generate csv files with reviews and scores: Manager -> Other tools -> Spreadsheet maker -> Reports
* [Agata Savary, 2020] While setting up a review form on SoftConf, include:
** All evaluation criteria, including diversity, in the review form
** Cutting edge vs. introductory category
** Duration (3 or 6 hours)
* [Agata Savary, 2020] Currently, the generic conference schedule assumes only 2 months between the call for tutorials and notification to proposers. If a solid 1-month review procedure is to take place, this means, that proposers only have 1 month between the call and the submission deadline. It would be good to shift the call to [T-12].
* [Agata Savary, 2020] For many past tutorials, no slides are available online. It is, therefore, important to promote Open Access of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation slides on the ACL Anthology. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]. As a tutorial chair, make sure that
** initial versions of tutorial slides are submitted to the proceedings together with the tutorial abstracts
** tutorial authors are encouraged to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]
* [Marine Capuat, 2019] Continue soliciting both cutting edge and introductory tutorials
* [Marine Capuat, 2019] Ask proposers to express their preferred location/conference in one place (i.e. either in proposal pdf, or in softconf form, not both) to avoid inconsistencies
* [ACL 2018 chairs] Issues with proceedings edition, when one joint SoftConf space is used for many conferences (see the [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs report])
* [ACL 2016 chairs] Avoid confusion about the submission procedures for the final tutorial descriptions, those were submitted by email in many different formats

Author: Agata Savary, July 2019-July 2020

Updates are welcome

[[Category:Conference Handbook]]

Tutorial chair handbook

2020-07-15T17:37:19Z

AgataSavary: /* Publication of tutorials */

== Duties ==
Tutorial chairs are nominated by the general chair of the main conference. Their duties are described in [[Tutorial chair duties|here]].

== Timeline ==
The timeline for the organisation of tutorials should follow the general [[conference planning schedule]]. Below, we complete this schedule with more detailed tasks, as defined for the ACL 2020 tutorials. The dates from the [[conference planning schedule]] are highlighted in '''bold'''. 
In recent years, calls and selection of tutorial proposals have been coordinated among some major CL conferences (usually a subset of ACL, NAACL, EMNLP, AACL and COLING). Note that if your conference is the first in the year to come, you will need to be the major driving force in the initial steps described below. 
Tutorial proposals are usually reviewed by the tutorial chairs of the coordinating conferences, but it may be a good idea to also rectruit a small group of external reviewers. They should have a good experience in reviewing and a large understanding of the recent progress in the CL domain.

* [T-12]
** Learn from the general chair which conferences will have their tutorials coordinated with yours.
** Establish contact with the tutorial chairs of these other conferences, and discuss
*** Recruitment of external reviewers
*** Proposal evaluation criteria
*** Soliciting tutorial proposals from particular potential lecturers on particular topics
*** Organizing a public poll about interest in the submitted tutorial
** Discuss general strategies of the main conference with the general chair and the program chairs
***promotion of some topics
***mailing lists and conference alert portals to be used for all announcements
** Set up a SoftConf space for proposal submission
*** Solicit the creation of a SoftConf space from [mailto:support@softconf.com support] (currently Rich Gerber). Note that submissions will be joint for several conferences but the proceedings will be edited for each conference separately. Therefore, a new SoftConf space will have to be created later for the selected proposals of each conference.
*** If you plan to have external tutorial reviewers, invite them via SoftConf.
*** Set up a submission page.
** Set up a joint email for tutorial chairs.
** Confirm the number (usually 6-8) and length (usually 3 hours) of tutorial slots with the ACL Business Manager (currently Priscilla Rasmussen).
** Draft the joint call for tutorials (based on [[#Past calls for tutorials|past calls]]).

* '''[T-11]''' ''(in the future, it would be good to extend the time slot between the call and the deadline to 2 months at least, especially if July/August is included)''
** Send out the call for tutorials.
** For posting the call on the ACL website, contact the ACL Business Manager (currently Priscilla Rasmussen).
** If you planned to encourage submission of proposals from particular potential lecturers, contact them now.
** Set up a SoftConf page for proposal reviews.

* [T-10]
** Deadline for tutorial proposal submission.
** If you decided to run a public poll about interest in the submitted tutorial, set it up and send out a call for expression of interest.
** Assign proposals to reviewers, send out invitations to review (via SoftConf).

* '''[T-9]'''
** Select tutorials jointly with other conference chairs.
** Notify the authors (via SoftConf).
** Liaise with local chair for space needs and scheduling.
** Send instructions to tutorial teachers about printed material, agenda/time management, payments, fees, etc.
** Liaise the conference website chair for updates about tutorials and authors' guidelines.
** Liaise with the ACL Office for the registration form.

* '''[T-4]'''
** Collect short descriptions for proceedings from tutorial teachers.
** Provide them to the webmaster and the general and program chairs .

* '''[T-3]'''
** Confirm the tutorials with authors.
** Authors submit the final tutorial descriptions for the proceedings.
** Arrange for printing of tutorial materials.

* [T-2]
** Arrange with the [mailto:support@softconf.com SoftConf support] the creation of a separate SoftConf space for the selected tutorials of your conference only. Indicate which tutorial submissions should be copied there.
** Set up the final submission form in the new SoftConf space. Include the submissions of slides, which will be published on the ACL Anthology. Include signing the CC-BY-4.0 license agreement.

* '''[T-1]'''
** Collect and check final tutorial materials.
** Create the tutorial proceedings, including slides.

* '''[T-0]''' Conference and tutorials

* [T+1]
** Invite the lecturers to update the slides at the ACL Anthology via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form] (under CC-BY-4.0).
** Update this tutorial page and the list of [[Past tutorials|past tutorials]] with the data from your conference. For access to the ACL [[Conference Handbook]], contact the ACL Secretary (currently Shiqi Zhou).

== Publication of tutorials ==
For NAACL 2019, ACL Anthology stores the tutorial descriptions, slides and video recordings. The same should go for ACL 2020 (which went fully virtual due to the COVID pandemic), except that recordings of live sessions are sometimes replaced with pre-recordings.

In tutorials organized until 2019, there was no homogeneous policy as to the publication of tutorial material. Data (abstracts, slides, code, etc.) from selected tutorials were usually scattered in 2 or 3 places:
* conference website
* ACL Anthology
* private repositories of the authors
The sustainability of these varies from venue to venue. The main websites of some conferences are not maintained, therefore ACL collects [https://www.aclweb.org/portal/node/2481 mirrors] of some of them (but not all).
The tutorial proceedings, published in the ACL Anthology, until recently included only abstracts but no tutorial slides.

As a result, for many [[past tutorials]], no slides are available online, so the community cannot benefit from these assets. It is, therefore, important to promote '''Open Access''' of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation '''slides in the ACL Anthology'''. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form].

It is recommended for tutorial chairs to:
* announce the publication of the tutorial slides under the CC-BY-4.0 license (used in the ACL Anthology)
* encourage the authors to also publish, under open licenses, their code and data used in the tutorials
* include the initial versions of tutorial slides, together with abstract, in the proceedings
* encourage the tutorial authors to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]

== Past tutorials ==
The [[Past tutorials|list of recent tutorials]] contains tutorial authors' names, titles, and to their summaries in conference websites and/or in the ACL Anthology.

== Organization of past tutorials ==
For the last years, the call and selection of tutorials has been coordinated among several major CL conferences. The table below summarizes some data collected from existing reports and from direct contacts with past tutorial chairs.

{| border="1" cellspacing="0" cellpadding="5"
|'''Year'''
|'''Coordinated conferences'''
|'''Tutorial chairs'''
|'''Submission platform'''
|'''# Submissions'''
|'''Reviewers'''
|'''# Reviews per proposal'''
|'''# Selected tutorials'''
|'''Selection criteria'''
|'''Useful links'''
|-
|2020
|ACL EMNLP AACL-IJCNLP COLING
|ACL: Agata Savary (France), Yue Zhang (China) EMNLP: Benjamin Van Durme (USA), Aline Villavicencio (UK & Brazil) AACL-IJCNLP: Tim Baldwin (Australia), Fei Xia (USA) COLING: Daniel Beck (Australia), Lucia Specia (UK)
|SoftConf
|43
|Tutorial chairs + external experts
|3
|ACL: 8 EMNLP: 7 ACCL-IJCNLP: 6 COLING: 7
|Clarity and preparedness Novelty or timely character of the topic Lecturers' experience Likely audience interest Open access of the teaching material Diversity aspects (multilingualism, gender, age and country of the lecturers) Compatibility with the required venues
|[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 Call] [https://acl2020.org/program/tutorials/ ACL tutorials] [https://2020.emnlp.org/tutorials EMNLP tutorials] [http://aacl2020.org/ ACCL-IJCNLP tutorials] [https://coling2020.org/pages/tutorials COLING tutorials]
|
|-
|2019
|NAACL ACL EMNLP
|ACL: Preslav Nakov (Qatar), Alexis Palmer (USA) NAACL: Anoop Sarkar (Canda), Michael Strube (Germany) EMNLP-IJCNLP: Marine Carpuat (USA), Tim Baldwin (Australia)
|SoftConf
|45
|Tutorial chairs
|2-3
|ACL: 9 NAACL: 6 EMNLP: 7
|Presenters (1-5) Description (1-5) Interest Level (1-5) Preparedness (1-5)
|[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials Call] [http://www.acl2019.org/EN/tutorials.xhtml ACL tutorials] [https://naacl2019.org/program/tutorials/ NAACL tutorials]
|-
|2018
|ACL NAACL EMNLP COLING
|ACL: Yoav Artzi (USA), Jacob Eisenstein (USA) COLING: Pascale Fung (Hong Kong), Donia Scott (UK), Marilyn Walker (USA) EMNLP: Mausam (India), Lu Wang (USA) NAACL: Mohit Bansal (USA), Rebecca Passonneau (USA)
|SoftConf
|49
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6 COLING: 6
|Quality of plan Diversity of topics Qualifications of presenter likely audience interest
|[https://acl2018.org/2017/09/11/joint-call-for-tutorials/ Call] 
[https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs Report] 
[https://acl2018.org/tutorials/ ACL tutorials] 
[https://emnlp2018.org/program/tutorials/ EMNLP tutorials] 
[https://coling2018.org/tutorial-list/ COLING tutorials]
|-
|2017
|ACL EACL EMNLP
|ACL: Maja Popović (Germany), Jordan Boyd-Graber (USA) EACL: Alex Klementiev (Germany), Lucia Specia (UK) EMNLP: Nathan Schneider (USA), Alexandra Birch (UK)
|SoftConf
|26
|Tutorial chairs
|3
|ACL: 6 EACL: 6 EMNLP: 7
|Clarity (description and outline) Topic
|[https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs Report] [http://acl2017.org/tutorials/ ACL tutorials] [http://eacl2017.org/index.php/tutorials EACL tutorials]
|-
|2016
|ACL NAACL EMNLP
|ACL: Alexandra Birch (UK), Willem Zuidema (Netherlands) NAACL: Alexander Rush and Bishan Yang EMNLP: Rebecca Hwa and Mohit Bansal
|SoftConf
|32
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6
|Relevance to ACL community (15) Quality of instructor (15) Outline and depth/breadth of the proposal (15) 
Potential attendance Newly emerging area not previously covered in an ACL-related tutorial (yes or no) Introduction into related fields (yes or no) 
Overall score (15) 1st preference for venue as indicated in proposal comments
|[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 Call] [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs Report] [http://naacl.org/naacl-hlt-2016/tutorials.html NAACL tutorials] [https://www.aclweb.org/mirror/emnlp2016/tutorials.html EMNLP tutorials]
|}

== Past calls for tutorials ==
*[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 ACL/AACL-IJCNLP/EMNLP/COLING 2020 joint call for tutorials]
*[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials NAACL/ACL/EMNLP 2019 joint call for tutorials]
*[https://www.aclweb.org/portal/content/final-joint-call-tutorial-proposals-aclcolingemnlpnaacl-2018 ACL/COLING/EMNLP/NAACL 2018 joint call for tutorials]
*[https://www.aclweb.org/portal/content/eacl-2017-call-tutorial-proposals EACL 2017 call for tutorials]
*[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 NAACL/ACL/EMNLP 2016 joint call for tutorials]
*[https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-naacl-2015-acl-2015-emnlp-2015-0 NAACL/ACL/EMNLP 2015 joint call for tutorials]

== Reports from past tutorial chairs ==
*[[2020Q3_Reports:_Tutorial_Chairs|ACL 2020 report]] (by Agata Savary and Yue Zhang)
*[[2019Q1_Reports:_ACL_2019#Tutorial_Co-Chairs|ACL 2019 report]] (by Preslav Nakov and Alexis Palmer)
*[[2018Q3 Reports: Tutorial Chairs|ACL 2018 report]] (by Yoav Artzi and Jacob Eisenstein)
*[[2017Q3 Reports: Tutorial Chairs|ACL 2017 report]] (by Maja Popović and Jordan Boyd-Graber)
*[[2016Q3 Reports: Tutorial Chairs|ACL 2016 report]] (by Alexandra Birch and Willem Zuidema)

== Lessons learned by past tutorial chairs ==
* [Agata Savary, 2020] If the conference is virtual:
** Remember that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]).
** Announce the individual modalities with the attendees early enough (for instance, if watching pre-recordings in advance is necessary for some tutorials because their live sessions only included question answering)
** If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to Zoom rooms only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.
* [Agata Savary, 2020] Use a common SoftConf space for joint tutorial submissions for all coordinated conference. Once the proposals have been selected and assigned to conferences, use one track per conference in the same SoftConf space.
* [Agata Savary, 2020] Publish the list of recommended readings on each tutorial page
* [Agata Savary, 2020] A good way to proceed while selecting tutorials for several venues is to have a common online meeting, in which
** Tutorials are clustered per themes (to distribute similar tutorials over different conferences)
** The chairs for each venue select their best choices in a round robin manner.
** Generate csv files with reviews and scores: Manager -> Other tools -> Spreadsheet maker -> Reports
* [Agata Savary, 2020] While setting up a review form on SoftConf, include:
** All evaluation criteria, including diversity, in the review form
** Cutting edge vs. introductory category
** Duration (3 or 6 hours)
* [Agata Savary, 2020] Currently, the generic conference schedule assumes only 2 months between the call for tutorials and notification to proposers. If a solid 1-month review procedure is to take place, this means, that proposers only have 1 month between the call and the submission deadline. It would be good to shift the call to [T-12].
* [Agata Savary, 2020] For many past tutorials, no slides are available online. It is, therefore, important to promote Open Access of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation slides on the ACL Anthology. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]. As a tutorial chair, make sure that
** initial versions of tutorial slides are submitted to the proceedings together with the tutorial abstracts
** tutorial authors are encouraged to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]
* [Marine Capuat, 2019] Continue soliciting both cutting edge and introductory tutorials
* [Marine Capuat, 2019] Ask proposers to express their preferred location/conference in one place (i.e. either in proposal pdf, or in softconf form, not both) to avoid inconsistencies
* [ACL 2018 chairs] Issues with proceedings edition, when one joint SoftConf space is used for many conferences (see the [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs report])
* [ACL 2016 chairs] Avoid confusion about the submission procedures for the final tutorial descriptions, those were submitted by email in many different formats

Author: Agata Savary, July 2019-July 2020

Updates are welcome

[[Category:Conference Handbook]]

Tutorial chair handbook

2020-07-15T17:33:37Z

AgataSavary: /* Publication of tutorials */

== Duties ==
Tutorial chairs are nominated by the general chair of the main conference. Their duties are described in [[Tutorial chair duties|here]].

== Timeline ==
The timeline for the organisation of tutorials should follow the general [[conference planning schedule]]. Below, we complete this schedule with more detailed tasks, as defined for the ACL 2020 tutorials. The dates from the [[conference planning schedule]] are highlighted in '''bold'''. 
In recent years, calls and selection of tutorial proposals have been coordinated among some major CL conferences (usually a subset of ACL, NAACL, EMNLP, AACL and COLING). Note that if your conference is the first in the year to come, you will need to be the major driving force in the initial steps described below. 
Tutorial proposals are usually reviewed by the tutorial chairs of the coordinating conferences, but it may be a good idea to also rectruit a small group of external reviewers. They should have a good experience in reviewing and a large understanding of the recent progress in the CL domain.

* [T-12]
** Learn from the general chair which conferences will have their tutorials coordinated with yours.
** Establish contact with the tutorial chairs of these other conferences, and discuss
*** Recruitment of external reviewers
*** Proposal evaluation criteria
*** Soliciting tutorial proposals from particular potential lecturers on particular topics
*** Organizing a public poll about interest in the submitted tutorial
** Discuss general strategies of the main conference with the general chair and the program chairs
***promotion of some topics
***mailing lists and conference alert portals to be used for all announcements
** Set up a SoftConf space for proposal submission
*** Solicit the creation of a SoftConf space from [mailto:support@softconf.com support] (currently Rich Gerber). Note that submissions will be joint for several conferences but the proceedings will be edited for each conference separately. Therefore, a new SoftConf space will have to be created later for the selected proposals of each conference.
*** If you plan to have external tutorial reviewers, invite them via SoftConf.
*** Set up a submission page.
** Set up a joint email for tutorial chairs.
** Confirm the number (usually 6-8) and length (usually 3 hours) of tutorial slots with the ACL Business Manager (currently Priscilla Rasmussen).
** Draft the joint call for tutorials (based on [[#Past calls for tutorials|past calls]]).

* '''[T-11]''' ''(in the future, it would be good to extend the time slot between the call and the deadline to 2 months at least, especially if July/August is included)''
** Send out the call for tutorials.
** For posting the call on the ACL website, contact the ACL Business Manager (currently Priscilla Rasmussen).
** If you planned to encourage submission of proposals from particular potential lecturers, contact them now.
** Set up a SoftConf page for proposal reviews.

* [T-10]
** Deadline for tutorial proposal submission.
** If you decided to run a public poll about interest in the submitted tutorial, set it up and send out a call for expression of interest.
** Assign proposals to reviewers, send out invitations to review (via SoftConf).

* '''[T-9]'''
** Select tutorials jointly with other conference chairs.
** Notify the authors (via SoftConf).
** Liaise with local chair for space needs and scheduling.
** Send instructions to tutorial teachers about printed material, agenda/time management, payments, fees, etc.
** Liaise the conference website chair for updates about tutorials and authors' guidelines.
** Liaise with the ACL Office for the registration form.

* '''[T-4]'''
** Collect short descriptions for proceedings from tutorial teachers.
** Provide them to the webmaster and the general and program chairs .

* '''[T-3]'''
** Confirm the tutorials with authors.
** Authors submit the final tutorial descriptions for the proceedings.
** Arrange for printing of tutorial materials.

* [T-2]
** Arrange with the [mailto:support@softconf.com SoftConf support] the creation of a separate SoftConf space for the selected tutorials of your conference only. Indicate which tutorial submissions should be copied there.
** Set up the final submission form in the new SoftConf space. Include the submissions of slides, which will be published on the ACL Anthology. Include signing the CC-BY-4.0 license agreement.

* '''[T-1]'''
** Collect and check final tutorial materials.
** Create the tutorial proceedings, including slides.

* '''[T-0]''' Conference and tutorials

* [T+1]
** Invite the lecturers to update the slides at the ACL Anthology via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form] (under CC-BY-4.0).
** Update this tutorial page and the list of [[Past tutorials|past tutorials]] with the data from your conference. For access to the ACL [[Conference Handbook]], contact the ACL Secretary (currently Shiqi Zhou).

== Publication of tutorials ==
For the 2019 and 2020 tutorials, archives of tutorial descriptions and/or slides exist at the ACL Anthology. For NAACL 2019 and ACL 2020 (which went fully virtual due to the COVID pandemic), video recordings and/or pre-recordings are additionally archived.

In tutorials organized until 2019, there was no homogeneous policy as to the publication of tutorial material. Data (abstracts, slides, code, etc.) from selected tutorials were usually scattered in 2 or 3 places:
* conference website
* ACL Anthology
* private repositories of the authors
The sustainability of these varies from venue to venue. The main websites of some conferences are not maintained, therefore ACL collects [https://www.aclweb.org/portal/node/2481 mirrors] of some of them (but not all).
The tutorial proceedings, published in the ACL Anthology, until recently included only abstracts but no tutorial slides.

As a result, for many [[past tutorials]], no slides are available online, so the community cannot benefit from these assets. It is, therefore, important to promote '''Open Access''' of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation '''slides in the ACL Anthology'''. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form].

It is recommended for tutorial chairs to:
* announce the publication of the tutorial slides under the CC-BY-4.0 license (used in the ACL Anthology)
* encourage the authors to also publish, under open licenses, their code and data used in the tutorials
* include the initial versions of tutorial slides, together with abstract, in the proceedings
* encourage the tutorial authors to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]

== Past tutorials ==
The [[Past tutorials|list of recent tutorials]] contains tutorial authors' names, titles, and to their summaries in conference websites and/or in the ACL Anthology.

== Organization of past tutorials ==
For the last years, the call and selection of tutorials has been coordinated among several major CL conferences. The table below summarizes some data collected from existing reports and from direct contacts with past tutorial chairs.

{| border="1" cellspacing="0" cellpadding="5"
|'''Year'''
|'''Coordinated conferences'''
|'''Tutorial chairs'''
|'''Submission platform'''
|'''# Submissions'''
|'''Reviewers'''
|'''# Reviews per proposal'''
|'''# Selected tutorials'''
|'''Selection criteria'''
|'''Useful links'''
|-
|2020
|ACL EMNLP AACL-IJCNLP COLING
|ACL: Agata Savary (France), Yue Zhang (China) EMNLP: Benjamin Van Durme (USA), Aline Villavicencio (UK & Brazil) AACL-IJCNLP: Tim Baldwin (Australia), Fei Xia (USA) COLING: Daniel Beck (Australia), Lucia Specia (UK)
|SoftConf
|43
|Tutorial chairs + external experts
|3
|ACL: 8 EMNLP: 7 ACCL-IJCNLP: 6 COLING: 7
|Clarity and preparedness Novelty or timely character of the topic Lecturers' experience Likely audience interest Open access of the teaching material Diversity aspects (multilingualism, gender, age and country of the lecturers) Compatibility with the required venues
|[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 Call] [https://acl2020.org/program/tutorials/ ACL tutorials] [https://2020.emnlp.org/tutorials EMNLP tutorials] [http://aacl2020.org/ ACCL-IJCNLP tutorials] [https://coling2020.org/pages/tutorials COLING tutorials]
|
|-
|2019
|NAACL ACL EMNLP
|ACL: Preslav Nakov (Qatar), Alexis Palmer (USA) NAACL: Anoop Sarkar (Canda), Michael Strube (Germany) EMNLP-IJCNLP: Marine Carpuat (USA), Tim Baldwin (Australia)
|SoftConf
|45
|Tutorial chairs
|2-3
|ACL: 9 NAACL: 6 EMNLP: 7
|Presenters (1-5) Description (1-5) Interest Level (1-5) Preparedness (1-5)
|[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials Call] [http://www.acl2019.org/EN/tutorials.xhtml ACL tutorials] [https://naacl2019.org/program/tutorials/ NAACL tutorials]
|-
|2018
|ACL NAACL EMNLP COLING
|ACL: Yoav Artzi (USA), Jacob Eisenstein (USA) COLING: Pascale Fung (Hong Kong), Donia Scott (UK), Marilyn Walker (USA) EMNLP: Mausam (India), Lu Wang (USA) NAACL: Mohit Bansal (USA), Rebecca Passonneau (USA)
|SoftConf
|49
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6 COLING: 6
|Quality of plan Diversity of topics Qualifications of presenter likely audience interest
|[https://acl2018.org/2017/09/11/joint-call-for-tutorials/ Call] 
[https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs Report] 
[https://acl2018.org/tutorials/ ACL tutorials] 
[https://emnlp2018.org/program/tutorials/ EMNLP tutorials] 
[https://coling2018.org/tutorial-list/ COLING tutorials]
|-
|2017
|ACL EACL EMNLP
|ACL: Maja Popović (Germany), Jordan Boyd-Graber (USA) EACL: Alex Klementiev (Germany), Lucia Specia (UK) EMNLP: Nathan Schneider (USA), Alexandra Birch (UK)
|SoftConf
|26
|Tutorial chairs
|3
|ACL: 6 EACL: 6 EMNLP: 7
|Clarity (description and outline) Topic
|[https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs Report] [http://acl2017.org/tutorials/ ACL tutorials] [http://eacl2017.org/index.php/tutorials EACL tutorials]
|-
|2016
|ACL NAACL EMNLP
|ACL: Alexandra Birch (UK), Willem Zuidema (Netherlands) NAACL: Alexander Rush and Bishan Yang EMNLP: Rebecca Hwa and Mohit Bansal
|SoftConf
|32
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6
|Relevance to ACL community (15) Quality of instructor (15) Outline and depth/breadth of the proposal (15) 
Potential attendance Newly emerging area not previously covered in an ACL-related tutorial (yes or no) Introduction into related fields (yes or no) 
Overall score (15) 1st preference for venue as indicated in proposal comments
|[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 Call] [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs Report] [http://naacl.org/naacl-hlt-2016/tutorials.html NAACL tutorials] [https://www.aclweb.org/mirror/emnlp2016/tutorials.html EMNLP tutorials]
|}

== Past calls for tutorials ==
*[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 ACL/AACL-IJCNLP/EMNLP/COLING 2020 joint call for tutorials]
*[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials NAACL/ACL/EMNLP 2019 joint call for tutorials]
*[https://www.aclweb.org/portal/content/final-joint-call-tutorial-proposals-aclcolingemnlpnaacl-2018 ACL/COLING/EMNLP/NAACL 2018 joint call for tutorials]
*[https://www.aclweb.org/portal/content/eacl-2017-call-tutorial-proposals EACL 2017 call for tutorials]
*[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 NAACL/ACL/EMNLP 2016 joint call for tutorials]
*[https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-naacl-2015-acl-2015-emnlp-2015-0 NAACL/ACL/EMNLP 2015 joint call for tutorials]

== Reports from past tutorial chairs ==
*[[2020Q3_Reports:_Tutorial_Chairs|ACL 2020 report]] (by Agata Savary and Yue Zhang)
*[[2019Q1_Reports:_ACL_2019#Tutorial_Co-Chairs|ACL 2019 report]] (by Preslav Nakov and Alexis Palmer)
*[[2018Q3 Reports: Tutorial Chairs|ACL 2018 report]] (by Yoav Artzi and Jacob Eisenstein)
*[[2017Q3 Reports: Tutorial Chairs|ACL 2017 report]] (by Maja Popović and Jordan Boyd-Graber)
*[[2016Q3 Reports: Tutorial Chairs|ACL 2016 report]] (by Alexandra Birch and Willem Zuidema)

== Lessons learned by past tutorial chairs ==
* [Agata Savary, 2020] If the conference is virtual:
** Remember that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]).
** Announce the individual modalities with the attendees early enough (for instance, if watching pre-recordings in advance is necessary for some tutorials because their live sessions only included question answering)
** If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to Zoom rooms only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.
* [Agata Savary, 2020] Use a common SoftConf space for joint tutorial submissions for all coordinated conference. Once the proposals have been selected and assigned to conferences, use one track per conference in the same SoftConf space.
* [Agata Savary, 2020] Publish the list of recommended readings on each tutorial page
* [Agata Savary, 2020] A good way to proceed while selecting tutorials for several venues is to have a common online meeting, in which
** Tutorials are clustered per themes (to distribute similar tutorials over different conferences)
** The chairs for each venue select their best choices in a round robin manner.
** Generate csv files with reviews and scores: Manager -> Other tools -> Spreadsheet maker -> Reports
* [Agata Savary, 2020] While setting up a review form on SoftConf, include:
** All evaluation criteria, including diversity, in the review form
** Cutting edge vs. introductory category
** Duration (3 or 6 hours)
* [Agata Savary, 2020] Currently, the generic conference schedule assumes only 2 months between the call for tutorials and notification to proposers. If a solid 1-month review procedure is to take place, this means, that proposers only have 1 month between the call and the submission deadline. It would be good to shift the call to [T-12].
* [Agata Savary, 2020] For many past tutorials, no slides are available online. It is, therefore, important to promote Open Access of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation slides on the ACL Anthology. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]. As a tutorial chair, make sure that
** initial versions of tutorial slides are submitted to the proceedings together with the tutorial abstracts
** tutorial authors are encouraged to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]
* [Marine Capuat, 2019] Continue soliciting both cutting edge and introductory tutorials
* [Marine Capuat, 2019] Ask proposers to express their preferred location/conference in one place (i.e. either in proposal pdf, or in softconf form, not both) to avoid inconsistencies
* [ACL 2018 chairs] Issues with proceedings edition, when one joint SoftConf space is used for many conferences (see the [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs report])
* [ACL 2016 chairs] Avoid confusion about the submission procedures for the final tutorial descriptions, those were submitted by email in many different formats

Author: Agata Savary, July 2019-July 2020

Updates are welcome

[[Category:Conference Handbook]]

Tutorial chair handbook

2020-07-15T17:32:50Z

AgataSavary: /* Publication of tutorials */

== Duties ==
Tutorial chairs are nominated by the general chair of the main conference. Their duties are described in [[Tutorial chair duties|here]].

== Timeline ==
The timeline for the organisation of tutorials should follow the general [[conference planning schedule]]. Below, we complete this schedule with more detailed tasks, as defined for the ACL 2020 tutorials. The dates from the [[conference planning schedule]] are highlighted in '''bold'''. 
In recent years, calls and selection of tutorial proposals have been coordinated among some major CL conferences (usually a subset of ACL, NAACL, EMNLP, AACL and COLING). Note that if your conference is the first in the year to come, you will need to be the major driving force in the initial steps described below. 
Tutorial proposals are usually reviewed by the tutorial chairs of the coordinating conferences, but it may be a good idea to also rectruit a small group of external reviewers. They should have a good experience in reviewing and a large understanding of the recent progress in the CL domain.

* [T-12]
** Learn from the general chair which conferences will have their tutorials coordinated with yours.
** Establish contact with the tutorial chairs of these other conferences, and discuss
*** Recruitment of external reviewers
*** Proposal evaluation criteria
*** Soliciting tutorial proposals from particular potential lecturers on particular topics
*** Organizing a public poll about interest in the submitted tutorial
** Discuss general strategies of the main conference with the general chair and the program chairs
***promotion of some topics
***mailing lists and conference alert portals to be used for all announcements
** Set up a SoftConf space for proposal submission
*** Solicit the creation of a SoftConf space from [mailto:support@softconf.com support] (currently Rich Gerber). Note that submissions will be joint for several conferences but the proceedings will be edited for each conference separately. Therefore, a new SoftConf space will have to be created later for the selected proposals of each conference.
*** If you plan to have external tutorial reviewers, invite them via SoftConf.
*** Set up a submission page.
** Set up a joint email for tutorial chairs.
** Confirm the number (usually 6-8) and length (usually 3 hours) of tutorial slots with the ACL Business Manager (currently Priscilla Rasmussen).
** Draft the joint call for tutorials (based on [[#Past calls for tutorials|past calls]]).

* '''[T-11]''' ''(in the future, it would be good to extend the time slot between the call and the deadline to 2 months at least, especially if July/August is included)''
** Send out the call for tutorials.
** For posting the call on the ACL website, contact the ACL Business Manager (currently Priscilla Rasmussen).
** If you planned to encourage submission of proposals from particular potential lecturers, contact them now.
** Set up a SoftConf page for proposal reviews.

* [T-10]
** Deadline for tutorial proposal submission.
** If you decided to run a public poll about interest in the submitted tutorial, set it up and send out a call for expression of interest.
** Assign proposals to reviewers, send out invitations to review (via SoftConf).

* '''[T-9]'''
** Select tutorials jointly with other conference chairs.
** Notify the authors (via SoftConf).
** Liaise with local chair for space needs and scheduling.
** Send instructions to tutorial teachers about printed material, agenda/time management, payments, fees, etc.
** Liaise the conference website chair for updates about tutorials and authors' guidelines.
** Liaise with the ACL Office for the registration form.

* '''[T-4]'''
** Collect short descriptions for proceedings from tutorial teachers.
** Provide them to the webmaster and the general and program chairs .

* '''[T-3]'''
** Confirm the tutorials with authors.
** Authors submit the final tutorial descriptions for the proceedings.
** Arrange for printing of tutorial materials.

* [T-2]
** Arrange with the [mailto:support@softconf.com SoftConf support] the creation of a separate SoftConf space for the selected tutorials of your conference only. Indicate which tutorial submissions should be copied there.
** Set up the final submission form in the new SoftConf space. Include the submissions of slides, which will be published on the ACL Anthology. Include signing the CC-BY-4.0 license agreement.

* '''[T-1]'''
** Collect and check final tutorial materials.
** Create the tutorial proceedings, including slides.

* '''[T-0]''' Conference and tutorials

* [T+1]
** Invite the lecturers to update the slides at the ACL Anthology via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form] (under CC-BY-4.0).
** Update this tutorial page and the list of [[Past tutorials|past tutorials]] with the data from your conference. For access to the ACL [[Conference Handbook]], contact the ACL Secretary (currently Shiqi Zhou).

== Publication of tutorials ==
For the 2019 and 2020 tutorials, archives exist at the ACL Anthology. Additionally, for NAACL 2019 and ACL 2020 (which went fully virtual due to the COVID pandemic), video recordings and/or pre-recordings are additionally archived.

In tutorials organized until 2019, there was no homogeneous policy as to the publication of tutorial material. Data (abstracts, slides, code, etc.) from selected tutorials were usually scattered in 2 or 3 places:
* conference website
* ACL Anthology
* private repositories of the authors
The sustainability of these varies from venue to venue. The main websites of some conferences are not maintained, therefore ACL collects [https://www.aclweb.org/portal/node/2481 mirrors] of some of them (but not all).
The tutorial proceedings, published in the ACL Anthology, until recently included only abstracts but no tutorial slides.

As a result, for many [[past tutorials]], no slides are available online, so the community cannot benefit from these assets. It is, therefore, important to promote '''Open Access''' of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation '''slides in the ACL Anthology'''. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form].

It is recommended for tutorial chairs to:
* announce the publication of the tutorial slides under the CC-BY-4.0 license (used in the ACL Anthology)
* encourage the authors to also publish, under open licenses, their code and data used in the tutorials
* include the initial versions of tutorial slides, together with abstract, in the proceedings
* encourage the tutorial authors to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]

== Past tutorials ==
The [[Past tutorials|list of recent tutorials]] contains tutorial authors' names, titles, and to their summaries in conference websites and/or in the ACL Anthology.

== Organization of past tutorials ==
For the last years, the call and selection of tutorials has been coordinated among several major CL conferences. The table below summarizes some data collected from existing reports and from direct contacts with past tutorial chairs.

{| border="1" cellspacing="0" cellpadding="5"
|'''Year'''
|'''Coordinated conferences'''
|'''Tutorial chairs'''
|'''Submission platform'''
|'''# Submissions'''
|'''Reviewers'''
|'''# Reviews per proposal'''
|'''# Selected tutorials'''
|'''Selection criteria'''
|'''Useful links'''
|-
|2020
|ACL EMNLP AACL-IJCNLP COLING
|ACL: Agata Savary (France), Yue Zhang (China) EMNLP: Benjamin Van Durme (USA), Aline Villavicencio (UK & Brazil) AACL-IJCNLP: Tim Baldwin (Australia), Fei Xia (USA) COLING: Daniel Beck (Australia), Lucia Specia (UK)
|SoftConf
|43
|Tutorial chairs + external experts
|3
|ACL: 8 EMNLP: 7 ACCL-IJCNLP: 6 COLING: 7
|Clarity and preparedness Novelty or timely character of the topic Lecturers' experience Likely audience interest Open access of the teaching material Diversity aspects (multilingualism, gender, age and country of the lecturers) Compatibility with the required venues
|[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 Call] [https://acl2020.org/program/tutorials/ ACL tutorials] [https://2020.emnlp.org/tutorials EMNLP tutorials] [http://aacl2020.org/ ACCL-IJCNLP tutorials] [https://coling2020.org/pages/tutorials COLING tutorials]
|
|-
|2019
|NAACL ACL EMNLP
|ACL: Preslav Nakov (Qatar), Alexis Palmer (USA) NAACL: Anoop Sarkar (Canda), Michael Strube (Germany) EMNLP-IJCNLP: Marine Carpuat (USA), Tim Baldwin (Australia)
|SoftConf
|45
|Tutorial chairs
|2-3
|ACL: 9 NAACL: 6 EMNLP: 7
|Presenters (1-5) Description (1-5) Interest Level (1-5) Preparedness (1-5)
|[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials Call] [http://www.acl2019.org/EN/tutorials.xhtml ACL tutorials] [https://naacl2019.org/program/tutorials/ NAACL tutorials]
|-
|2018
|ACL NAACL EMNLP COLING
|ACL: Yoav Artzi (USA), Jacob Eisenstein (USA) COLING: Pascale Fung (Hong Kong), Donia Scott (UK), Marilyn Walker (USA) EMNLP: Mausam (India), Lu Wang (USA) NAACL: Mohit Bansal (USA), Rebecca Passonneau (USA)
|SoftConf
|49
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6 COLING: 6
|Quality of plan Diversity of topics Qualifications of presenter likely audience interest
|[https://acl2018.org/2017/09/11/joint-call-for-tutorials/ Call] 
[https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs Report] 
[https://acl2018.org/tutorials/ ACL tutorials] 
[https://emnlp2018.org/program/tutorials/ EMNLP tutorials] 
[https://coling2018.org/tutorial-list/ COLING tutorials]
|-
|2017
|ACL EACL EMNLP
|ACL: Maja Popović (Germany), Jordan Boyd-Graber (USA) EACL: Alex Klementiev (Germany), Lucia Specia (UK) EMNLP: Nathan Schneider (USA), Alexandra Birch (UK)
|SoftConf
|26
|Tutorial chairs
|3
|ACL: 6 EACL: 6 EMNLP: 7
|Clarity (description and outline) Topic
|[https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs Report] [http://acl2017.org/tutorials/ ACL tutorials] [http://eacl2017.org/index.php/tutorials EACL tutorials]
|-
|2016
|ACL NAACL EMNLP
|ACL: Alexandra Birch (UK), Willem Zuidema (Netherlands) NAACL: Alexander Rush and Bishan Yang EMNLP: Rebecca Hwa and Mohit Bansal
|SoftConf
|32
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6
|Relevance to ACL community (15) Quality of instructor (15) Outline and depth/breadth of the proposal (15) 
Potential attendance Newly emerging area not previously covered in an ACL-related tutorial (yes or no) Introduction into related fields (yes or no) 
Overall score (15) 1st preference for venue as indicated in proposal comments
|[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 Call] [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs Report] [http://naacl.org/naacl-hlt-2016/tutorials.html NAACL tutorials] [https://www.aclweb.org/mirror/emnlp2016/tutorials.html EMNLP tutorials]
|}

== Past calls for tutorials ==
*[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 ACL/AACL-IJCNLP/EMNLP/COLING 2020 joint call for tutorials]
*[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials NAACL/ACL/EMNLP 2019 joint call for tutorials]
*[https://www.aclweb.org/portal/content/final-joint-call-tutorial-proposals-aclcolingemnlpnaacl-2018 ACL/COLING/EMNLP/NAACL 2018 joint call for tutorials]
*[https://www.aclweb.org/portal/content/eacl-2017-call-tutorial-proposals EACL 2017 call for tutorials]
*[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 NAACL/ACL/EMNLP 2016 joint call for tutorials]
*[https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-naacl-2015-acl-2015-emnlp-2015-0 NAACL/ACL/EMNLP 2015 joint call for tutorials]

== Reports from past tutorial chairs ==
*[[2020Q3_Reports:_Tutorial_Chairs|ACL 2020 report]] (by Agata Savary and Yue Zhang)
*[[2019Q1_Reports:_ACL_2019#Tutorial_Co-Chairs|ACL 2019 report]] (by Preslav Nakov and Alexis Palmer)
*[[2018Q3 Reports: Tutorial Chairs|ACL 2018 report]] (by Yoav Artzi and Jacob Eisenstein)
*[[2017Q3 Reports: Tutorial Chairs|ACL 2017 report]] (by Maja Popović and Jordan Boyd-Graber)
*[[2016Q3 Reports: Tutorial Chairs|ACL 2016 report]] (by Alexandra Birch and Willem Zuidema)

== Lessons learned by past tutorial chairs ==
* [Agata Savary, 2020] If the conference is virtual:
** Remember that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]).
** Announce the individual modalities with the attendees early enough (for instance, if watching pre-recordings in advance is necessary for some tutorials because their live sessions only included question answering)
** If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to Zoom rooms only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.
* [Agata Savary, 2020] Use a common SoftConf space for joint tutorial submissions for all coordinated conference. Once the proposals have been selected and assigned to conferences, use one track per conference in the same SoftConf space.
* [Agata Savary, 2020] Publish the list of recommended readings on each tutorial page
* [Agata Savary, 2020] A good way to proceed while selecting tutorials for several venues is to have a common online meeting, in which
** Tutorials are clustered per themes (to distribute similar tutorials over different conferences)
** The chairs for each venue select their best choices in a round robin manner.
** Generate csv files with reviews and scores: Manager -> Other tools -> Spreadsheet maker -> Reports
* [Agata Savary, 2020] While setting up a review form on SoftConf, include:
** All evaluation criteria, including diversity, in the review form
** Cutting edge vs. introductory category
** Duration (3 or 6 hours)
* [Agata Savary, 2020] Currently, the generic conference schedule assumes only 2 months between the call for tutorials and notification to proposers. If a solid 1-month review procedure is to take place, this means, that proposers only have 1 month between the call and the submission deadline. It would be good to shift the call to [T-12].
* [Agata Savary, 2020] For many past tutorials, no slides are available online. It is, therefore, important to promote Open Access of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation slides on the ACL Anthology. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]. As a tutorial chair, make sure that
** initial versions of tutorial slides are submitted to the proceedings together with the tutorial abstracts
** tutorial authors are encouraged to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]
* [Marine Capuat, 2019] Continue soliciting both cutting edge and introductory tutorials
* [Marine Capuat, 2019] Ask proposers to express their preferred location/conference in one place (i.e. either in proposal pdf, or in softconf form, not both) to avoid inconsistencies
* [ACL 2018 chairs] Issues with proceedings edition, when one joint SoftConf space is used for many conferences (see the [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs report])
* [ACL 2016 chairs] Avoid confusion about the submission procedures for the final tutorial descriptions, those were submitted by email in many different formats

Author: Agata Savary, July 2019-July 2020

Updates are welcome

[[Category:Conference Handbook]]

Tutorial chair handbook

2020-07-15T17:32:22Z

AgataSavary: /* Publication of tutorials */

== Duties ==
Tutorial chairs are nominated by the general chair of the main conference. Their duties are described in [[Tutorial chair duties|here]].

== Timeline ==
The timeline for the organisation of tutorials should follow the general [[conference planning schedule]]. Below, we complete this schedule with more detailed tasks, as defined for the ACL 2020 tutorials. The dates from the [[conference planning schedule]] are highlighted in '''bold'''. 
In recent years, calls and selection of tutorial proposals have been coordinated among some major CL conferences (usually a subset of ACL, NAACL, EMNLP, AACL and COLING). Note that if your conference is the first in the year to come, you will need to be the major driving force in the initial steps described below. 
Tutorial proposals are usually reviewed by the tutorial chairs of the coordinating conferences, but it may be a good idea to also rectruit a small group of external reviewers. They should have a good experience in reviewing and a large understanding of the recent progress in the CL domain.

* [T-12]
** Learn from the general chair which conferences will have their tutorials coordinated with yours.
** Establish contact with the tutorial chairs of these other conferences, and discuss
*** Recruitment of external reviewers
*** Proposal evaluation criteria
*** Soliciting tutorial proposals from particular potential lecturers on particular topics
*** Organizing a public poll about interest in the submitted tutorial
** Discuss general strategies of the main conference with the general chair and the program chairs
***promotion of some topics
***mailing lists and conference alert portals to be used for all announcements
** Set up a SoftConf space for proposal submission
*** Solicit the creation of a SoftConf space from [mailto:support@softconf.com support] (currently Rich Gerber). Note that submissions will be joint for several conferences but the proceedings will be edited for each conference separately. Therefore, a new SoftConf space will have to be created later for the selected proposals of each conference.
*** If you plan to have external tutorial reviewers, invite them via SoftConf.
*** Set up a submission page.
** Set up a joint email for tutorial chairs.
** Confirm the number (usually 6-8) and length (usually 3 hours) of tutorial slots with the ACL Business Manager (currently Priscilla Rasmussen).
** Draft the joint call for tutorials (based on [[#Past calls for tutorials|past calls]]).

* '''[T-11]''' ''(in the future, it would be good to extend the time slot between the call and the deadline to 2 months at least, especially if July/August is included)''
** Send out the call for tutorials.
** For posting the call on the ACL website, contact the ACL Business Manager (currently Priscilla Rasmussen).
** If you planned to encourage submission of proposals from particular potential lecturers, contact them now.
** Set up a SoftConf page for proposal reviews.

* [T-10]
** Deadline for tutorial proposal submission.
** If you decided to run a public poll about interest in the submitted tutorial, set it up and send out a call for expression of interest.
** Assign proposals to reviewers, send out invitations to review (via SoftConf).

* '''[T-9]'''
** Select tutorials jointly with other conference chairs.
** Notify the authors (via SoftConf).
** Liaise with local chair for space needs and scheduling.
** Send instructions to tutorial teachers about printed material, agenda/time management, payments, fees, etc.
** Liaise the conference website chair for updates about tutorials and authors' guidelines.
** Liaise with the ACL Office for the registration form.

* '''[T-4]'''
** Collect short descriptions for proceedings from tutorial teachers.
** Provide them to the webmaster and the general and program chairs .

* '''[T-3]'''
** Confirm the tutorials with authors.
** Authors submit the final tutorial descriptions for the proceedings.
** Arrange for printing of tutorial materials.

* [T-2]
** Arrange with the [mailto:support@softconf.com SoftConf support] the creation of a separate SoftConf space for the selected tutorials of your conference only. Indicate which tutorial submissions should be copied there.
** Set up the final submission form in the new SoftConf space. Include the submissions of slides, which will be published on the ACL Anthology. Include signing the CC-BY-4.0 license agreement.

* '''[T-1]'''
** Collect and check final tutorial materials.
** Create the tutorial proceedings, including slides.

* '''[T-0]''' Conference and tutorials

* [T+1]
** Invite the lecturers to update the slides at the ACL Anthology via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form] (under CC-BY-4.0).
** Update this tutorial page and the list of [[Past tutorials|past tutorials]] with the data from your conference. For access to the ACL [[Conference Handbook]], contact the ACL Secretary (currently Shiqi Zhou).

== Publication of tutorials ==
In 2019 and 2020 tutorials, archives exist at the ACL Anthology. Additionally, for NAACL 2019 and ACL 2020 (which went fully virtual due to the COVID pandemic), video recordings and/or pre-recordings are additionally archived.

In tutorials organized until 2019, there was no homogeneous policy as to the publication of tutorial material. Data (abstracts, slides, code, etc.) from selected tutorials were usually scattered in 2 or 3 places:
* conference website
* ACL Anthology
* private repositories of the authors
The sustainability of these varies from venue to venue. The main websites of some conferences are not maintained, therefore ACL collects [https://www.aclweb.org/portal/node/2481 mirrors] of some of them (but not all).
The tutorial proceedings, published in the ACL Anthology, until recently included only abstracts but no tutorial slides.

As a result, for many [[past tutorials]], no slides are available online, so the community cannot benefit from these assets. It is, therefore, important to promote '''Open Access''' of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation '''slides in the ACL Anthology'''. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form].

It is recommended for tutorial chairs to:
* announce the publication of the tutorial slides under the CC-BY-4.0 license (used in the ACL Anthology)
* encourage the authors to also publish, under open licenses, their code and data used in the tutorials
* include the initial versions of tutorial slides, together with abstract, in the proceedings
* encourage the tutorial authors to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]

== Past tutorials ==
The [[Past tutorials|list of recent tutorials]] contains tutorial authors' names, titles, and to their summaries in conference websites and/or in the ACL Anthology.

== Organization of past tutorials ==
For the last years, the call and selection of tutorials has been coordinated among several major CL conferences. The table below summarizes some data collected from existing reports and from direct contacts with past tutorial chairs.

{| border="1" cellspacing="0" cellpadding="5"
|'''Year'''
|'''Coordinated conferences'''
|'''Tutorial chairs'''
|'''Submission platform'''
|'''# Submissions'''
|'''Reviewers'''
|'''# Reviews per proposal'''
|'''# Selected tutorials'''
|'''Selection criteria'''
|'''Useful links'''
|-
|2020
|ACL EMNLP AACL-IJCNLP COLING
|ACL: Agata Savary (France), Yue Zhang (China) EMNLP: Benjamin Van Durme (USA), Aline Villavicencio (UK & Brazil) AACL-IJCNLP: Tim Baldwin (Australia), Fei Xia (USA) COLING: Daniel Beck (Australia), Lucia Specia (UK)
|SoftConf
|43
|Tutorial chairs + external experts
|3
|ACL: 8 EMNLP: 7 ACCL-IJCNLP: 6 COLING: 7
|Clarity and preparedness Novelty or timely character of the topic Lecturers' experience Likely audience interest Open access of the teaching material Diversity aspects (multilingualism, gender, age and country of the lecturers) Compatibility with the required venues
|[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 Call] [https://acl2020.org/program/tutorials/ ACL tutorials] [https://2020.emnlp.org/tutorials EMNLP tutorials] [http://aacl2020.org/ ACCL-IJCNLP tutorials] [https://coling2020.org/pages/tutorials COLING tutorials]
|
|-
|2019
|NAACL ACL EMNLP
|ACL: Preslav Nakov (Qatar), Alexis Palmer (USA) NAACL: Anoop Sarkar (Canda), Michael Strube (Germany) EMNLP-IJCNLP: Marine Carpuat (USA), Tim Baldwin (Australia)
|SoftConf
|45
|Tutorial chairs
|2-3
|ACL: 9 NAACL: 6 EMNLP: 7
|Presenters (1-5) Description (1-5) Interest Level (1-5) Preparedness (1-5)
|[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials Call] [http://www.acl2019.org/EN/tutorials.xhtml ACL tutorials] [https://naacl2019.org/program/tutorials/ NAACL tutorials]
|-
|2018
|ACL NAACL EMNLP COLING
|ACL: Yoav Artzi (USA), Jacob Eisenstein (USA) COLING: Pascale Fung (Hong Kong), Donia Scott (UK), Marilyn Walker (USA) EMNLP: Mausam (India), Lu Wang (USA) NAACL: Mohit Bansal (USA), Rebecca Passonneau (USA)
|SoftConf
|49
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6 COLING: 6
|Quality of plan Diversity of topics Qualifications of presenter likely audience interest
|[https://acl2018.org/2017/09/11/joint-call-for-tutorials/ Call] 
[https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs Report] 
[https://acl2018.org/tutorials/ ACL tutorials] 
[https://emnlp2018.org/program/tutorials/ EMNLP tutorials] 
[https://coling2018.org/tutorial-list/ COLING tutorials]
|-
|2017
|ACL EACL EMNLP
|ACL: Maja Popović (Germany), Jordan Boyd-Graber (USA) EACL: Alex Klementiev (Germany), Lucia Specia (UK) EMNLP: Nathan Schneider (USA), Alexandra Birch (UK)
|SoftConf
|26
|Tutorial chairs
|3
|ACL: 6 EACL: 6 EMNLP: 7
|Clarity (description and outline) Topic
|[https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs Report] [http://acl2017.org/tutorials/ ACL tutorials] [http://eacl2017.org/index.php/tutorials EACL tutorials]
|-
|2016
|ACL NAACL EMNLP
|ACL: Alexandra Birch (UK), Willem Zuidema (Netherlands) NAACL: Alexander Rush and Bishan Yang EMNLP: Rebecca Hwa and Mohit Bansal
|SoftConf
|32
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6
|Relevance to ACL community (15) Quality of instructor (15) Outline and depth/breadth of the proposal (15) 
Potential attendance Newly emerging area not previously covered in an ACL-related tutorial (yes or no) Introduction into related fields (yes or no) 
Overall score (15) 1st preference for venue as indicated in proposal comments
|[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 Call] [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs Report] [http://naacl.org/naacl-hlt-2016/tutorials.html NAACL tutorials] [https://www.aclweb.org/mirror/emnlp2016/tutorials.html EMNLP tutorials]
|}

== Past calls for tutorials ==
*[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 ACL/AACL-IJCNLP/EMNLP/COLING 2020 joint call for tutorials]
*[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials NAACL/ACL/EMNLP 2019 joint call for tutorials]
*[https://www.aclweb.org/portal/content/final-joint-call-tutorial-proposals-aclcolingemnlpnaacl-2018 ACL/COLING/EMNLP/NAACL 2018 joint call for tutorials]
*[https://www.aclweb.org/portal/content/eacl-2017-call-tutorial-proposals EACL 2017 call for tutorials]
*[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 NAACL/ACL/EMNLP 2016 joint call for tutorials]
*[https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-naacl-2015-acl-2015-emnlp-2015-0 NAACL/ACL/EMNLP 2015 joint call for tutorials]

== Reports from past tutorial chairs ==
*[[2020Q3_Reports:_Tutorial_Chairs|ACL 2020 report]] (by Agata Savary and Yue Zhang)
*[[2019Q1_Reports:_ACL_2019#Tutorial_Co-Chairs|ACL 2019 report]] (by Preslav Nakov and Alexis Palmer)
*[[2018Q3 Reports: Tutorial Chairs|ACL 2018 report]] (by Yoav Artzi and Jacob Eisenstein)
*[[2017Q3 Reports: Tutorial Chairs|ACL 2017 report]] (by Maja Popović and Jordan Boyd-Graber)
*[[2016Q3 Reports: Tutorial Chairs|ACL 2016 report]] (by Alexandra Birch and Willem Zuidema)

== Lessons learned by past tutorial chairs ==
* [Agata Savary, 2020] If the conference is virtual:
** Remember that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]).
** Announce the individual modalities with the attendees early enough (for instance, if watching pre-recordings in advance is necessary for some tutorials because their live sessions only included question answering)
** If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to Zoom rooms only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.
* [Agata Savary, 2020] Use a common SoftConf space for joint tutorial submissions for all coordinated conference. Once the proposals have been selected and assigned to conferences, use one track per conference in the same SoftConf space.
* [Agata Savary, 2020] Publish the list of recommended readings on each tutorial page
* [Agata Savary, 2020] A good way to proceed while selecting tutorials for several venues is to have a common online meeting, in which
** Tutorials are clustered per themes (to distribute similar tutorials over different conferences)
** The chairs for each venue select their best choices in a round robin manner.
** Generate csv files with reviews and scores: Manager -> Other tools -> Spreadsheet maker -> Reports
* [Agata Savary, 2020] While setting up a review form on SoftConf, include:
** All evaluation criteria, including diversity, in the review form
** Cutting edge vs. introductory category
** Duration (3 or 6 hours)
* [Agata Savary, 2020] Currently, the generic conference schedule assumes only 2 months between the call for tutorials and notification to proposers. If a solid 1-month review procedure is to take place, this means, that proposers only have 1 month between the call and the submission deadline. It would be good to shift the call to [T-12].
* [Agata Savary, 2020] For many past tutorials, no slides are available online. It is, therefore, important to promote Open Access of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation slides on the ACL Anthology. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]. As a tutorial chair, make sure that
** initial versions of tutorial slides are submitted to the proceedings together with the tutorial abstracts
** tutorial authors are encouraged to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]
* [Marine Capuat, 2019] Continue soliciting both cutting edge and introductory tutorials
* [Marine Capuat, 2019] Ask proposers to express their preferred location/conference in one place (i.e. either in proposal pdf, or in softconf form, not both) to avoid inconsistencies
* [ACL 2018 chairs] Issues with proceedings edition, when one joint SoftConf space is used for many conferences (see the [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs report])
* [ACL 2016 chairs] Avoid confusion about the submission procedures for the final tutorial descriptions, those were submitted by email in many different formats

Author: Agata Savary, July 2019-July 2020

Updates are welcome

[[Category:Conference Handbook]]

Tutorial chair handbook

2020-07-15T17:27:44Z

AgataSavary: /* Reports from past tutorial chairs */

== Duties ==
Tutorial chairs are nominated by the general chair of the main conference. Their duties are described in [[Tutorial chair duties|here]].

== Timeline ==
The timeline for the organisation of tutorials should follow the general [[conference planning schedule]]. Below, we complete this schedule with more detailed tasks, as defined for the ACL 2020 tutorials. The dates from the [[conference planning schedule]] are highlighted in '''bold'''. 
In recent years, calls and selection of tutorial proposals have been coordinated among some major CL conferences (usually a subset of ACL, NAACL, EMNLP, AACL and COLING). Note that if your conference is the first in the year to come, you will need to be the major driving force in the initial steps described below. 
Tutorial proposals are usually reviewed by the tutorial chairs of the coordinating conferences, but it may be a good idea to also rectruit a small group of external reviewers. They should have a good experience in reviewing and a large understanding of the recent progress in the CL domain.

* [T-12]
** Learn from the general chair which conferences will have their tutorials coordinated with yours.
** Establish contact with the tutorial chairs of these other conferences, and discuss
*** Recruitment of external reviewers
*** Proposal evaluation criteria
*** Soliciting tutorial proposals from particular potential lecturers on particular topics
*** Organizing a public poll about interest in the submitted tutorial
** Discuss general strategies of the main conference with the general chair and the program chairs
***promotion of some topics
***mailing lists and conference alert portals to be used for all announcements
** Set up a SoftConf space for proposal submission
*** Solicit the creation of a SoftConf space from [mailto:support@softconf.com support] (currently Rich Gerber). Note that submissions will be joint for several conferences but the proceedings will be edited for each conference separately. Therefore, a new SoftConf space will have to be created later for the selected proposals of each conference.
*** If you plan to have external tutorial reviewers, invite them via SoftConf.
*** Set up a submission page.
** Set up a joint email for tutorial chairs.
** Confirm the number (usually 6-8) and length (usually 3 hours) of tutorial slots with the ACL Business Manager (currently Priscilla Rasmussen).
** Draft the joint call for tutorials (based on [[#Past calls for tutorials|past calls]]).

* '''[T-11]''' ''(in the future, it would be good to extend the time slot between the call and the deadline to 2 months at least, especially if July/August is included)''
** Send out the call for tutorials.
** For posting the call on the ACL website, contact the ACL Business Manager (currently Priscilla Rasmussen).
** If you planned to encourage submission of proposals from particular potential lecturers, contact them now.
** Set up a SoftConf page for proposal reviews.

* [T-10]
** Deadline for tutorial proposal submission.
** If you decided to run a public poll about interest in the submitted tutorial, set it up and send out a call for expression of interest.
** Assign proposals to reviewers, send out invitations to review (via SoftConf).

* '''[T-9]'''
** Select tutorials jointly with other conference chairs.
** Notify the authors (via SoftConf).
** Liaise with local chair for space needs and scheduling.
** Send instructions to tutorial teachers about printed material, agenda/time management, payments, fees, etc.
** Liaise the conference website chair for updates about tutorials and authors' guidelines.
** Liaise with the ACL Office for the registration form.

* '''[T-4]'''
** Collect short descriptions for proceedings from tutorial teachers.
** Provide them to the webmaster and the general and program chairs .

* '''[T-3]'''
** Confirm the tutorials with authors.
** Authors submit the final tutorial descriptions for the proceedings.
** Arrange for printing of tutorial materials.

* [T-2]
** Arrange with the [mailto:support@softconf.com SoftConf support] the creation of a separate SoftConf space for the selected tutorials of your conference only. Indicate which tutorial submissions should be copied there.
** Set up the final submission form in the new SoftConf space. Include the submissions of slides, which will be published on the ACL Anthology. Include signing the CC-BY-4.0 license agreement.

* '''[T-1]'''
** Collect and check final tutorial materials.
** Create the tutorial proceedings, including slides.

* '''[T-0]''' Conference and tutorials

* [T+1]
** Invite the lecturers to update the slides at the ACL Anthology via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form] (under CC-BY-4.0).
** Update this tutorial page and the list of [[Past tutorials|past tutorials]] with the data from your conference. For access to the ACL [[Conference Handbook]], contact the ACL Secretary (currently Shiqi Zhou).

== Publication of tutorials ==
In tutorials organized until 2019, there was no homogeneous policy as to the publication of tutorial material. Data (abstracts, slides, code, etc.) from selected tutorials were usually scattered in 2 or 3 places:
* conference website
* ACL Anthology
* private repositories of the authors
The sustainability of these varies from venue to venue. The main websites of some conferences are not maintained, therefore ACL collects [https://www.aclweb.org/portal/node/2481 mirrors] of some of them (but not all).
The tutorial proceedings, published in the ACL Anthology, until recently included only abstracts but no tutorial slides.

As a result, for many [[past tutorials]], no slides are available online, so the community cannot benefit from these assets. It is, therefore, important to promote '''Open Access''' of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation '''slides in the ACL Anthology'''. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form].

It is recommended for tutorial chairs to:
* announce the publication of the tutorial slides under the CC-BY-4.0 license (used in the ACL Anthology)
* encourage the authors to also publish, under open licenses, their code and data used in the tutorials
* include the initial versions of tutorial slides, together with abstract, in the proceedings
* encourage the tutorial authors to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]

== Past tutorials ==
The [[Past tutorials|list of recent tutorials]] contains tutorial authors' names, titles, and to their summaries in conference websites and/or in the ACL Anthology.

== Organization of past tutorials ==
For the last years, the call and selection of tutorials has been coordinated among several major CL conferences. The table below summarizes some data collected from existing reports and from direct contacts with past tutorial chairs.

{| border="1" cellspacing="0" cellpadding="5"
|'''Year'''
|'''Coordinated conferences'''
|'''Tutorial chairs'''
|'''Submission platform'''
|'''# Submissions'''
|'''Reviewers'''
|'''# Reviews per proposal'''
|'''# Selected tutorials'''
|'''Selection criteria'''
|'''Useful links'''
|-
|2020
|ACL EMNLP AACL-IJCNLP COLING
|ACL: Agata Savary (France), Yue Zhang (China) EMNLP: Benjamin Van Durme (USA), Aline Villavicencio (UK & Brazil) AACL-IJCNLP: Tim Baldwin (Australia), Fei Xia (USA) COLING: Daniel Beck (Australia), Lucia Specia (UK)
|SoftConf
|43
|Tutorial chairs + external experts
|3
|ACL: 8 EMNLP: 7 ACCL-IJCNLP: 6 COLING: 7
|Clarity and preparedness Novelty or timely character of the topic Lecturers' experience Likely audience interest Open access of the teaching material Diversity aspects (multilingualism, gender, age and country of the lecturers) Compatibility with the required venues
|[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 Call] [https://acl2020.org/program/tutorials/ ACL tutorials] [https://2020.emnlp.org/tutorials EMNLP tutorials] [http://aacl2020.org/ ACCL-IJCNLP tutorials] [https://coling2020.org/pages/tutorials COLING tutorials]
|
|-
|2019
|NAACL ACL EMNLP
|ACL: Preslav Nakov (Qatar), Alexis Palmer (USA) NAACL: Anoop Sarkar (Canda), Michael Strube (Germany) EMNLP-IJCNLP: Marine Carpuat (USA), Tim Baldwin (Australia)
|SoftConf
|45
|Tutorial chairs
|2-3
|ACL: 9 NAACL: 6 EMNLP: 7
|Presenters (1-5) Description (1-5) Interest Level (1-5) Preparedness (1-5)
|[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials Call] [http://www.acl2019.org/EN/tutorials.xhtml ACL tutorials] [https://naacl2019.org/program/tutorials/ NAACL tutorials]
|-
|2018
|ACL NAACL EMNLP COLING
|ACL: Yoav Artzi (USA), Jacob Eisenstein (USA) COLING: Pascale Fung (Hong Kong), Donia Scott (UK), Marilyn Walker (USA) EMNLP: Mausam (India), Lu Wang (USA) NAACL: Mohit Bansal (USA), Rebecca Passonneau (USA)
|SoftConf
|49
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6 COLING: 6
|Quality of plan Diversity of topics Qualifications of presenter likely audience interest
|[https://acl2018.org/2017/09/11/joint-call-for-tutorials/ Call] 
[https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs Report] 
[https://acl2018.org/tutorials/ ACL tutorials] 
[https://emnlp2018.org/program/tutorials/ EMNLP tutorials] 
[https://coling2018.org/tutorial-list/ COLING tutorials]
|-
|2017
|ACL EACL EMNLP
|ACL: Maja Popović (Germany), Jordan Boyd-Graber (USA) EACL: Alex Klementiev (Germany), Lucia Specia (UK) EMNLP: Nathan Schneider (USA), Alexandra Birch (UK)
|SoftConf
|26
|Tutorial chairs
|3
|ACL: 6 EACL: 6 EMNLP: 7
|Clarity (description and outline) Topic
|[https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs Report] [http://acl2017.org/tutorials/ ACL tutorials] [http://eacl2017.org/index.php/tutorials EACL tutorials]
|-
|2016
|ACL NAACL EMNLP
|ACL: Alexandra Birch (UK), Willem Zuidema (Netherlands) NAACL: Alexander Rush and Bishan Yang EMNLP: Rebecca Hwa and Mohit Bansal
|SoftConf
|32
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6
|Relevance to ACL community (15) Quality of instructor (15) Outline and depth/breadth of the proposal (15) 
Potential attendance Newly emerging area not previously covered in an ACL-related tutorial (yes or no) Introduction into related fields (yes or no) 
Overall score (15) 1st preference for venue as indicated in proposal comments
|[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 Call] [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs Report] [http://naacl.org/naacl-hlt-2016/tutorials.html NAACL tutorials] [https://www.aclweb.org/mirror/emnlp2016/tutorials.html EMNLP tutorials]
|}

== Past calls for tutorials ==
*[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 ACL/AACL-IJCNLP/EMNLP/COLING 2020 joint call for tutorials]
*[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials NAACL/ACL/EMNLP 2019 joint call for tutorials]
*[https://www.aclweb.org/portal/content/final-joint-call-tutorial-proposals-aclcolingemnlpnaacl-2018 ACL/COLING/EMNLP/NAACL 2018 joint call for tutorials]
*[https://www.aclweb.org/portal/content/eacl-2017-call-tutorial-proposals EACL 2017 call for tutorials]
*[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 NAACL/ACL/EMNLP 2016 joint call for tutorials]
*[https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-naacl-2015-acl-2015-emnlp-2015-0 NAACL/ACL/EMNLP 2015 joint call for tutorials]

== Reports from past tutorial chairs ==
*[[2020Q3_Reports:_Tutorial_Chairs|ACL 2020 report]] (by Agata Savary and Yue Zhang)
*[[2019Q1_Reports:_ACL_2019#Tutorial_Co-Chairs|ACL 2019 report]] (by Preslav Nakov and Alexis Palmer)
*[[2018Q3 Reports: Tutorial Chairs|ACL 2018 report]] (by Yoav Artzi and Jacob Eisenstein)
*[[2017Q3 Reports: Tutorial Chairs|ACL 2017 report]] (by Maja Popović and Jordan Boyd-Graber)
*[[2016Q3 Reports: Tutorial Chairs|ACL 2016 report]] (by Alexandra Birch and Willem Zuidema)

== Lessons learned by past tutorial chairs ==
* [Agata Savary, 2020] If the conference is virtual:
** Remember that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]).
** Announce the individual modalities with the attendees early enough (for instance, if watching pre-recordings in advance is necessary for some tutorials because their live sessions only included question answering)
** If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to Zoom rooms only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.
* [Agata Savary, 2020] Use a common SoftConf space for joint tutorial submissions for all coordinated conference. Once the proposals have been selected and assigned to conferences, use one track per conference in the same SoftConf space.
* [Agata Savary, 2020] Publish the list of recommended readings on each tutorial page
* [Agata Savary, 2020] A good way to proceed while selecting tutorials for several venues is to have a common online meeting, in which
** Tutorials are clustered per themes (to distribute similar tutorials over different conferences)
** The chairs for each venue select their best choices in a round robin manner.
** Generate csv files with reviews and scores: Manager -> Other tools -> Spreadsheet maker -> Reports
* [Agata Savary, 2020] While setting up a review form on SoftConf, include:
** All evaluation criteria, including diversity, in the review form
** Cutting edge vs. introductory category
** Duration (3 or 6 hours)
* [Agata Savary, 2020] Currently, the generic conference schedule assumes only 2 months between the call for tutorials and notification to proposers. If a solid 1-month review procedure is to take place, this means, that proposers only have 1 month between the call and the submission deadline. It would be good to shift the call to [T-12].
* [Agata Savary, 2020] For many past tutorials, no slides are available online. It is, therefore, important to promote Open Access of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation slides on the ACL Anthology. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]. As a tutorial chair, make sure that
** initial versions of tutorial slides are submitted to the proceedings together with the tutorial abstracts
** tutorial authors are encouraged to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]
* [Marine Capuat, 2019] Continue soliciting both cutting edge and introductory tutorials
* [Marine Capuat, 2019] Ask proposers to express their preferred location/conference in one place (i.e. either in proposal pdf, or in softconf form, not both) to avoid inconsistencies
* [ACL 2018 chairs] Issues with proceedings edition, when one joint SoftConf space is used for many conferences (see the [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs report])
* [ACL 2016 chairs] Avoid confusion about the submission procedures for the final tutorial descriptions, those were submitted by email in many different formats

Author: Agata Savary, July 2019-July 2020

Updates are welcome

[[Category:Conference Handbook]]

Tutorial chair handbook

2020-07-15T17:26:25Z

AgataSavary: /* Reports from past tutorial chairs */

== Duties ==
Tutorial chairs are nominated by the general chair of the main conference. Their duties are described in [[Tutorial chair duties|here]].

== Timeline ==
The timeline for the organisation of tutorials should follow the general [[conference planning schedule]]. Below, we complete this schedule with more detailed tasks, as defined for the ACL 2020 tutorials. The dates from the [[conference planning schedule]] are highlighted in '''bold'''. 
In recent years, calls and selection of tutorial proposals have been coordinated among some major CL conferences (usually a subset of ACL, NAACL, EMNLP, AACL and COLING). Note that if your conference is the first in the year to come, you will need to be the major driving force in the initial steps described below. 
Tutorial proposals are usually reviewed by the tutorial chairs of the coordinating conferences, but it may be a good idea to also rectruit a small group of external reviewers. They should have a good experience in reviewing and a large understanding of the recent progress in the CL domain.

* [T-12]
** Learn from the general chair which conferences will have their tutorials coordinated with yours.
** Establish contact with the tutorial chairs of these other conferences, and discuss
*** Recruitment of external reviewers
*** Proposal evaluation criteria
*** Soliciting tutorial proposals from particular potential lecturers on particular topics
*** Organizing a public poll about interest in the submitted tutorial
** Discuss general strategies of the main conference with the general chair and the program chairs
***promotion of some topics
***mailing lists and conference alert portals to be used for all announcements
** Set up a SoftConf space for proposal submission
*** Solicit the creation of a SoftConf space from [mailto:support@softconf.com support] (currently Rich Gerber). Note that submissions will be joint for several conferences but the proceedings will be edited for each conference separately. Therefore, a new SoftConf space will have to be created later for the selected proposals of each conference.
*** If you plan to have external tutorial reviewers, invite them via SoftConf.
*** Set up a submission page.
** Set up a joint email for tutorial chairs.
** Confirm the number (usually 6-8) and length (usually 3 hours) of tutorial slots with the ACL Business Manager (currently Priscilla Rasmussen).
** Draft the joint call for tutorials (based on [[#Past calls for tutorials|past calls]]).

* '''[T-11]''' ''(in the future, it would be good to extend the time slot between the call and the deadline to 2 months at least, especially if July/August is included)''
** Send out the call for tutorials.
** For posting the call on the ACL website, contact the ACL Business Manager (currently Priscilla Rasmussen).
** If you planned to encourage submission of proposals from particular potential lecturers, contact them now.
** Set up a SoftConf page for proposal reviews.

* [T-10]
** Deadline for tutorial proposal submission.
** If you decided to run a public poll about interest in the submitted tutorial, set it up and send out a call for expression of interest.
** Assign proposals to reviewers, send out invitations to review (via SoftConf).

* '''[T-9]'''
** Select tutorials jointly with other conference chairs.
** Notify the authors (via SoftConf).
** Liaise with local chair for space needs and scheduling.
** Send instructions to tutorial teachers about printed material, agenda/time management, payments, fees, etc.
** Liaise the conference website chair for updates about tutorials and authors' guidelines.
** Liaise with the ACL Office for the registration form.

* '''[T-4]'''
** Collect short descriptions for proceedings from tutorial teachers.
** Provide them to the webmaster and the general and program chairs .

* '''[T-3]'''
** Confirm the tutorials with authors.
** Authors submit the final tutorial descriptions for the proceedings.
** Arrange for printing of tutorial materials.

* [T-2]
** Arrange with the [mailto:support@softconf.com SoftConf support] the creation of a separate SoftConf space for the selected tutorials of your conference only. Indicate which tutorial submissions should be copied there.
** Set up the final submission form in the new SoftConf space. Include the submissions of slides, which will be published on the ACL Anthology. Include signing the CC-BY-4.0 license agreement.

* '''[T-1]'''
** Collect and check final tutorial materials.
** Create the tutorial proceedings, including slides.

* '''[T-0]''' Conference and tutorials

* [T+1]
** Invite the lecturers to update the slides at the ACL Anthology via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form] (under CC-BY-4.0).
** Update this tutorial page and the list of [[Past tutorials|past tutorials]] with the data from your conference. For access to the ACL [[Conference Handbook]], contact the ACL Secretary (currently Shiqi Zhou).

== Publication of tutorials ==
In tutorials organized until 2019, there was no homogeneous policy as to the publication of tutorial material. Data (abstracts, slides, code, etc.) from selected tutorials were usually scattered in 2 or 3 places:
* conference website
* ACL Anthology
* private repositories of the authors
The sustainability of these varies from venue to venue. The main websites of some conferences are not maintained, therefore ACL collects [https://www.aclweb.org/portal/node/2481 mirrors] of some of them (but not all).
The tutorial proceedings, published in the ACL Anthology, until recently included only abstracts but no tutorial slides.

As a result, for many [[past tutorials]], no slides are available online, so the community cannot benefit from these assets. It is, therefore, important to promote '''Open Access''' of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation '''slides in the ACL Anthology'''. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form].

It is recommended for tutorial chairs to:
* announce the publication of the tutorial slides under the CC-BY-4.0 license (used in the ACL Anthology)
* encourage the authors to also publish, under open licenses, their code and data used in the tutorials
* include the initial versions of tutorial slides, together with abstract, in the proceedings
* encourage the tutorial authors to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]

== Past tutorials ==
The [[Past tutorials|list of recent tutorials]] contains tutorial authors' names, titles, and to their summaries in conference websites and/or in the ACL Anthology.

== Organization of past tutorials ==
For the last years, the call and selection of tutorials has been coordinated among several major CL conferences. The table below summarizes some data collected from existing reports and from direct contacts with past tutorial chairs.

{| border="1" cellspacing="0" cellpadding="5"
|'''Year'''
|'''Coordinated conferences'''
|'''Tutorial chairs'''
|'''Submission platform'''
|'''# Submissions'''
|'''Reviewers'''
|'''# Reviews per proposal'''
|'''# Selected tutorials'''
|'''Selection criteria'''
|'''Useful links'''
|-
|2020
|ACL EMNLP AACL-IJCNLP COLING
|ACL: Agata Savary (France), Yue Zhang (China) EMNLP: Benjamin Van Durme (USA), Aline Villavicencio (UK & Brazil) AACL-IJCNLP: Tim Baldwin (Australia), Fei Xia (USA) COLING: Daniel Beck (Australia), Lucia Specia (UK)
|SoftConf
|43
|Tutorial chairs + external experts
|3
|ACL: 8 EMNLP: 7 ACCL-IJCNLP: 6 COLING: 7
|Clarity and preparedness Novelty or timely character of the topic Lecturers' experience Likely audience interest Open access of the teaching material Diversity aspects (multilingualism, gender, age and country of the lecturers) Compatibility with the required venues
|[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 Call] [https://acl2020.org/program/tutorials/ ACL tutorials] [https://2020.emnlp.org/tutorials EMNLP tutorials] [http://aacl2020.org/ ACCL-IJCNLP tutorials] [https://coling2020.org/pages/tutorials COLING tutorials]
|
|-
|2019
|NAACL ACL EMNLP
|ACL: Preslav Nakov (Qatar), Alexis Palmer (USA) NAACL: Anoop Sarkar (Canda), Michael Strube (Germany) EMNLP-IJCNLP: Marine Carpuat (USA), Tim Baldwin (Australia)
|SoftConf
|45
|Tutorial chairs
|2-3
|ACL: 9 NAACL: 6 EMNLP: 7
|Presenters (1-5) Description (1-5) Interest Level (1-5) Preparedness (1-5)
|[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials Call] [http://www.acl2019.org/EN/tutorials.xhtml ACL tutorials] [https://naacl2019.org/program/tutorials/ NAACL tutorials]
|-
|2018
|ACL NAACL EMNLP COLING
|ACL: Yoav Artzi (USA), Jacob Eisenstein (USA) COLING: Pascale Fung (Hong Kong), Donia Scott (UK), Marilyn Walker (USA) EMNLP: Mausam (India), Lu Wang (USA) NAACL: Mohit Bansal (USA), Rebecca Passonneau (USA)
|SoftConf
|49
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6 COLING: 6
|Quality of plan Diversity of topics Qualifications of presenter likely audience interest
|[https://acl2018.org/2017/09/11/joint-call-for-tutorials/ Call] 
[https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs Report] 
[https://acl2018.org/tutorials/ ACL tutorials] 
[https://emnlp2018.org/program/tutorials/ EMNLP tutorials] 
[https://coling2018.org/tutorial-list/ COLING tutorials]
|-
|2017
|ACL EACL EMNLP
|ACL: Maja Popović (Germany), Jordan Boyd-Graber (USA) EACL: Alex Klementiev (Germany), Lucia Specia (UK) EMNLP: Nathan Schneider (USA), Alexandra Birch (UK)
|SoftConf
|26
|Tutorial chairs
|3
|ACL: 6 EACL: 6 EMNLP: 7
|Clarity (description and outline) Topic
|[https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs Report] [http://acl2017.org/tutorials/ ACL tutorials] [http://eacl2017.org/index.php/tutorials EACL tutorials]
|-
|2016
|ACL NAACL EMNLP
|ACL: Alexandra Birch (UK), Willem Zuidema (Netherlands) NAACL: Alexander Rush and Bishan Yang EMNLP: Rebecca Hwa and Mohit Bansal
|SoftConf
|32
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6
|Relevance to ACL community (15) Quality of instructor (15) Outline and depth/breadth of the proposal (15) 
Potential attendance Newly emerging area not previously covered in an ACL-related tutorial (yes or no) Introduction into related fields (yes or no) 
Overall score (15) 1st preference for venue as indicated in proposal comments
|[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 Call] [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs Report] [http://naacl.org/naacl-hlt-2016/tutorials.html NAACL tutorials] [https://www.aclweb.org/mirror/emnlp2016/tutorials.html EMNLP tutorials]
|}

== Past calls for tutorials ==
*[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 ACL/AACL-IJCNLP/EMNLP/COLING 2020 joint call for tutorials]
*[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials NAACL/ACL/EMNLP 2019 joint call for tutorials]
*[https://www.aclweb.org/portal/content/final-joint-call-tutorial-proposals-aclcolingemnlpnaacl-2018 ACL/COLING/EMNLP/NAACL 2018 joint call for tutorials]
*[https://www.aclweb.org/portal/content/eacl-2017-call-tutorial-proposals EACL 2017 call for tutorials]
*[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 NAACL/ACL/EMNLP 2016 joint call for tutorials]
*[https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-naacl-2015-acl-2015-emnlp-2015-0 NAACL/ACL/EMNLP 2015 joint call for tutorials]

== Reports from past tutorial chairs ==
*[[2020Q3_Reports:_ACL_2020#Tutorial_Chairs|ACL 2020 report]] (by Agata Savary and Yue Zhang)
*[[2019Q1_Reports:_ACL_2019#Tutorial_Co-Chairs|ACL 2019 report]] (by Preslav Nakov and Alexis Palmer)
*[[2018Q3 Reports: Tutorial Chairs|ACL 2018 report]] (by Yoav Artzi and Jacob Eisenstein)
*[[2017Q3 Reports: Tutorial Chairs|ACL 2017 report]] (by Maja Popović and Jordan Boyd-Graber)
*[[2016Q3 Reports: Tutorial Chairs|ACL 2016 report]] (by Alexandra Birch and Willem Zuidema)

== Lessons learned by past tutorial chairs ==
* [Agata Savary, 2020] If the conference is virtual:
** Remember that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]).
** Announce the individual modalities with the attendees early enough (for instance, if watching pre-recordings in advance is necessary for some tutorials because their live sessions only included question answering)
** If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to Zoom rooms only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.
* [Agata Savary, 2020] Use a common SoftConf space for joint tutorial submissions for all coordinated conference. Once the proposals have been selected and assigned to conferences, use one track per conference in the same SoftConf space.
* [Agata Savary, 2020] Publish the list of recommended readings on each tutorial page
* [Agata Savary, 2020] A good way to proceed while selecting tutorials for several venues is to have a common online meeting, in which
** Tutorials are clustered per themes (to distribute similar tutorials over different conferences)
** The chairs for each venue select their best choices in a round robin manner.
** Generate csv files with reviews and scores: Manager -> Other tools -> Spreadsheet maker -> Reports
* [Agata Savary, 2020] While setting up a review form on SoftConf, include:
** All evaluation criteria, including diversity, in the review form
** Cutting edge vs. introductory category
** Duration (3 or 6 hours)
* [Agata Savary, 2020] Currently, the generic conference schedule assumes only 2 months between the call for tutorials and notification to proposers. If a solid 1-month review procedure is to take place, this means, that proposers only have 1 month between the call and the submission deadline. It would be good to shift the call to [T-12].
* [Agata Savary, 2020] For many past tutorials, no slides are available online. It is, therefore, important to promote Open Access of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation slides on the ACL Anthology. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]. As a tutorial chair, make sure that
** initial versions of tutorial slides are submitted to the proceedings together with the tutorial abstracts
** tutorial authors are encouraged to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]
* [Marine Capuat, 2019] Continue soliciting both cutting edge and introductory tutorials
* [Marine Capuat, 2019] Ask proposers to express their preferred location/conference in one place (i.e. either in proposal pdf, or in softconf form, not both) to avoid inconsistencies
* [ACL 2018 chairs] Issues with proceedings edition, when one joint SoftConf space is used for many conferences (see the [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs report])
* [ACL 2016 chairs] Avoid confusion about the submission procedures for the final tutorial descriptions, those were submitted by email in many different formats

Author: Agata Savary, July 2019-July 2020

Updates are welcome

[[Category:Conference Handbook]]

Tutorial chair handbook

2020-07-15T17:24:44Z

AgataSavary: /* Lessons learned by past tutorial chairs */

== Duties ==
Tutorial chairs are nominated by the general chair of the main conference. Their duties are described in [[Tutorial chair duties|here]].

== Timeline ==
The timeline for the organisation of tutorials should follow the general [[conference planning schedule]]. Below, we complete this schedule with more detailed tasks, as defined for the ACL 2020 tutorials. The dates from the [[conference planning schedule]] are highlighted in '''bold'''. 
In recent years, calls and selection of tutorial proposals have been coordinated among some major CL conferences (usually a subset of ACL, NAACL, EMNLP, AACL and COLING). Note that if your conference is the first in the year to come, you will need to be the major driving force in the initial steps described below. 
Tutorial proposals are usually reviewed by the tutorial chairs of the coordinating conferences, but it may be a good idea to also rectruit a small group of external reviewers. They should have a good experience in reviewing and a large understanding of the recent progress in the CL domain.

* [T-12]
** Learn from the general chair which conferences will have their tutorials coordinated with yours.
** Establish contact with the tutorial chairs of these other conferences, and discuss
*** Recruitment of external reviewers
*** Proposal evaluation criteria
*** Soliciting tutorial proposals from particular potential lecturers on particular topics
*** Organizing a public poll about interest in the submitted tutorial
** Discuss general strategies of the main conference with the general chair and the program chairs
***promotion of some topics
***mailing lists and conference alert portals to be used for all announcements
** Set up a SoftConf space for proposal submission
*** Solicit the creation of a SoftConf space from [mailto:support@softconf.com support] (currently Rich Gerber). Note that submissions will be joint for several conferences but the proceedings will be edited for each conference separately. Therefore, a new SoftConf space will have to be created later for the selected proposals of each conference.
*** If you plan to have external tutorial reviewers, invite them via SoftConf.
*** Set up a submission page.
** Set up a joint email for tutorial chairs.
** Confirm the number (usually 6-8) and length (usually 3 hours) of tutorial slots with the ACL Business Manager (currently Priscilla Rasmussen).
** Draft the joint call for tutorials (based on [[#Past calls for tutorials|past calls]]).

* '''[T-11]''' ''(in the future, it would be good to extend the time slot between the call and the deadline to 2 months at least, especially if July/August is included)''
** Send out the call for tutorials.
** For posting the call on the ACL website, contact the ACL Business Manager (currently Priscilla Rasmussen).
** If you planned to encourage submission of proposals from particular potential lecturers, contact them now.
** Set up a SoftConf page for proposal reviews.

* [T-10]
** Deadline for tutorial proposal submission.
** If you decided to run a public poll about interest in the submitted tutorial, set it up and send out a call for expression of interest.
** Assign proposals to reviewers, send out invitations to review (via SoftConf).

* '''[T-9]'''
** Select tutorials jointly with other conference chairs.
** Notify the authors (via SoftConf).
** Liaise with local chair for space needs and scheduling.
** Send instructions to tutorial teachers about printed material, agenda/time management, payments, fees, etc.
** Liaise the conference website chair for updates about tutorials and authors' guidelines.
** Liaise with the ACL Office for the registration form.

* '''[T-4]'''
** Collect short descriptions for proceedings from tutorial teachers.
** Provide them to the webmaster and the general and program chairs .

* '''[T-3]'''
** Confirm the tutorials with authors.
** Authors submit the final tutorial descriptions for the proceedings.
** Arrange for printing of tutorial materials.

* [T-2]
** Arrange with the [mailto:support@softconf.com SoftConf support] the creation of a separate SoftConf space for the selected tutorials of your conference only. Indicate which tutorial submissions should be copied there.
** Set up the final submission form in the new SoftConf space. Include the submissions of slides, which will be published on the ACL Anthology. Include signing the CC-BY-4.0 license agreement.

* '''[T-1]'''
** Collect and check final tutorial materials.
** Create the tutorial proceedings, including slides.

* '''[T-0]''' Conference and tutorials

* [T+1]
** Invite the lecturers to update the slides at the ACL Anthology via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form] (under CC-BY-4.0).
** Update this tutorial page and the list of [[Past tutorials|past tutorials]] with the data from your conference. For access to the ACL [[Conference Handbook]], contact the ACL Secretary (currently Shiqi Zhou).

== Publication of tutorials ==
In tutorials organized until 2019, there was no homogeneous policy as to the publication of tutorial material. Data (abstracts, slides, code, etc.) from selected tutorials were usually scattered in 2 or 3 places:
* conference website
* ACL Anthology
* private repositories of the authors
The sustainability of these varies from venue to venue. The main websites of some conferences are not maintained, therefore ACL collects [https://www.aclweb.org/portal/node/2481 mirrors] of some of them (but not all).
The tutorial proceedings, published in the ACL Anthology, until recently included only abstracts but no tutorial slides.

As a result, for many [[past tutorials]], no slides are available online, so the community cannot benefit from these assets. It is, therefore, important to promote '''Open Access''' of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation '''slides in the ACL Anthology'''. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form].

It is recommended for tutorial chairs to:
* announce the publication of the tutorial slides under the CC-BY-4.0 license (used in the ACL Anthology)
* encourage the authors to also publish, under open licenses, their code and data used in the tutorials
* include the initial versions of tutorial slides, together with abstract, in the proceedings
* encourage the tutorial authors to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]

== Past tutorials ==
The [[Past tutorials|list of recent tutorials]] contains tutorial authors' names, titles, and to their summaries in conference websites and/or in the ACL Anthology.

== Organization of past tutorials ==
For the last years, the call and selection of tutorials has been coordinated among several major CL conferences. The table below summarizes some data collected from existing reports and from direct contacts with past tutorial chairs.

{| border="1" cellspacing="0" cellpadding="5"
|'''Year'''
|'''Coordinated conferences'''
|'''Tutorial chairs'''
|'''Submission platform'''
|'''# Submissions'''
|'''Reviewers'''
|'''# Reviews per proposal'''
|'''# Selected tutorials'''
|'''Selection criteria'''
|'''Useful links'''
|-
|2020
|ACL EMNLP AACL-IJCNLP COLING
|ACL: Agata Savary (France), Yue Zhang (China) EMNLP: Benjamin Van Durme (USA), Aline Villavicencio (UK & Brazil) AACL-IJCNLP: Tim Baldwin (Australia), Fei Xia (USA) COLING: Daniel Beck (Australia), Lucia Specia (UK)
|SoftConf
|43
|Tutorial chairs + external experts
|3
|ACL: 8 EMNLP: 7 ACCL-IJCNLP: 6 COLING: 7
|Clarity and preparedness Novelty or timely character of the topic Lecturers' experience Likely audience interest Open access of the teaching material Diversity aspects (multilingualism, gender, age and country of the lecturers) Compatibility with the required venues
|[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 Call] [https://acl2020.org/program/tutorials/ ACL tutorials] [https://2020.emnlp.org/tutorials EMNLP tutorials] [http://aacl2020.org/ ACCL-IJCNLP tutorials] [https://coling2020.org/pages/tutorials COLING tutorials]
|
|-
|2019
|NAACL ACL EMNLP
|ACL: Preslav Nakov (Qatar), Alexis Palmer (USA) NAACL: Anoop Sarkar (Canda), Michael Strube (Germany) EMNLP-IJCNLP: Marine Carpuat (USA), Tim Baldwin (Australia)
|SoftConf
|45
|Tutorial chairs
|2-3
|ACL: 9 NAACL: 6 EMNLP: 7
|Presenters (1-5) Description (1-5) Interest Level (1-5) Preparedness (1-5)
|[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials Call] [http://www.acl2019.org/EN/tutorials.xhtml ACL tutorials] [https://naacl2019.org/program/tutorials/ NAACL tutorials]
|-
|2018
|ACL NAACL EMNLP COLING
|ACL: Yoav Artzi (USA), Jacob Eisenstein (USA) COLING: Pascale Fung (Hong Kong), Donia Scott (UK), Marilyn Walker (USA) EMNLP: Mausam (India), Lu Wang (USA) NAACL: Mohit Bansal (USA), Rebecca Passonneau (USA)
|SoftConf
|49
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6 COLING: 6
|Quality of plan Diversity of topics Qualifications of presenter likely audience interest
|[https://acl2018.org/2017/09/11/joint-call-for-tutorials/ Call] 
[https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs Report] 
[https://acl2018.org/tutorials/ ACL tutorials] 
[https://emnlp2018.org/program/tutorials/ EMNLP tutorials] 
[https://coling2018.org/tutorial-list/ COLING tutorials]
|-
|2017
|ACL EACL EMNLP
|ACL: Maja Popović (Germany), Jordan Boyd-Graber (USA) EACL: Alex Klementiev (Germany), Lucia Specia (UK) EMNLP: Nathan Schneider (USA), Alexandra Birch (UK)
|SoftConf
|26
|Tutorial chairs
|3
|ACL: 6 EACL: 6 EMNLP: 7
|Clarity (description and outline) Topic
|[https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs Report] [http://acl2017.org/tutorials/ ACL tutorials] [http://eacl2017.org/index.php/tutorials EACL tutorials]
|-
|2016
|ACL NAACL EMNLP
|ACL: Alexandra Birch (UK), Willem Zuidema (Netherlands) NAACL: Alexander Rush and Bishan Yang EMNLP: Rebecca Hwa and Mohit Bansal
|SoftConf
|32
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6
|Relevance to ACL community (15) Quality of instructor (15) Outline and depth/breadth of the proposal (15) 
Potential attendance Newly emerging area not previously covered in an ACL-related tutorial (yes or no) Introduction into related fields (yes or no) 
Overall score (15) 1st preference for venue as indicated in proposal comments
|[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 Call] [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs Report] [http://naacl.org/naacl-hlt-2016/tutorials.html NAACL tutorials] [https://www.aclweb.org/mirror/emnlp2016/tutorials.html EMNLP tutorials]
|}

== Past calls for tutorials ==
*[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 ACL/AACL-IJCNLP/EMNLP/COLING 2020 joint call for tutorials]
*[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials NAACL/ACL/EMNLP 2019 joint call for tutorials]
*[https://www.aclweb.org/portal/content/final-joint-call-tutorial-proposals-aclcolingemnlpnaacl-2018 ACL/COLING/EMNLP/NAACL 2018 joint call for tutorials]
*[https://www.aclweb.org/portal/content/eacl-2017-call-tutorial-proposals EACL 2017 call for tutorials]
*[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 NAACL/ACL/EMNLP 2016 joint call for tutorials]
*[https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-naacl-2015-acl-2015-emnlp-2015-0 NAACL/ACL/EMNLP 2015 joint call for tutorials]

== Reports from past tutorial chairs ==
*[[2020Q1_Reports:_ACL_2020#Tutorial_Chairs|ACL 2020 report]] (by Agata Savary and Yue Zhang)
*[[2019Q1_Reports:_ACL_2019#Tutorial_Co-Chairs|ACL 2019 report]] (by Preslav Nakov and Alexis Palmer)
*[[2018Q3 Reports: Tutorial Chairs|ACL 2018 report]] (by Yoav Artzi and Jacob Eisenstein)
*[[2017Q3 Reports: Tutorial Chairs|ACL 2017 report]] (by Maja Popović and Jordan Boyd-Graber)
*[[2016Q3 Reports: Tutorial Chairs|ACL 2016 report]] (by Alexandra Birch and Willem Zuidema)

== Lessons learned by past tutorial chairs ==
* [Agata Savary, 2020] If the conference is virtual:
** Remember that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]).
** Announce the individual modalities with the attendees early enough (for instance, if watching pre-recordings in advance is necessary for some tutorials because their live sessions only included question answering)
** If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to Zoom rooms only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.
* [Agata Savary, 2020] Use a common SoftConf space for joint tutorial submissions for all coordinated conference. Once the proposals have been selected and assigned to conferences, use one track per conference in the same SoftConf space.
* [Agata Savary, 2020] Publish the list of recommended readings on each tutorial page
* [Agata Savary, 2020] A good way to proceed while selecting tutorials for several venues is to have a common online meeting, in which
** Tutorials are clustered per themes (to distribute similar tutorials over different conferences)
** The chairs for each venue select their best choices in a round robin manner.
** Generate csv files with reviews and scores: Manager -> Other tools -> Spreadsheet maker -> Reports
* [Agata Savary, 2020] While setting up a review form on SoftConf, include:
** All evaluation criteria, including diversity, in the review form
** Cutting edge vs. introductory category
** Duration (3 or 6 hours)
* [Agata Savary, 2020] Currently, the generic conference schedule assumes only 2 months between the call for tutorials and notification to proposers. If a solid 1-month review procedure is to take place, this means, that proposers only have 1 month between the call and the submission deadline. It would be good to shift the call to [T-12].
* [Agata Savary, 2020] For many past tutorials, no slides are available online. It is, therefore, important to promote Open Access of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation slides on the ACL Anthology. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]. As a tutorial chair, make sure that
** initial versions of tutorial slides are submitted to the proceedings together with the tutorial abstracts
** tutorial authors are encouraged to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]
* [Marine Capuat, 2019] Continue soliciting both cutting edge and introductory tutorials
* [Marine Capuat, 2019] Ask proposers to express their preferred location/conference in one place (i.e. either in proposal pdf, or in softconf form, not both) to avoid inconsistencies
* [ACL 2018 chairs] Issues with proceedings edition, when one joint SoftConf space is used for many conferences (see the [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs report])
* [ACL 2016 chairs] Avoid confusion about the submission procedures for the final tutorial descriptions, those were submitted by email in many different formats

Author: Agata Savary, July 2019-July 2020

Updates are welcome

[[Category:Conference Handbook]]

Tutorial chair handbook

2020-07-15T17:24:10Z

AgataSavary: /* Lessons learned by past tutorial chairs */

== Duties ==
Tutorial chairs are nominated by the general chair of the main conference. Their duties are described in [[Tutorial chair duties|here]].

== Timeline ==
The timeline for the organisation of tutorials should follow the general [[conference planning schedule]]. Below, we complete this schedule with more detailed tasks, as defined for the ACL 2020 tutorials. The dates from the [[conference planning schedule]] are highlighted in '''bold'''. 
In recent years, calls and selection of tutorial proposals have been coordinated among some major CL conferences (usually a subset of ACL, NAACL, EMNLP, AACL and COLING). Note that if your conference is the first in the year to come, you will need to be the major driving force in the initial steps described below. 
Tutorial proposals are usually reviewed by the tutorial chairs of the coordinating conferences, but it may be a good idea to also rectruit a small group of external reviewers. They should have a good experience in reviewing and a large understanding of the recent progress in the CL domain.

* [T-12]
** Learn from the general chair which conferences will have their tutorials coordinated with yours.
** Establish contact with the tutorial chairs of these other conferences, and discuss
*** Recruitment of external reviewers
*** Proposal evaluation criteria
*** Soliciting tutorial proposals from particular potential lecturers on particular topics
*** Organizing a public poll about interest in the submitted tutorial
** Discuss general strategies of the main conference with the general chair and the program chairs
***promotion of some topics
***mailing lists and conference alert portals to be used for all announcements
** Set up a SoftConf space for proposal submission
*** Solicit the creation of a SoftConf space from [mailto:support@softconf.com support] (currently Rich Gerber). Note that submissions will be joint for several conferences but the proceedings will be edited for each conference separately. Therefore, a new SoftConf space will have to be created later for the selected proposals of each conference.
*** If you plan to have external tutorial reviewers, invite them via SoftConf.
*** Set up a submission page.
** Set up a joint email for tutorial chairs.
** Confirm the number (usually 6-8) and length (usually 3 hours) of tutorial slots with the ACL Business Manager (currently Priscilla Rasmussen).
** Draft the joint call for tutorials (based on [[#Past calls for tutorials|past calls]]).

* '''[T-11]''' ''(in the future, it would be good to extend the time slot between the call and the deadline to 2 months at least, especially if July/August is included)''
** Send out the call for tutorials.
** For posting the call on the ACL website, contact the ACL Business Manager (currently Priscilla Rasmussen).
** If you planned to encourage submission of proposals from particular potential lecturers, contact them now.
** Set up a SoftConf page for proposal reviews.

* [T-10]
** Deadline for tutorial proposal submission.
** If you decided to run a public poll about interest in the submitted tutorial, set it up and send out a call for expression of interest.
** Assign proposals to reviewers, send out invitations to review (via SoftConf).

* '''[T-9]'''
** Select tutorials jointly with other conference chairs.
** Notify the authors (via SoftConf).
** Liaise with local chair for space needs and scheduling.
** Send instructions to tutorial teachers about printed material, agenda/time management, payments, fees, etc.
** Liaise the conference website chair for updates about tutorials and authors' guidelines.
** Liaise with the ACL Office for the registration form.

* '''[T-4]'''
** Collect short descriptions for proceedings from tutorial teachers.
** Provide them to the webmaster and the general and program chairs .

* '''[T-3]'''
** Confirm the tutorials with authors.
** Authors submit the final tutorial descriptions for the proceedings.
** Arrange for printing of tutorial materials.

* [T-2]
** Arrange with the [mailto:support@softconf.com SoftConf support] the creation of a separate SoftConf space for the selected tutorials of your conference only. Indicate which tutorial submissions should be copied there.
** Set up the final submission form in the new SoftConf space. Include the submissions of slides, which will be published on the ACL Anthology. Include signing the CC-BY-4.0 license agreement.

* '''[T-1]'''
** Collect and check final tutorial materials.
** Create the tutorial proceedings, including slides.

* '''[T-0]''' Conference and tutorials

* [T+1]
** Invite the lecturers to update the slides at the ACL Anthology via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form] (under CC-BY-4.0).
** Update this tutorial page and the list of [[Past tutorials|past tutorials]] with the data from your conference. For access to the ACL [[Conference Handbook]], contact the ACL Secretary (currently Shiqi Zhou).

== Publication of tutorials ==
In tutorials organized until 2019, there was no homogeneous policy as to the publication of tutorial material. Data (abstracts, slides, code, etc.) from selected tutorials were usually scattered in 2 or 3 places:
* conference website
* ACL Anthology
* private repositories of the authors
The sustainability of these varies from venue to venue. The main websites of some conferences are not maintained, therefore ACL collects [https://www.aclweb.org/portal/node/2481 mirrors] of some of them (but not all).
The tutorial proceedings, published in the ACL Anthology, until recently included only abstracts but no tutorial slides.

As a result, for many [[past tutorials]], no slides are available online, so the community cannot benefit from these assets. It is, therefore, important to promote '''Open Access''' of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation '''slides in the ACL Anthology'''. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form].

It is recommended for tutorial chairs to:
* announce the publication of the tutorial slides under the CC-BY-4.0 license (used in the ACL Anthology)
* encourage the authors to also publish, under open licenses, their code and data used in the tutorials
* include the initial versions of tutorial slides, together with abstract, in the proceedings
* encourage the tutorial authors to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]

== Past tutorials ==
The [[Past tutorials|list of recent tutorials]] contains tutorial authors' names, titles, and to their summaries in conference websites and/or in the ACL Anthology.

== Organization of past tutorials ==
For the last years, the call and selection of tutorials has been coordinated among several major CL conferences. The table below summarizes some data collected from existing reports and from direct contacts with past tutorial chairs.

{| border="1" cellspacing="0" cellpadding="5"
|'''Year'''
|'''Coordinated conferences'''
|'''Tutorial chairs'''
|'''Submission platform'''
|'''# Submissions'''
|'''Reviewers'''
|'''# Reviews per proposal'''
|'''# Selected tutorials'''
|'''Selection criteria'''
|'''Useful links'''
|-
|2020
|ACL EMNLP AACL-IJCNLP COLING
|ACL: Agata Savary (France), Yue Zhang (China) EMNLP: Benjamin Van Durme (USA), Aline Villavicencio (UK & Brazil) AACL-IJCNLP: Tim Baldwin (Australia), Fei Xia (USA) COLING: Daniel Beck (Australia), Lucia Specia (UK)
|SoftConf
|43
|Tutorial chairs + external experts
|3
|ACL: 8 EMNLP: 7 ACCL-IJCNLP: 6 COLING: 7
|Clarity and preparedness Novelty or timely character of the topic Lecturers' experience Likely audience interest Open access of the teaching material Diversity aspects (multilingualism, gender, age and country of the lecturers) Compatibility with the required venues
|[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 Call] [https://acl2020.org/program/tutorials/ ACL tutorials] [https://2020.emnlp.org/tutorials EMNLP tutorials] [http://aacl2020.org/ ACCL-IJCNLP tutorials] [https://coling2020.org/pages/tutorials COLING tutorials]
|
|-
|2019
|NAACL ACL EMNLP
|ACL: Preslav Nakov (Qatar), Alexis Palmer (USA) NAACL: Anoop Sarkar (Canda), Michael Strube (Germany) EMNLP-IJCNLP: Marine Carpuat (USA), Tim Baldwin (Australia)
|SoftConf
|45
|Tutorial chairs
|2-3
|ACL: 9 NAACL: 6 EMNLP: 7
|Presenters (1-5) Description (1-5) Interest Level (1-5) Preparedness (1-5)
|[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials Call] [http://www.acl2019.org/EN/tutorials.xhtml ACL tutorials] [https://naacl2019.org/program/tutorials/ NAACL tutorials]
|-
|2018
|ACL NAACL EMNLP COLING
|ACL: Yoav Artzi (USA), Jacob Eisenstein (USA) COLING: Pascale Fung (Hong Kong), Donia Scott (UK), Marilyn Walker (USA) EMNLP: Mausam (India), Lu Wang (USA) NAACL: Mohit Bansal (USA), Rebecca Passonneau (USA)
|SoftConf
|49
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6 COLING: 6
|Quality of plan Diversity of topics Qualifications of presenter likely audience interest
|[https://acl2018.org/2017/09/11/joint-call-for-tutorials/ Call] 
[https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs Report] 
[https://acl2018.org/tutorials/ ACL tutorials] 
[https://emnlp2018.org/program/tutorials/ EMNLP tutorials] 
[https://coling2018.org/tutorial-list/ COLING tutorials]
|-
|2017
|ACL EACL EMNLP
|ACL: Maja Popović (Germany), Jordan Boyd-Graber (USA) EACL: Alex Klementiev (Germany), Lucia Specia (UK) EMNLP: Nathan Schneider (USA), Alexandra Birch (UK)
|SoftConf
|26
|Tutorial chairs
|3
|ACL: 6 EACL: 6 EMNLP: 7
|Clarity (description and outline) Topic
|[https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs Report] [http://acl2017.org/tutorials/ ACL tutorials] [http://eacl2017.org/index.php/tutorials EACL tutorials]
|-
|2016
|ACL NAACL EMNLP
|ACL: Alexandra Birch (UK), Willem Zuidema (Netherlands) NAACL: Alexander Rush and Bishan Yang EMNLP: Rebecca Hwa and Mohit Bansal
|SoftConf
|32
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6
|Relevance to ACL community (15) Quality of instructor (15) Outline and depth/breadth of the proposal (15) 
Potential attendance Newly emerging area not previously covered in an ACL-related tutorial (yes or no) Introduction into related fields (yes or no) 
Overall score (15) 1st preference for venue as indicated in proposal comments
|[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 Call] [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs Report] [http://naacl.org/naacl-hlt-2016/tutorials.html NAACL tutorials] [https://www.aclweb.org/mirror/emnlp2016/tutorials.html EMNLP tutorials]
|}

== Past calls for tutorials ==
*[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 ACL/AACL-IJCNLP/EMNLP/COLING 2020 joint call for tutorials]
*[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials NAACL/ACL/EMNLP 2019 joint call for tutorials]
*[https://www.aclweb.org/portal/content/final-joint-call-tutorial-proposals-aclcolingemnlpnaacl-2018 ACL/COLING/EMNLP/NAACL 2018 joint call for tutorials]
*[https://www.aclweb.org/portal/content/eacl-2017-call-tutorial-proposals EACL 2017 call for tutorials]
*[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 NAACL/ACL/EMNLP 2016 joint call for tutorials]
*[https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-naacl-2015-acl-2015-emnlp-2015-0 NAACL/ACL/EMNLP 2015 joint call for tutorials]

== Reports from past tutorial chairs ==
*[[2020Q1_Reports:_ACL_2020#Tutorial_Chairs|ACL 2020 report]] (by Agata Savary and Yue Zhang)
*[[2019Q1_Reports:_ACL_2019#Tutorial_Co-Chairs|ACL 2019 report]] (by Preslav Nakov and Alexis Palmer)
*[[2018Q3 Reports: Tutorial Chairs|ACL 2018 report]] (by Yoav Artzi and Jacob Eisenstein)
*[[2017Q3 Reports: Tutorial Chairs|ACL 2017 report]] (by Maja Popović and Jordan Boyd-Graber)
*[[2016Q3 Reports: Tutorial Chairs|ACL 2016 report]] (by Alexandra Birch and Willem Zuidema)

== Lessons learned by past tutorial chairs ==
* [Agata Savary, 2020] If the conference is virtual:
** Remember that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also [https://acl2020.org/blog/detailed-modalities-of-tutorials/ this post]).
** Announce the individual modalities with the attendees early enough (for instance, if watching pre-recordings in advance is necessary for some tutorials because their live sessions only included question answering)
** If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to Zoom rooms only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.
* [Agata Savary, 2020] Use a common SoftConf space for joint tutorial submissions for all coordinated conference. Once the proposals have been selected and assigned to conferences, use one track per conference in the same SoftConf space.
* [Agata Savary, 2020] Publish the list of recommended readings on each tutorial page
* [Agata Savary, 2020] A good way to proceed while selecting tutorials for several venues is to have a common online meeting, in which
** Tutorials are clustered per themes (to distribute similar tutorials over different conferences)
** The chairs for each venue select their best choices in a round robin manner.
** Generate csv files with reviews and scores: Manager -> Other tools -> Spreadsheet maker -> Reports
* [Agata Savary, 2020] While setting up a review form on SoftConf, include:
** All evaluation criteria, including diversity, in the review form
** Cutting edge vs. introductory category
** Duration (3 or 6 hours)
* [Agata Savary, 2020] Currently, the generic conference schedule assumes only 2 months between the call for tutorials and notification to proposers. If a solid 1-month review procedure is to take place, this means, that proposers only have 1 month between the call and the submission deadline. It would be good to shift the call to [T-12].
* [Agata Savary, 2020] For many past tutorials, no slides are available online. It is, therefore, important to promote Open Access of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation slides on the ACL Anthology. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]. As a tutorial chair, make sure that
** initial versions of tutorial slides are submitted to the proceedings together with the tutorial abstracts
** tutorial authors are encouraged to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]
* [Marine Capuat, 2019] Continue soliciting both cutting edge and introductory tutorials
* [Marine Capuat, 2019] Ask proposers to express their preferred location/conference in one place (i.e. either in proposal pdf, or in softconf form, not both) to avoid inconsistencies
* [ACL 2018 chairs] Issues with proceedings edition, when one joint SoftConf space is used for many conferences (see the [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs report])
* [ACL 2016 chairs] Avoid confusion about the submission procedures for the final tutorial descriptions, those were submitted by email in many different formats

Author: Agata Savary, July 2019

Updates are welcome

[[Category:Conference Handbook]]

Tutorial chair handbook

2020-07-15T17:23:43Z

AgataSavary: /* Lessons learned by past tutorial chairs */

== Duties ==
Tutorial chairs are nominated by the general chair of the main conference. Their duties are described in [[Tutorial chair duties|here]].

== Timeline ==
The timeline for the organisation of tutorials should follow the general [[conference planning schedule]]. Below, we complete this schedule with more detailed tasks, as defined for the ACL 2020 tutorials. The dates from the [[conference planning schedule]] are highlighted in '''bold'''. 
In recent years, calls and selection of tutorial proposals have been coordinated among some major CL conferences (usually a subset of ACL, NAACL, EMNLP, AACL and COLING). Note that if your conference is the first in the year to come, you will need to be the major driving force in the initial steps described below. 
Tutorial proposals are usually reviewed by the tutorial chairs of the coordinating conferences, but it may be a good idea to also rectruit a small group of external reviewers. They should have a good experience in reviewing and a large understanding of the recent progress in the CL domain.

* [T-12]
** Learn from the general chair which conferences will have their tutorials coordinated with yours.
** Establish contact with the tutorial chairs of these other conferences, and discuss
*** Recruitment of external reviewers
*** Proposal evaluation criteria
*** Soliciting tutorial proposals from particular potential lecturers on particular topics
*** Organizing a public poll about interest in the submitted tutorial
** Discuss general strategies of the main conference with the general chair and the program chairs
***promotion of some topics
***mailing lists and conference alert portals to be used for all announcements
** Set up a SoftConf space for proposal submission
*** Solicit the creation of a SoftConf space from [mailto:support@softconf.com support] (currently Rich Gerber). Note that submissions will be joint for several conferences but the proceedings will be edited for each conference separately. Therefore, a new SoftConf space will have to be created later for the selected proposals of each conference.
*** If you plan to have external tutorial reviewers, invite them via SoftConf.
*** Set up a submission page.
** Set up a joint email for tutorial chairs.
** Confirm the number (usually 6-8) and length (usually 3 hours) of tutorial slots with the ACL Business Manager (currently Priscilla Rasmussen).
** Draft the joint call for tutorials (based on [[#Past calls for tutorials|past calls]]).

* '''[T-11]''' ''(in the future, it would be good to extend the time slot between the call and the deadline to 2 months at least, especially if July/August is included)''
** Send out the call for tutorials.
** For posting the call on the ACL website, contact the ACL Business Manager (currently Priscilla Rasmussen).
** If you planned to encourage submission of proposals from particular potential lecturers, contact them now.
** Set up a SoftConf page for proposal reviews.

* [T-10]
** Deadline for tutorial proposal submission.
** If you decided to run a public poll about interest in the submitted tutorial, set it up and send out a call for expression of interest.
** Assign proposals to reviewers, send out invitations to review (via SoftConf).

* '''[T-9]'''
** Select tutorials jointly with other conference chairs.
** Notify the authors (via SoftConf).
** Liaise with local chair for space needs and scheduling.
** Send instructions to tutorial teachers about printed material, agenda/time management, payments, fees, etc.
** Liaise the conference website chair for updates about tutorials and authors' guidelines.
** Liaise with the ACL Office for the registration form.

* '''[T-4]'''
** Collect short descriptions for proceedings from tutorial teachers.
** Provide them to the webmaster and the general and program chairs .

* '''[T-3]'''
** Confirm the tutorials with authors.
** Authors submit the final tutorial descriptions for the proceedings.
** Arrange for printing of tutorial materials.

* [T-2]
** Arrange with the [mailto:support@softconf.com SoftConf support] the creation of a separate SoftConf space for the selected tutorials of your conference only. Indicate which tutorial submissions should be copied there.
** Set up the final submission form in the new SoftConf space. Include the submissions of slides, which will be published on the ACL Anthology. Include signing the CC-BY-4.0 license agreement.

* '''[T-1]'''
** Collect and check final tutorial materials.
** Create the tutorial proceedings, including slides.

* '''[T-0]''' Conference and tutorials

* [T+1]
** Invite the lecturers to update the slides at the ACL Anthology via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form] (under CC-BY-4.0).
** Update this tutorial page and the list of [[Past tutorials|past tutorials]] with the data from your conference. For access to the ACL [[Conference Handbook]], contact the ACL Secretary (currently Shiqi Zhou).

== Publication of tutorials ==
In tutorials organized until 2019, there was no homogeneous policy as to the publication of tutorial material. Data (abstracts, slides, code, etc.) from selected tutorials were usually scattered in 2 or 3 places:
* conference website
* ACL Anthology
* private repositories of the authors
The sustainability of these varies from venue to venue. The main websites of some conferences are not maintained, therefore ACL collects [https://www.aclweb.org/portal/node/2481 mirrors] of some of them (but not all).
The tutorial proceedings, published in the ACL Anthology, until recently included only abstracts but no tutorial slides.

As a result, for many [[past tutorials]], no slides are available online, so the community cannot benefit from these assets. It is, therefore, important to promote '''Open Access''' of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation '''slides in the ACL Anthology'''. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form].

It is recommended for tutorial chairs to:
* announce the publication of the tutorial slides under the CC-BY-4.0 license (used in the ACL Anthology)
* encourage the authors to also publish, under open licenses, their code and data used in the tutorials
* include the initial versions of tutorial slides, together with abstract, in the proceedings
* encourage the tutorial authors to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]

== Past tutorials ==
The [[Past tutorials|list of recent tutorials]] contains tutorial authors' names, titles, and to their summaries in conference websites and/or in the ACL Anthology.

== Organization of past tutorials ==
For the last years, the call and selection of tutorials has been coordinated among several major CL conferences. The table below summarizes some data collected from existing reports and from direct contacts with past tutorial chairs.

{| border="1" cellspacing="0" cellpadding="5"
|'''Year'''
|'''Coordinated conferences'''
|'''Tutorial chairs'''
|'''Submission platform'''
|'''# Submissions'''
|'''Reviewers'''
|'''# Reviews per proposal'''
|'''# Selected tutorials'''
|'''Selection criteria'''
|'''Useful links'''
|-
|2020
|ACL EMNLP AACL-IJCNLP COLING
|ACL: Agata Savary (France), Yue Zhang (China) EMNLP: Benjamin Van Durme (USA), Aline Villavicencio (UK & Brazil) AACL-IJCNLP: Tim Baldwin (Australia), Fei Xia (USA) COLING: Daniel Beck (Australia), Lucia Specia (UK)
|SoftConf
|43
|Tutorial chairs + external experts
|3
|ACL: 8 EMNLP: 7 ACCL-IJCNLP: 6 COLING: 7
|Clarity and preparedness Novelty or timely character of the topic Lecturers' experience Likely audience interest Open access of the teaching material Diversity aspects (multilingualism, gender, age and country of the lecturers) Compatibility with the required venues
|[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 Call] [https://acl2020.org/program/tutorials/ ACL tutorials] [https://2020.emnlp.org/tutorials EMNLP tutorials] [http://aacl2020.org/ ACCL-IJCNLP tutorials] [https://coling2020.org/pages/tutorials COLING tutorials]
|
|-
|2019
|NAACL ACL EMNLP
|ACL: Preslav Nakov (Qatar), Alexis Palmer (USA) NAACL: Anoop Sarkar (Canda), Michael Strube (Germany) EMNLP-IJCNLP: Marine Carpuat (USA), Tim Baldwin (Australia)
|SoftConf
|45
|Tutorial chairs
|2-3
|ACL: 9 NAACL: 6 EMNLP: 7
|Presenters (1-5) Description (1-5) Interest Level (1-5) Preparedness (1-5)
|[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials Call] [http://www.acl2019.org/EN/tutorials.xhtml ACL tutorials] [https://naacl2019.org/program/tutorials/ NAACL tutorials]
|-
|2018
|ACL NAACL EMNLP COLING
|ACL: Yoav Artzi (USA), Jacob Eisenstein (USA) COLING: Pascale Fung (Hong Kong), Donia Scott (UK), Marilyn Walker (USA) EMNLP: Mausam (India), Lu Wang (USA) NAACL: Mohit Bansal (USA), Rebecca Passonneau (USA)
|SoftConf
|49
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6 COLING: 6
|Quality of plan Diversity of topics Qualifications of presenter likely audience interest
|[https://acl2018.org/2017/09/11/joint-call-for-tutorials/ Call] 
[https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs Report] 
[https://acl2018.org/tutorials/ ACL tutorials] 
[https://emnlp2018.org/program/tutorials/ EMNLP tutorials] 
[https://coling2018.org/tutorial-list/ COLING tutorials]
|-
|2017
|ACL EACL EMNLP
|ACL: Maja Popović (Germany), Jordan Boyd-Graber (USA) EACL: Alex Klementiev (Germany), Lucia Specia (UK) EMNLP: Nathan Schneider (USA), Alexandra Birch (UK)
|SoftConf
|26
|Tutorial chairs
|3
|ACL: 6 EACL: 6 EMNLP: 7
|Clarity (description and outline) Topic
|[https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs Report] [http://acl2017.org/tutorials/ ACL tutorials] [http://eacl2017.org/index.php/tutorials EACL tutorials]
|-
|2016
|ACL NAACL EMNLP
|ACL: Alexandra Birch (UK), Willem Zuidema (Netherlands) NAACL: Alexander Rush and Bishan Yang EMNLP: Rebecca Hwa and Mohit Bansal
|SoftConf
|32
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6
|Relevance to ACL community (15) Quality of instructor (15) Outline and depth/breadth of the proposal (15) 
Potential attendance Newly emerging area not previously covered in an ACL-related tutorial (yes or no) Introduction into related fields (yes or no) 
Overall score (15) 1st preference for venue as indicated in proposal comments
|[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 Call] [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs Report] [http://naacl.org/naacl-hlt-2016/tutorials.html NAACL tutorials] [https://www.aclweb.org/mirror/emnlp2016/tutorials.html EMNLP tutorials]
|}

== Past calls for tutorials ==
*[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 ACL/AACL-IJCNLP/EMNLP/COLING 2020 joint call for tutorials]
*[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials NAACL/ACL/EMNLP 2019 joint call for tutorials]
*[https://www.aclweb.org/portal/content/final-joint-call-tutorial-proposals-aclcolingemnlpnaacl-2018 ACL/COLING/EMNLP/NAACL 2018 joint call for tutorials]
*[https://www.aclweb.org/portal/content/eacl-2017-call-tutorial-proposals EACL 2017 call for tutorials]
*[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 NAACL/ACL/EMNLP 2016 joint call for tutorials]
*[https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-naacl-2015-acl-2015-emnlp-2015-0 NAACL/ACL/EMNLP 2015 joint call for tutorials]

== Reports from past tutorial chairs ==
*[[2020Q1_Reports:_ACL_2020#Tutorial_Chairs|ACL 2020 report]] (by Agata Savary and Yue Zhang)
*[[2019Q1_Reports:_ACL_2019#Tutorial_Co-Chairs|ACL 2019 report]] (by Preslav Nakov and Alexis Palmer)
*[[2018Q3 Reports: Tutorial Chairs|ACL 2018 report]] (by Yoav Artzi and Jacob Eisenstein)
*[[2017Q3 Reports: Tutorial Chairs|ACL 2017 report]] (by Maja Popović and Jordan Boyd-Graber)
*[[2016Q3 Reports: Tutorial Chairs|ACL 2016 report]] (by Alexandra Birch and Willem Zuidema)

== Lessons learned by past tutorial chairs ==
* [Agata Savary, 2020] If the conference is virtual:
** Remember that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also [https://acl2020.org/blog/detailed-modalities-of-tutorials/|this post]).
** Announce the individual modalities with the attendees early enough (for instance, if watching pre-recordings in advance is necessary for some tutorials because their live sessions only included question answering)
** If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to Zoom rooms only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.
* [Agata Savary, 2020] Use a common SoftConf space for joint tutorial submissions for all coordinated conference. Once the proposals have been selected and assigned to conferences, use one track per conference in the same SoftConf space.
* [Agata Savary, 2020] Publish the list of recommended readings on each tutorial page
* [Agata Savary, 2020] A good way to proceed while selecting tutorials for several venues is to have a common online meeting, in which
** Tutorials are clustered per themes (to distribute similar tutorials over different conferences)
** The chairs for each venue select their best choices in a round robin manner.
** Generate csv files with reviews and scores: Manager -> Other tools -> Spreadsheet maker -> Reports
* [Agata Savary, 2020] While setting up a review form on SoftConf, include:
** All evaluation criteria, including diversity, in the review form
** Cutting edge vs. introductory category
** Duration (3 or 6 hours)
* [Agata Savary, 2020] Currently, the generic conference schedule assumes only 2 months between the call for tutorials and notification to proposers. If a solid 1-month review procedure is to take place, this means, that proposers only have 1 month between the call and the submission deadline. It would be good to shift the call to [T-12].
* [Agata Savary, 2020] For many past tutorials, no slides are available online. It is, therefore, important to promote Open Access of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation slides on the ACL Anthology. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]. As a tutorial chair, make sure that
** initial versions of tutorial slides are submitted to the proceedings together with the tutorial abstracts
** tutorial authors are encouraged to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]
* [Marine Capuat, 2019] Continue soliciting both cutting edge and introductory tutorials
* [Marine Capuat, 2019] Ask proposers to express their preferred location/conference in one place (i.e. either in proposal pdf, or in softconf form, not both) to avoid inconsistencies
* [ACL 2018 chairs] Issues with proceedings edition, when one joint SoftConf space is used for many conferences (see the [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs report])
* [ACL 2016 chairs] Avoid confusion about the submission procedures for the final tutorial descriptions, those were submitted by email in many different formats

Author: Agata Savary, July 2019

Updates are welcome

[[Category:Conference Handbook]]

Tutorial chair handbook

2020-07-15T17:22:25Z

AgataSavary: /* Lessons learned by past tutorial chairs */

== Duties ==
Tutorial chairs are nominated by the general chair of the main conference. Their duties are described in [[Tutorial chair duties|here]].

== Timeline ==
The timeline for the organisation of tutorials should follow the general [[conference planning schedule]]. Below, we complete this schedule with more detailed tasks, as defined for the ACL 2020 tutorials. The dates from the [[conference planning schedule]] are highlighted in '''bold'''. 
In recent years, calls and selection of tutorial proposals have been coordinated among some major CL conferences (usually a subset of ACL, NAACL, EMNLP, AACL and COLING). Note that if your conference is the first in the year to come, you will need to be the major driving force in the initial steps described below. 
Tutorial proposals are usually reviewed by the tutorial chairs of the coordinating conferences, but it may be a good idea to also rectruit a small group of external reviewers. They should have a good experience in reviewing and a large understanding of the recent progress in the CL domain.

* [T-12]
** Learn from the general chair which conferences will have their tutorials coordinated with yours.
** Establish contact with the tutorial chairs of these other conferences, and discuss
*** Recruitment of external reviewers
*** Proposal evaluation criteria
*** Soliciting tutorial proposals from particular potential lecturers on particular topics
*** Organizing a public poll about interest in the submitted tutorial
** Discuss general strategies of the main conference with the general chair and the program chairs
***promotion of some topics
***mailing lists and conference alert portals to be used for all announcements
** Set up a SoftConf space for proposal submission
*** Solicit the creation of a SoftConf space from [mailto:support@softconf.com support] (currently Rich Gerber). Note that submissions will be joint for several conferences but the proceedings will be edited for each conference separately. Therefore, a new SoftConf space will have to be created later for the selected proposals of each conference.
*** If you plan to have external tutorial reviewers, invite them via SoftConf.
*** Set up a submission page.
** Set up a joint email for tutorial chairs.
** Confirm the number (usually 6-8) and length (usually 3 hours) of tutorial slots with the ACL Business Manager (currently Priscilla Rasmussen).
** Draft the joint call for tutorials (based on [[#Past calls for tutorials|past calls]]).

* '''[T-11]''' ''(in the future, it would be good to extend the time slot between the call and the deadline to 2 months at least, especially if July/August is included)''
** Send out the call for tutorials.
** For posting the call on the ACL website, contact the ACL Business Manager (currently Priscilla Rasmussen).
** If you planned to encourage submission of proposals from particular potential lecturers, contact them now.
** Set up a SoftConf page for proposal reviews.

* [T-10]
** Deadline for tutorial proposal submission.
** If you decided to run a public poll about interest in the submitted tutorial, set it up and send out a call for expression of interest.
** Assign proposals to reviewers, send out invitations to review (via SoftConf).

* '''[T-9]'''
** Select tutorials jointly with other conference chairs.
** Notify the authors (via SoftConf).
** Liaise with local chair for space needs and scheduling.
** Send instructions to tutorial teachers about printed material, agenda/time management, payments, fees, etc.
** Liaise the conference website chair for updates about tutorials and authors' guidelines.
** Liaise with the ACL Office for the registration form.

* '''[T-4]'''
** Collect short descriptions for proceedings from tutorial teachers.
** Provide them to the webmaster and the general and program chairs .

* '''[T-3]'''
** Confirm the tutorials with authors.
** Authors submit the final tutorial descriptions for the proceedings.
** Arrange for printing of tutorial materials.

* [T-2]
** Arrange with the [mailto:support@softconf.com SoftConf support] the creation of a separate SoftConf space for the selected tutorials of your conference only. Indicate which tutorial submissions should be copied there.
** Set up the final submission form in the new SoftConf space. Include the submissions of slides, which will be published on the ACL Anthology. Include signing the CC-BY-4.0 license agreement.

* '''[T-1]'''
** Collect and check final tutorial materials.
** Create the tutorial proceedings, including slides.

* '''[T-0]''' Conference and tutorials

* [T+1]
** Invite the lecturers to update the slides at the ACL Anthology via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form] (under CC-BY-4.0).
** Update this tutorial page and the list of [[Past tutorials|past tutorials]] with the data from your conference. For access to the ACL [[Conference Handbook]], contact the ACL Secretary (currently Shiqi Zhou).

== Publication of tutorials ==
In tutorials organized until 2019, there was no homogeneous policy as to the publication of tutorial material. Data (abstracts, slides, code, etc.) from selected tutorials were usually scattered in 2 or 3 places:
* conference website
* ACL Anthology
* private repositories of the authors
The sustainability of these varies from venue to venue. The main websites of some conferences are not maintained, therefore ACL collects [https://www.aclweb.org/portal/node/2481 mirrors] of some of them (but not all).
The tutorial proceedings, published in the ACL Anthology, until recently included only abstracts but no tutorial slides.

As a result, for many [[past tutorials]], no slides are available online, so the community cannot benefit from these assets. It is, therefore, important to promote '''Open Access''' of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation '''slides in the ACL Anthology'''. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form].

It is recommended for tutorial chairs to:
* announce the publication of the tutorial slides under the CC-BY-4.0 license (used in the ACL Anthology)
* encourage the authors to also publish, under open licenses, their code and data used in the tutorials
* include the initial versions of tutorial slides, together with abstract, in the proceedings
* encourage the tutorial authors to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]

== Past tutorials ==
The [[Past tutorials|list of recent tutorials]] contains tutorial authors' names, titles, and to their summaries in conference websites and/or in the ACL Anthology.

== Organization of past tutorials ==
For the last years, the call and selection of tutorials has been coordinated among several major CL conferences. The table below summarizes some data collected from existing reports and from direct contacts with past tutorial chairs.

{| border="1" cellspacing="0" cellpadding="5"
|'''Year'''
|'''Coordinated conferences'''
|'''Tutorial chairs'''
|'''Submission platform'''
|'''# Submissions'''
|'''Reviewers'''
|'''# Reviews per proposal'''
|'''# Selected tutorials'''
|'''Selection criteria'''
|'''Useful links'''
|-
|2020
|ACL EMNLP AACL-IJCNLP COLING
|ACL: Agata Savary (France), Yue Zhang (China) EMNLP: Benjamin Van Durme (USA), Aline Villavicencio (UK & Brazil) AACL-IJCNLP: Tim Baldwin (Australia), Fei Xia (USA) COLING: Daniel Beck (Australia), Lucia Specia (UK)
|SoftConf
|43
|Tutorial chairs + external experts
|3
|ACL: 8 EMNLP: 7 ACCL-IJCNLP: 6 COLING: 7
|Clarity and preparedness Novelty or timely character of the topic Lecturers' experience Likely audience interest Open access of the teaching material Diversity aspects (multilingualism, gender, age and country of the lecturers) Compatibility with the required venues
|[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 Call] [https://acl2020.org/program/tutorials/ ACL tutorials] [https://2020.emnlp.org/tutorials EMNLP tutorials] [http://aacl2020.org/ ACCL-IJCNLP tutorials] [https://coling2020.org/pages/tutorials COLING tutorials]
|
|-
|2019
|NAACL ACL EMNLP
|ACL: Preslav Nakov (Qatar), Alexis Palmer (USA) NAACL: Anoop Sarkar (Canda), Michael Strube (Germany) EMNLP-IJCNLP: Marine Carpuat (USA), Tim Baldwin (Australia)
|SoftConf
|45
|Tutorial chairs
|2-3
|ACL: 9 NAACL: 6 EMNLP: 7
|Presenters (1-5) Description (1-5) Interest Level (1-5) Preparedness (1-5)
|[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials Call] [http://www.acl2019.org/EN/tutorials.xhtml ACL tutorials] [https://naacl2019.org/program/tutorials/ NAACL tutorials]
|-
|2018
|ACL NAACL EMNLP COLING
|ACL: Yoav Artzi (USA), Jacob Eisenstein (USA) COLING: Pascale Fung (Hong Kong), Donia Scott (UK), Marilyn Walker (USA) EMNLP: Mausam (India), Lu Wang (USA) NAACL: Mohit Bansal (USA), Rebecca Passonneau (USA)
|SoftConf
|49
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6 COLING: 6
|Quality of plan Diversity of topics Qualifications of presenter likely audience interest
|[https://acl2018.org/2017/09/11/joint-call-for-tutorials/ Call] 
[https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs Report] 
[https://acl2018.org/tutorials/ ACL tutorials] 
[https://emnlp2018.org/program/tutorials/ EMNLP tutorials] 
[https://coling2018.org/tutorial-list/ COLING tutorials]
|-
|2017
|ACL EACL EMNLP
|ACL: Maja Popović (Germany), Jordan Boyd-Graber (USA) EACL: Alex Klementiev (Germany), Lucia Specia (UK) EMNLP: Nathan Schneider (USA), Alexandra Birch (UK)
|SoftConf
|26
|Tutorial chairs
|3
|ACL: 6 EACL: 6 EMNLP: 7
|Clarity (description and outline) Topic
|[https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs Report] [http://acl2017.org/tutorials/ ACL tutorials] [http://eacl2017.org/index.php/tutorials EACL tutorials]
|-
|2016
|ACL NAACL EMNLP
|ACL: Alexandra Birch (UK), Willem Zuidema (Netherlands) NAACL: Alexander Rush and Bishan Yang EMNLP: Rebecca Hwa and Mohit Bansal
|SoftConf
|32
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6
|Relevance to ACL community (15) Quality of instructor (15) Outline and depth/breadth of the proposal (15) 
Potential attendance Newly emerging area not previously covered in an ACL-related tutorial (yes or no) Introduction into related fields (yes or no) 
Overall score (15) 1st preference for venue as indicated in proposal comments
|[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 Call] [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs Report] [http://naacl.org/naacl-hlt-2016/tutorials.html NAACL tutorials] [https://www.aclweb.org/mirror/emnlp2016/tutorials.html EMNLP tutorials]
|}

== Past calls for tutorials ==
*[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 ACL/AACL-IJCNLP/EMNLP/COLING 2020 joint call for tutorials]
*[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials NAACL/ACL/EMNLP 2019 joint call for tutorials]
*[https://www.aclweb.org/portal/content/final-joint-call-tutorial-proposals-aclcolingemnlpnaacl-2018 ACL/COLING/EMNLP/NAACL 2018 joint call for tutorials]
*[https://www.aclweb.org/portal/content/eacl-2017-call-tutorial-proposals EACL 2017 call for tutorials]
*[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 NAACL/ACL/EMNLP 2016 joint call for tutorials]
*[https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-naacl-2015-acl-2015-emnlp-2015-0 NAACL/ACL/EMNLP 2015 joint call for tutorials]

== Reports from past tutorial chairs ==
*[[2020Q1_Reports:_ACL_2020#Tutorial_Chairs|ACL 2020 report]] (by Agata Savary and Yue Zhang)
*[[2019Q1_Reports:_ACL_2019#Tutorial_Co-Chairs|ACL 2019 report]] (by Preslav Nakov and Alexis Palmer)
*[[2018Q3 Reports: Tutorial Chairs|ACL 2018 report]] (by Yoav Artzi and Jacob Eisenstein)
*[[2017Q3 Reports: Tutorial Chairs|ACL 2017 report]] (by Maja Popović and Jordan Boyd-Graber)
*[[2016Q3 Reports: Tutorial Chairs|ACL 2016 report]] (by Alexandra Birch and Willem Zuidema)

== Lessons learned by past tutorial chairs ==
* [Agata Savary, 2020] If the conference is virtual:
** Remember that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also this post).
** Announce the individual modalities with the attendees early enough (for instance, if watching pre-recordings in advance is necessary for some tutorials because their live sessions only included question answering)
** If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to Zoom rooms only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.
* [Agata Savary, 2020] Use a common SoftConf space for joint tutorial submissions for all coordinated conference. Once the proposals have been selected and assigned to conferences, use one track per conference in the same SoftConf space.
* [Agata Savary, 2020] Publish the list of recommended readings on each tutorial page
* [Agata Savary, 2020] A good way to proceed while selecting tutorials for several venues is to have a common online meeting, in which
** Tutorials are clustered per themes (to distribute similar tutorials over different conferences)
** The chairs for each venue select their best choices in a round robin manner.
** Generate csv files with reviews and scores: Manager -> Other tools -> Spreadsheet maker -> Reports
* [Agata Savary, 2020] While setting up a review form on SoftConf, include:
** All evaluation criteria, including diversity, in the review form
** Cutting edge vs. introductory category
** Duration (3 or 6 hours)
* [Agata Savary, 2020] Currently, the generic conference schedule assumes only 2 months between the call for tutorials and notification to proposers. If a solid 1-month review procedure is to take place, this means, that proposers only have 1 month between the call and the submission deadline. It would be good to shift the call to [T-12].
* [Agata Savary, 2020] For many past tutorials, no slides are available online. It is, therefore, important to promote Open Access of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation slides on the ACL Anthology. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]. As a tutorial chair, make sure that
** initial versions of tutorial slides are submitted to the proceedings together with the tutorial abstracts
** tutorial authors are encouraged to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]
* [Marine Capuat, 2019] Continue soliciting both cutting edge and introductory tutorials
* [Marine Capuat, 2019] Ask proposers to express their preferred location/conference in one place (i.e. either in proposal pdf, or in softconf form, not both) to avoid inconsistencies
* [ACL 2018 chairs] Issues with proceedings edition, when one joint SoftConf space is used for many conferences (see the [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs report])
* [ACL 2016 chairs] Avoid confusion about the submission procedures for the final tutorial descriptions, those were submitted by email in many different formats

Author: Agata Savary, July 2019

Updates are welcome

[[Category:Conference Handbook]]

Tutorial chair handbook

2020-07-15T17:21:56Z

AgataSavary:

== Duties ==
Tutorial chairs are nominated by the general chair of the main conference. Their duties are described in [[Tutorial chair duties|here]].

== Timeline ==
The timeline for the organisation of tutorials should follow the general [[conference planning schedule]]. Below, we complete this schedule with more detailed tasks, as defined for the ACL 2020 tutorials. The dates from the [[conference planning schedule]] are highlighted in '''bold'''. 
In recent years, calls and selection of tutorial proposals have been coordinated among some major CL conferences (usually a subset of ACL, NAACL, EMNLP, AACL and COLING). Note that if your conference is the first in the year to come, you will need to be the major driving force in the initial steps described below. 
Tutorial proposals are usually reviewed by the tutorial chairs of the coordinating conferences, but it may be a good idea to also rectruit a small group of external reviewers. They should have a good experience in reviewing and a large understanding of the recent progress in the CL domain.

* [T-12]
** Learn from the general chair which conferences will have their tutorials coordinated with yours.
** Establish contact with the tutorial chairs of these other conferences, and discuss
*** Recruitment of external reviewers
*** Proposal evaluation criteria
*** Soliciting tutorial proposals from particular potential lecturers on particular topics
*** Organizing a public poll about interest in the submitted tutorial
** Discuss general strategies of the main conference with the general chair and the program chairs
***promotion of some topics
***mailing lists and conference alert portals to be used for all announcements
** Set up a SoftConf space for proposal submission
*** Solicit the creation of a SoftConf space from [mailto:support@softconf.com support] (currently Rich Gerber). Note that submissions will be joint for several conferences but the proceedings will be edited for each conference separately. Therefore, a new SoftConf space will have to be created later for the selected proposals of each conference.
*** If you plan to have external tutorial reviewers, invite them via SoftConf.
*** Set up a submission page.
** Set up a joint email for tutorial chairs.
** Confirm the number (usually 6-8) and length (usually 3 hours) of tutorial slots with the ACL Business Manager (currently Priscilla Rasmussen).
** Draft the joint call for tutorials (based on [[#Past calls for tutorials|past calls]]).

* '''[T-11]''' ''(in the future, it would be good to extend the time slot between the call and the deadline to 2 months at least, especially if July/August is included)''
** Send out the call for tutorials.
** For posting the call on the ACL website, contact the ACL Business Manager (currently Priscilla Rasmussen).
** If you planned to encourage submission of proposals from particular potential lecturers, contact them now.
** Set up a SoftConf page for proposal reviews.

* [T-10]
** Deadline for tutorial proposal submission.
** If you decided to run a public poll about interest in the submitted tutorial, set it up and send out a call for expression of interest.
** Assign proposals to reviewers, send out invitations to review (via SoftConf).

* '''[T-9]'''
** Select tutorials jointly with other conference chairs.
** Notify the authors (via SoftConf).
** Liaise with local chair for space needs and scheduling.
** Send instructions to tutorial teachers about printed material, agenda/time management, payments, fees, etc.
** Liaise the conference website chair for updates about tutorials and authors' guidelines.
** Liaise with the ACL Office for the registration form.

* '''[T-4]'''
** Collect short descriptions for proceedings from tutorial teachers.
** Provide them to the webmaster and the general and program chairs .

* '''[T-3]'''
** Confirm the tutorials with authors.
** Authors submit the final tutorial descriptions for the proceedings.
** Arrange for printing of tutorial materials.

* [T-2]
** Arrange with the [mailto:support@softconf.com SoftConf support] the creation of a separate SoftConf space for the selected tutorials of your conference only. Indicate which tutorial submissions should be copied there.
** Set up the final submission form in the new SoftConf space. Include the submissions of slides, which will be published on the ACL Anthology. Include signing the CC-BY-4.0 license agreement.

* '''[T-1]'''
** Collect and check final tutorial materials.
** Create the tutorial proceedings, including slides.

* '''[T-0]''' Conference and tutorials

* [T+1]
** Invite the lecturers to update the slides at the ACL Anthology via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form] (under CC-BY-4.0).
** Update this tutorial page and the list of [[Past tutorials|past tutorials]] with the data from your conference. For access to the ACL [[Conference Handbook]], contact the ACL Secretary (currently Shiqi Zhou).

== Publication of tutorials ==
In tutorials organized until 2019, there was no homogeneous policy as to the publication of tutorial material. Data (abstracts, slides, code, etc.) from selected tutorials were usually scattered in 2 or 3 places:
* conference website
* ACL Anthology
* private repositories of the authors
The sustainability of these varies from venue to venue. The main websites of some conferences are not maintained, therefore ACL collects [https://www.aclweb.org/portal/node/2481 mirrors] of some of them (but not all).
The tutorial proceedings, published in the ACL Anthology, until recently included only abstracts but no tutorial slides.

As a result, for many [[past tutorials]], no slides are available online, so the community cannot benefit from these assets. It is, therefore, important to promote '''Open Access''' of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation '''slides in the ACL Anthology'''. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form].

It is recommended for tutorial chairs to:
* announce the publication of the tutorial slides under the CC-BY-4.0 license (used in the ACL Anthology)
* encourage the authors to also publish, under open licenses, their code and data used in the tutorials
* include the initial versions of tutorial slides, together with abstract, in the proceedings
* encourage the tutorial authors to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]

== Past tutorials ==
The [[Past tutorials|list of recent tutorials]] contains tutorial authors' names, titles, and to their summaries in conference websites and/or in the ACL Anthology.

== Organization of past tutorials ==
For the last years, the call and selection of tutorials has been coordinated among several major CL conferences. The table below summarizes some data collected from existing reports and from direct contacts with past tutorial chairs.

{| border="1" cellspacing="0" cellpadding="5"
|'''Year'''
|'''Coordinated conferences'''
|'''Tutorial chairs'''
|'''Submission platform'''
|'''# Submissions'''
|'''Reviewers'''
|'''# Reviews per proposal'''
|'''# Selected tutorials'''
|'''Selection criteria'''
|'''Useful links'''
|-
|2020
|ACL EMNLP AACL-IJCNLP COLING
|ACL: Agata Savary (France), Yue Zhang (China) EMNLP: Benjamin Van Durme (USA), Aline Villavicencio (UK & Brazil) AACL-IJCNLP: Tim Baldwin (Australia), Fei Xia (USA) COLING: Daniel Beck (Australia), Lucia Specia (UK)
|SoftConf
|43
|Tutorial chairs + external experts
|3
|ACL: 8 EMNLP: 7 ACCL-IJCNLP: 6 COLING: 7
|Clarity and preparedness Novelty or timely character of the topic Lecturers' experience Likely audience interest Open access of the teaching material Diversity aspects (multilingualism, gender, age and country of the lecturers) Compatibility with the required venues
|[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 Call] [https://acl2020.org/program/tutorials/ ACL tutorials] [https://2020.emnlp.org/tutorials EMNLP tutorials] [http://aacl2020.org/ ACCL-IJCNLP tutorials] [https://coling2020.org/pages/tutorials COLING tutorials]
|
|-
|2019
|NAACL ACL EMNLP
|ACL: Preslav Nakov (Qatar), Alexis Palmer (USA) NAACL: Anoop Sarkar (Canda), Michael Strube (Germany) EMNLP-IJCNLP: Marine Carpuat (USA), Tim Baldwin (Australia)
|SoftConf
|45
|Tutorial chairs
|2-3
|ACL: 9 NAACL: 6 EMNLP: 7
|Presenters (1-5) Description (1-5) Interest Level (1-5) Preparedness (1-5)
|[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials Call] [http://www.acl2019.org/EN/tutorials.xhtml ACL tutorials] [https://naacl2019.org/program/tutorials/ NAACL tutorials]
|-
|2018
|ACL NAACL EMNLP COLING
|ACL: Yoav Artzi (USA), Jacob Eisenstein (USA) COLING: Pascale Fung (Hong Kong), Donia Scott (UK), Marilyn Walker (USA) EMNLP: Mausam (India), Lu Wang (USA) NAACL: Mohit Bansal (USA), Rebecca Passonneau (USA)
|SoftConf
|49
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6 COLING: 6
|Quality of plan Diversity of topics Qualifications of presenter likely audience interest
|[https://acl2018.org/2017/09/11/joint-call-for-tutorials/ Call] 
[https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs Report] 
[https://acl2018.org/tutorials/ ACL tutorials] 
[https://emnlp2018.org/program/tutorials/ EMNLP tutorials] 
[https://coling2018.org/tutorial-list/ COLING tutorials]
|-
|2017
|ACL EACL EMNLP
|ACL: Maja Popović (Germany), Jordan Boyd-Graber (USA) EACL: Alex Klementiev (Germany), Lucia Specia (UK) EMNLP: Nathan Schneider (USA), Alexandra Birch (UK)
|SoftConf
|26
|Tutorial chairs
|3
|ACL: 6 EACL: 6 EMNLP: 7
|Clarity (description and outline) Topic
|[https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs Report] [http://acl2017.org/tutorials/ ACL tutorials] [http://eacl2017.org/index.php/tutorials EACL tutorials]
|-
|2016
|ACL NAACL EMNLP
|ACL: Alexandra Birch (UK), Willem Zuidema (Netherlands) NAACL: Alexander Rush and Bishan Yang EMNLP: Rebecca Hwa and Mohit Bansal
|SoftConf
|32
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6
|Relevance to ACL community (15) Quality of instructor (15) Outline and depth/breadth of the proposal (15) 
Potential attendance Newly emerging area not previously covered in an ACL-related tutorial (yes or no) Introduction into related fields (yes or no) 
Overall score (15) 1st preference for venue as indicated in proposal comments
|[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 Call] [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs Report] [http://naacl.org/naacl-hlt-2016/tutorials.html NAACL tutorials] [https://www.aclweb.org/mirror/emnlp2016/tutorials.html EMNLP tutorials]
|}

== Past calls for tutorials ==
*[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 ACL/AACL-IJCNLP/EMNLP/COLING 2020 joint call for tutorials]
*[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials NAACL/ACL/EMNLP 2019 joint call for tutorials]
*[https://www.aclweb.org/portal/content/final-joint-call-tutorial-proposals-aclcolingemnlpnaacl-2018 ACL/COLING/EMNLP/NAACL 2018 joint call for tutorials]
*[https://www.aclweb.org/portal/content/eacl-2017-call-tutorial-proposals EACL 2017 call for tutorials]
*[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 NAACL/ACL/EMNLP 2016 joint call for tutorials]
*[https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-naacl-2015-acl-2015-emnlp-2015-0 NAACL/ACL/EMNLP 2015 joint call for tutorials]

== Reports from past tutorial chairs ==
*[[2020Q1_Reports:_ACL_2020#Tutorial_Chairs|ACL 2020 report]] (by Agata Savary and Yue Zhang)
*[[2019Q1_Reports:_ACL_2019#Tutorial_Co-Chairs|ACL 2019 report]] (by Preslav Nakov and Alexis Palmer)
*[[2018Q3 Reports: Tutorial Chairs|ACL 2018 report]] (by Yoav Artzi and Jacob Eisenstein)
*[[2017Q3 Reports: Tutorial Chairs|ACL 2017 report]] (by Maja Popović and Jordan Boyd-Graber)
*[[2016Q3 Reports: Tutorial Chairs|ACL 2016 report]] (by Alexandra Birch and Willem Zuidema)

== Lessons learned by past tutorial chairs ==
* [Agata Savary, 2020] If the conference is virtual:
** Remember that the modalities of tutorials largely vary from those of main conference papers. Tutorials slots are long (3.5-hours each), have a larger audience and may require a high degree of interaction, depending on the topics and the teachers’ preferences. Therefore, the preparation of tutorials should be considered on an individual basis (see also this post).
** Announce the individual modalities with the attendees early enough (for instance, if watching pre-recordings in advance is necessary for some tutorials because their live sessions only included question answering)
** If services of SlidesLive or alike are used, it is crucial that the teachers understand the technical modalities (what is the added value from these services as compared to Zoom rooms only, how are the live sessions set up based on pre-recorded material and live interaction, how are they technically coordinated, etc.). If these technical details are unclear, some teachers may be reluctant to cooperate.
* [Agata Savary, 2020] Use a common SoftConf space for joint tutorial submissions for all coordinated conference. Once the proposals have been selected and assigned to conferences, use one track per conference in the same SoftConf space.
* [Agata Savary, 2020] Publish the list of recommended readings on the tutorial page
* [Agata Savary, 2020] A good way to proceed while selecting tutorials for several venues is to have a common online meeting, in which
** Tutorials are clustered per themes (to distribute similar tutorials over different conferences)
** The chairs for each venue select their best choices in a round robin manner.
** Generate csv files with reviews and scores: Manager -> Other tools -> Spreadsheet maker -> Reports
* [Agata Savary, 2020] While setting up a review form on SoftConf, include:
** All evaluation criteria, including diversity, in the review form
** Cutting edge vs. introductory category
** Duration (3 or 6 hours)
* [Agata Savary, 2020] Currently, the generic conference schedule assumes only 2 months between the call for tutorials and notification to proposers. If a solid 1-month review procedure is to take place, this means, that proposers only have 1 month between the call and the submission deadline. It would be good to shift the call to [T-12].
* [Agata Savary, 2020] For many past tutorials, no slides are available online. It is, therefore, important to promote Open Access of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation slides on the ACL Anthology. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]. As a tutorial chair, make sure that
** initial versions of tutorial slides are submitted to the proceedings together with the tutorial abstracts
** tutorial authors are encouraged to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]
* [Marine Capuat, 2019] Continue soliciting both cutting edge and introductory tutorials
* [Marine Capuat, 2019] Ask proposers to express their preferred location/conference in one place (i.e. either in proposal pdf, or in softconf form, not both) to avoid inconsistencies
* [ACL 2018 chairs] Issues with proceedings edition, when one joint SoftConf space is used for many conferences (see the [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs report])
* [ACL 2016 chairs] Avoid confusion about the submission procedures for the final tutorial descriptions, those were submitted by email in many different formats

Author: Agata Savary, July 2019

Updates are welcome

[[Category:Conference Handbook]]

Tutorial chair handbook

2020-07-15T17:18:50Z

AgataSavary: /* Lessons learned by past tutorial chairs */

== Duties ==
Tutorial chairs are nominated by the general chair of the main conference. Their duties are described in [[Tutorial chair duties|here]].

== Timeline ==
The timeline for the organisation of tutorials should follow the general [[conference planning schedule]]. Below, we complete this schedule with more detailed tasks, as defined for the ACL 2020 tutorials. The dates from the [[conference planning schedule]] are highlighted in '''bold'''. 
In recent years, calls and selection of tutorial proposals have been coordinated among some major CL conferences (usually a subset of ACL, NAACL, EMNLP, AACL and COLING). Note that if your conference is the first in the year to come, you will need to be the major driving force in the initial steps described below. 
Tutorial proposals are usually reviewed by the tutorial chairs of the coordinating conferences, but it may be a good idea to also rectruit a small group of external reviewers. They should have a good experience in reviewing and a large understanding of the recent progress in the CL domain.

* [T-12]
** Learn from the general chair which conferences will have their tutorials coordinated with yours.
** Establish contact with the tutorial chairs of these other conferences, and discuss
*** Recruitment of external reviewers
*** Proposal evaluation criteria
*** Soliciting tutorial proposals from particular potential lecturers on particular topics
*** Organizing a public poll about interest in the submitted tutorial
** Discuss general strategies of the main conference with the general chair and the program chairs
***promotion of some topics
***mailing lists and conference alert portals to be used for all announcements
** Set up a SoftConf space for proposal submission
*** Solicit the creation of a SoftConf space from [mailto:support@softconf.com support] (currently Rich Gerber). Note that submissions will be joint for several conferences but the proceedings will be edited for each conference separately. Therefore, a new SoftConf space will have to be created later for the selected proposals of each conference.
*** If you plan to have external tutorial reviewers, invite them via SoftConf.
*** Set up a submission page.
** Set up a joint email for tutorial chairs.
** Confirm the number (usually 6-8) and length (usually 3 hours) of tutorial slots with the ACL Business Manager (currently Priscilla Rasmussen).
** Draft the joint call for tutorials (based on [[#Past calls for tutorials|past calls]]).

* '''[T-11]''' ''(in the future, it would be good to extend the time slot between the call and the deadline to 2 months at least, especially if July/August is included)''
** Send out the call for tutorials.
** For posting the call on the ACL website, contact the ACL Business Manager (currently Priscilla Rasmussen).
** If you planned to encourage submission of proposals from particular potential lecturers, contact them now.
** Set up a SoftConf page for proposal reviews.

* [T-10]
** Deadline for tutorial proposal submission.
** If you decided to run a public poll about interest in the submitted tutorial, set it up and send out a call for expression of interest.
** Assign proposals to reviewers, send out invitations to review (via SoftConf).

* '''[T-9]'''
** Select tutorials jointly with other conference chairs.
** Notify the authors (via SoftConf).
** Liaise with local chair for space needs and scheduling.
** Send instructions to tutorial teachers about printed material, agenda/time management, payments, fees, etc.
** Liaise the conference website chair for updates about tutorials and authors' guidelines.
** Liaise with the ACL Office for the registration form.

* '''[T-4]'''
** Collect short descriptions for proceedings from tutorial teachers.
** Provide them to the webmaster and the general and program chairs .

* '''[T-3]'''
** Confirm the tutorials with authors.
** Authors submit the final tutorial descriptions for the proceedings.
** Arrange for printing of tutorial materials.

* [T-2]
** Arrange with the [mailto:support@softconf.com SoftConf support] the creation of a separate SoftConf space for the selected tutorials of your conference only. Indicate which tutorial submissions should be copied there.
** Set up the final submission form in the new SoftConf space. Include the submissions of slides, which will be published on the ACL Anthology. Include signing the CC-BY-4.0 license agreement.

* '''[T-1]'''
** Collect and check final tutorial materials.
** Create the tutorial proceedings, including slides.

* '''[T-0]''' Conference and tutorials

* [T+1]
** Invite the lecturers to update the slides at the ACL Anthology via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form] (under CC-BY-4.0).
** Update this tutorial page and the list of [[Past tutorials|past tutorials]] with the data from your conference. For access to the ACL [[Conference Handbook]], contact the ACL Secretary (currently Shiqi Zhou).

== Publication of tutorials ==
In tutorials organized until 2019, there was no homogeneous policy as to the publication of tutorial material. Data (abstracts, slides, code, etc.) from selected tutorials were usually scattered in 2 or 3 places:
* conference website
* ACL Anthology
* private repositories of the authors
The sustainability of these varies from venue to venue. The main websites of some conferences are not maintained, therefore ACL collects [https://www.aclweb.org/portal/node/2481 mirrors] of some of them (but not all).
The tutorial proceedings, published in the ACL Anthology, until recently included only abstracts but no tutorial slides.

As a result, for many [[past tutorials]], no slides are available online, so the community cannot benefit from these assets. It is, therefore, important to promote '''Open Access''' of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation '''slides in the ACL Anthology'''. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form].

It is recommended for tutorial chairs to:
* announce the publication of the tutorial slides under the CC-BY-4.0 license (used in the ACL Anthology)
* encourage the authors to also publish, under open licenses, their code and data used in the tutorials
* include the initial versions of tutorial slides, together with abstract, in the proceedings
* encourage the tutorial authors to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]

== Past tutorials ==
The [[Past tutorials|list of recent tutorials]] contains tutorial authors' names, titles, and to their summaries in conference websites and/or in the ACL Anthology.

== Organization of past tutorials ==
For the last years, the call and selection of tutorials has been coordinated among several major CL conferences. The table below summarizes some data collected from existing reports and from direct contacts with past tutorial chairs.

{| border="1" cellspacing="0" cellpadding="5"
|'''Year'''
|'''Coordinated conferences'''
|'''Tutorial chairs'''
|'''Submission platform'''
|'''# Submissions'''
|'''Reviewers'''
|'''# Reviews per proposal'''
|'''# Selected tutorials'''
|'''Selection criteria'''
|'''Useful links'''
|-
|2020
|ACL EMNLP AACL-IJCNLP COLING
|ACL: Agata Savary (France), Yue Zhang (China) EMNLP: Benjamin Van Durme (USA), Aline Villavicencio (UK & Brazil) AACL-IJCNLP: Tim Baldwin (Australia), Fei Xia (USA) COLING: Daniel Beck (Australia), Lucia Specia (UK)
|SoftConf
|43
|Tutorial chairs + external experts
|3
|ACL: 8 EMNLP: 7 ACCL-IJCNLP: 6 COLING: 7
|Clarity and preparedness Novelty or timely character of the topic Lecturers' experience Likely audience interest Open access of the teaching material Diversity aspects (multilingualism, gender, age and country of the lecturers) Compatibility with the required venues
|[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 Call] [https://acl2020.org/program/tutorials/ ACL tutorials] [https://2020.emnlp.org/tutorials EMNLP tutorials] [http://aacl2020.org/ ACCL-IJCNLP tutorials] [https://coling2020.org/pages/tutorials COLING tutorials]
|
|-
|2019
|NAACL ACL EMNLP
|ACL: Preslav Nakov (Qatar), Alexis Palmer (USA) NAACL: Anoop Sarkar (Canda), Michael Strube (Germany) EMNLP-IJCNLP: Marine Carpuat (USA), Tim Baldwin (Australia)
|SoftConf
|45
|Tutorial chairs
|2-3
|ACL: 9 NAACL: 6 EMNLP: 7
|Presenters (1-5) Description (1-5) Interest Level (1-5) Preparedness (1-5)
|[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials Call] [http://www.acl2019.org/EN/tutorials.xhtml ACL tutorials] [https://naacl2019.org/program/tutorials/ NAACL tutorials]
|-
|2018
|ACL NAACL EMNLP COLING
|ACL: Yoav Artzi (USA), Jacob Eisenstein (USA) COLING: Pascale Fung (Hong Kong), Donia Scott (UK), Marilyn Walker (USA) EMNLP: Mausam (India), Lu Wang (USA) NAACL: Mohit Bansal (USA), Rebecca Passonneau (USA)
|SoftConf
|49
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6 COLING: 6
|Quality of plan Diversity of topics Qualifications of presenter likely audience interest
|[https://acl2018.org/2017/09/11/joint-call-for-tutorials/ Call] 
[https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs Report] 
[https://acl2018.org/tutorials/ ACL tutorials] 
[https://emnlp2018.org/program/tutorials/ EMNLP tutorials] 
[https://coling2018.org/tutorial-list/ COLING tutorials]
|-
|2017
|ACL EACL EMNLP
|ACL: Maja Popović (Germany), Jordan Boyd-Graber (USA) EACL: Alex Klementiev (Germany), Lucia Specia (UK) EMNLP: Nathan Schneider (USA), Alexandra Birch (UK)
|SoftConf
|26
|Tutorial chairs
|3
|ACL: 6 EACL: 6 EMNLP: 7
|Clarity (description and outline) Topic
|[https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs Report] [http://acl2017.org/tutorials/ ACL tutorials] [http://eacl2017.org/index.php/tutorials EACL tutorials]
|-
|2016
|ACL NAACL EMNLP
|ACL: Alexandra Birch (UK), Willem Zuidema (Netherlands) NAACL: Alexander Rush and Bishan Yang EMNLP: Rebecca Hwa and Mohit Bansal
|SoftConf
|32
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6
|Relevance to ACL community (15) Quality of instructor (15) Outline and depth/breadth of the proposal (15) 
Potential attendance Newly emerging area not previously covered in an ACL-related tutorial (yes or no) Introduction into related fields (yes or no) 
Overall score (15) 1st preference for venue as indicated in proposal comments
|[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 Call] [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs Report] [http://naacl.org/naacl-hlt-2016/tutorials.html NAACL tutorials] [https://www.aclweb.org/mirror/emnlp2016/tutorials.html EMNLP tutorials]
|}

== Past calls for tutorials ==
*[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 ACL/AACL-IJCNLP/EMNLP/COLING 2020 joint call for tutorials]
*[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials NAACL/ACL/EMNLP 2019 joint call for tutorials]
*[https://www.aclweb.org/portal/content/final-joint-call-tutorial-proposals-aclcolingemnlpnaacl-2018 ACL/COLING/EMNLP/NAACL 2018 joint call for tutorials]
*[https://www.aclweb.org/portal/content/eacl-2017-call-tutorial-proposals EACL 2017 call for tutorials]
*[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 NAACL/ACL/EMNLP 2016 joint call for tutorials]
*[https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-naacl-2015-acl-2015-emnlp-2015-0 NAACL/ACL/EMNLP 2015 joint call for tutorials]

== Reports from past tutorial chairs ==
*[[2020Q1_Reports:_ACL_2020#Tutorial_Chairs|ACL 2020 report]] (by Agata Savary and Yue Zhang)
*[[2019Q1_Reports:_ACL_2019#Tutorial_Co-Chairs|ACL 2019 report]] (by Preslav Nakov and Alexis Palmer)
*[[2018Q3 Reports: Tutorial Chairs|ACL 2018 report]] (by Yoav Artzi and Jacob Eisenstein)
*[[2017Q3 Reports: Tutorial Chairs|ACL 2017 report]] (by Maja Popović and Jordan Boyd-Graber)
*[[2016Q3 Reports: Tutorial Chairs|ACL 2016 report]] (by Alexandra Birch and Willem Zuidema)

== Lessons learned by past tutorial chairs ==
* [Agata Savary, 2020] Use a common SoftConf space for joint tutorial submissions for all coordinated conference. Once the proposals have been selected and assigned to conferences, use one track per conference in the same SoftConf space.
* [Agata Savary, 2020] Publish the list of recommended readings on the tutorial page
* [Agata Savary, 2020] A good way to proceed while selecting tutorials for several venues is to have a common online meeting, in which
** Tutorials are clustered per themes (to distribute similar tutorials over different conferences)
** The chairs for each venue select their best choices in a round robin manner.
** Generate csv files with reviews and scores: Manager -> Other tools -> Spreadsheet maker -> Reports
* [Agata Savary, 2020] While setting up a review form on SoftConf, include:
** All evaluation criteria, including diversity, in the review form
** Cutting edge vs. introductory category
** Duration (3 or 6 hours)
* [Agata Savary, 2020] Currently, the generic conference schedule assumes only 2 months between the call for tutorials and notification to proposers. If a solid 1-month review procedure is to take place, this means, that proposers only have 1 month between the call and the submission deadline. It would be good to shift the call to [T-12].
* [Agata Savary, 2020] For many past tutorials, no slides are available online. It is, therefore, important to promote Open Access of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation slides on the ACL Anthology. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]. As a tutorial chair, make sure that
** initial versions of tutorial slides are submitted to the proceedings together with the tutorial abstracts
** tutorial authors are encouraged to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]
* [Marine Capuat, 2019] Continue soliciting both cutting edge and introductory tutorials
* [Marine Capuat, 2019] Ask proposers to express their preferred location/conference in one place (i.e. either in proposal pdf, or in softconf form, not both) to avoid inconsistencies
* [ACL 2018 chairs] Issues with proceedings edition, when one joint SoftConf space is used for many conferences (see the [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs report])
* [ACL 2016 chairs] Avoid confusion about the submission procedures for the final tutorial descriptions, those were submitted by email in many different formats

Author: Agata Savary, July 2019

Updates are welcome

[[Category:Conference Handbook]]

Tutorial chair handbook

2020-07-15T17:18:03Z

AgataSavary: /* Lessons learned by past tutorial chairs */

== Duties ==
Tutorial chairs are nominated by the general chair of the main conference. Their duties are described in [[Tutorial chair duties|here]].

== Timeline ==
The timeline for the organisation of tutorials should follow the general [[conference planning schedule]]. Below, we complete this schedule with more detailed tasks, as defined for the ACL 2020 tutorials. The dates from the [[conference planning schedule]] are highlighted in '''bold'''. 
In recent years, calls and selection of tutorial proposals have been coordinated among some major CL conferences (usually a subset of ACL, NAACL, EMNLP, AACL and COLING). Note that if your conference is the first in the year to come, you will need to be the major driving force in the initial steps described below. 
Tutorial proposals are usually reviewed by the tutorial chairs of the coordinating conferences, but it may be a good idea to also rectruit a small group of external reviewers. They should have a good experience in reviewing and a large understanding of the recent progress in the CL domain.

* [T-12]
** Learn from the general chair which conferences will have their tutorials coordinated with yours.
** Establish contact with the tutorial chairs of these other conferences, and discuss
*** Recruitment of external reviewers
*** Proposal evaluation criteria
*** Soliciting tutorial proposals from particular potential lecturers on particular topics
*** Organizing a public poll about interest in the submitted tutorial
** Discuss general strategies of the main conference with the general chair and the program chairs
***promotion of some topics
***mailing lists and conference alert portals to be used for all announcements
** Set up a SoftConf space for proposal submission
*** Solicit the creation of a SoftConf space from [mailto:support@softconf.com support] (currently Rich Gerber). Note that submissions will be joint for several conferences but the proceedings will be edited for each conference separately. Therefore, a new SoftConf space will have to be created later for the selected proposals of each conference.
*** If you plan to have external tutorial reviewers, invite them via SoftConf.
*** Set up a submission page.
** Set up a joint email for tutorial chairs.
** Confirm the number (usually 6-8) and length (usually 3 hours) of tutorial slots with the ACL Business Manager (currently Priscilla Rasmussen).
** Draft the joint call for tutorials (based on [[#Past calls for tutorials|past calls]]).

* '''[T-11]''' ''(in the future, it would be good to extend the time slot between the call and the deadline to 2 months at least, especially if July/August is included)''
** Send out the call for tutorials.
** For posting the call on the ACL website, contact the ACL Business Manager (currently Priscilla Rasmussen).
** If you planned to encourage submission of proposals from particular potential lecturers, contact them now.
** Set up a SoftConf page for proposal reviews.

* [T-10]
** Deadline for tutorial proposal submission.
** If you decided to run a public poll about interest in the submitted tutorial, set it up and send out a call for expression of interest.
** Assign proposals to reviewers, send out invitations to review (via SoftConf).

* '''[T-9]'''
** Select tutorials jointly with other conference chairs.
** Notify the authors (via SoftConf).
** Liaise with local chair for space needs and scheduling.
** Send instructions to tutorial teachers about printed material, agenda/time management, payments, fees, etc.
** Liaise the conference website chair for updates about tutorials and authors' guidelines.
** Liaise with the ACL Office for the registration form.

* '''[T-4]'''
** Collect short descriptions for proceedings from tutorial teachers.
** Provide them to the webmaster and the general and program chairs .

* '''[T-3]'''
** Confirm the tutorials with authors.
** Authors submit the final tutorial descriptions for the proceedings.
** Arrange for printing of tutorial materials.

* [T-2]
** Arrange with the [mailto:support@softconf.com SoftConf support] the creation of a separate SoftConf space for the selected tutorials of your conference only. Indicate which tutorial submissions should be copied there.
** Set up the final submission form in the new SoftConf space. Include the submissions of slides, which will be published on the ACL Anthology. Include signing the CC-BY-4.0 license agreement.

* '''[T-1]'''
** Collect and check final tutorial materials.
** Create the tutorial proceedings, including slides.

* '''[T-0]''' Conference and tutorials

* [T+1]
** Invite the lecturers to update the slides at the ACL Anthology via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form] (under CC-BY-4.0).
** Update this tutorial page and the list of [[Past tutorials|past tutorials]] with the data from your conference. For access to the ACL [[Conference Handbook]], contact the ACL Secretary (currently Shiqi Zhou).

== Publication of tutorials ==
In tutorials organized until 2019, there was no homogeneous policy as to the publication of tutorial material. Data (abstracts, slides, code, etc.) from selected tutorials were usually scattered in 2 or 3 places:
* conference website
* ACL Anthology
* private repositories of the authors
The sustainability of these varies from venue to venue. The main websites of some conferences are not maintained, therefore ACL collects [https://www.aclweb.org/portal/node/2481 mirrors] of some of them (but not all).
The tutorial proceedings, published in the ACL Anthology, until recently included only abstracts but no tutorial slides.

As a result, for many [[past tutorials]], no slides are available online, so the community cannot benefit from these assets. It is, therefore, important to promote '''Open Access''' of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation '''slides in the ACL Anthology'''. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form].

It is recommended for tutorial chairs to:
* announce the publication of the tutorial slides under the CC-BY-4.0 license (used in the ACL Anthology)
* encourage the authors to also publish, under open licenses, their code and data used in the tutorials
* include the initial versions of tutorial slides, together with abstract, in the proceedings
* encourage the tutorial authors to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]

== Past tutorials ==
The [[Past tutorials|list of recent tutorials]] contains tutorial authors' names, titles, and to their summaries in conference websites and/or in the ACL Anthology.

== Organization of past tutorials ==
For the last years, the call and selection of tutorials has been coordinated among several major CL conferences. The table below summarizes some data collected from existing reports and from direct contacts with past tutorial chairs.

{| border="1" cellspacing="0" cellpadding="5"
|'''Year'''
|'''Coordinated conferences'''
|'''Tutorial chairs'''
|'''Submission platform'''
|'''# Submissions'''
|'''Reviewers'''
|'''# Reviews per proposal'''
|'''# Selected tutorials'''
|'''Selection criteria'''
|'''Useful links'''
|-
|2020
|ACL EMNLP AACL-IJCNLP COLING
|ACL: Agata Savary (France), Yue Zhang (China) EMNLP: Benjamin Van Durme (USA), Aline Villavicencio (UK & Brazil) AACL-IJCNLP: Tim Baldwin (Australia), Fei Xia (USA) COLING: Daniel Beck (Australia), Lucia Specia (UK)
|SoftConf
|43
|Tutorial chairs + external experts
|3
|ACL: 8 EMNLP: 7 ACCL-IJCNLP: 6 COLING: 7
|Clarity and preparedness Novelty or timely character of the topic Lecturers' experience Likely audience interest Open access of the teaching material Diversity aspects (multilingualism, gender, age and country of the lecturers) Compatibility with the required venues
|[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 Call] [https://acl2020.org/program/tutorials/ ACL tutorials] [https://2020.emnlp.org/tutorials EMNLP tutorials] [http://aacl2020.org/ ACCL-IJCNLP tutorials] [https://coling2020.org/pages/tutorials COLING tutorials]
|
|-
|2019
|NAACL ACL EMNLP
|ACL: Preslav Nakov (Qatar), Alexis Palmer (USA) NAACL: Anoop Sarkar (Canda), Michael Strube (Germany) EMNLP-IJCNLP: Marine Carpuat (USA), Tim Baldwin (Australia)
|SoftConf
|45
|Tutorial chairs
|2-3
|ACL: 9 NAACL: 6 EMNLP: 7
|Presenters (1-5) Description (1-5) Interest Level (1-5) Preparedness (1-5)
|[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials Call] [http://www.acl2019.org/EN/tutorials.xhtml ACL tutorials] [https://naacl2019.org/program/tutorials/ NAACL tutorials]
|-
|2018
|ACL NAACL EMNLP COLING
|ACL: Yoav Artzi (USA), Jacob Eisenstein (USA) COLING: Pascale Fung (Hong Kong), Donia Scott (UK), Marilyn Walker (USA) EMNLP: Mausam (India), Lu Wang (USA) NAACL: Mohit Bansal (USA), Rebecca Passonneau (USA)
|SoftConf
|49
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6 COLING: 6
|Quality of plan Diversity of topics Qualifications of presenter likely audience interest
|[https://acl2018.org/2017/09/11/joint-call-for-tutorials/ Call] 
[https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs Report] 
[https://acl2018.org/tutorials/ ACL tutorials] 
[https://emnlp2018.org/program/tutorials/ EMNLP tutorials] 
[https://coling2018.org/tutorial-list/ COLING tutorials]
|-
|2017
|ACL EACL EMNLP
|ACL: Maja Popović (Germany), Jordan Boyd-Graber (USA) EACL: Alex Klementiev (Germany), Lucia Specia (UK) EMNLP: Nathan Schneider (USA), Alexandra Birch (UK)
|SoftConf
|26
|Tutorial chairs
|3
|ACL: 6 EACL: 6 EMNLP: 7
|Clarity (description and outline) Topic
|[https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs Report] [http://acl2017.org/tutorials/ ACL tutorials] [http://eacl2017.org/index.php/tutorials EACL tutorials]
|-
|2016
|ACL NAACL EMNLP
|ACL: Alexandra Birch (UK), Willem Zuidema (Netherlands) NAACL: Alexander Rush and Bishan Yang EMNLP: Rebecca Hwa and Mohit Bansal
|SoftConf
|32
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6
|Relevance to ACL community (15) Quality of instructor (15) Outline and depth/breadth of the proposal (15) 
Potential attendance Newly emerging area not previously covered in an ACL-related tutorial (yes or no) Introduction into related fields (yes or no) 
Overall score (15) 1st preference for venue as indicated in proposal comments
|[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 Call] [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs Report] [http://naacl.org/naacl-hlt-2016/tutorials.html NAACL tutorials] [https://www.aclweb.org/mirror/emnlp2016/tutorials.html EMNLP tutorials]
|}

== Past calls for tutorials ==
*[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 ACL/AACL-IJCNLP/EMNLP/COLING 2020 joint call for tutorials]
*[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials NAACL/ACL/EMNLP 2019 joint call for tutorials]
*[https://www.aclweb.org/portal/content/final-joint-call-tutorial-proposals-aclcolingemnlpnaacl-2018 ACL/COLING/EMNLP/NAACL 2018 joint call for tutorials]
*[https://www.aclweb.org/portal/content/eacl-2017-call-tutorial-proposals EACL 2017 call for tutorials]
*[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 NAACL/ACL/EMNLP 2016 joint call for tutorials]
*[https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-naacl-2015-acl-2015-emnlp-2015-0 NAACL/ACL/EMNLP 2015 joint call for tutorials]

== Reports from past tutorial chairs ==
*[[2020Q1_Reports:_ACL_2020#Tutorial_Chairs|ACL 2020 report]] (by Agata Savary and Yue Zhang)
*[[2019Q1_Reports:_ACL_2019#Tutorial_Co-Chairs|ACL 2019 report]] (by Preslav Nakov and Alexis Palmer)
*[[2018Q3 Reports: Tutorial Chairs|ACL 2018 report]] (by Yoav Artzi and Jacob Eisenstein)
*[[2017Q3 Reports: Tutorial Chairs|ACL 2017 report]] (by Maja Popović and Jordan Boyd-Graber)
*[[2016Q3 Reports: Tutorial Chairs|ACL 2016 report]] (by Alexandra Birch and Willem Zuidema)

== Lessons learned by past tutorial chairs ==
* [Agata Savary, 2020] Use a common SoftConf space for joint tutorial submissions for all coordinated conference. Once the proposals have been selected and assigned to conferences, use one track per conference in the same SoftConf space.
* [Agata Savary, 2020] Publish the list of recommended readings on the tutorial page
* [Agata Savary, 2020] A good way to proceed while selecting tutorials for several venues is to have a common online meeting, in which
* Tutorials are clustered per themes (to distribute similar tutorials over different conferences)
* The chairs for each venue select their best choices in a round robin manner.
* Generate csv files with reviews and scores: Manager -> Other tools -> Spreadsheet maker -> Reports
* [Agata Savary, 2020] While setting up a review form on SoftConf, include:
* All evaluation criteria, including diversity, in the review form
* Cutting edge vs. introductory category
* Duration (3 or 6 hours)
* [Agata Savary, 2020] Currently, the generic conference schedule assumes only 2 months between the call for tutorials and notification to proposers. If a solid 1-month review procedure is to take place, this means, that proposers only have 1 month between the call and the submission deadline. It would be good to shift the call to [T-12].
* [Agata Savary, 2020] For many past tutorials, no slides are available online. It is, therefore, important to promote Open Access of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation slides on the ACL Anthology. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]. As a tutorial chair, make sure that
** initial versions of tutorial slides are submitted to the proceedings together with the tutorial abstracts
** tutorial authors are encouraged to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]
* [Marine Capuat, 2019] Continue soliciting both cutting edge and introductory tutorials
* [Marine Capuat, 2019] Ask proposers to express their preferred location/conference in one place (i.e. either in proposal pdf, or in softconf form, not both) to avoid inconsistencies
* [ACL 2018 chairs] Issues with proceedings edition, when one joint SoftConf space is used for many conferences (see the [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs report])
* [ACL 2016 chairs] Avoid confusion about the submission procedures for the final tutorial descriptions, those were submitted by email in many different formats

Author: Agata Savary, July 2019

Updates are welcome

[[Category:Conference Handbook]]

Tutorial chair handbook

2020-07-15T17:12:50Z

AgataSavary: /* Past calls for tutorials */

== Duties ==
Tutorial chairs are nominated by the general chair of the main conference. Their duties are described in [[Tutorial chair duties|here]].

== Timeline ==
The timeline for the organisation of tutorials should follow the general [[conference planning schedule]]. Below, we complete this schedule with more detailed tasks, as defined for the ACL 2020 tutorials. The dates from the [[conference planning schedule]] are highlighted in '''bold'''. 
In recent years, calls and selection of tutorial proposals have been coordinated among some major CL conferences (usually a subset of ACL, NAACL, EMNLP, AACL and COLING). Note that if your conference is the first in the year to come, you will need to be the major driving force in the initial steps described below. 
Tutorial proposals are usually reviewed by the tutorial chairs of the coordinating conferences, but it may be a good idea to also rectruit a small group of external reviewers. They should have a good experience in reviewing and a large understanding of the recent progress in the CL domain.

* [T-12]
** Learn from the general chair which conferences will have their tutorials coordinated with yours.
** Establish contact with the tutorial chairs of these other conferences, and discuss
*** Recruitment of external reviewers
*** Proposal evaluation criteria
*** Soliciting tutorial proposals from particular potential lecturers on particular topics
*** Organizing a public poll about interest in the submitted tutorial
** Discuss general strategies of the main conference with the general chair and the program chairs
***promotion of some topics
***mailing lists and conference alert portals to be used for all announcements
** Set up a SoftConf space for proposal submission
*** Solicit the creation of a SoftConf space from [mailto:support@softconf.com support] (currently Rich Gerber). Note that submissions will be joint for several conferences but the proceedings will be edited for each conference separately. Therefore, a new SoftConf space will have to be created later for the selected proposals of each conference.
*** If you plan to have external tutorial reviewers, invite them via SoftConf.
*** Set up a submission page.
** Set up a joint email for tutorial chairs.
** Confirm the number (usually 6-8) and length (usually 3 hours) of tutorial slots with the ACL Business Manager (currently Priscilla Rasmussen).
** Draft the joint call for tutorials (based on [[#Past calls for tutorials|past calls]]).

* '''[T-11]''' ''(in the future, it would be good to extend the time slot between the call and the deadline to 2 months at least, especially if July/August is included)''
** Send out the call for tutorials.
** For posting the call on the ACL website, contact the ACL Business Manager (currently Priscilla Rasmussen).
** If you planned to encourage submission of proposals from particular potential lecturers, contact them now.
** Set up a SoftConf page for proposal reviews.

* [T-10]
** Deadline for tutorial proposal submission.
** If you decided to run a public poll about interest in the submitted tutorial, set it up and send out a call for expression of interest.
** Assign proposals to reviewers, send out invitations to review (via SoftConf).

* '''[T-9]'''
** Select tutorials jointly with other conference chairs.
** Notify the authors (via SoftConf).
** Liaise with local chair for space needs and scheduling.
** Send instructions to tutorial teachers about printed material, agenda/time management, payments, fees, etc.
** Liaise the conference website chair for updates about tutorials and authors' guidelines.
** Liaise with the ACL Office for the registration form.

* '''[T-4]'''
** Collect short descriptions for proceedings from tutorial teachers.
** Provide them to the webmaster and the general and program chairs .

* '''[T-3]'''
** Confirm the tutorials with authors.
** Authors submit the final tutorial descriptions for the proceedings.
** Arrange for printing of tutorial materials.

* [T-2]
** Arrange with the [mailto:support@softconf.com SoftConf support] the creation of a separate SoftConf space for the selected tutorials of your conference only. Indicate which tutorial submissions should be copied there.
** Set up the final submission form in the new SoftConf space. Include the submissions of slides, which will be published on the ACL Anthology. Include signing the CC-BY-4.0 license agreement.

* '''[T-1]'''
** Collect and check final tutorial materials.
** Create the tutorial proceedings, including slides.

* '''[T-0]''' Conference and tutorials

* [T+1]
** Invite the lecturers to update the slides at the ACL Anthology via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form] (under CC-BY-4.0).
** Update this tutorial page and the list of [[Past tutorials|past tutorials]] with the data from your conference. For access to the ACL [[Conference Handbook]], contact the ACL Secretary (currently Shiqi Zhou).

== Publication of tutorials ==
In tutorials organized until 2019, there was no homogeneous policy as to the publication of tutorial material. Data (abstracts, slides, code, etc.) from selected tutorials were usually scattered in 2 or 3 places:
* conference website
* ACL Anthology
* private repositories of the authors
The sustainability of these varies from venue to venue. The main websites of some conferences are not maintained, therefore ACL collects [https://www.aclweb.org/portal/node/2481 mirrors] of some of them (but not all).
The tutorial proceedings, published in the ACL Anthology, until recently included only abstracts but no tutorial slides.

As a result, for many [[past tutorials]], no slides are available online, so the community cannot benefit from these assets. It is, therefore, important to promote '''Open Access''' of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation '''slides in the ACL Anthology'''. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form].

It is recommended for tutorial chairs to:
* announce the publication of the tutorial slides under the CC-BY-4.0 license (used in the ACL Anthology)
* encourage the authors to also publish, under open licenses, their code and data used in the tutorials
* include the initial versions of tutorial slides, together with abstract, in the proceedings
* encourage the tutorial authors to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]

== Past tutorials ==
The [[Past tutorials|list of recent tutorials]] contains tutorial authors' names, titles, and to their summaries in conference websites and/or in the ACL Anthology.

== Organization of past tutorials ==
For the last years, the call and selection of tutorials has been coordinated among several major CL conferences. The table below summarizes some data collected from existing reports and from direct contacts with past tutorial chairs.

{| border="1" cellspacing="0" cellpadding="5"
|'''Year'''
|'''Coordinated conferences'''
|'''Tutorial chairs'''
|'''Submission platform'''
|'''# Submissions'''
|'''Reviewers'''
|'''# Reviews per proposal'''
|'''# Selected tutorials'''
|'''Selection criteria'''
|'''Useful links'''
|-
|2020
|ACL EMNLP AACL-IJCNLP COLING
|ACL: Agata Savary (France), Yue Zhang (China) EMNLP: Benjamin Van Durme (USA), Aline Villavicencio (UK & Brazil) AACL-IJCNLP: Tim Baldwin (Australia), Fei Xia (USA) COLING: Daniel Beck (Australia), Lucia Specia (UK)
|SoftConf
|43
|Tutorial chairs + external experts
|3
|ACL: 8 EMNLP: 7 ACCL-IJCNLP: 6 COLING: 7
|Clarity and preparedness Novelty or timely character of the topic Lecturers' experience Likely audience interest Open access of the teaching material Diversity aspects (multilingualism, gender, age and country of the lecturers) Compatibility with the required venues
|[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 Call] [https://acl2020.org/program/tutorials/ ACL tutorials] [https://2020.emnlp.org/tutorials EMNLP tutorials] [http://aacl2020.org/ ACCL-IJCNLP tutorials] [https://coling2020.org/pages/tutorials COLING tutorials]
|
|-
|2019
|NAACL ACL EMNLP
|ACL: Preslav Nakov (Qatar), Alexis Palmer (USA) NAACL: Anoop Sarkar (Canda), Michael Strube (Germany) EMNLP-IJCNLP: Marine Carpuat (USA), Tim Baldwin (Australia)
|SoftConf
|45
|Tutorial chairs
|2-3
|ACL: 9 NAACL: 6 EMNLP: 7
|Presenters (1-5) Description (1-5) Interest Level (1-5) Preparedness (1-5)
|[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials Call] [http://www.acl2019.org/EN/tutorials.xhtml ACL tutorials] [https://naacl2019.org/program/tutorials/ NAACL tutorials]
|-
|2018
|ACL NAACL EMNLP COLING
|ACL: Yoav Artzi (USA), Jacob Eisenstein (USA) COLING: Pascale Fung (Hong Kong), Donia Scott (UK), Marilyn Walker (USA) EMNLP: Mausam (India), Lu Wang (USA) NAACL: Mohit Bansal (USA), Rebecca Passonneau (USA)
|SoftConf
|49
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6 COLING: 6
|Quality of plan Diversity of topics Qualifications of presenter likely audience interest
|[https://acl2018.org/2017/09/11/joint-call-for-tutorials/ Call] 
[https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs Report] 
[https://acl2018.org/tutorials/ ACL tutorials] 
[https://emnlp2018.org/program/tutorials/ EMNLP tutorials] 
[https://coling2018.org/tutorial-list/ COLING tutorials]
|-
|2017
|ACL EACL EMNLP
|ACL: Maja Popović (Germany), Jordan Boyd-Graber (USA) EACL: Alex Klementiev (Germany), Lucia Specia (UK) EMNLP: Nathan Schneider (USA), Alexandra Birch (UK)
|SoftConf
|26
|Tutorial chairs
|3
|ACL: 6 EACL: 6 EMNLP: 7
|Clarity (description and outline) Topic
|[https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs Report] [http://acl2017.org/tutorials/ ACL tutorials] [http://eacl2017.org/index.php/tutorials EACL tutorials]
|-
|2016
|ACL NAACL EMNLP
|ACL: Alexandra Birch (UK), Willem Zuidema (Netherlands) NAACL: Alexander Rush and Bishan Yang EMNLP: Rebecca Hwa and Mohit Bansal
|SoftConf
|32
|Tutorial chairs
|2
|ACL: 8 NAACL: 6 EMNLP: 6
|Relevance to ACL community (15) Quality of instructor (15) Outline and depth/breadth of the proposal (15) 
Potential attendance Newly emerging area not previously covered in an ACL-related tutorial (yes or no) Introduction into related fields (yes or no) 
Overall score (15) 1st preference for venue as indicated in proposal comments
|[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 Call] [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs Report] [http://naacl.org/naacl-hlt-2016/tutorials.html NAACL tutorials] [https://www.aclweb.org/mirror/emnlp2016/tutorials.html EMNLP tutorials]
|}

== Past calls for tutorials ==
*[https://www.aclweb.org/portal/content/second-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020-0 ACL/AACL-IJCNLP/EMNLP/COLING 2020 joint call for tutorials]
*[https://www.aclweb.org/portal/content/naaclaclemnlp-tutorials NAACL/ACL/EMNLP 2019 joint call for tutorials]
*[https://www.aclweb.org/portal/content/final-joint-call-tutorial-proposals-aclcolingemnlpnaacl-2018 ACL/COLING/EMNLP/NAACL 2018 joint call for tutorials]
*[https://www.aclweb.org/portal/content/eacl-2017-call-tutorial-proposals EACL 2017 call for tutorials]
*[https://www.aclweb.org/portal/content/last-joint-call-tutorial-proposals-naaclaclemnlp-2016 NAACL/ACL/EMNLP 2016 joint call for tutorials]
*[https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-naacl-2015-acl-2015-emnlp-2015-0 NAACL/ACL/EMNLP 2015 joint call for tutorials]

== Reports from past tutorial chairs ==
*[[2020Q1_Reports:_ACL_2020#Tutorial_Chairs|ACL 2020 report]] (by Agata Savary and Yue Zhang)
*[[2019Q1_Reports:_ACL_2019#Tutorial_Co-Chairs|ACL 2019 report]] (by Preslav Nakov and Alexis Palmer)
*[[2018Q3 Reports: Tutorial Chairs|ACL 2018 report]] (by Yoav Artzi and Jacob Eisenstein)
*[[2017Q3 Reports: Tutorial Chairs|ACL 2017 report]] (by Maja Popović and Jordan Boyd-Graber)
*[[2016Q3 Reports: Tutorial Chairs|ACL 2016 report]] (by Alexandra Birch and Willem Zuidema)

== Lessons learned by past tutorial chairs ==
* [Agata Savary, 2020] Currently, the generic conference schedule assumes only 2 months between the call for tutorials and notification to proposers. If a solid 1-month review procedure is to take place, this means, that proposers only have 1 month between the call and the submission deadline. It would be good to shift the call to [T-12].
* [Agata Savary, 2020] For many past tutorials, no slides are available online. It is, therefore, important to promote Open Access of tutorial material (slides, codes, etc.), ideally via a centralized platform. Since May 2019, it is possible to publish presentation slides on the ACL Anthology. Past tutorial (and paper) authors can submit and update slides via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]. As a tutorial chair, make sure that
** initial versions of tutorial slides are submitted to the proceedings together with the tutorial abstracts
** tutorial authors are encouraged to upload the final versions of their slides after the conference via [https://forms.office.com/Pages/ResponsePage.aspx?id=DQSIkWdsW0yxEjajBLZtrQAAAAAAAAAAAAMAABqTSThUN0I2VEdZMTk4Sks3S042MVkxUEZQUVdOUS4u this form]
* [Marine Capuat, 2019] Continue soliciting both cutting edge and introductory tutorials
* [Marine Capuat, 2019] Ask proposers to express their preferred location/conference in one place (i.e. either in proposal pdf, or in softconf form, not both) to avoid inconsistencies
* [ACL 2018 chairs] Issues with proceedings edition, when one joint SoftConf space is used for many conferences (see the [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs report])
* [ACL 2016 chairs] Avoid confusion about the submission procedures for the final tutorial descriptions, those were submitted by email in many different formats

Author: Agata Savary, July 2019

Updates are welcome

[[Category:Conference Handbook]]

Past tutorials

2020-07-15T17:07:57Z

AgataSavary:

This page belongs to the [[tutorial chair handbook]]. It summarizes data on tutorials which took place at some recent ACL, EACL, NAACL, EMNLP and COLING conferences.

==2020 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Interpretability and Analysis in Neural NLP
|Yonatan Belinkov, Sebastian Gehrmann and Ellie Pavlick
|ACL 2020
|[https://acl2020.org/program/tutorials/#t1-interpretability-and-analysis-in-neural-nlp-cutting-edge-]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.1/]
|-
|Multi-modal Information Extraction from Text, Semi-structured, and Tabular Data on the Web
|Xin Luna Dong, Hannaneh Hajishirzi, Colin Lockard and Prashant Shiralkar
|ACL 2020
|[https://acl2020.org/program/tutorials/#t2-multi-modal-information-extraction-from-text-semi-structured-and-tabular-data-on-the-web-cutting-edge-]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.6/]
|-
|Reviewing Natural Language Processing Research
|Kevin Cohen, Karën Fort, Margot Mieskes and Aurélie Névéol
|ACL 2020
|[https://acl2020.org/program/tutorials/#t3-reviewing-natural-language-processing-research-introductory-]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.4/]
|-
|Stylized Text Generation: Approaches and Applications
|Lili Mou and Olga Vechtomova
|ACL 2020
|[https://acl2020.org/program/tutorials/#t4-stylized-text-generation-approaches-and-applications-cutting-edge-]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.5/]
|-
|Achieving Common Ground in Multi-modal Dialogue
|Malihe Alikhani and Matthew Stone
|ACL 2020
|[https://acl2020.org/program/tutorials/#t5-achieving-common-ground-in-multi-modal-dialogue-cutting-edge-]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.3/]
|-
|Commonsense Reasoning for Natural Language Processing
|Maarten Sap, Vered Shwartz, Antoine Bosselut, Yejin Choi and Dan Roth
|ACL 2020
|[https://acl2020.org/program/tutorials/#t6-commonsense-reasoning-for-natural-language-processing-introductory-]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.7/]
|-
|Integrating Ethics into the NLP Curriculum
|Emily M. Bender, Dirk Hovy and Alexandra Schofield
|ACL 2020
|[https://acl2020.org/program/tutorials/#t7-integrating-ethics-into-the-nlp-curriculum-introductory-]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.2/]
|-
|Open-Domain Question Answering
|Danqi Chen and Scott Wen-tau Yih
|ACL 2020
|[https://acl2020.org/program/tutorials/#t8-open-domain-question-answering-cutting-edge-]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.8/]
|-
|TO BE COMPLETED for AACL, EMNLP and COLING 2020
|
|
|
|}

==2019 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Latent Structure Models for Natural Language Processing
|André F. T. Martins, Tsvetomila Mihaylova, Nikita Nangia and Vlad Niculae
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T1]
|[https://www.aclweb.org/anthology/P19-4001/]
|-
|Graph-Based Meaning Representations: Design and Processing
|Alexander Koller, Stephan Oepen and Weiwei Sun
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T2]
|[https://www.aclweb.org/anthology/P19-4002/]
|-
|Discourse Analysis and Its Applications
|Shafiq Joty, Giuseppe Carenini, Raymond Ng and Gabriel Murray
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T3]
|[https://www.aclweb.org/anthology/P19-4003/]
|-
|Computational Analysis of Political Texts: Bridging Research Efforts Across Communities
|Goran Glavaš, Federico Nanni and Simone Paolo Ponzetto
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T4]
|[https://www.aclweb.org/anthology/P19-4004/]
|-
|Wikipedia as a Resource for Text Analysis and Retrieval
|Marius Pasca
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T5]
|[https://www.aclweb.org/anthology/P19-4005/]
|-
|Deep Bayesian Natural Language Processing
|Jen-Tzung Chien
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T6]
|[https://www.aclweb.org/anthology/P19-4006/]
|-
|Unsupervised Cross-Lingual Representation Learning
|Sebastian Ruder, Anders Søgaard and Ivan Vulić
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T7]
|[https://www.aclweb.org/anthology/P19-4007/]
|-
|Advances in Argument Mining
|Katarzyna Budzynska and Chris Reed
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T8]
|[https://www.aclweb.org/anthology/P19-4008/]
|-
|Storytelling from Structured Data and Knowledge Graphs : An NLG Perspective
|Abhijit Mishra, Anirban Laha, Karthik Sankaranarayanan, Parag Jain and Saravanan Krishnan
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T9]
|[https://www.aclweb.org/anthology/P19-4009/]
|-
|Deep Adversarial Learning for NLP
|William Yang Wang, Sameer Singh and Jiwei Li
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t1-deep-adversarial-learning-for-nlp]
|[https://aclweb.org/anthology/N19-5001/]
|-
|Deep Learning for Natural Language Inference
|Samuel Bowman and Xiaodan Zhu
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t2-deep-learning-for-natural-language-inference]
|[https://aclweb.org/anthology/N19-5002/]
|-
|Measuring and Modeling Language Change
|Jacob Eisenstein
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t3-measuring-and-modeling-language-change]
|[https://aclweb.org/anthology/N19-5003/]
|-
|Transfer Learning in Natural Language Processing
|Sebastian Ruder, Matthew Peters, Swabha Swayamdipta and Thomas Wolf
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t4-transfer-learning-in-natural-language-processing]
|[https://aclweb.org/anthology/N19-5004/]
|-
|Language Learning and Processing in People and Machines
|Aida Nematzadeh, Richard Futrell and Roger Levy
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t5-language-learning-and-processing-in-people-and-machines]
|[https://aclweb.org/anthology/N19-5005/]
|-
|Applications of Natural Language Processing in Clinical Research and Practice
|Yanshan Wang, Ahmad Tafti, Sunghwan Sohn and Rui Zhang
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t6-applications-of-natural-language-processing-in-clinical-research-and-practice]
|[https://aclweb.org/anthology/N19-5006/]
|}

==2018 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Joint models for NLP
|Yue Zhang
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Graph Formalisms for Meaning Representations
|Adam Lopez and Sorcha Gilroy
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Writing Code for NLP Research
|Matt Gardner, Mark Neumann, Joel Grus, and Nicholas Lourie
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Deep Latent Variable Models of Natural Language
|Alexander Rush, Yoon Kim, and Sam Wiseman
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Standardized Tests as benchmarks for Artificial Intelligence
|Mrinmaya Sachan, Minjoon Seo, Hannaneh Hajishirzi, and Eric Xing
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Deep Chit-Chat: Deep Learning for ChatBots
|Wei Wu and Rui Yan
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|100 Things You Always Wanted to Know about Semantics & Pragmatics But Were Afraid to Ask
|Emily M. Bender
|ACL 2018
|[https://acl2018.org/tutorials/#100-things-you-always-wante]
|[https://aclweb.org/anthology/P18-5001/]
|-
|Neural Approaches to Conversational AI
|Jianfeng Gao, Michel Galley and Lihong Li
|ACL 2018
|[https://acl2018.org/tutorials/#neural-approaches-to-conver]
|[https://aclweb.org/anthology/P18-5002/]
|-
|Variational Inference and Deep Generative Models
|Wilker Aziz and Philip Schulz
|ACL 2018
|[https://acl2018.org/tutorials/#variational-inference-and-d]
|[https://aclweb.org/anthology/P18-5003/]
|-
|Connecting Language and Vision to Actions
|Peter Anderson, Abhishek Das and Qi Wu
|ACL 2018
|[https://acl2018.org/tutorials/#connecting-language-and-vis]
|[https://aclweb.org/anthology/P18-5004/]
|-
|Beyond Multiword Expressions: Processing Idioms and Metaphors
|Valia Kordoni
|ACL 2018
|[https://acl2018.org/tutorials/#beyond-multiword-expression]
|[https://aclweb.org/anthology/P18-5005/]
|-
|Neural Semantic Parsing
|Luke Zettlemoyer, Matt Gardner, Pradeep Dasigi, Srinivasan Iyer and Alane Suhr
|ACL 2018
|[https://acl2018.org/tutorials/#neural-semantic-parsing]
|[https://aclweb.org/anthology/P18-5006/]
|-
|Deep Reinforcement Learning for NLP
|William Yang Wang, Jiwei Li and Xiaodong He
|ACL 2018
|[https://acl2018.org/tutorials/#deep-reinforcement-learning]
|[https://aclweb.org/anthology/P18-5007/]
|-
|Multi-lingual Entity Discovery and Linking
|Avirup Sil, Heng Ji, Dan Roth and Silviu-Petru Cucerzan
|ACL 2018
|[https://acl2018.org/tutorials/#multi-lingual-entity-discov]
|[https://aclweb.org/anthology/P18-5008/]
|-
|Modelling Natural Language, Programs, and their Intersection
|Graham Neubig and Miltiadis Allamanis
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6001/]
|-
|Deep Learning Approaches to Text Production
|Claire Gardent and Shashi Narayan
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6002/]
|-
|Scalable Construction and Reasoning of Massive Knowledge Bases
|Xiang Ren, Nanyun Peng and William Yang Wang
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6003/]
|-
|The interplay between lexical resources and Natural Language Processing
|Jose Camacho-Collados, Luis Espinosa Anke and Mohammad Taher Pilehvar
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6004/]
|-
|Socially Responsible NLP
|Yulia Tsvetkov, Vinodkumar Prabhakaran and Rob Voigt
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6005/]
|-
|Deep Learning for Conversational AI
|Pei-Hao Su, Nikola Mrkšić, Iñigo Casanueva, Ivan Vulić
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6006/]
|-
|NLP for Conversations: Sentiment, Summarization, and Group Dynamics
|Gabriel Murray, Giuseppe Carenini and Shafiq Joty
|COLING 2018
|[https://sites.google.com/view/nlpforconversations]
|[https://aclweb.org/anthology/C18-3001/]
|-
|Practical Parsing for Downstream Applications
|Daniel Dakota and Sandra Kübler
|COLING 2018
|[http://cl.indiana.edu/colingParsing18/]
|[https://aclweb.org/anthology/C18-3002/]
|-
|Frame Semantics across Languages: Towards a Multilingual FrameNet
|Collin Baker, Michael Ellsworth, Miriam R L Petruck and Swabha Swayamdipta
|COLING 2018
|[https://framenet.icsi.berkeley.edu/fndrupal/node/5552/]
|[https://aclweb.org/anthology/C18-3003/]
|-
|Deep Bayesian Learning and Understanding
|Jen-Tzung Chien
|COLING 2018
|[http://chien.cm.nctu.edu.tw/home/coling/]
|[https://aclweb.org/anthology/C18-3004/]
|-
|Data-Driven Text Simplification
|Sanja Štajner and Horacio Saggion
|COLING 2018
|[http://taln.upf.edu/pages/coling2018simplification/]
|[https://aclweb.org/anthology/C18-3005/]
|-
|Deep Learning for Dialogue Systems
|Yun-Nung Chen, Asli Celikyilmaz and Dilek Hakkani-Tur
|COLING 2018
|[http://deepdialogue.miulab.tw/]
|[https://aclweb.org/anthology/C18-3006/]
|}

==2017 tutorials==

EMNLP 2017 website is no longer available. There are no traces of the AMNLP 2017 tutorials, except this [https://www.facebook.com/emnlp2017/posts/well-have-7-tutorials-at-emnlp2017-concepthier-sarcasm-graphrep-srl-memaugnn-str/1860468284191872/ Facebook post].

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Universal Dependencies
|Joakim Nivre, Daniel Zeman, Filip Ginter, and Francis Tyers
|EACL 2017
|[http://universaldependencies.org/eacl17tutorial/]
|
|-
|Practical Neural Machine Translation
|Rico Sennrich and Barry Haddow
|EACL 2017
|[http://www.statmt.org/eacl2017/practical-nmt.pdf]
|
|-
|Imitation learning for structured prediction in natural language processing
|Andreas Vlachos, Gerasimos Lampouras and Sebastian Riedel
|EACL 2017
|[https://sheffieldnlp.github.io/ImitationLearningTutorialEACL2017/]
|
|-
|Word Vector Space Specialisation
|Ivan Vulić, Nikola Mrkšić, and Mohammad Taher Pilehvar
|EACL 2017
|[http://people.ds.cam.ac.uk/iv250/tutorial/wv-tutorial.pdf]
|
|-
|Integer Linear Programming formulations in Natural Language Processing
|Dan Roth and Vivek Srikumar
|EACL 2017
|[https://sheffieldnlp.github.io/ImitationLearningTutorialEACL2017/]
|
|-
|Building Multimodal Simulations for Natural Language
|James Pustejovsky and Nikhil Krishnaswamy
|EACL 2017
|[http://www.voxicon.net]
|
|-
|Natural Language Processing for Precision Medicine
|Hoifung Poon, Chris Quirk, Kristina Toutanova, and Wen-tau Yih
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5001/]
|-
|Multimodal Machine Learning
|Louis-Philippe Morency and Tadas Baltrusaitis
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5002/]
|-
|Deep Learning for Semantic Composition
|Xiaodan Zhu and Edward Grefenstette
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5003/]
|-
|Deep Learning for Dialogue Systems
|Yun-Nung Chen, Asli Celikyilmaz, and Dilek Hakkani-Tur
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5004/]
|-
|Beyond Words: Deep Learning for Multi-word Expressions and Collocations
|Valia Kordoni
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5005/]
|-
|Making Better Use of the Crowd
|Jennifer Wortman Vaughan
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5006/]
|}

==2016 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Multimodal Learning and Reasoning
|Desmond Elliott, Douwe Kiela and Angeliki Lazaridou
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index4d04.html?article_id=59]
|
|-
|NLP Approaches to Computational Argumentation
|Noam Slonim, Iryna Gurevych, Chris Reed and Benno Stein
|ACL 2016
|[http://mirror.aclweb.org/acl2016/indexfc69.html?article_id=56]
|
|-
|Computer Aided Translation
|Philipp Koehn
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index6ea2.html?article_id=62]
|
|-
|Semantic Representations of Word Senses and Concepts
|José Camacho-Collados, Ignacio Iacobacci, Roberto Navigli and Mohammad Taher Pilehvar
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index6b9c.html?article_id=58]
|
|-
|Neural Machine Translation
|Thang Luong, Kyunghyun Cho and Christopher D. Manning
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index2e51.html?article_id=55]
|
|-
|Game Theory and Natural Language: Origin, Evolution and Processing
|Rocco Tripodi and Marcello Pelillo
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index56cf.html?article_id=57]
|
|-
|Understanding Short Texts
|Zhongyuan Wang and Haixun Wang
|ACL 2016
|[http://mirror.aclweb.org/acl2016/indexedf3.html?article_id=60]
|
|-
|MetaNet: Repository, Identification System, and Applications
|Miriam R L Petruck and Ellen K Dodge
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index6a61.html?article_id=61]
|
|-
|English Resource Semantics
|Dan Flickinger, Emily M. Bender, and Woodley Packard
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t1.html]
|[https://aclweb.org/anthology/N16-4001/]
|-
|Multilingual Multimodal Language Processing Using Neural Networks
|Mitesh M Khapra and Sarath Chandar
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t2.html]
|[https://aclweb.org/anthology/N16-4002/]
|-
|Question Answering with Knowledge Base, Web and Beyond
|Scott Wen-tau Yih & Hao Ma
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t3.html]
|[https://aclweb.org/anthology/N16-4003/]
|-
|Recent Progress in Deep Learning for NLP
|Zhengdong Lu and Hang Li
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t4.html]
|[https://aclweb.org/anthology/N16-4004/]
|-
|Scalable Statistical Relational Learning for NLP
|William Yang Wang and William W. Cohen
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t5.html]
|[https://aclweb.org/anthology/N16-4005/]
|-
|Statistical Machine Translation between Related Languages
|Pushpak Bhattacharyya, Mitesh Khapra, and Anoop Kunchukuttan
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t6.html]
|[https://aclweb.org/anthology/N16-4006/]
|-
|Practical Neural Networks for NLP: From Theory to Code
|Chris Dyer, Yoav Goldberg and Graham Neubig
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#practical]
|
|-
|Advanced Markov Logic Techniques for Scalable Joint Inference in NLP
|Deepak Venugopal, Vibhav Gogate and Vincent Ng
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#advanced]
|
|-
|Lifelong Machine Learning for Natural Language Processing
|Zhiyuan Chen and Bing Liu
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#lifelong]
|
|-
|Neural Networks for Sentiment Analysis
|Yue Zhang and Duy Tin Vo
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#neural]
|
|-
|Continuous Vector Spaces for Cross-language NLP Applications
|Rafael E. Banchs
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#continuous]
|
|-
|Methods and Theories for Large-scale Structured Prediction
|Xu Sun and Yansong Feng
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#methods]
|
|-
|Compositional Distributional Models of Meaning
|Mehrnoosh Sadrzadeh and Dimitri Kartsaklis
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T1/]
|[https://aclweb.org/anthology/C16-3001/]
|-
|Chinese Textual Sentiment Analysis: Datasets, Resources and Tools
|Lun-Wei Ku and Wei-Fan Chen
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T2]
|[https://aclweb.org/anthology/C16-3002/]
|-
|Natural Language Processing for Intelligent Access to Scientific Information
|Horacio Saggion and Francesco Ronzano
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T3]
|[https://aclweb.org/anthology/C16-3003/]
|-
|Quality Estimation for Language Output Applications
|Carolina Scarton, Gustavo Henrique Paetzold, and Lucia Specia
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T4]
|[https://aclweb.org/anthology/C16-3004/]
|-
|Translationese: Between Human and Machine Translation
|Shuly Wintner
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T5/]
|[https://aclweb.org/anthology/C16-3005/]
|-
|Succinct Data Structures for NLP-at-Scale
|Matthias Petri and Trevor Cohn
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T6/]
|[https://aclweb.org/anthology/C16-3006/]
|-
|The Role of Wikipedia in Text Analysis and Retrieval
|Marius Pasca
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T7]
|[https://aclweb.org/anthology/C16-3007/]
|}

Author: Agata Savary, July 2019

Updates are welcome

Past tutorials

2020-07-15T17:07:15Z

AgataSavary:

This page belongs to the [[tutorial chair handbook]]. It summarizes data on tutorials which took place at some recent ACL, EACL, NAACL, EMNLP and COLING conferences.

==2020 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Interpretability and Analysis in Neural NLP
|Yonatan Belinkov, Sebastian Gehrmann and Ellie Pavlick
|ACL 2020
|[https://acl2020.org/program/tutorials/#t1-interpretability-and-analysis-in-neural-nlp-cutting-edge-]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.1/]
|-
|Multi-modal Information Extraction from Text, Semi-structured, and Tabular Data on the Web
|Xin Luna Dong, Hannaneh Hajishirzi, Colin Lockard and Prashant Shiralkar
|ACL 2020
|[https://acl2020.org/program/tutorials/#t2-multi-modal-information-extraction-from-text-semi-structured-and-tabular-data-on-the-web-cutting-edge-]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.6/]
|-
|Reviewing Natural Language Processing Research
|Kevin Cohen, Karën Fort, Margot Mieskes and Aurélie Névéol
|ACL 2020
|[https://acl2020.org/program/tutorials/#t3-reviewing-natural-language-processing-research-introductory-]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.4/]
|-
|Stylized Text Generation: Approaches and Applications
|Lili Mou and Olga Vechtomova
|ACL 2020
|[https://acl2020.org/program/tutorials/#t4-stylized-text-generation-approaches-and-applications-cutting-edge-]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.5/]
|-
|Achieving Common Ground in Multi-modal Dialogue
|Malihe Alikhani and Matthew Stone
|ACL 2020
|[https://acl2020.org/program/tutorials/#t5-achieving-common-ground-in-multi-modal-dialogue-cutting-edge-]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.3/]
|-
|Commonsense Reasoning for Natural Language Processing
|Maarten Sap, Vered Shwartz, Antoine Bosselut, Yejin Choi and Dan Roth
|ACL 2020
|[https://acl2020.org/program/tutorials/#t6-commonsense-reasoning-for-natural-language-processing-introductory-]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.7/]
|-
|Integrating Ethics into the NLP Curriculum
|Emily M. Bender, Dirk Hovy and Alexandra Schofield
|ACL 2020
|[https://acl2020.org/program/tutorials/#t7-integrating-ethics-into-the-nlp-curriculum-introductory-]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.2/]
|-
|Open-Domain Question Answering
|Danqi Chen and Scott Wen-tau Yih
|ACL 2020
|[https://acl2020.org/program/tutorials/#t8-open-domain-question-answering-cutting-edge-]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.8/]
|-
|TO BE COMPLETED fro AACL, EMNLP and COLING 2020
|
|
|
|}

==2019 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Latent Structure Models for Natural Language Processing
|André F. T. Martins, Tsvetomila Mihaylova, Nikita Nangia and Vlad Niculae
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T1]
|[https://www.aclweb.org/anthology/P19-4001/]
|-
|Graph-Based Meaning Representations: Design and Processing
|Alexander Koller, Stephan Oepen and Weiwei Sun
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T2]
|[https://www.aclweb.org/anthology/P19-4002/]
|-
|Discourse Analysis and Its Applications
|Shafiq Joty, Giuseppe Carenini, Raymond Ng and Gabriel Murray
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T3]
|[https://www.aclweb.org/anthology/P19-4003/]
|-
|Computational Analysis of Political Texts: Bridging Research Efforts Across Communities
|Goran Glavaš, Federico Nanni and Simone Paolo Ponzetto
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T4]
|[https://www.aclweb.org/anthology/P19-4004/]
|-
|Wikipedia as a Resource for Text Analysis and Retrieval
|Marius Pasca
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T5]
|[https://www.aclweb.org/anthology/P19-4005/]
|-
|Deep Bayesian Natural Language Processing
|Jen-Tzung Chien
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T6]
|[https://www.aclweb.org/anthology/P19-4006/]
|-
|Unsupervised Cross-Lingual Representation Learning
|Sebastian Ruder, Anders Søgaard and Ivan Vulić
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T7]
|[https://www.aclweb.org/anthology/P19-4007/]
|-
|Advances in Argument Mining
|Katarzyna Budzynska and Chris Reed
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T8]
|[https://www.aclweb.org/anthology/P19-4008/]
|-
|Storytelling from Structured Data and Knowledge Graphs : An NLG Perspective
|Abhijit Mishra, Anirban Laha, Karthik Sankaranarayanan, Parag Jain and Saravanan Krishnan
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T9]
|[https://www.aclweb.org/anthology/P19-4009/]
|-
|Deep Adversarial Learning for NLP
|William Yang Wang, Sameer Singh and Jiwei Li
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t1-deep-adversarial-learning-for-nlp]
|[https://aclweb.org/anthology/N19-5001/]
|-
|Deep Learning for Natural Language Inference
|Samuel Bowman and Xiaodan Zhu
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t2-deep-learning-for-natural-language-inference]
|[https://aclweb.org/anthology/N19-5002/]
|-
|Measuring and Modeling Language Change
|Jacob Eisenstein
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t3-measuring-and-modeling-language-change]
|[https://aclweb.org/anthology/N19-5003/]
|-
|Transfer Learning in Natural Language Processing
|Sebastian Ruder, Matthew Peters, Swabha Swayamdipta and Thomas Wolf
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t4-transfer-learning-in-natural-language-processing]
|[https://aclweb.org/anthology/N19-5004/]
|-
|Language Learning and Processing in People and Machines
|Aida Nematzadeh, Richard Futrell and Roger Levy
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t5-language-learning-and-processing-in-people-and-machines]
|[https://aclweb.org/anthology/N19-5005/]
|-
|Applications of Natural Language Processing in Clinical Research and Practice
|Yanshan Wang, Ahmad Tafti, Sunghwan Sohn and Rui Zhang
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t6-applications-of-natural-language-processing-in-clinical-research-and-practice]
|[https://aclweb.org/anthology/N19-5006/]
|}

==2018 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Joint models for NLP
|Yue Zhang
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Graph Formalisms for Meaning Representations
|Adam Lopez and Sorcha Gilroy
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Writing Code for NLP Research
|Matt Gardner, Mark Neumann, Joel Grus, and Nicholas Lourie
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Deep Latent Variable Models of Natural Language
|Alexander Rush, Yoon Kim, and Sam Wiseman
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Standardized Tests as benchmarks for Artificial Intelligence
|Mrinmaya Sachan, Minjoon Seo, Hannaneh Hajishirzi, and Eric Xing
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Deep Chit-Chat: Deep Learning for ChatBots
|Wei Wu and Rui Yan
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|100 Things You Always Wanted to Know about Semantics & Pragmatics But Were Afraid to Ask
|Emily M. Bender
|ACL 2018
|[https://acl2018.org/tutorials/#100-things-you-always-wante]
|[https://aclweb.org/anthology/P18-5001/]
|-
|Neural Approaches to Conversational AI
|Jianfeng Gao, Michel Galley and Lihong Li
|ACL 2018
|[https://acl2018.org/tutorials/#neural-approaches-to-conver]
|[https://aclweb.org/anthology/P18-5002/]
|-
|Variational Inference and Deep Generative Models
|Wilker Aziz and Philip Schulz
|ACL 2018
|[https://acl2018.org/tutorials/#variational-inference-and-d]
|[https://aclweb.org/anthology/P18-5003/]
|-
|Connecting Language and Vision to Actions
|Peter Anderson, Abhishek Das and Qi Wu
|ACL 2018
|[https://acl2018.org/tutorials/#connecting-language-and-vis]
|[https://aclweb.org/anthology/P18-5004/]
|-
|Beyond Multiword Expressions: Processing Idioms and Metaphors
|Valia Kordoni
|ACL 2018
|[https://acl2018.org/tutorials/#beyond-multiword-expression]
|[https://aclweb.org/anthology/P18-5005/]
|-
|Neural Semantic Parsing
|Luke Zettlemoyer, Matt Gardner, Pradeep Dasigi, Srinivasan Iyer and Alane Suhr
|ACL 2018
|[https://acl2018.org/tutorials/#neural-semantic-parsing]
|[https://aclweb.org/anthology/P18-5006/]
|-
|Deep Reinforcement Learning for NLP
|William Yang Wang, Jiwei Li and Xiaodong He
|ACL 2018
|[https://acl2018.org/tutorials/#deep-reinforcement-learning]
|[https://aclweb.org/anthology/P18-5007/]
|-
|Multi-lingual Entity Discovery and Linking
|Avirup Sil, Heng Ji, Dan Roth and Silviu-Petru Cucerzan
|ACL 2018
|[https://acl2018.org/tutorials/#multi-lingual-entity-discov]
|[https://aclweb.org/anthology/P18-5008/]
|-
|Modelling Natural Language, Programs, and their Intersection
|Graham Neubig and Miltiadis Allamanis
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6001/]
|-
|Deep Learning Approaches to Text Production
|Claire Gardent and Shashi Narayan
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6002/]
|-
|Scalable Construction and Reasoning of Massive Knowledge Bases
|Xiang Ren, Nanyun Peng and William Yang Wang
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6003/]
|-
|The interplay between lexical resources and Natural Language Processing
|Jose Camacho-Collados, Luis Espinosa Anke and Mohammad Taher Pilehvar
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6004/]
|-
|Socially Responsible NLP
|Yulia Tsvetkov, Vinodkumar Prabhakaran and Rob Voigt
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6005/]
|-
|Deep Learning for Conversational AI
|Pei-Hao Su, Nikola Mrkšić, Iñigo Casanueva, Ivan Vulić
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6006/]
|-
|NLP for Conversations: Sentiment, Summarization, and Group Dynamics
|Gabriel Murray, Giuseppe Carenini and Shafiq Joty
|COLING 2018
|[https://sites.google.com/view/nlpforconversations]
|[https://aclweb.org/anthology/C18-3001/]
|-
|Practical Parsing for Downstream Applications
|Daniel Dakota and Sandra Kübler
|COLING 2018
|[http://cl.indiana.edu/colingParsing18/]
|[https://aclweb.org/anthology/C18-3002/]
|-
|Frame Semantics across Languages: Towards a Multilingual FrameNet
|Collin Baker, Michael Ellsworth, Miriam R L Petruck and Swabha Swayamdipta
|COLING 2018
|[https://framenet.icsi.berkeley.edu/fndrupal/node/5552/]
|[https://aclweb.org/anthology/C18-3003/]
|-
|Deep Bayesian Learning and Understanding
|Jen-Tzung Chien
|COLING 2018
|[http://chien.cm.nctu.edu.tw/home/coling/]
|[https://aclweb.org/anthology/C18-3004/]
|-
|Data-Driven Text Simplification
|Sanja Štajner and Horacio Saggion
|COLING 2018
|[http://taln.upf.edu/pages/coling2018simplification/]
|[https://aclweb.org/anthology/C18-3005/]
|-
|Deep Learning for Dialogue Systems
|Yun-Nung Chen, Asli Celikyilmaz and Dilek Hakkani-Tur
|COLING 2018
|[http://deepdialogue.miulab.tw/]
|[https://aclweb.org/anthology/C18-3006/]
|}

==2017 tutorials==

EMNLP 2017 website is no longer available. There are no traces of the AMNLP 2017 tutorials, except this [https://www.facebook.com/emnlp2017/posts/well-have-7-tutorials-at-emnlp2017-concepthier-sarcasm-graphrep-srl-memaugnn-str/1860468284191872/ Facebook post].

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Universal Dependencies
|Joakim Nivre, Daniel Zeman, Filip Ginter, and Francis Tyers
|EACL 2017
|[http://universaldependencies.org/eacl17tutorial/]
|
|-
|Practical Neural Machine Translation
|Rico Sennrich and Barry Haddow
|EACL 2017
|[http://www.statmt.org/eacl2017/practical-nmt.pdf]
|
|-
|Imitation learning for structured prediction in natural language processing
|Andreas Vlachos, Gerasimos Lampouras and Sebastian Riedel
|EACL 2017
|[https://sheffieldnlp.github.io/ImitationLearningTutorialEACL2017/]
|
|-
|Word Vector Space Specialisation
|Ivan Vulić, Nikola Mrkšić, and Mohammad Taher Pilehvar
|EACL 2017
|[http://people.ds.cam.ac.uk/iv250/tutorial/wv-tutorial.pdf]
|
|-
|Integer Linear Programming formulations in Natural Language Processing
|Dan Roth and Vivek Srikumar
|EACL 2017
|[https://sheffieldnlp.github.io/ImitationLearningTutorialEACL2017/]
|
|-
|Building Multimodal Simulations for Natural Language
|James Pustejovsky and Nikhil Krishnaswamy
|EACL 2017
|[http://www.voxicon.net]
|
|-
|Natural Language Processing for Precision Medicine
|Hoifung Poon, Chris Quirk, Kristina Toutanova, and Wen-tau Yih
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5001/]
|-
|Multimodal Machine Learning
|Louis-Philippe Morency and Tadas Baltrusaitis
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5002/]
|-
|Deep Learning for Semantic Composition
|Xiaodan Zhu and Edward Grefenstette
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5003/]
|-
|Deep Learning for Dialogue Systems
|Yun-Nung Chen, Asli Celikyilmaz, and Dilek Hakkani-Tur
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5004/]
|-
|Beyond Words: Deep Learning for Multi-word Expressions and Collocations
|Valia Kordoni
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5005/]
|-
|Making Better Use of the Crowd
|Jennifer Wortman Vaughan
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5006/]
|}

==2016 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Multimodal Learning and Reasoning
|Desmond Elliott, Douwe Kiela and Angeliki Lazaridou
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index4d04.html?article_id=59]
|
|-
|NLP Approaches to Computational Argumentation
|Noam Slonim, Iryna Gurevych, Chris Reed and Benno Stein
|ACL 2016
|[http://mirror.aclweb.org/acl2016/indexfc69.html?article_id=56]
|
|-
|Computer Aided Translation
|Philipp Koehn
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index6ea2.html?article_id=62]
|
|-
|Semantic Representations of Word Senses and Concepts
|José Camacho-Collados, Ignacio Iacobacci, Roberto Navigli and Mohammad Taher Pilehvar
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index6b9c.html?article_id=58]
|
|-
|Neural Machine Translation
|Thang Luong, Kyunghyun Cho and Christopher D. Manning
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index2e51.html?article_id=55]
|
|-
|Game Theory and Natural Language: Origin, Evolution and Processing
|Rocco Tripodi and Marcello Pelillo
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index56cf.html?article_id=57]
|
|-
|Understanding Short Texts
|Zhongyuan Wang and Haixun Wang
|ACL 2016
|[http://mirror.aclweb.org/acl2016/indexedf3.html?article_id=60]
|
|-
|MetaNet: Repository, Identification System, and Applications
|Miriam R L Petruck and Ellen K Dodge
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index6a61.html?article_id=61]
|
|-
|English Resource Semantics
|Dan Flickinger, Emily M. Bender, and Woodley Packard
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t1.html]
|[https://aclweb.org/anthology/N16-4001/]
|-
|Multilingual Multimodal Language Processing Using Neural Networks
|Mitesh M Khapra and Sarath Chandar
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t2.html]
|[https://aclweb.org/anthology/N16-4002/]
|-
|Question Answering with Knowledge Base, Web and Beyond
|Scott Wen-tau Yih & Hao Ma
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t3.html]
|[https://aclweb.org/anthology/N16-4003/]
|-
|Recent Progress in Deep Learning for NLP
|Zhengdong Lu and Hang Li
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t4.html]
|[https://aclweb.org/anthology/N16-4004/]
|-
|Scalable Statistical Relational Learning for NLP
|William Yang Wang and William W. Cohen
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t5.html]
|[https://aclweb.org/anthology/N16-4005/]
|-
|Statistical Machine Translation between Related Languages
|Pushpak Bhattacharyya, Mitesh Khapra, and Anoop Kunchukuttan
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t6.html]
|[https://aclweb.org/anthology/N16-4006/]
|-
|Practical Neural Networks for NLP: From Theory to Code
|Chris Dyer, Yoav Goldberg and Graham Neubig
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#practical]
|
|-
|Advanced Markov Logic Techniques for Scalable Joint Inference in NLP
|Deepak Venugopal, Vibhav Gogate and Vincent Ng
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#advanced]
|
|-
|Lifelong Machine Learning for Natural Language Processing
|Zhiyuan Chen and Bing Liu
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#lifelong]
|
|-
|Neural Networks for Sentiment Analysis
|Yue Zhang and Duy Tin Vo
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#neural]
|
|-
|Continuous Vector Spaces for Cross-language NLP Applications
|Rafael E. Banchs
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#continuous]
|
|-
|Methods and Theories for Large-scale Structured Prediction
|Xu Sun and Yansong Feng
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#methods]
|
|-
|Compositional Distributional Models of Meaning
|Mehrnoosh Sadrzadeh and Dimitri Kartsaklis
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T1/]
|[https://aclweb.org/anthology/C16-3001/]
|-
|Chinese Textual Sentiment Analysis: Datasets, Resources and Tools
|Lun-Wei Ku and Wei-Fan Chen
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T2]
|[https://aclweb.org/anthology/C16-3002/]
|-
|Natural Language Processing for Intelligent Access to Scientific Information
|Horacio Saggion and Francesco Ronzano
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T3]
|[https://aclweb.org/anthology/C16-3003/]
|-
|Quality Estimation for Language Output Applications
|Carolina Scarton, Gustavo Henrique Paetzold, and Lucia Specia
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T4]
|[https://aclweb.org/anthology/C16-3004/]
|-
|Translationese: Between Human and Machine Translation
|Shuly Wintner
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T5/]
|[https://aclweb.org/anthology/C16-3005/]
|-
|Succinct Data Structures for NLP-at-Scale
|Matthias Petri and Trevor Cohn
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T6/]
|[https://aclweb.org/anthology/C16-3006/]
|-
|The Role of Wikipedia in Text Analysis and Retrieval
|Marius Pasca
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T7]
|[https://aclweb.org/anthology/C16-3007/]
|}

Author: Agata Savary, July 2019

Updates are welcome

Past tutorials

2020-07-15T17:05:51Z

AgataSavary:

This page belongs to the [[tutorial chair handbook]]. It summarizes data on tutorials which took place at some recent ACL, EACL, NAACL, EMNLP and COLING conferences.

==2020 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Interpretability and Analysis in Neural NLP
|Yonatan Belinkov, Sebastian Gehrmann and Ellie Pavlick
|ACL 2020
|[https://acl2020.org/program/tutorials/#t1-interpretability-and-analysis-in-neural-nlp-cutting-edge-]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.1/]
|-
|Multi-modal Information Extraction from Text, Semi-structured, and Tabular Data on the Web
|Xin Luna Dong, Hannaneh Hajishirzi, Colin Lockard and Prashant Shiralkar
|ACL 2020
|[https://acl2020.org/program/tutorials/#t2-multi-modal-information-extraction-from-text-semi-structured-and-tabular-data-on-the-web-cutting-edge-]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.6/]
|-
|Reviewing Natural Language Processing Research
|Kevin Cohen, Karën Fort, Margot Mieskes and Aurélie Névéol
|ACL 2020
|[https://acl2020.org/program/tutorials/#t3-reviewing-natural-language-processing-research-introductory-]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.4/]
|-
|Stylized Text Generation: Approaches and Applications
|Lili Mou and Olga Vechtomova
|ACL 2020
|[https://acl2020.org/program/tutorials/#t4-stylized-text-generation-approaches-and-applications-cutting-edge-]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.5/]
|-
|Achieving Common Ground in Multi-modal Dialogue
|Malihe Alikhani and Matthew Stone
|ACL 2020
|[https://acl2020.org/program/tutorials/#t5-achieving-common-ground-in-multi-modal-dialogue-cutting-edge-]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.3/]
|-
|Commonsense Reasoning for Natural Language Processing
|Maarten Sap, Vered Shwartz, Antoine Bosselut, Dan Roth and Yejin Choi
|ACL 2020
|[https://acl2020.org/program/tutorials/#t6-commonsense-reasoning-for-natural-language-processing-introductory-]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.7/]
|-
|Integrating Ethics into the NLP Curriculum
|Emily M. Bender, Dirk Hovy and Alexandra Schofield
|ACL 2020
|[https://acl2020.org/program/tutorials/#t7-integrating-ethics-into-the-nlp-curriculum-introductory-]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.2/]
|-
|Open-Domain Question Answering
|Danqi Chen and Scott Wen-tau Yih
|ACL 2020
|[https://acl2020.org/program/tutorials/#t8-open-domain-question-answering-cutting-edge-]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.8/]
|-
|TO BE COMPLETED fro AACL, EMNLP and COLING 2020
|
|
|
|}

==2019 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Latent Structure Models for Natural Language Processing
|André F. T. Martins, Tsvetomila Mihaylova, Nikita Nangia and Vlad Niculae
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T1]
|[https://www.aclweb.org/anthology/P19-4001/]
|-
|Graph-Based Meaning Representations: Design and Processing
|Alexander Koller, Stephan Oepen and Weiwei Sun
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T2]
|[https://www.aclweb.org/anthology/P19-4002/]
|-
|Discourse Analysis and Its Applications
|Shafiq Joty, Giuseppe Carenini, Raymond Ng and Gabriel Murray
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T3]
|[https://www.aclweb.org/anthology/P19-4003/]
|-
|Computational Analysis of Political Texts: Bridging Research Efforts Across Communities
|Goran Glavaš, Federico Nanni and Simone Paolo Ponzetto
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T4]
|[https://www.aclweb.org/anthology/P19-4004/]
|-
|Wikipedia as a Resource for Text Analysis and Retrieval
|Marius Pasca
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T5]
|[https://www.aclweb.org/anthology/P19-4005/]
|-
|Deep Bayesian Natural Language Processing
|Jen-Tzung Chien
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T6]
|[https://www.aclweb.org/anthology/P19-4006/]
|-
|Unsupervised Cross-Lingual Representation Learning
|Sebastian Ruder, Anders Søgaard and Ivan Vulić
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T7]
|[https://www.aclweb.org/anthology/P19-4007/]
|-
|Advances in Argument Mining
|Katarzyna Budzynska and Chris Reed
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T8]
|[https://www.aclweb.org/anthology/P19-4008/]
|-
|Storytelling from Structured Data and Knowledge Graphs : An NLG Perspective
|Abhijit Mishra, Anirban Laha, Karthik Sankaranarayanan, Parag Jain and Saravanan Krishnan
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T9]
|[https://www.aclweb.org/anthology/P19-4009/]
|-
|Deep Adversarial Learning for NLP
|William Yang Wang, Sameer Singh and Jiwei Li
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t1-deep-adversarial-learning-for-nlp]
|[https://aclweb.org/anthology/N19-5001/]
|-
|Deep Learning for Natural Language Inference
|Samuel Bowman and Xiaodan Zhu
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t2-deep-learning-for-natural-language-inference]
|[https://aclweb.org/anthology/N19-5002/]
|-
|Measuring and Modeling Language Change
|Jacob Eisenstein
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t3-measuring-and-modeling-language-change]
|[https://aclweb.org/anthology/N19-5003/]
|-
|Transfer Learning in Natural Language Processing
|Sebastian Ruder, Matthew Peters, Swabha Swayamdipta and Thomas Wolf
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t4-transfer-learning-in-natural-language-processing]
|[https://aclweb.org/anthology/N19-5004/]
|-
|Language Learning and Processing in People and Machines
|Aida Nematzadeh, Richard Futrell and Roger Levy
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t5-language-learning-and-processing-in-people-and-machines]
|[https://aclweb.org/anthology/N19-5005/]
|-
|Applications of Natural Language Processing in Clinical Research and Practice
|Yanshan Wang, Ahmad Tafti, Sunghwan Sohn and Rui Zhang
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t6-applications-of-natural-language-processing-in-clinical-research-and-practice]
|[https://aclweb.org/anthology/N19-5006/]
|}

==2018 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Joint models for NLP
|Yue Zhang
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Graph Formalisms for Meaning Representations
|Adam Lopez and Sorcha Gilroy
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Writing Code for NLP Research
|Matt Gardner, Mark Neumann, Joel Grus, and Nicholas Lourie
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Deep Latent Variable Models of Natural Language
|Alexander Rush, Yoon Kim, and Sam Wiseman
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Standardized Tests as benchmarks for Artificial Intelligence
|Mrinmaya Sachan, Minjoon Seo, Hannaneh Hajishirzi, and Eric Xing
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Deep Chit-Chat: Deep Learning for ChatBots
|Wei Wu and Rui Yan
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|100 Things You Always Wanted to Know about Semantics & Pragmatics But Were Afraid to Ask
|Emily M. Bender
|ACL 2018
|[https://acl2018.org/tutorials/#100-things-you-always-wante]
|[https://aclweb.org/anthology/P18-5001/]
|-
|Neural Approaches to Conversational AI
|Jianfeng Gao, Michel Galley and Lihong Li
|ACL 2018
|[https://acl2018.org/tutorials/#neural-approaches-to-conver]
|[https://aclweb.org/anthology/P18-5002/]
|-
|Variational Inference and Deep Generative Models
|Wilker Aziz and Philip Schulz
|ACL 2018
|[https://acl2018.org/tutorials/#variational-inference-and-d]
|[https://aclweb.org/anthology/P18-5003/]
|-
|Connecting Language and Vision to Actions
|Peter Anderson, Abhishek Das and Qi Wu
|ACL 2018
|[https://acl2018.org/tutorials/#connecting-language-and-vis]
|[https://aclweb.org/anthology/P18-5004/]
|-
|Beyond Multiword Expressions: Processing Idioms and Metaphors
|Valia Kordoni
|ACL 2018
|[https://acl2018.org/tutorials/#beyond-multiword-expression]
|[https://aclweb.org/anthology/P18-5005/]
|-
|Neural Semantic Parsing
|Luke Zettlemoyer, Matt Gardner, Pradeep Dasigi, Srinivasan Iyer and Alane Suhr
|ACL 2018
|[https://acl2018.org/tutorials/#neural-semantic-parsing]
|[https://aclweb.org/anthology/P18-5006/]
|-
|Deep Reinforcement Learning for NLP
|William Yang Wang, Jiwei Li and Xiaodong He
|ACL 2018
|[https://acl2018.org/tutorials/#deep-reinforcement-learning]
|[https://aclweb.org/anthology/P18-5007/]
|-
|Multi-lingual Entity Discovery and Linking
|Avirup Sil, Heng Ji, Dan Roth and Silviu-Petru Cucerzan
|ACL 2018
|[https://acl2018.org/tutorials/#multi-lingual-entity-discov]
|[https://aclweb.org/anthology/P18-5008/]
|-
|Modelling Natural Language, Programs, and their Intersection
|Graham Neubig and Miltiadis Allamanis
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6001/]
|-
|Deep Learning Approaches to Text Production
|Claire Gardent and Shashi Narayan
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6002/]
|-
|Scalable Construction and Reasoning of Massive Knowledge Bases
|Xiang Ren, Nanyun Peng and William Yang Wang
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6003/]
|-
|The interplay between lexical resources and Natural Language Processing
|Jose Camacho-Collados, Luis Espinosa Anke and Mohammad Taher Pilehvar
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6004/]
|-
|Socially Responsible NLP
|Yulia Tsvetkov, Vinodkumar Prabhakaran and Rob Voigt
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6005/]
|-
|Deep Learning for Conversational AI
|Pei-Hao Su, Nikola Mrkšić, Iñigo Casanueva, Ivan Vulić
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6006/]
|-
|NLP for Conversations: Sentiment, Summarization, and Group Dynamics
|Gabriel Murray, Giuseppe Carenini and Shafiq Joty
|COLING 2018
|[https://sites.google.com/view/nlpforconversations]
|[https://aclweb.org/anthology/C18-3001/]
|-
|Practical Parsing for Downstream Applications
|Daniel Dakota and Sandra Kübler
|COLING 2018
|[http://cl.indiana.edu/colingParsing18/]
|[https://aclweb.org/anthology/C18-3002/]
|-
|Frame Semantics across Languages: Towards a Multilingual FrameNet
|Collin Baker, Michael Ellsworth, Miriam R L Petruck and Swabha Swayamdipta
|COLING 2018
|[https://framenet.icsi.berkeley.edu/fndrupal/node/5552/]
|[https://aclweb.org/anthology/C18-3003/]
|-
|Deep Bayesian Learning and Understanding
|Jen-Tzung Chien
|COLING 2018
|[http://chien.cm.nctu.edu.tw/home/coling/]
|[https://aclweb.org/anthology/C18-3004/]
|-
|Data-Driven Text Simplification
|Sanja Štajner and Horacio Saggion
|COLING 2018
|[http://taln.upf.edu/pages/coling2018simplification/]
|[https://aclweb.org/anthology/C18-3005/]
|-
|Deep Learning for Dialogue Systems
|Yun-Nung Chen, Asli Celikyilmaz and Dilek Hakkani-Tur
|COLING 2018
|[http://deepdialogue.miulab.tw/]
|[https://aclweb.org/anthology/C18-3006/]
|}

==2017 tutorials==

EMNLP 2017 website is no longer available. There are no traces of the AMNLP 2017 tutorials, except this [https://www.facebook.com/emnlp2017/posts/well-have-7-tutorials-at-emnlp2017-concepthier-sarcasm-graphrep-srl-memaugnn-str/1860468284191872/ Facebook post].

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Universal Dependencies
|Joakim Nivre, Daniel Zeman, Filip Ginter, and Francis Tyers
|EACL 2017
|[http://universaldependencies.org/eacl17tutorial/]
|
|-
|Practical Neural Machine Translation
|Rico Sennrich and Barry Haddow
|EACL 2017
|[http://www.statmt.org/eacl2017/practical-nmt.pdf]
|
|-
|Imitation learning for structured prediction in natural language processing
|Andreas Vlachos, Gerasimos Lampouras and Sebastian Riedel
|EACL 2017
|[https://sheffieldnlp.github.io/ImitationLearningTutorialEACL2017/]
|
|-
|Word Vector Space Specialisation
|Ivan Vulić, Nikola Mrkšić, and Mohammad Taher Pilehvar
|EACL 2017
|[http://people.ds.cam.ac.uk/iv250/tutorial/wv-tutorial.pdf]
|
|-
|Integer Linear Programming formulations in Natural Language Processing
|Dan Roth and Vivek Srikumar
|EACL 2017
|[https://sheffieldnlp.github.io/ImitationLearningTutorialEACL2017/]
|
|-
|Building Multimodal Simulations for Natural Language
|James Pustejovsky and Nikhil Krishnaswamy
|EACL 2017
|[http://www.voxicon.net]
|
|-
|Natural Language Processing for Precision Medicine
|Hoifung Poon, Chris Quirk, Kristina Toutanova, and Wen-tau Yih
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5001/]
|-
|Multimodal Machine Learning
|Louis-Philippe Morency and Tadas Baltrusaitis
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5002/]
|-
|Deep Learning for Semantic Composition
|Xiaodan Zhu and Edward Grefenstette
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5003/]
|-
|Deep Learning for Dialogue Systems
|Yun-Nung Chen, Asli Celikyilmaz, and Dilek Hakkani-Tur
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5004/]
|-
|Beyond Words: Deep Learning for Multi-word Expressions and Collocations
|Valia Kordoni
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5005/]
|-
|Making Better Use of the Crowd
|Jennifer Wortman Vaughan
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5006/]
|}

==2016 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Multimodal Learning and Reasoning
|Desmond Elliott, Douwe Kiela and Angeliki Lazaridou
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index4d04.html?article_id=59]
|
|-
|NLP Approaches to Computational Argumentation
|Noam Slonim, Iryna Gurevych, Chris Reed and Benno Stein
|ACL 2016
|[http://mirror.aclweb.org/acl2016/indexfc69.html?article_id=56]
|
|-
|Computer Aided Translation
|Philipp Koehn
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index6ea2.html?article_id=62]
|
|-
|Semantic Representations of Word Senses and Concepts
|José Camacho-Collados, Ignacio Iacobacci, Roberto Navigli and Mohammad Taher Pilehvar
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index6b9c.html?article_id=58]
|
|-
|Neural Machine Translation
|Thang Luong, Kyunghyun Cho and Christopher D. Manning
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index2e51.html?article_id=55]
|
|-
|Game Theory and Natural Language: Origin, Evolution and Processing
|Rocco Tripodi and Marcello Pelillo
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index56cf.html?article_id=57]
|
|-
|Understanding Short Texts
|Zhongyuan Wang and Haixun Wang
|ACL 2016
|[http://mirror.aclweb.org/acl2016/indexedf3.html?article_id=60]
|
|-
|MetaNet: Repository, Identification System, and Applications
|Miriam R L Petruck and Ellen K Dodge
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index6a61.html?article_id=61]
|
|-
|English Resource Semantics
|Dan Flickinger, Emily M. Bender, and Woodley Packard
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t1.html]
|[https://aclweb.org/anthology/N16-4001/]
|-
|Multilingual Multimodal Language Processing Using Neural Networks
|Mitesh M Khapra and Sarath Chandar
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t2.html]
|[https://aclweb.org/anthology/N16-4002/]
|-
|Question Answering with Knowledge Base, Web and Beyond
|Scott Wen-tau Yih & Hao Ma
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t3.html]
|[https://aclweb.org/anthology/N16-4003/]
|-
|Recent Progress in Deep Learning for NLP
|Zhengdong Lu and Hang Li
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t4.html]
|[https://aclweb.org/anthology/N16-4004/]
|-
|Scalable Statistical Relational Learning for NLP
|William Yang Wang and William W. Cohen
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t5.html]
|[https://aclweb.org/anthology/N16-4005/]
|-
|Statistical Machine Translation between Related Languages
|Pushpak Bhattacharyya, Mitesh Khapra, and Anoop Kunchukuttan
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t6.html]
|[https://aclweb.org/anthology/N16-4006/]
|-
|Practical Neural Networks for NLP: From Theory to Code
|Chris Dyer, Yoav Goldberg and Graham Neubig
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#practical]
|
|-
|Advanced Markov Logic Techniques for Scalable Joint Inference in NLP
|Deepak Venugopal, Vibhav Gogate and Vincent Ng
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#advanced]
|
|-
|Lifelong Machine Learning for Natural Language Processing
|Zhiyuan Chen and Bing Liu
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#lifelong]
|
|-
|Neural Networks for Sentiment Analysis
|Yue Zhang and Duy Tin Vo
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#neural]
|
|-
|Continuous Vector Spaces for Cross-language NLP Applications
|Rafael E. Banchs
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#continuous]
|
|-
|Methods and Theories for Large-scale Structured Prediction
|Xu Sun and Yansong Feng
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#methods]
|
|-
|Compositional Distributional Models of Meaning
|Mehrnoosh Sadrzadeh and Dimitri Kartsaklis
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T1/]
|[https://aclweb.org/anthology/C16-3001/]
|-
|Chinese Textual Sentiment Analysis: Datasets, Resources and Tools
|Lun-Wei Ku and Wei-Fan Chen
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T2]
|[https://aclweb.org/anthology/C16-3002/]
|-
|Natural Language Processing for Intelligent Access to Scientific Information
|Horacio Saggion and Francesco Ronzano
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T3]
|[https://aclweb.org/anthology/C16-3003/]
|-
|Quality Estimation for Language Output Applications
|Carolina Scarton, Gustavo Henrique Paetzold, and Lucia Specia
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T4]
|[https://aclweb.org/anthology/C16-3004/]
|-
|Translationese: Between Human and Machine Translation
|Shuly Wintner
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T5/]
|[https://aclweb.org/anthology/C16-3005/]
|-
|Succinct Data Structures for NLP-at-Scale
|Matthias Petri and Trevor Cohn
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T6/]
|[https://aclweb.org/anthology/C16-3006/]
|-
|The Role of Wikipedia in Text Analysis and Retrieval
|Marius Pasca
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T7]
|[https://aclweb.org/anthology/C16-3007/]
|}

Author: Agata Savary, July 2019

Updates are welcome

Past tutorials

2020-07-15T17:01:12Z

AgataSavary:

This page belongs to the [[tutorial chair handbook]]. It summarizes data on tutorials which took place at some recent ACL, EACL, NAACL, EMNLP and COLING conferences.

==2020 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Interpretability and Analysis in Neural NLP
|Yonatan Belinkov, Sebastian Gehrmann and Ellie Pavlick
|ACL 2020
|[https://acl2020.org/program/tutorials/#t1-interpretability-and-analysis-in-neural-nlp-cutting-edge-]
|[https://www.aclweb.org/anthology/2020.acl-tutorials.1/]
|-
|Multi-modal Information Extraction from Text, Semi-structured, and Tabular Data on the Web
|Xin Luna Dong, Hannaneh Hajishirzi, Colin Lockard and Prashant Shiralkar
|ACL 2020
|[https://acl2020.org/program/tutorials/#t2-multi-modal-information-extraction-from-text-semi-structured-and-tabular-data-on-the-web-cutting-edge-]
|
|-
|Reviewing Natural Language Processing Research
|Kevin Cohen, Karën Fort, Margot Mieskes and Aurélie Névéol
|ACL 2020
|[https://acl2020.org/program/tutorials/#t3-reviewing-natural-language-processing-research-introductory-]
|
|-
|Stylized Text Generation: Approaches and Applications
|Lili Mou and Olga Vechtomova
|ACL 2020
|[https://acl2020.org/program/tutorials/#t4-stylized-text-generation-approaches-and-applications-cutting-edge-]
|
|-
|Achieving Common Ground in Multi-modal Dialogue
|Malihe Alikhani and Matthew Stone
|ACL 2020
|[https://acl2020.org/program/tutorials/#t5-achieving-common-ground-in-multi-modal-dialogue-cutting-edge-]
|
|-
|Commonsense Reasoning for Natural Language Processing
|Maarten Sap, Vered Shwartz, Antoine Bosselut, Dan Roth and Yejin Choi
|ACL 2020
|[https://acl2020.org/program/tutorials/#t6-commonsense-reasoning-for-natural-language-processing-introductory-]
|
|-
|Integrating Ethics into the NLP Curriculum
|Emily M. Bender, Dirk Hovy and Alexandra Schofield
|ACL 2020
|[https://acl2020.org/program/tutorials/#t7-integrating-ethics-into-the-nlp-curriculum-introductory-]
|
|-
|Recent Advances in Open-Domain Question Answering
|Danqi Chen and Scott Wen-tau Yih
|ACL 2020
|[https://acl2020.org/program/tutorials/#t8-open-domain-question-answering-cutting-edge-]
|
|-
|TO BE COMPLETED
|
|
|
|}

==2019 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Latent Structure Models for Natural Language Processing
|André F. T. Martins, Tsvetomila Mihaylova, Nikita Nangia and Vlad Niculae
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T1]
|[https://www.aclweb.org/anthology/P19-4001/]
|-
|Graph-Based Meaning Representations: Design and Processing
|Alexander Koller, Stephan Oepen and Weiwei Sun
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T2]
|[https://www.aclweb.org/anthology/P19-4002/]
|-
|Discourse Analysis and Its Applications
|Shafiq Joty, Giuseppe Carenini, Raymond Ng and Gabriel Murray
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T3]
|[https://www.aclweb.org/anthology/P19-4003/]
|-
|Computational Analysis of Political Texts: Bridging Research Efforts Across Communities
|Goran Glavaš, Federico Nanni and Simone Paolo Ponzetto
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T4]
|[https://www.aclweb.org/anthology/P19-4004/]
|-
|Wikipedia as a Resource for Text Analysis and Retrieval
|Marius Pasca
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T5]
|[https://www.aclweb.org/anthology/P19-4005/]
|-
|Deep Bayesian Natural Language Processing
|Jen-Tzung Chien
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T6]
|[https://www.aclweb.org/anthology/P19-4006/]
|-
|Unsupervised Cross-Lingual Representation Learning
|Sebastian Ruder, Anders Søgaard and Ivan Vulić
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T7]
|[https://www.aclweb.org/anthology/P19-4007/]
|-
|Advances in Argument Mining
|Katarzyna Budzynska and Chris Reed
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T8]
|[https://www.aclweb.org/anthology/P19-4008/]
|-
|Storytelling from Structured Data and Knowledge Graphs : An NLG Perspective
|Abhijit Mishra, Anirban Laha, Karthik Sankaranarayanan, Parag Jain and Saravanan Krishnan
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T9]
|[https://www.aclweb.org/anthology/P19-4009/]
|-
|Deep Adversarial Learning for NLP
|William Yang Wang, Sameer Singh and Jiwei Li
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t1-deep-adversarial-learning-for-nlp]
|[https://aclweb.org/anthology/N19-5001/]
|-
|Deep Learning for Natural Language Inference
|Samuel Bowman and Xiaodan Zhu
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t2-deep-learning-for-natural-language-inference]
|[https://aclweb.org/anthology/N19-5002/]
|-
|Measuring and Modeling Language Change
|Jacob Eisenstein
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t3-measuring-and-modeling-language-change]
|[https://aclweb.org/anthology/N19-5003/]
|-
|Transfer Learning in Natural Language Processing
|Sebastian Ruder, Matthew Peters, Swabha Swayamdipta and Thomas Wolf
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t4-transfer-learning-in-natural-language-processing]
|[https://aclweb.org/anthology/N19-5004/]
|-
|Language Learning and Processing in People and Machines
|Aida Nematzadeh, Richard Futrell and Roger Levy
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t5-language-learning-and-processing-in-people-and-machines]
|[https://aclweb.org/anthology/N19-5005/]
|-
|Applications of Natural Language Processing in Clinical Research and Practice
|Yanshan Wang, Ahmad Tafti, Sunghwan Sohn and Rui Zhang
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t6-applications-of-natural-language-processing-in-clinical-research-and-practice]
|[https://aclweb.org/anthology/N19-5006/]
|}

==2018 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Joint models for NLP
|Yue Zhang
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Graph Formalisms for Meaning Representations
|Adam Lopez and Sorcha Gilroy
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Writing Code for NLP Research
|Matt Gardner, Mark Neumann, Joel Grus, and Nicholas Lourie
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Deep Latent Variable Models of Natural Language
|Alexander Rush, Yoon Kim, and Sam Wiseman
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Standardized Tests as benchmarks for Artificial Intelligence
|Mrinmaya Sachan, Minjoon Seo, Hannaneh Hajishirzi, and Eric Xing
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Deep Chit-Chat: Deep Learning for ChatBots
|Wei Wu and Rui Yan
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|100 Things You Always Wanted to Know about Semantics & Pragmatics But Were Afraid to Ask
|Emily M. Bender
|ACL 2018
|[https://acl2018.org/tutorials/#100-things-you-always-wante]
|[https://aclweb.org/anthology/P18-5001/]
|-
|Neural Approaches to Conversational AI
|Jianfeng Gao, Michel Galley and Lihong Li
|ACL 2018
|[https://acl2018.org/tutorials/#neural-approaches-to-conver]
|[https://aclweb.org/anthology/P18-5002/]
|-
|Variational Inference and Deep Generative Models
|Wilker Aziz and Philip Schulz
|ACL 2018
|[https://acl2018.org/tutorials/#variational-inference-and-d]
|[https://aclweb.org/anthology/P18-5003/]
|-
|Connecting Language and Vision to Actions
|Peter Anderson, Abhishek Das and Qi Wu
|ACL 2018
|[https://acl2018.org/tutorials/#connecting-language-and-vis]
|[https://aclweb.org/anthology/P18-5004/]
|-
|Beyond Multiword Expressions: Processing Idioms and Metaphors
|Valia Kordoni
|ACL 2018
|[https://acl2018.org/tutorials/#beyond-multiword-expression]
|[https://aclweb.org/anthology/P18-5005/]
|-
|Neural Semantic Parsing
|Luke Zettlemoyer, Matt Gardner, Pradeep Dasigi, Srinivasan Iyer and Alane Suhr
|ACL 2018
|[https://acl2018.org/tutorials/#neural-semantic-parsing]
|[https://aclweb.org/anthology/P18-5006/]
|-
|Deep Reinforcement Learning for NLP
|William Yang Wang, Jiwei Li and Xiaodong He
|ACL 2018
|[https://acl2018.org/tutorials/#deep-reinforcement-learning]
|[https://aclweb.org/anthology/P18-5007/]
|-
|Multi-lingual Entity Discovery and Linking
|Avirup Sil, Heng Ji, Dan Roth and Silviu-Petru Cucerzan
|ACL 2018
|[https://acl2018.org/tutorials/#multi-lingual-entity-discov]
|[https://aclweb.org/anthology/P18-5008/]
|-
|Modelling Natural Language, Programs, and their Intersection
|Graham Neubig and Miltiadis Allamanis
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6001/]
|-
|Deep Learning Approaches to Text Production
|Claire Gardent and Shashi Narayan
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6002/]
|-
|Scalable Construction and Reasoning of Massive Knowledge Bases
|Xiang Ren, Nanyun Peng and William Yang Wang
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6003/]
|-
|The interplay between lexical resources and Natural Language Processing
|Jose Camacho-Collados, Luis Espinosa Anke and Mohammad Taher Pilehvar
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6004/]
|-
|Socially Responsible NLP
|Yulia Tsvetkov, Vinodkumar Prabhakaran and Rob Voigt
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6005/]
|-
|Deep Learning for Conversational AI
|Pei-Hao Su, Nikola Mrkšić, Iñigo Casanueva, Ivan Vulić
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6006/]
|-
|NLP for Conversations: Sentiment, Summarization, and Group Dynamics
|Gabriel Murray, Giuseppe Carenini and Shafiq Joty
|COLING 2018
|[https://sites.google.com/view/nlpforconversations]
|[https://aclweb.org/anthology/C18-3001/]
|-
|Practical Parsing for Downstream Applications
|Daniel Dakota and Sandra Kübler
|COLING 2018
|[http://cl.indiana.edu/colingParsing18/]
|[https://aclweb.org/anthology/C18-3002/]
|-
|Frame Semantics across Languages: Towards a Multilingual FrameNet
|Collin Baker, Michael Ellsworth, Miriam R L Petruck and Swabha Swayamdipta
|COLING 2018
|[https://framenet.icsi.berkeley.edu/fndrupal/node/5552/]
|[https://aclweb.org/anthology/C18-3003/]
|-
|Deep Bayesian Learning and Understanding
|Jen-Tzung Chien
|COLING 2018
|[http://chien.cm.nctu.edu.tw/home/coling/]
|[https://aclweb.org/anthology/C18-3004/]
|-
|Data-Driven Text Simplification
|Sanja Štajner and Horacio Saggion
|COLING 2018
|[http://taln.upf.edu/pages/coling2018simplification/]
|[https://aclweb.org/anthology/C18-3005/]
|-
|Deep Learning for Dialogue Systems
|Yun-Nung Chen, Asli Celikyilmaz and Dilek Hakkani-Tur
|COLING 2018
|[http://deepdialogue.miulab.tw/]
|[https://aclweb.org/anthology/C18-3006/]
|}

==2017 tutorials==

EMNLP 2017 website is no longer available. There are no traces of the AMNLP 2017 tutorials, except this [https://www.facebook.com/emnlp2017/posts/well-have-7-tutorials-at-emnlp2017-concepthier-sarcasm-graphrep-srl-memaugnn-str/1860468284191872/ Facebook post].

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Universal Dependencies
|Joakim Nivre, Daniel Zeman, Filip Ginter, and Francis Tyers
|EACL 2017
|[http://universaldependencies.org/eacl17tutorial/]
|
|-
|Practical Neural Machine Translation
|Rico Sennrich and Barry Haddow
|EACL 2017
|[http://www.statmt.org/eacl2017/practical-nmt.pdf]
|
|-
|Imitation learning for structured prediction in natural language processing
|Andreas Vlachos, Gerasimos Lampouras and Sebastian Riedel
|EACL 2017
|[https://sheffieldnlp.github.io/ImitationLearningTutorialEACL2017/]
|
|-
|Word Vector Space Specialisation
|Ivan Vulić, Nikola Mrkšić, and Mohammad Taher Pilehvar
|EACL 2017
|[http://people.ds.cam.ac.uk/iv250/tutorial/wv-tutorial.pdf]
|
|-
|Integer Linear Programming formulations in Natural Language Processing
|Dan Roth and Vivek Srikumar
|EACL 2017
|[https://sheffieldnlp.github.io/ImitationLearningTutorialEACL2017/]
|
|-
|Building Multimodal Simulations for Natural Language
|James Pustejovsky and Nikhil Krishnaswamy
|EACL 2017
|[http://www.voxicon.net]
|
|-
|Natural Language Processing for Precision Medicine
|Hoifung Poon, Chris Quirk, Kristina Toutanova, and Wen-tau Yih
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5001/]
|-
|Multimodal Machine Learning
|Louis-Philippe Morency and Tadas Baltrusaitis
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5002/]
|-
|Deep Learning for Semantic Composition
|Xiaodan Zhu and Edward Grefenstette
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5003/]
|-
|Deep Learning for Dialogue Systems
|Yun-Nung Chen, Asli Celikyilmaz, and Dilek Hakkani-Tur
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5004/]
|-
|Beyond Words: Deep Learning for Multi-word Expressions and Collocations
|Valia Kordoni
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5005/]
|-
|Making Better Use of the Crowd
|Jennifer Wortman Vaughan
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5006/]
|}

==2016 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Multimodal Learning and Reasoning
|Desmond Elliott, Douwe Kiela and Angeliki Lazaridou
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index4d04.html?article_id=59]
|
|-
|NLP Approaches to Computational Argumentation
|Noam Slonim, Iryna Gurevych, Chris Reed and Benno Stein
|ACL 2016
|[http://mirror.aclweb.org/acl2016/indexfc69.html?article_id=56]
|
|-
|Computer Aided Translation
|Philipp Koehn
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index6ea2.html?article_id=62]
|
|-
|Semantic Representations of Word Senses and Concepts
|José Camacho-Collados, Ignacio Iacobacci, Roberto Navigli and Mohammad Taher Pilehvar
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index6b9c.html?article_id=58]
|
|-
|Neural Machine Translation
|Thang Luong, Kyunghyun Cho and Christopher D. Manning
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index2e51.html?article_id=55]
|
|-
|Game Theory and Natural Language: Origin, Evolution and Processing
|Rocco Tripodi and Marcello Pelillo
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index56cf.html?article_id=57]
|
|-
|Understanding Short Texts
|Zhongyuan Wang and Haixun Wang
|ACL 2016
|[http://mirror.aclweb.org/acl2016/indexedf3.html?article_id=60]
|
|-
|MetaNet: Repository, Identification System, and Applications
|Miriam R L Petruck and Ellen K Dodge
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index6a61.html?article_id=61]
|
|-
|English Resource Semantics
|Dan Flickinger, Emily M. Bender, and Woodley Packard
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t1.html]
|[https://aclweb.org/anthology/N16-4001/]
|-
|Multilingual Multimodal Language Processing Using Neural Networks
|Mitesh M Khapra and Sarath Chandar
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t2.html]
|[https://aclweb.org/anthology/N16-4002/]
|-
|Question Answering with Knowledge Base, Web and Beyond
|Scott Wen-tau Yih & Hao Ma
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t3.html]
|[https://aclweb.org/anthology/N16-4003/]
|-
|Recent Progress in Deep Learning for NLP
|Zhengdong Lu and Hang Li
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t4.html]
|[https://aclweb.org/anthology/N16-4004/]
|-
|Scalable Statistical Relational Learning for NLP
|William Yang Wang and William W. Cohen
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t5.html]
|[https://aclweb.org/anthology/N16-4005/]
|-
|Statistical Machine Translation between Related Languages
|Pushpak Bhattacharyya, Mitesh Khapra, and Anoop Kunchukuttan
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t6.html]
|[https://aclweb.org/anthology/N16-4006/]
|-
|Practical Neural Networks for NLP: From Theory to Code
|Chris Dyer, Yoav Goldberg and Graham Neubig
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#practical]
|
|-
|Advanced Markov Logic Techniques for Scalable Joint Inference in NLP
|Deepak Venugopal, Vibhav Gogate and Vincent Ng
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#advanced]
|
|-
|Lifelong Machine Learning for Natural Language Processing
|Zhiyuan Chen and Bing Liu
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#lifelong]
|
|-
|Neural Networks for Sentiment Analysis
|Yue Zhang and Duy Tin Vo
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#neural]
|
|-
|Continuous Vector Spaces for Cross-language NLP Applications
|Rafael E. Banchs
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#continuous]
|
|-
|Methods and Theories for Large-scale Structured Prediction
|Xu Sun and Yansong Feng
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#methods]
|
|-
|Compositional Distributional Models of Meaning
|Mehrnoosh Sadrzadeh and Dimitri Kartsaklis
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T1/]
|[https://aclweb.org/anthology/C16-3001/]
|-
|Chinese Textual Sentiment Analysis: Datasets, Resources and Tools
|Lun-Wei Ku and Wei-Fan Chen
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T2]
|[https://aclweb.org/anthology/C16-3002/]
|-
|Natural Language Processing for Intelligent Access to Scientific Information
|Horacio Saggion and Francesco Ronzano
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T3]
|[https://aclweb.org/anthology/C16-3003/]
|-
|Quality Estimation for Language Output Applications
|Carolina Scarton, Gustavo Henrique Paetzold, and Lucia Specia
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T4]
|[https://aclweb.org/anthology/C16-3004/]
|-
|Translationese: Between Human and Machine Translation
|Shuly Wintner
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T5/]
|[https://aclweb.org/anthology/C16-3005/]
|-
|Succinct Data Structures for NLP-at-Scale
|Matthias Petri and Trevor Cohn
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T6/]
|[https://aclweb.org/anthology/C16-3006/]
|-
|The Role of Wikipedia in Text Analysis and Retrieval
|Marius Pasca
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T7]
|[https://aclweb.org/anthology/C16-3007/]
|}

Author: Agata Savary, July 2019

Updates are welcome

Past tutorials

2020-07-15T16:59:36Z

AgataSavary:

This page belongs to the [[tutorial chair handbook]]. It summarizes data on tutorials which took place at some recent ACL, EACL, NAACL, EMNLP and COLING conferences.

==2020 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Interpretability and Analysis in Neural NLP
|Yonatan Belinkov, Sebastian Gehrmann and Ellie Pavlick
|ACL 2020
|[https://acl2020.org/program/tutorials/#t1-interpretability-and-analysis-in-neural-nlp-cutting-edge-]
|
|-
|Multi-modal Information Extraction from Text, Semi-structured, and Tabular Data on the Web
|Xin Luna Dong, Hannaneh Hajishirzi, Colin Lockard and Prashant Shiralkar
|ACL 2020
|[https://acl2020.org/program/tutorials/#t2-multi-modal-information-extraction-from-text-semi-structured-and-tabular-data-on-the-web-cutting-edge-]
|
|-
|Reviewing Natural Language Processing Research
|Kevin Cohen, Karën Fort, Margot Mieskes and Aurélie Névéol
|ACL 2020
|[https://acl2020.org/program/tutorials/#t3-reviewing-natural-language-processing-research-introductory-]
|
|-
|Stylized Text Generation: Approaches and Applications
|Lili Mou and Olga Vechtomova
|ACL 2020
|[https://acl2020.org/program/tutorials/#t4-stylized-text-generation-approaches-and-applications-cutting-edge-]
|
|-
|Achieving Common Ground in Multi-modal Dialogue
|Malihe Alikhani and Matthew Stone
|ACL 2020
|[https://acl2020.org/program/tutorials/#t5-achieving-common-ground-in-multi-modal-dialogue-cutting-edge-]
|
|-
|Commonsense Reasoning for Natural Language Processing
|Maarten Sap, Vered Shwartz, Antoine Bosselut, Dan Roth and Yejin Choi
|ACL 2020
|[https://acl2020.org/program/tutorials/#t6-commonsense-reasoning-for-natural-language-processing-introductory-]
|
|-
|Integrating Ethics into the NLP Curriculum
|Emily M. Bender, Dirk Hovy and Alexandra Schofield
|ACL 2020
|[https://acl2020.org/program/tutorials/#t7-integrating-ethics-into-the-nlp-curriculum-introductory-]
|
|-
|Recent Advances in Open-Domain Question Answering
|Danqi Chen and Scott Wen-tau Yih
|ACL 2020
|[https://acl2020.org/program/tutorials/#t8-open-domain-question-answering-cutting-edge-]
|
|-
|TO BE COMPLETED
|
|
|
|}

==2019 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Latent Structure Models for Natural Language Processing
|André F. T. Martins, Tsvetomila Mihaylova, Nikita Nangia and Vlad Niculae
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T1]
|[https://www.aclweb.org/anthology/P19-4001/]
|-
|Graph-Based Meaning Representations: Design and Processing
|Alexander Koller, Stephan Oepen and Weiwei Sun
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T2]
|[https://www.aclweb.org/anthology/P19-4002/]
|-
|Discourse Analysis and Its Applications
|Shafiq Joty, Giuseppe Carenini, Raymond Ng and Gabriel Murray
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T3]
|[https://www.aclweb.org/anthology/P19-4003/]
|-
|Computational Analysis of Political Texts: Bridging Research Efforts Across Communities
|Goran Glavaš, Federico Nanni and Simone Paolo Ponzetto
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T4]
|[https://www.aclweb.org/anthology/P19-4004/]
|-
|Wikipedia as a Resource for Text Analysis and Retrieval
|Marius Pasca
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T5]
|[https://www.aclweb.org/anthology/P19-4005/]
|-
|Deep Bayesian Natural Language Processing
|Jen-Tzung Chien
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T6]
|[https://www.aclweb.org/anthology/P19-4006/]
|-
|Unsupervised Cross-Lingual Representation Learning
|Sebastian Ruder, Anders Søgaard and Ivan Vulić
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T7]
|[https://www.aclweb.org/anthology/P19-4007/]
|-
|Advances in Argument Mining
|Katarzyna Budzynska and Chris Reed
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T8]
|[https://www.aclweb.org/anthology/P19-4008/]
|-
|Storytelling from Structured Data and Knowledge Graphs : An NLG Perspective
|Abhijit Mishra, Anirban Laha, Karthik Sankaranarayanan, Parag Jain and Saravanan Krishnan
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T9]
|[https://www.aclweb.org/anthology/P19-4009/]
|-
|Deep Adversarial Learning for NLP
|William Yang Wang, Sameer Singh and Jiwei Li
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t1-deep-adversarial-learning-for-nlp]
|[https://aclweb.org/anthology/N19-5001/]
|-
|Deep Learning for Natural Language Inference
|Samuel Bowman and Xiaodan Zhu
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t2-deep-learning-for-natural-language-inference]
|[https://aclweb.org/anthology/N19-5002/]
|-
|Measuring and Modeling Language Change
|Jacob Eisenstein
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t3-measuring-and-modeling-language-change]
|[https://aclweb.org/anthology/N19-5003/]
|-
|Transfer Learning in Natural Language Processing
|Sebastian Ruder, Matthew Peters, Swabha Swayamdipta and Thomas Wolf
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t4-transfer-learning-in-natural-language-processing]
|[https://aclweb.org/anthology/N19-5004/]
|-
|Language Learning and Processing in People and Machines
|Aida Nematzadeh, Richard Futrell and Roger Levy
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t5-language-learning-and-processing-in-people-and-machines]
|[https://aclweb.org/anthology/N19-5005/]
|-
|Applications of Natural Language Processing in Clinical Research and Practice
|Yanshan Wang, Ahmad Tafti, Sunghwan Sohn and Rui Zhang
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t6-applications-of-natural-language-processing-in-clinical-research-and-practice]
|[https://aclweb.org/anthology/N19-5006/]
|}

==2018 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Joint models for NLP
|Yue Zhang
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Graph Formalisms for Meaning Representations
|Adam Lopez and Sorcha Gilroy
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Writing Code for NLP Research
|Matt Gardner, Mark Neumann, Joel Grus, and Nicholas Lourie
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Deep Latent Variable Models of Natural Language
|Alexander Rush, Yoon Kim, and Sam Wiseman
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Standardized Tests as benchmarks for Artificial Intelligence
|Mrinmaya Sachan, Minjoon Seo, Hannaneh Hajishirzi, and Eric Xing
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Deep Chit-Chat: Deep Learning for ChatBots
|Wei Wu and Rui Yan
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|100 Things You Always Wanted to Know about Semantics & Pragmatics But Were Afraid to Ask
|Emily M. Bender
|ACL 2018
|[https://acl2018.org/tutorials/#100-things-you-always-wante]
|[https://aclweb.org/anthology/P18-5001/]
|-
|Neural Approaches to Conversational AI
|Jianfeng Gao, Michel Galley and Lihong Li
|ACL 2018
|[https://acl2018.org/tutorials/#neural-approaches-to-conver]
|[https://aclweb.org/anthology/P18-5002/]
|-
|Variational Inference and Deep Generative Models
|Wilker Aziz and Philip Schulz
|ACL 2018
|[https://acl2018.org/tutorials/#variational-inference-and-d]
|[https://aclweb.org/anthology/P18-5003/]
|-
|Connecting Language and Vision to Actions
|Peter Anderson, Abhishek Das and Qi Wu
|ACL 2018
|[https://acl2018.org/tutorials/#connecting-language-and-vis]
|[https://aclweb.org/anthology/P18-5004/]
|-
|Beyond Multiword Expressions: Processing Idioms and Metaphors
|Valia Kordoni
|ACL 2018
|[https://acl2018.org/tutorials/#beyond-multiword-expression]
|[https://aclweb.org/anthology/P18-5005/]
|-
|Neural Semantic Parsing
|Luke Zettlemoyer, Matt Gardner, Pradeep Dasigi, Srinivasan Iyer and Alane Suhr
|ACL 2018
|[https://acl2018.org/tutorials/#neural-semantic-parsing]
|[https://aclweb.org/anthology/P18-5006/]
|-
|Deep Reinforcement Learning for NLP
|William Yang Wang, Jiwei Li and Xiaodong He
|ACL 2018
|[https://acl2018.org/tutorials/#deep-reinforcement-learning]
|[https://aclweb.org/anthology/P18-5007/]
|-
|Multi-lingual Entity Discovery and Linking
|Avirup Sil, Heng Ji, Dan Roth and Silviu-Petru Cucerzan
|ACL 2018
|[https://acl2018.org/tutorials/#multi-lingual-entity-discov]
|[https://aclweb.org/anthology/P18-5008/]
|-
|Modelling Natural Language, Programs, and their Intersection
|Graham Neubig and Miltiadis Allamanis
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6001/]
|-
|Deep Learning Approaches to Text Production
|Claire Gardent and Shashi Narayan
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6002/]
|-
|Scalable Construction and Reasoning of Massive Knowledge Bases
|Xiang Ren, Nanyun Peng and William Yang Wang
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6003/]
|-
|The interplay between lexical resources and Natural Language Processing
|Jose Camacho-Collados, Luis Espinosa Anke and Mohammad Taher Pilehvar
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6004/]
|-
|Socially Responsible NLP
|Yulia Tsvetkov, Vinodkumar Prabhakaran and Rob Voigt
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6005/]
|-
|Deep Learning for Conversational AI
|Pei-Hao Su, Nikola Mrkšić, Iñigo Casanueva, Ivan Vulić
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6006/]
|-
|NLP for Conversations: Sentiment, Summarization, and Group Dynamics
|Gabriel Murray, Giuseppe Carenini and Shafiq Joty
|COLING 2018
|[https://sites.google.com/view/nlpforconversations]
|[https://aclweb.org/anthology/C18-3001/]
|-
|Practical Parsing for Downstream Applications
|Daniel Dakota and Sandra Kübler
|COLING 2018
|[http://cl.indiana.edu/colingParsing18/]
|[https://aclweb.org/anthology/C18-3002/]
|-
|Frame Semantics across Languages: Towards a Multilingual FrameNet
|Collin Baker, Michael Ellsworth, Miriam R L Petruck and Swabha Swayamdipta
|COLING 2018
|[https://framenet.icsi.berkeley.edu/fndrupal/node/5552/]
|[https://aclweb.org/anthology/C18-3003/]
|-
|Deep Bayesian Learning and Understanding
|Jen-Tzung Chien
|COLING 2018
|[http://chien.cm.nctu.edu.tw/home/coling/]
|[https://aclweb.org/anthology/C18-3004/]
|-
|Data-Driven Text Simplification
|Sanja Štajner and Horacio Saggion
|COLING 2018
|[http://taln.upf.edu/pages/coling2018simplification/]
|[https://aclweb.org/anthology/C18-3005/]
|-
|Deep Learning for Dialogue Systems
|Yun-Nung Chen, Asli Celikyilmaz and Dilek Hakkani-Tur
|COLING 2018
|[http://deepdialogue.miulab.tw/]
|[https://aclweb.org/anthology/C18-3006/]
|}

==2017 tutorials==

EMNLP 2017 website is no longer available. There are no traces of the AMNLP 2017 tutorials, except this [https://www.facebook.com/emnlp2017/posts/well-have-7-tutorials-at-emnlp2017-concepthier-sarcasm-graphrep-srl-memaugnn-str/1860468284191872/ Facebook post].

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Universal Dependencies
|Joakim Nivre, Daniel Zeman, Filip Ginter, and Francis Tyers
|EACL 2017
|[http://universaldependencies.org/eacl17tutorial/]
|
|-
|Practical Neural Machine Translation
|Rico Sennrich and Barry Haddow
|EACL 2017
|[http://www.statmt.org/eacl2017/practical-nmt.pdf]
|
|-
|Imitation learning for structured prediction in natural language processing
|Andreas Vlachos, Gerasimos Lampouras and Sebastian Riedel
|EACL 2017
|[https://sheffieldnlp.github.io/ImitationLearningTutorialEACL2017/]
|
|-
|Word Vector Space Specialisation
|Ivan Vulić, Nikola Mrkšić, and Mohammad Taher Pilehvar
|EACL 2017
|[http://people.ds.cam.ac.uk/iv250/tutorial/wv-tutorial.pdf]
|
|-
|Integer Linear Programming formulations in Natural Language Processing
|Dan Roth and Vivek Srikumar
|EACL 2017
|[https://sheffieldnlp.github.io/ImitationLearningTutorialEACL2017/]
|
|-
|Building Multimodal Simulations for Natural Language
|James Pustejovsky and Nikhil Krishnaswamy
|EACL 2017
|[http://www.voxicon.net]
|
|-
|Natural Language Processing for Precision Medicine
|Hoifung Poon, Chris Quirk, Kristina Toutanova, and Wen-tau Yih
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5001/]
|-
|Multimodal Machine Learning
|Louis-Philippe Morency and Tadas Baltrusaitis
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5002/]
|-
|Deep Learning for Semantic Composition
|Xiaodan Zhu and Edward Grefenstette
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5003/]
|-
|Deep Learning for Dialogue Systems
|Yun-Nung Chen, Asli Celikyilmaz, and Dilek Hakkani-Tur
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5004/]
|-
|Beyond Words: Deep Learning for Multi-word Expressions and Collocations
|Valia Kordoni
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5005/]
|-
|Making Better Use of the Crowd
|Jennifer Wortman Vaughan
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5006/]
|}

==2016 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Multimodal Learning and Reasoning
|Desmond Elliott, Douwe Kiela and Angeliki Lazaridou
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index4d04.html?article_id=59]
|
|-
|NLP Approaches to Computational Argumentation
|Noam Slonim, Iryna Gurevych, Chris Reed and Benno Stein
|ACL 2016
|[http://mirror.aclweb.org/acl2016/indexfc69.html?article_id=56]
|
|-
|Computer Aided Translation
|Philipp Koehn
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index6ea2.html?article_id=62]
|
|-
|Semantic Representations of Word Senses and Concepts
|José Camacho-Collados, Ignacio Iacobacci, Roberto Navigli and Mohammad Taher Pilehvar
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index6b9c.html?article_id=58]
|
|-
|Neural Machine Translation
|Thang Luong, Kyunghyun Cho and Christopher D. Manning
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index2e51.html?article_id=55]
|
|-
|Game Theory and Natural Language: Origin, Evolution and Processing
|Rocco Tripodi and Marcello Pelillo
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index56cf.html?article_id=57]
|
|-
|Understanding Short Texts
|Zhongyuan Wang and Haixun Wang
|ACL 2016
|[http://mirror.aclweb.org/acl2016/indexedf3.html?article_id=60]
|
|-
|MetaNet: Repository, Identification System, and Applications
|Miriam R L Petruck and Ellen K Dodge
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index6a61.html?article_id=61]
|
|-
|English Resource Semantics
|Dan Flickinger, Emily M. Bender, and Woodley Packard
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t1.html]
|[https://aclweb.org/anthology/N16-4001/]
|-
|Multilingual Multimodal Language Processing Using Neural Networks
|Mitesh M Khapra and Sarath Chandar
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t2.html]
|[https://aclweb.org/anthology/N16-4002/]
|-
|Question Answering with Knowledge Base, Web and Beyond
|Scott Wen-tau Yih & Hao Ma
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t3.html]
|[https://aclweb.org/anthology/N16-4003/]
|-
|Recent Progress in Deep Learning for NLP
|Zhengdong Lu and Hang Li
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t4.html]
|[https://aclweb.org/anthology/N16-4004/]
|-
|Scalable Statistical Relational Learning for NLP
|William Yang Wang and William W. Cohen
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t5.html]
|[https://aclweb.org/anthology/N16-4005/]
|-
|Statistical Machine Translation between Related Languages
|Pushpak Bhattacharyya, Mitesh Khapra, and Anoop Kunchukuttan
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t6.html]
|[https://aclweb.org/anthology/N16-4006/]
|-
|Practical Neural Networks for NLP: From Theory to Code
|Chris Dyer, Yoav Goldberg and Graham Neubig
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#practical]
|
|-
|Advanced Markov Logic Techniques for Scalable Joint Inference in NLP
|Deepak Venugopal, Vibhav Gogate and Vincent Ng
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#advanced]
|
|-
|Lifelong Machine Learning for Natural Language Processing
|Zhiyuan Chen and Bing Liu
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#lifelong]
|
|-
|Neural Networks for Sentiment Analysis
|Yue Zhang and Duy Tin Vo
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#neural]
|
|-
|Continuous Vector Spaces for Cross-language NLP Applications
|Rafael E. Banchs
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#continuous]
|
|-
|Methods and Theories for Large-scale Structured Prediction
|Xu Sun and Yansong Feng
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#methods]
|
|-
|Compositional Distributional Models of Meaning
|Mehrnoosh Sadrzadeh and Dimitri Kartsaklis
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T1/]
|[https://aclweb.org/anthology/C16-3001/]
|-
|Chinese Textual Sentiment Analysis: Datasets, Resources and Tools
|Lun-Wei Ku and Wei-Fan Chen
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T2]
|[https://aclweb.org/anthology/C16-3002/]
|-
|Natural Language Processing for Intelligent Access to Scientific Information
|Horacio Saggion and Francesco Ronzano
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T3]
|[https://aclweb.org/anthology/C16-3003/]
|-
|Quality Estimation for Language Output Applications
|Carolina Scarton, Gustavo Henrique Paetzold, and Lucia Specia
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T4]
|[https://aclweb.org/anthology/C16-3004/]
|-
|Translationese: Between Human and Machine Translation
|Shuly Wintner
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T5/]
|[https://aclweb.org/anthology/C16-3005/]
|-
|Succinct Data Structures for NLP-at-Scale
|Matthias Petri and Trevor Cohn
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T6/]
|[https://aclweb.org/anthology/C16-3006/]
|-
|The Role of Wikipedia in Text Analysis and Retrieval
|Marius Pasca
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T7]
|[https://aclweb.org/anthology/C16-3007/]
|}

Author: Agata Savary, July 2019

Updates are welcome

2020Q1 Reports: ACL 2020

2020-02-20T08:35:57Z

AgataSavary: /* Tutorial Chairs */

== General Chair ==
Dan Jurafsky, Stanford University

The 58th annual meeting of the Association for Computational Linguistics (ACL) will take place in Seattle, Washington at the Hyatt Regency Seattle in downtown Seattle from July 5th through July 10th, 2020.

We have a great set of chairs! We are continuing 2019's new roles (Diversity and Inclusion chairs, Remote Presentation Chairs, AV Chairs) and adding new ones: (Sustainability chair), and we are doing well in demographic representation among our chairs (gender and region).

Following advice from last year, we have been using Slack for most intra-committee communication (and we put the Slack channel into the ACL pro space, so it can be preserved for future years), and using email only when absolutely necessary.

As usual, the growing size of the conference (both in papers and attendees) is a challenge, but both in papers and space we have been doing well (see the individual chair summaries below).

[this summary in progress]

== Program Chairs ==
Joyce Chai, University of Michigan

Natalie Schluter, IT University of Copenhagen, Denmark

Joel Tetreault, Dataminr, USA

== Local Organisation Chairs ==
Priscilla Rasmussen, ACL

With advice from:

Jianfeng Gao, Microsoft Research

Luke Zettlemoyer, University of Washington

== Tutorial Chairs ==

Agata Savary, University of Tours, France

Yue Zhang, Westlake University, Hangzhou, China

The call, submission, reviewing and selection of tutorials was coordinated jointly for 4 conferences: ACL, AACL-IJCNLP, COLING and EMNLP.

Before drafting the call, we collected lists of tutorials offered within the past 4 years. We analysed previous calls for tutorials and reports from tutorial chairs (from [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs 2016], [https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs 2017], [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs 2018] and [http://aclweb.org/adminwiki/index.php?title=2019Q1_Reports:_ACL_2019 2019]). We consulted previous tutorial chairs with a questionnaire including questions about: the number of submissions, encouraging submissions on specific topics or from specific lecturers, the review procedure, the evaluation criteria, the post-tutorial availability of the slides/codes, and lessons learned from tutorial coordination. We also discussed the publication of slides and video recordings from future tutorials with the persons in charge of the ACL Anthology. As a result of these steps, we created two new sections for the [https://www.aclweb.org/adminwiki/index.php?title=Conference_Handbook ACL Conference Handbook] (future chairs might consider updating these documents yearly):
* the list of [https://www.aclweb.org/adminwiki/index.php?title=Past_tutorials past tutorials] at ACL, COLING, EACL, EMNLP, and NAACL in 2016-2019
* a [https://www.aclweb.org/adminwiki/index.php?title=Tutorial_chair_handbook tutorial chair handbook]

The final [https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020 call] differs from previous calls in several aspects: (i) the expectations about tutorial proposals were made clearer, (ii) following the central ACL decision, the teachers' payment policy was replaced by a fee-waiving policy, (iii) the required submission details include two new items: diversity considerations and agreement for open access publication of slides, codes, data and video recordings, (iv) the evaluation criteria (see below) are announced.

We recruited a review committee of 19 members, including the 8 tutorial chairs and 11 external members selected for their large understanding of the NLP domain and a good experience in reviewing and/or tutorial teaching:

'''Review Committee'''
* Timothy Baldwin (University of Melbourne, Australia) - AACL-IJCNLP 2020 tutorial chair
* Daniel Beck (University of Melbourne, Australia) - COLING 2020 tutorial chair
* Emily M. Bender (University of Washington, WA, USA)
* Erik Cambria (Nanyang Technological University, Singapore)
* Gaël Dias (University of Caen Normandie, France)
* Stefan Evert (Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany)
* Yang Liu (Tsinghua University, Beijing, China)
* Agata Savary (University of Tours, France) - ACL 2020 tutorial chair
* João Sedoc (Johns Hopkins University, Baltimore, MD, USA)
* Lucia Specia (Sheffield University, UK) - COLING 2020 tutorial chair
* Xu SUN (Peking University, China)
* Yulia Tsvetkov (Carnegie Mellon University, Pittsburgh, PA, USA)
* Benjamin Van Durme (Johns Hopkins University, Baltimore, MD, USA) - EMNLP 2020 tutorial chair
* Aline Villavicencio (University of Sheffield, UK and Federal University of Rio Grande do Sul, Brazil) - EMNLP 2020 tutorial chair
* Taro Watanabe (Google, Inc., Tokyo, Japan)
* Aaron Steven White (University of Rochester, NY, USA)
* Fei Xia (University of Washington, WA, USA) - AACL-IJCNLP 2020 tutorial chair
* Yue Zhang (Westlake University, Hangzhou, China) - ACL 2020 tutorial chair
* Meishan Zhang (Tianjin University, China)

In total, we received 43 submissions for the 4 conferences. Each reviewer was assigned 6-7 proposals and each proposal received 3 reviews. The selection criteria included: clarity and preparedness, novelty or timely character of the topic, lecturers' experience, likely audience interest, open access of the teaching material, diversity aspects (multilingualism, gender, age and country of the lecturers), and compatibility with the preferred venues.
We accepted 31 proposals.

The decision making was handled via an online meeting of the 8 tutorial chairs. In particular, the selection of tutorials for each conference was done via the expression of interest of the tutorial chairs on a round-robin basis. Some slight adjustments were also performed after the meeting to better fit the authors' preferences. In total, 8, 8, 8 and 7 proposals were selected for ACL, AACL-IJCNLP, COLING and EMNLP, respectively. Upon the announcement the results, 2 of the proposals accepted for AACL-IJCNLP were withdrawn.

The submission, review, selection and collection of final material for all tutorials was handled via a dedicated SoftConf space, shared by the 4 coordinating conferences. After the selection of proposals, a separate track was created on SoftConf for each conference. The final submission page (one per conference) was set up so as to collect all the necessary data including notably: the tutorial slides, URLs for course material (if any), printable material (if any) and agreement for open access publication.

The final selection for ACL 2020 consists of the following 8 tutorials of 3 hours each (each of them had ACL as the preferred or the second preferred venue):

'''Morning Tutorials'''

'''T1: Interpretability and Analysis in Neural NLP''' (cutting-edge) 
Yonatan Belinkov, Sebastian Gehrmann and Ellie Pavlick 
While deep learning has transformed the NLP field and impacted the larger computational linguistics community, the rise of neural networks is stained by their opaque nature: It is challenging to interpret the inner workings of neural network models, and explicate their behavior. Therefore, in the last few years, an increasingly large body of work has been devoted to the analysis and interpretation of neural network models in NLP. 
This body of work is so far lacking a common framework and methodology. Moreover, approaching the analysis of modern neural networks can be difficult for newcomers to the field. This tutorial aims to fill this gap and introduce the nascent field of interpretability and analysis of neural networks in NLP. 
The tutorial covers the main lines of analysis work, such as probing classifier, behavior studies and test suites, psycholinguistic methods, visualizations, adversarial examples, and other methods. We highlight not only the most commonly applied analysis methods, but also the specific limitations and shortcomings of current approaches, in order to inform participants where to focus future efforts.

'''T2: Multi-modal Information Extraction from Text, Semi-structured, and Tabular Data on the Web''' (cutting-edge) 
Xin Luna Dong, Hannaneh Hajishirzi, Colin Lockard and Prashant Shiralkar 
The World Wide Web contains vast quantities of textual information in several forms: unstructured text, template-based semi-structured webpages (which present data in key-value pairs and lists), and tables. Methods for extracting information from these sources and converting it to a structured form have been a target of research from the natural language processing (NLP), data mining, and database communities. While these researchers have largely separated extraction from web data into different problems based on the modality of the data, they have faced similar problems such as learning with limited labeled data, defining (or avoiding defining) ontologies, making use of prior knowledge, and scaling solutions to deal with the size of the Web. 
In this tutorial we take a holistic view toward information extraction, exploring the commonalities in the challenges and solutions developed to address these different forms of text. We will explore the approaches targeted at unstructured text that largely rely on learning syntactic or semantic textual patterns, approaches targeted at semi-structured documents that learn to identify structural patterns in the template, and approaches targeting web tables which rely heavily on entity linking and type information. 
While these different data modalities have largely been considered separately in the past, recent research has started taking a more inclusive approach toward textual extraction, in which the multiple signals offered by textual, layout, and visual clues are combined into a single extraction model made possible by new deep learning approaches. At the same time, trends within purely textual extraction have shifted toward full-document understanding rather than considering sentences as independent units. With this in mind, it is worth considering the information extraction problem as a whole to motivate solutions that harness textual semantics along with visual and semi-structured layout information. We will discuss these approaches and suggest avenues for future work.

'''T3: Reviewing Natural Language Processing Research''' (introductory) 
Kevin Cohen, Karën Fort, Margot Mieskes and Aurélie Névéol 
As the demand for reviewing grows, so must the pool of reviewers. As the [http://www.livecongress.it/aol/indexSA.php?id=E2EAED7D&ticket= survey] presented by Graham Neubig at the 2019 ACL showed, a considerable number of reviewers are junior researchers, who might lack the experience and expertise necessary for high-quality reviews. Some of them might not have the environment or lack opportunities that allow them to learn the skills necessary. A tutorial on reviewing for the NLP community might increase reviewers’ confidence, as well as the quality of the reviews. This introductory tutorial will cover the goals, processes, and evaluation of reviewing research papers in natural language processing.

'''T4: Stylized Text Generation: Approaches and Applications''' (cutting-edge) 
Lili Mou and Olga Vechtomova 
Text generation has played an important role in various applications of natural language processing (NLP), and kn recent studies, researchers are paying increasing attention to modeling and manipulating the style of the generation text, which we call stylized text generation. In this tutorial, we will provide a comprehensive literature review in this direction. We start from the definition of style and different settings of stylized text generation, illustrated with various applications. Then, we present different settings of stylized generation, such as parallel supervised, style label-supervised, and unsupervised. In each setting, we delve deep into machine learning methods, including embedding learning techniques to represent style}, adversarial learning and reinforcement learning with cycle consistency to match content but to distinguish different styles. We also introduce current approaches of evaluating stylized text generation systems. We conclude our tutorial by presenting the challenges of stylized text generation and discussing future directions, such as small-data training, non-categorical style modeling, and a generalized scope of style transfer (e.g., controlling the syntax as a style).

'''Afternoon Tutorials'''

'''T5: Achieving Common Ground in Multi-modal Dialogue''' (cutting-edge) 
Malihe Alikhani and Matthew Stone 
All communication aims at achieving common ground (grounding): interlocutors can work together effectively only with mutual beliefs about what the state of the world is, about what their goals are, and about how they plan to make their goals a reality. Computational dialogue research offers some classic results on grouding, which unfortunately offer scant guidance to the design of grounding modules and behaviors in cutting-edge systems. In this tutorial, we focus on three main topic areas: 1) grounding in human-human communication; 2) grounding in dialogue systems; and 3) grounding in multi-modal interactive systems, including image-oriented conversations and human-robot interactions. We highlight a number of achievements of recent computational research in coordinating complex content, show how these results lead to rich and challenging opportunities for doing grounding in more flexible and powerful ways, and canvass relevant insights from the literature on human--human conversation. We expect that the tutorial will be of interest to researchers in dialogue systems, computational semantics and cognitive modeling, and hope that it will catalyze research and system building that more directly explores the creative, strategic ways conversational agents might be able to seek and offer evidence about their understanding of their interlocutors.

'''T6: Commonsense Reasoning for Natural Language Processing''' (introductory) 
Maarten Sap, Vered Shwartz, Antoine Bosselut, Dan Roth and Yejin Choi 
In our tutorial, we (1) outline the various types of commonsense (e.g., physical, social), and (2) discuss techniques to gather and represent commonsense knowledge, while highlighting the challenges specific to this type of knowledge (e.g., reporting bias). We will then (3) discuss the types of commonsense knowledge captured by modern NLP systems (e.g., large pretrained language models), and (4) present ways to measure systems' commonsense reasoning abilities. We finish with (5) a discussion of various ways in which commonsense reasoning can be used to improve performance on NLP tasks, exemplified by an (6) interactive session on integrating commonsense into a downstream task.

'''T7: Integrating Ethics into the NLP Curriculum''' (introductory) 
Emily M. Bender, Dirk Hovy and Alexandra Schofield 
Our goal in this tutorial is to empower NLP researchers and practitioners with tools and resources to teach others about how to ethically apply NLP techniques. Our tutorial will present both high-level strategies for developing an ethics-oriented curriculum, based on experience and best practices, as well as specific sample exercises that can be brought to a classroom. We plan to make this a highly interactive work session culminating in a shared online resource page that pools lesson plans, assignments, exercise ideas, reading suggestions, and ideas from the attendees. We consider three primary topics with our session that frequently underlie ethical issues in NLP research: Dual use, bias and privacy. 
In this setting, a key lesson is that there is no single approach to ethical NLP: each project requires thoughtful consideration about what steps can be taken to best support people affected by that project. However, we can learn (and teach) what kinds of issues to be aware of and what kinds of strategies are available for mitigating harm. To teach this process, we apply and promote interactive exercises that provide an opportunity to ideate, discuss, and reflect. We plan to facilitate this in a way that encourages positive discussion, emphasizing the creation of ideas for the future instead of negative opinions of previous work.

'''T8: Recent Advances in Open-Domain Question Answering''' (cutting-edge) 
Danqi Chen and Scott Wen-tau Yih 
Open-domain (textual) question answering (QA), the task of finding answers to open-domain questions by searching a large collection of documents, has been a long-standing problem in NLP, information retrieval (IR) and related fields (Voorhees et al., 1999; Moldovan et al., 2000; Brill et al.,2002; Ferrucci et al., 2010). Traditional QA systems were usually constructed as a pipeline, consisting of many different components such as question processing, document/passage retrieval and answer processing. With the rapid development of neural reading comprehension (Chen, 2018), modern open-domain QA systems have been restructured by combining traditional IR techniques and neural reading comprehension models (Chen et al., 2017; Yang et al., 2019) or even implemented in a fully end-to-end fashion (Lee et al., 2019; Seo et al., 2019). While the system architecture has been drastically simplified, two technical challenges remain critical:(1) “Retriever”: finding documents that (might)contain an answer from a large collection of documents; (2) “Reader”: finding the answer in a given paragraph or a document. 
In this tutorial, we aim to provide a comprehensive and coherent overview of recent advances in this line of research. We will start by first giving a brief historical background of open-domain question answering, discussing the basic setup and core technical challenges of the research problem.The focus will then shift to modern techniques and resources proposed for open-domain QA, including the basics of latest neural reading comprehension systems, new datasets and models. The scope will also be broadened to cover the information retrieval component on how to effectively identify passages relevant to the questions. Moreover, in-depth discussions will be given on the use of traditional / neural IR modules, as well as the trade-offs between modular design and end-to-end training. If time permits, we also plan to discuss some hybrid approaches for answering questions using both text and large knowledge bases (e.g. (Sun et al., 2018)) and give a critical review on how structured data complements the information from unstructured text. 
At the end of our tutorial, we will discuss some important questions, including (1) How much progress have we made compared to the QA systems developed in the last decade?(2) What are the main challenges and limitations of cur-rent approaches? (3) How to trade off the efficiency (computational time and memory requirements) and accuracy in the deep learning era? We hope that our tutorial will not only serve as a useful resource for the audience to efficiently acquire the up-to-date knowledge, but also provide new perspectives to stimulate the advances of open-domain QA research in the next phase.

== Workshop Chairs ==

Milica Gašić, Heinrich Heine University Düsseldorf

Dilek Hakkani-Tur, Amazon Alexa AI

Saif M. Mohammad, National Research Council Canada

Ves Stoyanov, Facebook AI

== Student Research Workshop Chairs and Faculty Advisors==

'''Student Research Workshop Co-chairs'''

Rotem Dror, Technion - Israel Institute of Technology

Jiangming Liu, The University of Edinburgh

Shruti Rijhwani, Carnegie Mellon University

'''Student Research Workshop Faculty Advisors'''

Omri Abend, Hebrew University of Jerusalem

Sujian Li, Peking University

Zhou Yu, University of California, Davis

Information about the Student Research Workshop (SRW) has posted on the workshop's website: https://sites.google.com/view/acl20studentresearchworkshop/. The SRW Call for Papers has been distributed to ACL mailing lists, as well as on our official Twitter account (@acl_srw) and the ACL meeting's Twitter account (@acl_meeting).

'''Pre-submission Mentoring Phase (completed mid-February 2020)'''

Before submission to the main deadline, the SRW offered pre-submission mentoring by experienced researchers of the ACL community. The pre-submission mentoring primarily serves to provide feedback on the writing style, readability and presentation of the paper.

We recruited 30 mentors for providing pre-submission feedback. The deadline for the pre-submission phase was January 17, 2020. We had 57 pre-submissions in total.

Mentors were matched to pre-submissions according to their research areas. All mentors have already provided feedback for the submissions and it was sent to the authors mid-February 2020. The majority of mentors have also offered to participate in follow-up discussions with the authors via email until the main submission deadline.

Vouchers for one month's free use of Grammarly Premium have been sent to all the pre-submission authors. These were provided by the ACL 2020 Diversity and Inclusion Committee.

'''Main submission'''

For the main submission, the START (softconf) submission page has been set up. Currently, we have recruited 200 members of the ACL community (both students and senior researchers) to serve as the Program Committee for reviewing submissions to the SRW. We plan on inviting more PC members, as the number of submissions is likely to be larger than originally estimated.

Submission deadlines for the SRW are as follows:

* Paper submission deadline: March 6, 2020

* Review deadline: April 10, 2020

* Acceptance notification: April 15, 2020

* Camera-ready deadline: May 6, 2020

* Travel grant application deadline: to be decided.

* Travel grant notification: to be decided.

We also plan to have a post-acceptance mentoring process, for all papers accepted to the SRW.

'''Funding'''

The SRW has applied for an NSF grant of $18,000. The Don and Betty Walker international fund will also be able to provide student support. The SRW organizers have made contact with a number of industry companies to obtain sponsorship, but not yet secured additional funding. Contact has been made with the ACL 2020 sponsorship chairs and with Priscilla to investigate other funding opportunities, as well as the Student Volunteer Program, which helps students cover registration fee to the main conference.

== Audio-Video Chairs ==

Hamid Palangi, Microsoft Research, Redmond

Lianhui Qin, University of Washington

== Conference Handbook Chair ==

Nanyun Peng, University of Southern California

== Demo Chairs ==

Asli Celikyilmaz, Microsoft Research, Redmond

Shawn Wen, PolyAI

== Diversity & Inclusion (D&I) Chairs ==

Cecilia Ovesdotter Alm, Rochester Institute of Technology

Vinodkumar Prabhakaran, Google

== Local Sponsorship Chairs ==

Hoifung Poon, Microsoft

Kristina Toutanova, Google

== Publication Chairs ==

Steven Bethard, University of Arizona

Ryan Cotterrell, University of Cambridge

Rui Yan, Peking University

Starting from the style files from ACL 2019, we have produced new LaTeX style files for ACL 2020. Most of the description was retained, but the order of sections was overhauled to make sure that important information wasn't scattered so haphazardly across the document. Other improvements were also made, like using the recommended citation style consistently throughout the LaTeX source, and separating out all the LaTeX-specific stuff into clearly marked sections. The MS Word version was derived from these LaTeX versions to match as closely as possible. The LaTeX version was also posted to the Overleaf gallery. The most recent .bib file for the entire ACL Anthology was included in the style file distribution to encourage authors to use the official citations for ACL Anthology publications. All style file changes were merged into https://github.com/acl-org/acl-pub/tree/gh-pages/paper_styles.

== Publicity Chair ==

Emily M. Bender, University of Washington

=== Dissemination ===

Durable accounts for the ACL meeting on Twitter and Facebook have been created:
* https://twitter.com/aclmeeting
* https://www.facebook.com/aclmeeting/

These will be passed along to the ACL 2021 publicity chair(s) so that they don't have to build up followers separately. As of Feb 4, 2020 the Twitter account has 4,061 followers and the Facebook account has 181. We have not yet been making use of the Instagram account, but we have been using the Twitter and Facebook accounts to publicize important dates as well as blog posts. The Twitter account especially has been useful for fielding questions from the community. Calls for papers have also gone out over the ACL member portal and several mailing lists, as well as websites such as WikiCFP. (These are maintained in a spreadsheet which can be handed off to the ACL 2021 publicity chair(s)).

=== Next Steps ===

* Recruit co-chairs, especially to coordinate live-tweeting of the conference
* Contact local media for coverage
* Develop land acknowledgement in consultation with the Duwamish Tribe (on whose land the meeting will take place). The Duwamish publish this information about land acknowledgments: https://www.duwamishtribe.org/land-acknowledgement

== Remote Presentation Chairs ==

Hao Fang, Microsoft Semantic Machines

Yi Luan, Google AI Language

== Sustainability Chairs ==

Ananya Ganesh, Educational Testing Service

Klaus Zechner, Educational Testing Service

Our main goal for this new focus area is to engage the ACL community in discussions about how best to reduce the carbon footprint of future ACL conferences in order to contribute to sustainable and livable conditions on this planet.
One of the main directions we are currently envisioning is to encourage and support conference attendees in virtual participation using live streaming of conference events as air travel is the main contributor to the carbon footprint of international conferences.

== Website & Conference App Chairs ==

Sudha Rao, Microsoft Research, Redmond

Yizhe Zhang, Microsoft Research, Redmond

== Business Office ==

Priscilla Rasmussen, ACL

2020Q1 Reports: ACL 2020

2020-02-18T13:50:01Z

AgataSavary: /* Tutorial Chairs */

== General Chair ==
Dan Jurafsky, Stanford University

The 58th annual meeting of the Association for Computational Linguistics (ACL) will take place in Seattle, Washington at the Hyatt Regency Seattle in downtown Seattle from July 5th through July 10th, 2020.

We have a great set of chairs! We are continuing 2019's new roles (Diversity and Inclusion chairs, Remote Presentation Chairs, AV Chairs) and adding new ones: (Sustainability chair), and we are doing well in demographic representation among our chairs (gender and region).

Following advice from last year, we have been using Slack for most intra-committee communication (and we put the Slack channel into the ACL pro space, so it can be preserved for future years), and using email only when absolutely necessary.

As usual, the growing size of the conference (both in papers and attendees) is a challenge, but both in papers and space we have been doing well (see the individual chair summaries below).

[this summary in progress]

== Program Chairs ==
Joyce Chai, University of Michigan

Natalie Schluter, IT University of Copenhagen, Denmark

Joel Tetreault, Dataminr, USA

== Local Organisation Chairs ==
Priscilla Rasmussen, ACL

With advice from:

Jianfeng Gao, Microsoft Research

Luke Zettlemoyer, University of Washington

== Tutorial Chairs ==

Agata Savary, University of Tours, France

Yue Zhang, Westlake University, Hangzhou, China

The call, submission, reviewing and selection of tutorials was coordinated jointly for 4 conferences: ACL, AACL-IJCNLP, COLING and EMNLP.

Before drafting the call, we collected lists of tutorials offered within the past 4 years. We analysed previous calls for tutorials and reports from tutorial chairs (from [https://aclweb.org/adminwiki/index.php?title=2016Q3_Reports:_Tutorial_Chairs 2016], [https://aclweb.org/adminwiki/index.php?title=2017Q3_Reports:_Tutorial_Chairs 2017], [https://aclweb.org/adminwiki/index.php?title=2018Q3_Reports:_Tutorial_Chairs 2018] and [http://aclweb.org/adminwiki/index.php?title=2019Q1_Reports:_ACL_2019 2019]). We consulted previous tutorial chairs with a questionnaire including questions about: the number of submissions, encouraging submissions on specific topics or from specific lecturers, the review procedure, the evaluation criteria, the post-tutorial availability of the slides/codes, and lessons learned from tutorial coordination. We also discussed the publication of slides and video recordings from future tutorials with the persons in charge of the ACL Anthology. As a result of these steps, we created two new sections for the [https://www.aclweb.org/adminwiki/index.php?title=Conference_Handbook ACL Conference Handbook] (future chairs might consider updating these documents yearly):
* the list of [https://www.aclweb.org/adminwiki/index.php?title=Past_tutorials past tutorials] at ACL, COLING, EACL, EMNLP, and NAACL in 2016-2019
* a [https://www.aclweb.org/adminwiki/index.php?title=Tutorial_chair_handbook tutorial chair handbook]

The final [https://www.aclweb.org/portal/content/joint-call-tutorial-proposals-aclaacl-ijcnlpemnlpcoling-2020 call] differs from previous calls in several aspects: (i) the expectations about tutorial proposals were made clearer, (ii) following the central ACL decision, the teachers' payment policy was replaced by a fee-waiving policy, (iii) the required submission details include two new items: diversity considerations and agreement for open access publication of slides, codes, data and video recordings, (iv) the evaluation criteria (see below) are announced.

We recruited a review committee of 19 members, including the 8 tutorial chairs and 11 external members selected for their large understanding of the NLP domain and a good experience in reviewing and/or tutorial teaching:

'''Review Committee'''
* Timothy Baldwin (University of Melbourne, Australia) - AACL-IJCNLP 2020 tutorial chair
* Daniel Beck (University of Melbourne, Australia) - COLING 2020 tutorial chair
* Emily M. Bender (University of Washington, WA, USA)
* Erik Cambria (Nanyang Technological University, Singapore)
* Gaël Dias (University of Caen Normandie, France)
* Stefan Evert (Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany)
* Yang Liu (Tsinghua University, Beijing, China)
* Agata Savary (University of Tours, France) - ACL 2020 tutorial chair
* João Sedoc (Johns Hopkins University, Baltimore, MD, USA)
* Lucia Specia (Sheffield University, UK) - COLING 2020 tutorial chair
* Xu SUN (Peking University, China)
* Yulia Tsvetkov (Carnegie Mellon University, Pittsburgh, PA, USA)
* Benjamin Van Durme (Johns Hopkins University, Baltimore, MD, USA) - EMNLP 2020 tutorial chair
* Aline Villavicencio (University of Sheffield, UK and Federal University of Rio Grande do Sul, Brazil) - EMNLP 2020 tutorial chair
* Taro Watanabe (Google, Inc., Tokyo, Japan)
* Aaron Steven White (University of Rochester, NY, USA)
* Fei Xia (University of Washington, WA, USA) - AACL-IJCNLP 2020 tutorial chair
* Yue Zhang (Westlake University, Hangzhou, China) - ACL 2020 tutorial chair
* Meishan Zhang (Tianjin University, China)

In total, we received 43 submissions for the 4 conferences. Each reviewer was assigned 6-7 proposals and each proposal received 3 reviews. The selection criteria included: clarity and preparedness, novelty or timely character of the topic, lecturers' experience, likely audience interest, open access of the teaching material, diversity aspects (multilingualism, gender, age and country of the lecturers), and compatibility with the preferred venues.
We accepted 31 proposals, 2 proposals were further withdrawn by the authors.

The decision making was handled via an online meeting of the 8 tutorial chairs. In particular, the selection of tutorials for each conference was done via the expression of interest of the tutorial chairs on a round-robin basis. Some slight adjustments were also performed after the meeting to better fit the authors' preferences. In total, 8, 8, 8 and 7 proposals were selected for ACL, AACL-IJCNLP, COLING and EMNLP, respectively. Upon the announcement the results, 2 of the proposals accepted for AACL-IJCNLP were withdrawn.

The submission, review, selection and collection of final material for all tutorials was handled via a dedicated SoftConf space, shared by the 4 coordinating conferences. After the selection of proposals, a separate track was created on SoftConf for each conference. The final submission page (one per conference) was set up so as to collect all the necessary data including notably: the tutorial slides, URLs for course material (if any), printable material (if any) and agreement for open access publication.

The final selection for ACL 2020 consists of the following 8 tutorials of 3 hours each (each of them had ACL as the preferred or the second preferred venue):

'''Morning Tutorials'''

'''T1: Interpretability and Analysis in Neural NLP''' (cutting-edge) 
Yonatan Belinkov, Sebastian Gehrmann and Ellie Pavlick 
While deep learning has transformed the NLP field and impacted the larger computational linguistics community, the rise of neural networks is stained by their opaque nature: It is challenging to interpret the inner workings of neural network models, and explicate their behavior. Therefore, in the last few years, an increasingly large body of work has been devoted to the analysis and interpretation of neural network models in NLP. 
This body of work is so far lacking a common framework and methodology. Moreover, approaching the analysis of modern neural networks can be difficult for newcomers to the field. This tutorial aims to fill this gap and introduce the nascent field of interpretability and analysis of neural networks in NLP. 
The tutorial covers the main lines of analysis work, such as probing classifier, behavior studies and test suites, psycholinguistic methods, visualizations, adversarial examples, and other methods. We highlight not only the most commonly applied analysis methods, but also the specific limitations and shortcomings of current approaches, in order to inform participants where to focus future efforts.

'''T2: Multi-modal Information Extraction from Text, Semi-structured, and Tabular Data on the Web''' (cutting-edge) 
Xin Luna Dong, Hannaneh Hajishirzi, Colin Lockard and Prashant Shiralkar 
The World Wide Web contains vast quantities of textual information in several forms: unstructured text, template-based semi-structured webpages (which present data in key-value pairs and lists), and tables. Methods for extracting information from these sources and converting it to a structured form have been a target of research from the natural language processing (NLP), data mining, and database communities. While these researchers have largely separated extraction from web data into different problems based on the modality of the data, they have faced similar problems such as learning with limited labeled data, defining (or avoiding defining) ontologies, making use of prior knowledge, and scaling solutions to deal with the size of the Web. 
In this tutorial we take a holistic view toward information extraction, exploring the commonalities in the challenges and solutions developed to address these different forms of text. We will explore the approaches targeted at unstructured text that largely rely on learning syntactic or semantic textual patterns, approaches targeted at semi-structured documents that learn to identify structural patterns in the template, and approaches targeting web tables which rely heavily on entity linking and type information. 
While these different data modalities have largely been considered separately in the past, recent research has started taking a more inclusive approach toward textual extraction, in which the multiple signals offered by textual, layout, and visual clues are combined into a single extraction model made possible by new deep learning approaches. At the same time, trends within purely textual extraction have shifted toward full-document understanding rather than considering sentences as independent units. With this in mind, it is worth considering the information extraction problem as a whole to motivate solutions that harness textual semantics along with visual and semi-structured layout information. We will discuss these approaches and suggest avenues for future work.

'''T3: Reviewing Natural Language Processing Research''' (introductory) 
Kevin Cohen, Karën Fort, Margot Mieskes and Aurélie Névéol 
As the demand for reviewing grows, so must the pool of reviewers. As the [http://www.livecongress.it/aol/indexSA.php?id=E2EAED7D&ticket= survey] presented by Graham Neubig at the 2019 ACL showed, a considerable number of reviewers are junior researchers, who might lack the experience and expertise necessary for high-quality reviews. Some of them might not have the environment or lack opportunities that allow them to learn the skills necessary. A tutorial on reviewing for the NLP community might increase reviewers’ confidence, as well as the quality of the reviews. This introductory tutorial will cover the goals, processes, and evaluation of reviewing research papers in natural language processing.

'''T4: Stylized Text Generation: Approaches and Applications''' (cutting-edge) 
Lili Mou and Olga Vechtomova 
Text generation has played an important role in various applications of natural language processing (NLP), and kn recent studies, researchers are paying increasing attention to modeling and manipulating the style of the generation text, which we call stylized text generation. In this tutorial, we will provide a comprehensive literature review in this direction. We start from the definition of style and different settings of stylized text generation, illustrated with various applications. Then, we present different settings of stylized generation, such as parallel supervised, style label-supervised, and unsupervised. In each setting, we delve deep into machine learning methods, including embedding learning techniques to represent style}, adversarial learning and reinforcement learning with cycle consistency to match content but to distinguish different styles. We also introduce current approaches of evaluating stylized text generation systems. We conclude our tutorial by presenting the challenges of stylized text generation and discussing future directions, such as small-data training, non-categorical style modeling, and a generalized scope of style transfer (e.g., controlling the syntax as a style).

'''Afternoon Tutorials'''

'''T5: Achieving Common Ground in Multi-modal Dialogue''' (cutting-edge) 
Malihe Alikhani and Matthew Stone 
All communication aims at achieving common ground (grounding): interlocutors can work together effectively only with mutual beliefs about what the state of the world is, about what their goals are, and about how they plan to make their goals a reality. Computational dialogue research offers some classic results on grouding, which unfortunately offer scant guidance to the design of grounding modules and behaviors in cutting-edge systems. In this tutorial, we focus on three main topic areas: 1) grounding in human-human communication; 2) grounding in dialogue systems; and 3) grounding in multi-modal interactive systems, including image-oriented conversations and human-robot interactions. We highlight a number of achievements of recent computational research in coordinating complex content, show how these results lead to rich and challenging opportunities for doing grounding in more flexible and powerful ways, and canvass relevant insights from the literature on human--human conversation. We expect that the tutorial will be of interest to researchers in dialogue systems, computational semantics and cognitive modeling, and hope that it will catalyze research and system building that more directly explores the creative, strategic ways conversational agents might be able to seek and offer evidence about their understanding of their interlocutors.

'''T6: Commonsense Reasoning for Natural Language Processing''' (introductory) 
Maarten Sap, Vered Shwartz, Antoine Bosselut, Dan Roth and Yejin Choi 
In our tutorial, we (1) outline the various types of commonsense (e.g., physical, social), and (2) discuss techniques to gather and represent commonsense knowledge, while highlighting the challenges specific to this type of knowledge (e.g., reporting bias). We will then (3) discuss the types of commonsense knowledge captured by modern NLP systems (e.g., large pretrained language models), and (4) present ways to measure systems' commonsense reasoning abilities. We finish with (5) a discussion of various ways in which commonsense reasoning can be used to improve performance on NLP tasks, exemplified by an (6) interactive session on integrating commonsense into a downstream task.

'''T7: Integrating Ethics into the NLP Curriculum''' (introductory) 
Emily M. Bender, Dirk Hovy and Alexandra Schofield 
Our goal in this tutorial is to empower NLP researchers and practitioners with tools and resources to teach others about how to ethically apply NLP techniques. Our tutorial will present both high-level strategies for developing an ethics-oriented curriculum, based on experience and best practices, as well as specific sample exercises that can be brought to a classroom. We plan to make this a highly interactive work session culminating in a shared online resource page that pools lesson plans, assignments, exercise ideas, reading suggestions, and ideas from the attendees. We consider three primary topics with our session that frequently underlie ethical issues in NLP research: Dual use, bias and privacy. 
In this setting, a key lesson is that there is no single approach to ethical NLP: each project requires thoughtful consideration about what steps can be taken to best support people affected by that project. However, we can learn (and teach) what kinds of issues to be aware of and what kinds of strategies are available for mitigating harm. To teach this process, we apply and promote interactive exercises that provide an opportunity to ideate, discuss, and reflect. We plan to facilitate this in a way that encourages positive discussion, emphasizing the creation of ideas for the future instead of negative opinions of previous work.

'''T8: Recent Advances in Open-Domain Question Answering''' (cutting-edge) 
Danqi Chen and Scott Wen-tau Yih 
Open-domain (textual) question answering (QA), the task of finding answers to open-domain questions by searching a large collection of documents, has been a long-standing problem in NLP, information retrieval (IR) and related fields (Voorhees et al., 1999; Moldovan et al., 2000; Brill et al.,2002; Ferrucci et al., 2010). Traditional QA systems were usually constructed as a pipeline, consisting of many different components such as question processing, document/passage retrieval and answer processing. With the rapid development of neural reading comprehension (Chen, 2018), modern open-domain QA systems have been restructured by combining traditional IR techniques and neural reading comprehension models (Chen et al., 2017; Yang et al., 2019) or even implemented in a fully end-to-end fashion (Lee et al., 2019; Seo et al., 2019). While the system architecture has been drastically simplified, two technical challenges remain critical:(1) “Retriever”: finding documents that (might)contain an answer from a large collection of documents; (2) “Reader”: finding the answer in a given paragraph or a document. 
In this tutorial, we aim to provide a comprehensive and coherent overview of recent advances in this line of research. We will start by first giving a brief historical background of open-domain question answering, discussing the basic setup and core technical challenges of the research problem.The focus will then shift to modern techniques and resources proposed for open-domain QA, including the basics of latest neural reading comprehension systems, new datasets and models. The scope will also be broadened to cover the information retrieval component on how to effectively identify passages relevant to the questions. Moreover, in-depth discussions will be given on the use of traditional / neural IR modules, as well as the trade-offs between modular design and end-to-end training. If time permits, we also plan to discuss some hybrid approaches for answering questions using both text and large knowledge bases (e.g. (Sun et al., 2018)) and give a critical review on how structured data complements the information from unstructured text. 
At the end of our tutorial, we will discuss some important questions, including (1) How much progress have we made compared to the QA systems developed in the last decade?(2) What are the main challenges and limitations of cur-rent approaches? (3) How to trade off the efficiency (computational time and memory requirements) and accuracy in the deep learning era? We hope that our tutorial will not only serve as a useful resource for the audience to efficiently acquire the up-to-date knowledge, but also provide new perspectives to stimulate the advances of open-domain QA research in the next phase.

== Workshop Chairs ==

Milica Gašić, Heinrich Heine University Düsseldorf

Dilek Hakkani-Tur, Amazon Alexa AI

Saif M. Mohammad, National Research Council Canada

Ves Stoyanov, Facebook AI

== Student Research Workshop Chairs and Faculty Advisors==

Rotem Dror, Technion - Israel Institute of Technology

Jiangming Liu, The University of Edinburgh

Shruti Rijhwani, Carnegie Mellon University

Omri Abend, Hebrew University of Jerusalem

Sujian Li, Peking University

Zhou Yu, University of California, Davis

== Audio-Video Chairs ==

Hamid Palangi, Microsoft Research, Redmond

Lianhui Qin, University of Washington

== Conference Handbook Chair ==

Nanyun Peng, University of Southern California

== Demo Chairs ==

Asli Celikyilmaz, Microsoft Research, Redmond

Shawn Wen, PolyAI

== Diversity & Inclusion (D&I) Chairs ==

Cecilia Ovesdotter Alm, Rochester Institute of Technology

Vinodkumar Prabhakaran, Google

== Local Sponsorship Chairs ==

Hoifung Poon, Microsoft

Kristina Toutanova, Google

== Publication Chairs ==

Steven Bethard, University of Arizona

Ryan Cotterrell, University of Cambridge

Rui Yan, Peking University

Starting from the style files from ACL 2019, we have produced new LaTeX style files for ACL 2020. Most of the description was retained, but the order of sections was overhauled to make sure that important information wasn't scattered so haphazardly across the document. Other improvements were also made, like using the recommended citation style consistently throughout the LaTeX source, and separating out all the LaTeX-specific stuff into clearly marked sections. The MS Word version was derived from these LaTeX versions to match as closely as possible. The LaTeX version was also posted to the Overleaf gallery. The most recent .bib file for the entire ACL Anthology was included in the style file distribution to encourage authors to use the official citations for ACL Anthology publications. All style file changes were merged into https://github.com/acl-org/acl-pub/tree/gh-pages/paper_styles.

== Publicity Chair ==

Emily M. Bender, University of Washington

=== Dissemination ===

Durable accounts for the ACL meeting on Twitter and Facebook have been created:
* https://twitter.com/aclmeeting
* https://www.facebook.com/aclmeeting/

These will be passed along to the ACL 2021 publicity chair(s) so that they don't have to build up followers separately. As of Feb 4, 2020 the Twitter account has 4,061 followers and the Facebook account has 181. We have not yet been making use of the Instagram account, but we have been using the Twitter and Facebook accounts to publicize important dates as well as blog posts. The Twitter account especially has been useful for fielding questions from the community. Calls for papers have also gone out over the ACL member portal and several mailing lists, as well as websites such as WikiCFP. (These are maintained in a spreadsheet which can be handed off to the ACL 2021 publicity chair(s)).

=== Next Steps ===

* Recruit co-chairs, especially to coordinate live-tweeting of the conference
* Contact local media for coverage
* Develop land acknowledgement in consultation with the Duwamish Tribe (on whose land the meeting will take place). The Duwamish publish this information about land acknowledgments: https://www.duwamishtribe.org/land-acknowledgement

== Remote Presentation Chairs ==

Hao Fang, Microsoft Semantic Machines

Yi Luan, Google AI Language

== Sustainability Chairs ==

Ananya Ganesh, Educational Testing Service

Klaus Zechner, Educational Testing Service

Our main goal for this new focus area is to engage the ACL community in discussions about how best to reduce the carbon footprint of future ACL conferences in order to contribute to sustainable and livable conditions on this planet.
One of the main directions we are currently envisioning is to encourage and support conference attendees in virtual participation using live streaming of conference events as air travel is the main contributor to the carbon footprint of international conferences.

== Website & Conference App Chairs ==

Sudha Rao, Microsoft Research, Redmond

Yizhe Zhang, Microsoft Research, Redmond

== Business Office ==

Priscilla Rasmussen, ACL

Past tutorials

2020-02-18T13:42:47Z

AgataSavary: /* 2020 tutorials */

This page belongs to the [[tutorial chair handbook]]. It summarizes data on tutorials which took place at some recent ACL, EACL, NAACL, EMNLP and COLING conferences.

==2020 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Interpretability and Analysis in Neural NLP
|Yonatan Belinkov, Sebastian Gehrmann and Ellie Pavlick
|ACL 2020
|[https://acl2020.org/program/tutorials/]
|
|-
|Multi-modal Information Extraction from Text, Semi-structured, and Tabular Data on the Web
|Xin Luna Dong, Hannaneh Hajishirzi, Colin Lockard and Prashant Shiralkar
|ACL 2020
|[https://acl2020.org/program/tutorials/]
|
|-
|Reviewing Natural Language Processing Research
|Kevin Cohen, Karën Fort, Margot Mieskes and Aurélie Névéol
|ACL 2020
|[https://acl2020.org/program/tutorials/]
|
|-
|Stylized Text Generation: Approaches and Applications
|Lili Mou and Olga Vechtomova
|ACL 2020
|[https://acl2020.org/program/tutorials/]
|
|-
|Achieving Common Ground in Multi-modal Dialogue
|Malihe Alikhani and Matthew Stone
|ACL 2020
|[https://acl2020.org/program/tutorials/]
|
|-
|Commonsense Reasoning for Natural Language Processing
|Maarten Sap, Vered Shwartz, Antoine Bosselut, Dan Roth and Yejin Choi
|ACL 2020
|[https://acl2020.org/program/tutorials/]
|
|-
|Integrating Ethics into the NLP Curriculum
|Emily M. Bender, Dirk Hovy and Alexandra Schofield
|ACL 2020
|[https://acl2020.org/program/tutorials/]
|
|-
|Recent Advances in Open-Domain Question Answering
|Danqi Chen and Scott Wen-tau Yih
|ACL 2020
|[https://acl2020.org/program/tutorials/]
|
|-
|TO BE COMPLETED
|
|
|
|}

==2019 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Latent Structure Models for Natural Language Processing
|André F. T. Martins, Tsvetomila Mihaylova, Nikita Nangia and Vlad Niculae
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T1]
|[https://www.aclweb.org/anthology/P19-4001/]
|-
|Graph-Based Meaning Representations: Design and Processing
|Alexander Koller, Stephan Oepen and Weiwei Sun
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T2]
|[https://www.aclweb.org/anthology/P19-4002/]
|-
|Discourse Analysis and Its Applications
|Shafiq Joty, Giuseppe Carenini, Raymond Ng and Gabriel Murray
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T3]
|[https://www.aclweb.org/anthology/P19-4003/]
|-
|Computational Analysis of Political Texts: Bridging Research Efforts Across Communities
|Goran Glavaš, Federico Nanni and Simone Paolo Ponzetto
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T4]
|[https://www.aclweb.org/anthology/P19-4004/]
|-
|Wikipedia as a Resource for Text Analysis and Retrieval
|Marius Pasca
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T5]
|[https://www.aclweb.org/anthology/P19-4005/]
|-
|Deep Bayesian Natural Language Processing
|Jen-Tzung Chien
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T6]
|[https://www.aclweb.org/anthology/P19-4006/]
|-
|Unsupervised Cross-Lingual Representation Learning
|Sebastian Ruder, Anders Søgaard and Ivan Vulić
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T7]
|[https://www.aclweb.org/anthology/P19-4007/]
|-
|Advances in Argument Mining
|Katarzyna Budzynska and Chris Reed
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T8]
|[https://www.aclweb.org/anthology/P19-4008/]
|-
|Storytelling from Structured Data and Knowledge Graphs : An NLG Perspective
|Abhijit Mishra, Anirban Laha, Karthik Sankaranarayanan, Parag Jain and Saravanan Krishnan
|ACL 2019
|[http://www.acl2019.org/EN/tutorials.xhtml#T9]
|[https://www.aclweb.org/anthology/P19-4009/]
|-
|Deep Adversarial Learning for NLP
|William Yang Wang, Sameer Singh and Jiwei Li
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t1-deep-adversarial-learning-for-nlp]
|[https://aclweb.org/anthology/N19-5001/]
|-
|Deep Learning for Natural Language Inference
|Samuel Bowman and Xiaodan Zhu
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t2-deep-learning-for-natural-language-inference]
|[https://aclweb.org/anthology/N19-5002/]
|-
|Measuring and Modeling Language Change
|Jacob Eisenstein
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t3-measuring-and-modeling-language-change]
|[https://aclweb.org/anthology/N19-5003/]
|-
|Transfer Learning in Natural Language Processing
|Sebastian Ruder, Matthew Peters, Swabha Swayamdipta and Thomas Wolf
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t4-transfer-learning-in-natural-language-processing]
|[https://aclweb.org/anthology/N19-5004/]
|-
|Language Learning and Processing in People and Machines
|Aida Nematzadeh, Richard Futrell and Roger Levy
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t5-language-learning-and-processing-in-people-and-machines]
|[https://aclweb.org/anthology/N19-5005/]
|-
|Applications of Natural Language Processing in Clinical Research and Practice
|Yanshan Wang, Ahmad Tafti, Sunghwan Sohn and Rui Zhang
|NAACL 2019
|[https://naacl2019.org/program/tutorials/#t6-applications-of-natural-language-processing-in-clinical-research-and-practice]
|[https://aclweb.org/anthology/N19-5006/]
|}

==2018 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Joint models for NLP
|Yue Zhang
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Graph Formalisms for Meaning Representations
|Adam Lopez and Sorcha Gilroy
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Writing Code for NLP Research
|Matt Gardner, Mark Neumann, Joel Grus, and Nicholas Lourie
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Deep Latent Variable Models of Natural Language
|Alexander Rush, Yoon Kim, and Sam Wiseman
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Standardized Tests as benchmarks for Artificial Intelligence
|Mrinmaya Sachan, Minjoon Seo, Hannaneh Hajishirzi, and Eric Xing
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|Deep Chit-Chat: Deep Learning for ChatBots
|Wei Wu and Rui Yan
|EMNLP 2018
|[https://emnlp2018.org/program/tutorials/]
|
|-
|100 Things You Always Wanted to Know about Semantics & Pragmatics But Were Afraid to Ask
|Emily M. Bender
|ACL 2018
|[https://acl2018.org/tutorials/#100-things-you-always-wante]
|[https://aclweb.org/anthology/P18-5001/]
|-
|Neural Approaches to Conversational AI
|Jianfeng Gao, Michel Galley and Lihong Li
|ACL 2018
|[https://acl2018.org/tutorials/#neural-approaches-to-conver]
|[https://aclweb.org/anthology/P18-5002/]
|-
|Variational Inference and Deep Generative Models
|Wilker Aziz and Philip Schulz
|ACL 2018
|[https://acl2018.org/tutorials/#variational-inference-and-d]
|[https://aclweb.org/anthology/P18-5003/]
|-
|Connecting Language and Vision to Actions
|Peter Anderson, Abhishek Das and Qi Wu
|ACL 2018
|[https://acl2018.org/tutorials/#connecting-language-and-vis]
|[https://aclweb.org/anthology/P18-5004/]
|-
|Beyond Multiword Expressions: Processing Idioms and Metaphors
|Valia Kordoni
|ACL 2018
|[https://acl2018.org/tutorials/#beyond-multiword-expression]
|[https://aclweb.org/anthology/P18-5005/]
|-
|Neural Semantic Parsing
|Luke Zettlemoyer, Matt Gardner, Pradeep Dasigi, Srinivasan Iyer and Alane Suhr
|ACL 2018
|[https://acl2018.org/tutorials/#neural-semantic-parsing]
|[https://aclweb.org/anthology/P18-5006/]
|-
|Deep Reinforcement Learning for NLP
|William Yang Wang, Jiwei Li and Xiaodong He
|ACL 2018
|[https://acl2018.org/tutorials/#deep-reinforcement-learning]
|[https://aclweb.org/anthology/P18-5007/]
|-
|Multi-lingual Entity Discovery and Linking
|Avirup Sil, Heng Ji, Dan Roth and Silviu-Petru Cucerzan
|ACL 2018
|[https://acl2018.org/tutorials/#multi-lingual-entity-discov]
|[https://aclweb.org/anthology/P18-5008/]
|-
|Modelling Natural Language, Programs, and their Intersection
|Graham Neubig and Miltiadis Allamanis
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6001/]
|-
|Deep Learning Approaches to Text Production
|Claire Gardent and Shashi Narayan
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6002/]
|-
|Scalable Construction and Reasoning of Massive Knowledge Bases
|Xiang Ren, Nanyun Peng and William Yang Wang
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6003/]
|-
|The interplay between lexical resources and Natural Language Processing
|Jose Camacho-Collados, Luis Espinosa Anke and Mohammad Taher Pilehvar
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6004/]
|-
|Socially Responsible NLP
|Yulia Tsvetkov, Vinodkumar Prabhakaran and Rob Voigt
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6005/]
|-
|Deep Learning for Conversational AI
|Pei-Hao Su, Nikola Mrkšić, Iñigo Casanueva, Ivan Vulić
|NAACL 2018
|
|[https://aclweb.org/anthology/N18-6006/]
|-
|NLP for Conversations: Sentiment, Summarization, and Group Dynamics
|Gabriel Murray, Giuseppe Carenini and Shafiq Joty
|COLING 2018
|[https://sites.google.com/view/nlpforconversations]
|[https://aclweb.org/anthology/C18-3001/]
|-
|Practical Parsing for Downstream Applications
|Daniel Dakota and Sandra Kübler
|COLING 2018
|[http://cl.indiana.edu/colingParsing18/]
|[https://aclweb.org/anthology/C18-3002/]
|-
|Frame Semantics across Languages: Towards a Multilingual FrameNet
|Collin Baker, Michael Ellsworth, Miriam R L Petruck and Swabha Swayamdipta
|COLING 2018
|[https://framenet.icsi.berkeley.edu/fndrupal/node/5552/]
|[https://aclweb.org/anthology/C18-3003/]
|-
|Deep Bayesian Learning and Understanding
|Jen-Tzung Chien
|COLING 2018
|[http://chien.cm.nctu.edu.tw/home/coling/]
|[https://aclweb.org/anthology/C18-3004/]
|-
|Data-Driven Text Simplification
|Sanja Štajner and Horacio Saggion
|COLING 2018
|[http://taln.upf.edu/pages/coling2018simplification/]
|[https://aclweb.org/anthology/C18-3005/]
|-
|Deep Learning for Dialogue Systems
|Yun-Nung Chen, Asli Celikyilmaz and Dilek Hakkani-Tur
|COLING 2018
|[http://deepdialogue.miulab.tw/]
|[https://aclweb.org/anthology/C18-3006/]
|}

==2017 tutorials==

EMNLP 2017 website is no longer available. There are no traces of the AMNLP 2017 tutorials, except this [https://www.facebook.com/emnlp2017/posts/well-have-7-tutorials-at-emnlp2017-concepthier-sarcasm-graphrep-srl-memaugnn-str/1860468284191872/ Facebook post].

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Universal Dependencies
|Joakim Nivre, Daniel Zeman, Filip Ginter, and Francis Tyers
|EACL 2017
|[http://universaldependencies.org/eacl17tutorial/]
|
|-
|Practical Neural Machine Translation
|Rico Sennrich and Barry Haddow
|EACL 2017
|[http://www.statmt.org/eacl2017/practical-nmt.pdf]
|
|-
|Imitation learning for structured prediction in natural language processing
|Andreas Vlachos, Gerasimos Lampouras and Sebastian Riedel
|EACL 2017
|[https://sheffieldnlp.github.io/ImitationLearningTutorialEACL2017/]
|
|-
|Word Vector Space Specialisation
|Ivan Vulić, Nikola Mrkšić, and Mohammad Taher Pilehvar
|EACL 2017
|[http://people.ds.cam.ac.uk/iv250/tutorial/wv-tutorial.pdf]
|
|-
|Integer Linear Programming formulations in Natural Language Processing
|Dan Roth and Vivek Srikumar
|EACL 2017
|[https://sheffieldnlp.github.io/ImitationLearningTutorialEACL2017/]
|
|-
|Building Multimodal Simulations for Natural Language
|James Pustejovsky and Nikhil Krishnaswamy
|EACL 2017
|[http://www.voxicon.net]
|
|-
|Natural Language Processing for Precision Medicine
|Hoifung Poon, Chris Quirk, Kristina Toutanova, and Wen-tau Yih
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5001/]
|-
|Multimodal Machine Learning
|Louis-Philippe Morency and Tadas Baltrusaitis
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5002/]
|-
|Deep Learning for Semantic Composition
|Xiaodan Zhu and Edward Grefenstette
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5003/]
|-
|Deep Learning for Dialogue Systems
|Yun-Nung Chen, Asli Celikyilmaz, and Dilek Hakkani-Tur
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5004/]
|-
|Beyond Words: Deep Learning for Multi-word Expressions and Collocations
|Valia Kordoni
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5005/]
|-
|Making Better Use of the Crowd
|Jennifer Wortman Vaughan
|ACL 2017
|[http://acl2017.org/tutorials/]
|[https://aclweb.org/anthology/P17-5006/]
|}

==2016 tutorials==

{| border="1" cellspacing="0" cellpadding="5"
|'''Title'''
|'''Trainers'''
|'''Conference'''
|'''Conference link'''
|'''ACL Anthology link'''
|-
|Multimodal Learning and Reasoning
|Desmond Elliott, Douwe Kiela and Angeliki Lazaridou
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index4d04.html?article_id=59]
|
|-
|NLP Approaches to Computational Argumentation
|Noam Slonim, Iryna Gurevych, Chris Reed and Benno Stein
|ACL 2016
|[http://mirror.aclweb.org/acl2016/indexfc69.html?article_id=56]
|
|-
|Computer Aided Translation
|Philipp Koehn
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index6ea2.html?article_id=62]
|
|-
|Semantic Representations of Word Senses and Concepts
|José Camacho-Collados, Ignacio Iacobacci, Roberto Navigli and Mohammad Taher Pilehvar
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index6b9c.html?article_id=58]
|
|-
|Neural Machine Translation
|Thang Luong, Kyunghyun Cho and Christopher D. Manning
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index2e51.html?article_id=55]
|
|-
|Game Theory and Natural Language: Origin, Evolution and Processing
|Rocco Tripodi and Marcello Pelillo
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index56cf.html?article_id=57]
|
|-
|Understanding Short Texts
|Zhongyuan Wang and Haixun Wang
|ACL 2016
|[http://mirror.aclweb.org/acl2016/indexedf3.html?article_id=60]
|
|-
|MetaNet: Repository, Identification System, and Applications
|Miriam R L Petruck and Ellen K Dodge
|ACL 2016
|[http://mirror.aclweb.org/acl2016/index6a61.html?article_id=61]
|
|-
|English Resource Semantics
|Dan Flickinger, Emily M. Bender, and Woodley Packard
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t1.html]
|[https://aclweb.org/anthology/N16-4001/]
|-
|Multilingual Multimodal Language Processing Using Neural Networks
|Mitesh M Khapra and Sarath Chandar
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t2.html]
|[https://aclweb.org/anthology/N16-4002/]
|-
|Question Answering with Knowledge Base, Web and Beyond
|Scott Wen-tau Yih & Hao Ma
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t3.html]
|[https://aclweb.org/anthology/N16-4003/]
|-
|Recent Progress in Deep Learning for NLP
|Zhengdong Lu and Hang Li
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t4.html]
|[https://aclweb.org/anthology/N16-4004/]
|-
|Scalable Statistical Relational Learning for NLP
|William Yang Wang and William W. Cohen
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t5.html]
|[https://aclweb.org/anthology/N16-4005/]
|-
|Statistical Machine Translation between Related Languages
|Pushpak Bhattacharyya, Mitesh Khapra, and Anoop Kunchukuttan
|NAACL 2016
|[http://naacl.org/naacl-hlt-2016/t6.html]
|[https://aclweb.org/anthology/N16-4006/]
|-
|Practical Neural Networks for NLP: From Theory to Code
|Chris Dyer, Yoav Goldberg and Graham Neubig
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#practical]
|
|-
|Advanced Markov Logic Techniques for Scalable Joint Inference in NLP
|Deepak Venugopal, Vibhav Gogate and Vincent Ng
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#advanced]
|
|-
|Lifelong Machine Learning for Natural Language Processing
|Zhiyuan Chen and Bing Liu
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#lifelong]
|
|-
|Neural Networks for Sentiment Analysis
|Yue Zhang and Duy Tin Vo
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#neural]
|
|-
|Continuous Vector Spaces for Cross-language NLP Applications
|Rafael E. Banchs
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#continuous]
|
|-
|Methods and Theories for Large-scale Structured Prediction
|Xu Sun and Yansong Feng
|EMNLP 2016
|[https://www.aclweb.org/mirror/emnlp2016/tutorials.html#methods]
|
|-
|Compositional Distributional Models of Meaning
|Mehrnoosh Sadrzadeh and Dimitri Kartsaklis
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T1/]
|[https://aclweb.org/anthology/C16-3001/]
|-
|Chinese Textual Sentiment Analysis: Datasets, Resources and Tools
|Lun-Wei Ku and Wei-Fan Chen
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T2]
|[https://aclweb.org/anthology/C16-3002/]
|-
|Natural Language Processing for Intelligent Access to Scientific Information
|Horacio Saggion and Francesco Ronzano
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T3]
|[https://aclweb.org/anthology/C16-3003/]
|-
|Quality Estimation for Language Output Applications
|Carolina Scarton, Gustavo Henrique Paetzold, and Lucia Specia
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T4]
|[https://aclweb.org/anthology/C16-3004/]
|-
|Translationese: Between Human and Machine Translation
|Shuly Wintner
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T5/]
|[https://aclweb.org/anthology/C16-3005/]
|-
|Succinct Data Structures for NLP-at-Scale
|Matthias Petri and Trevor Cohn
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T6/]
|[https://aclweb.org/anthology/C16-3006/]
|-
|The Role of Wikipedia in Text Analysis and Retrieval
|Marius Pasca
|COLING 2016
|[https://coling2016.anlp.jp/tutorials/T7]
|[https://aclweb.org/anthology/C16-3007/]
|}

Author: Agata Savary, July 2019

Updates are welcome