The 12th Workshop on the Innovative Use of NLP for Building Educational Applications (BEA12)

Event Notification Type: 
Call for Papers
Abbreviated Title: 
BEA12
Location: 
(co-located with EMNLP)
Friday, 8 September 2017
State: 
Country: 
Denmark
City: 
Copenhagen
Contact: 
Joel Tetreault
Submission Deadline: 
Sunday, 11 June 2017

SECOND CALL FOR PAPERS

The 12th Workshop on the Innovative Use of NLP for Building Educational Applications (BEA12)

Copenhagen, Denmark; September 08, 2017

(co-located with EMNLP)

https://www.cs.rochester.edu/~tetreaul/emnlp-bea12.html

*Submission Deadline: June 11, 2017*

WORKSHOP DESCRIPTION

The BEA Workshop, one of the largest one-day workshops in the ACL community, is a leading venue for NLP innovation for educational applications. The workshop’s continuous growth illustrates an alignment between societal need and technological advances. NLP capabilities now support an array of learning domains, including writing, speaking, reading, science, and mathematics, as well as the related intra- (e.g., self-confidence) and inter-personal (e.g., collaboration) domains that support achievement in those domains. Automated writing evaluation (AWE) and speech scoring applications are commercially deployed in high-stakes assessment and instructional contexts, including primary and secondary educational settings. Commercially-deployed plagiarism detection is also prevalent. The current educational and assessment landscape in K-12, higher education, and adult learning (in academic and workplace settings) fosters a strong interest in technologies that yield analytics to support proficiency measures for complex constructs. For writing, there is a focus on innovation that supports writing tasks requiring source use, argumentative discourse, and factual content accuracy. For speech, there is an interest in advancing automated scoring to include the evaluation of discourse and content features in responses to spoken assessments. General advances in speech technology have promoted a renewed interest in spoken dialog and multimodal systems for instruction and assessment in, for example, workplace interviews and simulated teaching environments. The explosive growth of mobile applications for game-based and simulation applications for instruction and assessment is another place where NLP has begun to play a large role, especially in language learning.

NLP for educational applications has gained visibility outside of the NLP community. First, the Hewlett Foundation reached out to public and private sectors and sponsored two competitions – one for automated essay scoring, and another for scoring of short response items – to engage the larger scientific community in this enterprise. Subsequently, EdX, a non-profit enterprise founded by Harvard and MIT, announced the release of software designed to automatically grade text using AI technologies. Learning@Scale, a relatively new venue for NLP research in education, promotes interdisciplinary research on learning and teaching. Massive Open Online Courses (MOOCs) now incorporate AWE systems to manage the thousands of assignments that may be received during a single MOOC course. MOOCs for Refugees have more recently popped up in response to current social situations. Courses include language learning, and AWE and other NLP capabilities could support the coursework.

Research into education-related problems has also continued to grow within the NLP community. The last five years saw eight shared tasks (including five on grammatical error correction), two workshops on NLP and Computer-Assisted Language Learning (TEA series), and two sessions of the ACL devoted exclusively to the role of NLP in assisting language learners. Three of the shared tasks were sponsored by BEA. At ACL 2016, there were over 12 papers devoted to NLP in education, the highest number ever at a conference of this caliber. All of this has served to increase the visibility of, and interest in, the field of education applications in NLP.

The 12th workshop will have oral presentation sessions and a large poster session in order to maximize the amount of original work presented (the workshop normally has 50+ attendees). We will invite both full papers and short papers on topics including: automated scoring of textual and spoken responses, intelligent tutoring, peer review, grammatical error detection/correction, learner cognition, spoken dialog, multimodal applications, tools for teachers and test developers, and use of corpora. Research that incorporates NLP for use with mobile and game-based platforms will be of special interest. Specific topics include:

* Automated scoring/evaluation for written student responses
o Content analysis for scoring/assessment
o Analysis of the structure of argumentation
o Grammatical error detection and correction
o Discourse and stylistic analysis
o Plagiarism detection
o Machine translation for assessment, instruction and curriculum development
o Detection of non-literal language (e.g., metaphor)
o Sentiment analysis
o Non-traditional genres (beyond essay scoring)

* Intelligent Tutoring (IT) and Game-based assessment that incorporates NLP
o Game-based learning
o Dialogue systems in education
o Hypothesis formation and testing
o Multi-modal communication between students and computers
o Generation of tutorial responses
o Knowledge representation in learning systems
o Concept visualization in learning systems

* Learner cognition
o Assessment of learners' language and cognitive skill levels
o Systems that detect and adapt to learners' cognitive or emotional states
o Tools for learners with special needs

* Use of corpora in educational tools
o Data mining of learner and other corpora for tool building
o Annotation standards and schemas / annotator agreement

* Tools and applications for classroom teachers and/or test developers

* NLP tools for second and foreign language learners
o Semantic-based access to instructional materials to identify appropriate texts
o Tools that automatically generate test questions
o Processing of and access to lecture materials across topics and genres
o Adaptation of instructional text to individual learners' grade levels
o Tools for text-based curriculum development
o E-learning tools for personalized course content
o Language-based educational games

* Descriptions and proposals for shared tasks

* Retrospective or survey papers on a particular NLP/Edu topic or field

* Vision papers about ideas discussing how the field should develop

NLI SHARED TASK 2017

The workshop will also host a Shared Task on Native Language Identification (NLI). NLI is the process of automatically identify the native language (L1) of a non-native speaker based solely on language that he or she produces in another language. Two previous shared tasks on NLI have been organized in which the task was to identify the native language of non-native speakers of English-based on essays and spoken responses they provided during a standardized assessment of academic English proficiency. The first shared task was based on the essays only and was also held with the BEA workshop in 2013. It was a total success with 29 teams competing, making it one of the largest shared tasks that year. Three years later, Computational Paralinguistics Challenge at Interspeech 2016 hosted a sub-challenge on identifying the native language based solely on the spoken responses.

This year's shared task combines the inputs from the two previous tasks. There will be three tracks: NLI on the essay only, NLI on the speech response only, and NLI using both responses from a test taker. We feel this will make for a more challenging shared task while building on the methods and results from the previous two shared tasks. The training and development data for the shared task was just released.

The organizing committee for the shared task is: Aoife Cahill (Educational Testing Service), Keelan Evanini (Educational Testing Service), Shervin Malmasi (Harvard Medical School), Joel Tetreault (Grammarly).

To register and participate in the shared task, please visit the NLI website:
https://sites.google.com/site/nlisharedtask/home

SUBMISSION INFORMATION

We will be using the ACL Submission Guidelines for the BEA12 Workshop this year. Authors are invited to submit a full paper of up to 8 pages of content with up to 2 additional pages for references. We also invite short papers of up to 4 pages of content, including 2 additional pages for references. Final camera ready versions of accepted papers will be given an additional page of content to address reviewer comments.

Papers which describe systems are also invited to give a demo of their system. If you would like to present a demo in addition to presenting the paper, please make sure to select either "full paper + demo" or "short paper + demo" under "Submission Category" in the START submission page.

Previously published papers cannot be accepted. The submissions will be reviewed by the program committee. As reviewing will be blind, please ensure that papers are anonymous. Self-references that reveal the author's identity, e.g., "We previously showed (Smith, 1991) ...", should be avoided. Instead, use citations such as "Smith previously showed (Smith, 1991) ...".

We have also included conflict of interest in the submission form. You should mark all potential reviewers who have been authors on the paper, are from the same research group or institution, or who have seen versions of this paper or discussed it with you.

We will be using the START conference system to manage submissions.

IMPORTANT DATES

Submission Deadline: Sunday, June 11 - 23:59 EST
Notification of Acceptance: Friday, June 30
Camera-ready Papers Due: Friday, July 14
Workshop: September 07 or 08

ORGANIZING COMMITTEE

* Joel Tetreault, Grammarly (primary contact)
* Jill Burstein, Educational Testing Service
* Claudia Leacock, Consultant
* Helen Yannakoudakis, University of Cambridge