The Twelfth Dialog System Technology Challenge (DSTC12)
======================================================================
Call for Track Proposals
======================================================================
Following on the success of the DSTC shared tasks since 2013, the DSTC organizing committees would like to invite track proposals for the 12th Dialog System Technology Challenge (DSTC12) which will be held in 2024-2025.
TOPICS OF INTEREST
------------------
The theme of DSTC12 is “LLM-Empowered Dialog Systems”. The challenge will consist of 3-5 tracks on the dialog research problems including, but not restricted to, the following topics:
Adapting LLMs for Spoken Dialogue Systems
Proposals that involve the application of current SOTA LLMs to various spoken dialog systems which require more conversational and proactive responses versus more verbose responses that are prevalent in current LLMs.
RAG for Dialog systems
Proposals for projects focusing on Retrieval Augmented Generation (RAG), for conversational interfaces. Being able to integrate external knowledge sources or tools into LLMs.
Guardrails for LLM dialog systems
Proposals are invited to explore guardrails, which ensure harmless responses in Language Model (LLM)-based conversational interfaces while still being helpful. This also involves preventing these LLMs to produce hallucinated responses.
LLM evaluation
Proposals that involve automatic and manual evaluation of different aspects of LLMs, such as instruction-following, safety, factuality, general conversational abilities, reasoning, planning, etc.
Integration of LLMs with Multi-Modal Inputs
Proposals that demonstrate innovative ways to combine text, speech, images, and other modalities with the linguistic prowess of LLMs to achieve more coherent and contextually relevant dialog systems.
Multi-lingual dialog processing
Proposals that explore novel approaches to multi-lingual dialog systems, including but not limited to the development of robust cross-lingual embeddings, transfer learning strategies for low-resource languages, and the integration of cultural and contextual nuances into dialog models, and handling code-switching scenarios.
As in prior editions of DSTC, we also welcome task proposals on end-to-end dialog systems, dialog state tracking, dialog policy learning, spoken language understanding, natural language generation for dialog systems, multi-lingual dialog processing, multi-modal dialog processing, knowledge-grounded dialog processing, dialog evaluation, any other dialog-related tasks.
PROPOSAL CONTENT
----------------
Track proposals should be up to three (3) pages of main content, including the following:
* A title and a brief description of the proposed tasks.
* A description of the main novelties or innovations of the proposal. If the track is an extension of a track at a previous DSTC, explain the novelty of the extended track.
* A brief description of the datasets, and the status of their collection (from planned to already collected).
* A brief description of the evaluation methods.
* The names, short biographies, affiliations, and full contact information of the track organizers. Biographies can be included in the appendix.
* An ethics section: discuss the risks and any societal benefits related to the task and data of the proposed track.
Track organizers are also encouraged to provide examples of input and output data in the appendix. The ethics section, references, and appendix do not count towards the 3-page limit.
SUBMISSION
----------
Track proposals should be submitted by filling out the submission form. PDF format is preferred for your proposal document.
Note: the submission form contains many questions, so please read the form and share it with your team well before the deadline. Here is the list of all the questions.
PRESENTATION
------------
The organizers of every submitted proposal are invited to present their track details at the DSTC12. Presentation is not a requirement, and we will try to accommodate remote presentation (e.g., via Zoom or recorded video).
IMPORTANT DATES
---------------
* Track Proposal Submission Deadline: 5/4/2024
* Track Acceptance Notification: 6/7/2024
ORGANIZING COMMITTEE
--------------------
General Chair: Behnam Hedayatnia (Amazon)
Workshop Chairs: Yun-Nung (Vivian) Chen (NTU), Chen Zhang (NUS)
Publication Chair: Raghav Gupta (Google)
Publicity Chair: Michel Galley (Microsoft)
CONTACT
-------
Please feel free to post any question or comment to the DSTC mailing list list [at] dstc.community.
To join the mailing list: visit https://groups.google.com/a/dstc.community/forum/#!forum/list/join.