shared task

BanglaKnight: Shared Task on Adversarial Bangla Prompt Classification at BLP 2027 (NAACL 2027): Call for Participation

Abbreviated Title: 
BanglaKnight
ACL-Sponsored Event: 
No
Event Notification Type: 
Call for Participation
Location: 
BLP 2027 Workshop at NAACL 2027
File attachments: 
AttachmentSize
PDF icon BanglaKnight poster (PDF)921.79 KB
Event Dates: 
Tuesday, 1 June 2027 to Saturday, 5 June 2027
Submission Deadline: 
Sunday, 24 January 2027
Contact: 
BLP 2027 organizers (questions via the CodaBench forum)
Contact Email: 
City: 
San Francisco
State: 
California
Country: 
USA
Description/Details: 

BanglaKnight is Shared Task 1 at the Third Workshop on Bangla Language Processing (BLP 2027), co-located with NAACL 2027 in San Francisco, June 1-5, 2027.
Website: https://banglaknight.github.io/Register on CodaBench: https://www.codabench.org/competitions/18411/
The task
Safety classifiers that screen requests before an LLM answers are mostly trained on English. Bangla is often typed in Latin letters or mixed with English, Hindi or Urdu, and a harmful request can get past a filter when it is spelled differently.
BanglaKnight asks participants to label each Bangla prompt with one of four risk levels: 0 Safe, 1 Offensive, 2 Harmful, 3 Severe. Every prompt appears in three written forms (Bangla script, romanized Bangla and code-mixed text) with the same label, and a system should give the same answer for all three. Systems are ranked by Defense Score: the mean macro-F1 over the three forms, multiplied by (1 - the share of unsafe prompts labeled Safe).
Available now

  • Training and development data
  • A blind dev-test set for practice submissions on CodaBench
  • A starter notebook with a BanglaBERT baseline that runs in about 8 minutes on a free Colab GPU

Important dates (all deadlines 11:59 PM AoE)

  • Development phase: now until January 7, 2027
  • Test phase: January 8-24, 2027
  • Registration deadline: January 17, 2027
  • Official results: February 3, 2027
  • System papers (up to 4 pages) due: February 15, 2027
  • Workshop: June 1-5, 2027, at NAACL 2027 in San Francisco

Individuals and teams of up to 5 are welcome. Please send questions to the CodaBench forum.
On behalf of the BLP 2027 organizers (https://blp-workshop.github.io/)

Field Matters 2027 Shared Task on Automatic Interlinear Glossing

ACL-Sponsored Event: 
No
Event Notification Type: 
Call for Participation
Location: 
EACL 2027
City: 
Athens
Country: 
Greece
Description/Details: 

## Important links ##
- Shared Task webpage: https://field-matters.github.io/sharedtask2027
- Announcements and general contact (Google group): https://groups.google.com/g/fm-shared-task-glossing-2027
- GitHub repository: https://github.com/field-matters/ST2027

## Description of the task ##

The Field Matters 2027 workshop at EACL 2027 (https://field-matters.github.io) offers a shared task on automatic interlinear glossing for field linguistic annotations.

1 Zbz sc tqwaipele tqnginipenge kraesz'mz
2 zbz sc tq-wai-pe-le tq-ngini-pe-nge kraesz'mz
3 so.then PFV RL-do-COS-3MINIA RL-become-COS-1MINII sixth
4 'So then that's how I became the sixth.' (Natugu)

Figure 1: An Interlinear Glossed Text (IGT) for a sentence in Natugu.
1: Source sentence in Natugu; 2: Morphological segmentation; 3: Interlinear gloss; 4: Translation in English.

The task is to predict interlinear glosses, which are morpheme-level linguistic annotations, as shown in the third line of the figure above, notably used in field linguistics.
Ultimately, such glossing systems aim to help field linguists with the annotation task for languages under documentation, as it is mainly performed manually with little automation.

This Shared Task follows the SIGMORPHON 2023 Shared Task on Interlinear Glossing (https://github.com/sigmorphon/2023GlossingST).
In our Shared Task, the input is mainly the unsegmented source sentence (first line). Some instances will be accompanied by their translation in a high-resource language (usually, in English; fourth line).
The goal is to predict the glossed sentence (third line).

## Languages ##

The task will cover the following languages:
- [Phase 1] 4 from the previous SIGMORPHON 2023 Shared Task on Interlinear Glossing: Gitksan (git), Lezgi (lez), Natugu (ntu), and Nyangbo (nyb).
- [Phase 1] + 1 language from the Glossed Audio Corpus of Ainu Folklore (https://ainu.ninjal.ac.jp/folklore/en/): Hokkaido Ainu (ain).
- [Phase 2]: TBA on 03 November 2026 (cf. website)

## Summary of the shared task rules ##

Submissions to the shared task should follow the rules below:
- Model: Any type of model is allowed, provided that the system is based on open-source or open-weight models (if LLM-based).
Closed-source models can be used for other purposes (e.g., summarising a grammar book, generating pseudo-training data).

- Dataset: Regarding the glossing dataset, we consider the GlossLM/PolyGloss dataset as the main labelled dataset to use. Any data without gloss information (e.g., monolingual sentences, dictionaries) can be used otherwise when publicly available.

- Evaluation: We will mainly evaluate the quality of the predicted gloss. However, we encourage submissions to be able to handle morphological segmentation (second line of the example figure), since field linguists usually analyse sentences into morphemes before producing glosses.
We will mainly use the Morpheme Error Rate (MER) to evaluate the predicted gloss quality.

- Participation tracks: We consider two sets of languages:
- existing_set (from phase 1; SIGMORPHON languages + Ainu) and
- new_set (from phase 2; new languages).
Participation can be in only one of the two tracks, or in both. However, in any case, a submission should contain predictions for all languages of a set.

## How to participate ##
If you are interested in the shared task or for more details, please join our Google Group: https://groups.google.com/g/fm-shared-task-glossing-2027 .

## Important dates ##
- Phase 1: Beginning of the development phase: 26 October 2026
- Phase 2: Information about the new languages: 03 November 2026
- Phase 3: Test set release (beginning of the test phase): 17 November 2026
- Output submission deadline: 08 December 2026
- System description deadline: 15 December 2026
- Notification of acceptance: 11 January 2027
- Camera-ready deadline: 22 January 2027
- Workshop dates: 9-14 March 2027 at EACL 2027

All deadlines are 23:59 UTC-12 ('Anywhere on Earth').

## Organisers ##
Field Matters workshop organisers and dataset contributors

## Contact ##
We suggest using the Google group for general questions that may interest all participants.
For private questions, please write to Shu Okabe (shu dot okabe at tum dot de).

Second CFP: SHROOM-Visions Shared Task @UncertaiNLP (EMNLP 2026)

Abbreviated Title: 
CfP: SHROOM-Visions Shared Task
ACL-Sponsored Event: 
No
Event Notification Type: 
Call for Participation
Event Dates: 
Thursday, 29 October 2026
Submission Deadline: 
Friday, 31 July 2026
Contact: 
Raúl Vázquez
Timothee Mickus
Chuyuan Li
Contact Email: 
raul.vazquez [at] helsinki.fi
timothee.mickus [at] helsinki.fi
lisa27chuyuan [at] gmail.com
City: 
Budapest
Country: 
Hungary
Description/Details: 

*apologies for cross-postings*

TL;DR
SHROOM-visions (https://helsinki-nlp.github.io/shroom/2026) is a shared task to advance model-agnostic evaluation of hallucination detection for Vision-and-Language Models (VLMs). Participate in detecting fine-grained hallucination spans across 4 languages (Chinese, English, French, Italian). Stay informed by joining our Google group (https://groups.google.com/g/shroom-visions/) or our Slack (https://join.slack.com/t/shroom-shared-task/shared_invite/zt-2mmn4i8h2-H...)!

Full Invitation
We are excited to announce the SHROOM-visions shared task on vision-language hallucination detection (link to website: https://helsinki-nlp.github.io/shroom/2026). We invite participants to detect and classify hallucination spans in a multilingual, multimodal context, using a dataset designed for enduring evaluation.

About
As new foundational models emerge monthly, how do we create hallucination evaluations that remain relevant? Current benchmarks are often tied to the idiosyncrasies of specific LLMs/VLMs, risking quick obsolescence. This shared task builds upon the *SHROOM (https://helsinki-nlp.github.io/shroom/) series of hallucination detection tasks and datasets, venturing into vision-language multilingual hallucination-span prediction. With this shared-task we aim to advance detection methods that generalize across model generations and focus on the core phenomenon of hallucination.

We provide a dataset of 20,000 samples annotated with a fine-grained, span-level labeling scheme:
- A train set of ~15,200 samples from 5 different LVLMs.
- A closed test set of 4,800 crafted samples.
- A submission platform (https://helsinki-nlp.github.io/shroom/) to evaluate the performance of your systems.
- Balanced coverage across 4 languages: Chinese, English, French, Italian.
- Each sample annotated by 3 annotators using a four-class taxonomy: Invention, Mischaracterization, OCR Problem, Miscounting.

Participants are invited to develop systems that accurately identify and classify hallucinated text spans in image-conditioned outputs. Participants will be invited to submit system description papers, with the option to present them at the UncertaiNLP workshop (https://uncertainlp.github.io/, co-located with EMNLP 2026). All authors of paper submissions will be asked to review peers' submissions (max 2 papers per author).

Key Dates:
All deadlines are “anywhere on Earth” (23:59 UTC-12).
- Train set available by: 10.05.2026
- Submission platform open by: 20.05.2026
- Evaluation phase ends: 31.07.2026
- System description papers due: 10.08.2026 (TBC)
- Notification of acceptance: 10.09.2026 (TBC)
- Camera-ready due: 20.09.2026 (TBC)
- UncertaiNLP workshop: end of October 2026 (co-located with EMNLP)

Evaluation Metrics:
Participants’ models must be able to produce spans corresponding to hallucinations in the text, classified along five possible categories (invention, mischaracterization, OCR problems, miscounting, and other hallucinations). The evaluation will rely on two metrics, evaluating labelled and unlabelled performances separately. Rankings and submissions will be handled separately per language: you are welcome to focus on the languages of your choice!

How to Participate:
- Register: Please register your team before making a submission on https://shroom.pythonanywhere.com
- Submit results: use our platform to submit your results before 31.07.2026
- Submit your system description: system description papers should be submitted by 10.08.2026 (TBC, further details will be announced at a later date).

Want to be kept in the loop?
Join our Google group mailing list or the shared task Slack! We are also open to hosting Q&A sessions for groups interested in participating, you just need to send us an email. We look forward to your participation and to the exciting research that will emerge from this task.

Best regards,
Raúl Vázquez and Timothee Mickus
On behalf of ALL the SHROOM-Visions organizers

The 19th NII Testbeds and Community for Information Access Research

Abbreviated Title: 
NTCIR-19
ACL-Sponsored Event: 
No
Event Notification Type: 
Call for Participation
Location: 
Tokyo, Japan
Contact: 
Eugene Yang
Contact Email: 
ntc19-pcc [at] nii.ac.jp
eugene.yang [at] jhu.edu
City: 
Tokyo
Country: 
Japan
Description/Details: 

■□■───────────────────────────────────────────■□■
CALL FOR NTCIR-19 TASK PARTICIPATION   
■□■───────────────────────────────────────────■□■

NTCIR-19 (Dec 8-10, 2026, Tokyo, Japan) now calls for task participation of anyone interested in research on information access technologies and their evaluation, such as retrieval from a large amount of document collections, question answering and natural language processing. We welcome students, young researchers, professors who supervise students, researchers working for a company, and anyone who is interested in informatics.

--------------------------------------------------
■ What is NTCIR?
--------------------------------------------------
NTCIR (NII Testbeds and Community for Information Access Research) is a project aimed at creating a large-scale evaluation framework for technologies that facilitate access to desired information, understanding, and utilization from vast amounts of data. These technologies include information retrieval, question answering, summarization, text mining, and machine translation. It provides a shared common foundation for numerous researchers both domestically and internationally, enabling them to advance their respective research, conduct validation and comparative evaluations, and learn from one another within a collaborative forum. Since it started in 1998, NTCIR has been striving to realize richer information access technologies and create future value. The project operates on a cycle of 1.5 years, during which several tasks (research areas) are selected for focused activities.

--------------------------------------------------
■ NTCIR-19 Tasks
--------------------------------------------------
NTCIR-19 Program Committee has selected 14 tasks.
The overview slide of each task can be available at the
kick-off page: https://research.nii.ac.jp/ntcir/ntcir-19/kickoffcfp.html
For more details, please visit the websites of each task:
Task Overview page: https://research.nii.ac.jp/ntcir/ntcir-19/tasks.html
--------------------------------------------------
== Core Tasks ==
- AEOLLM-2: Automatic Evaluation of LLMs 2
https://huggingface.co/spaces/THUIR/AEOLLM
- FinArg-3: Argument Quality Assessment of Financial Forward-Looking Statements
https://sites.google.com/nlg.csie.ntu.edu.tw/finarg3/
- HIDDEN-RAD2: Hidden Causal Reasoning in Radiology Report Generation
https://sites.google.com/view/hidden-rad2/
- Lifelog-7: Advancing Lifelog Analytics and Retrieval at NTCIR19
https://lifelog-ntcir-project-bd9eae.gitlab.io/
- MedNLP-CALL: Medical Natural Language Processing for Emergency Call
https://sociocom.naist.jp/mednlp-call/
- ToT: Tip-of-the-Tongue
https://ntcir-tot.github.io/

== Pilot Tasks ==
- AgenticInstruction: Instruction Generation for Agentic Search
https://geniie-lab.github.io/ntcir/
- CAMEO: Composed Access to Multimodal E-commerce Objects
https://sites.google.com/view/ntcir-cameo26/
- DAGRI: Data Analytics for aGRicultural Information
https://sites.google.com/view/dagri/
- FEHU: Fact-based Event-centric Human-value Understanding
https://sites.google.com/view/ntcir19fehu
- ModelRetrieval: Pre-trained Model Retrieval
https://pre-trained-model-retrieval.github.io/
- R2C2: RAG Responses Confident and Correct?
http://sakailab.com/r2c2/
- RegCom: Multinational, Multilingual, Multi-Industry Regulatory Compliance Checking
https://sites.google.com/view/ntcir19regcom/
- SciClaimEval: Cross-modal Claim Verification in Scientific Papers
https://sciclaimeval.github.io/

--------------------------------------------------
■ How to Participate
--------------------------------------------------
1. Please carefully read "How to Participate to NTCIR-19 Task(s)":
https://research.nii.ac.jp/ntcir/ntcir-19/howto20250930.html

2. Please register at (https://forms.office.com/pages/responsepage.aspx?id=4sLBp80l1USD33EvWbAu...)
The datasets of each task will be delivered to the team after registration (the date may vary depending on the task).
--------------------------------------------------
■ Registration Due (Depend on the task)
- Apr 17, 2026
R2C2: RAG Responses Confident and Correct?
- May 12, 2026
Lifelog-7: Advancing Lifelog Analytics and Retrieval at NTCIR19
- Jun 1, 2026
AgenticInstruction: Instruction Generation for Agentic Search
DAGRI: Data Analytics for aGRicultural Information
FEHU: Fact-based Event-centric Human-value Understanding
ModelRetrieval: Pre-trained Model Retrieval
RegCom: Multinational, Multilingual, Multi-Industry Regulatory Compliance Checking
SciClaimEval: Cross-modal Claim Verification in Scientific Papers
ToT: Tip-of-the-Tongue
- Jul 1, 2026
MedNLP-CALL: Medical Natural Language Processing for Emergency Call
- Jul 10, 2026
AEOLLM-2: Automatic Evaluation of LLMs 2
CAMEO: Composed Access to Multimodal E-commerce Objects
FinArg-3: Argument Quality Assessment of Financial Forward-Looking Statements
HIDDEN-RAD2: Hidden Causal Reasoning in Radiology Report Generation

--------------------------------------------------
■ Schedule
--------------------------------------------------
(*Schedule can be different in different tasks. Please visit
webpages of each task for the details.)

Jan-Jun 2026 Dry run*
Mar-Jul 2026 Formal run*
Aug 1, 2026 Evaluation results return / Task overview release (draft)
Sep 1, 2026 Submission due of participant papers (draft)
Nov 1, 2026 Camera-ready participant paper due
Dec 8-10, 2026 NTCIR-19 Conference (Tokyo, NII)

(At the NTCIR-19 Conference, an online presentation will be available)

--------------------------------------------------
■ Questions?
--------------------------------------------------
For information regarding the task specifications etc.,
please contact the NTCIR Program Chairs: ntc19-pcc@nii.ac.jp or task organizers.

For information regarding the online registration and previous NTCIR
test collections,
please contact the NTCIR office: ntc-secretariat@nii.ac.jp

CFP: CLEF Shared Task on Person-Place Relation Extraction from Multilingual Historical Texts (HIPE 2026)

Abbreviated Title: 
CLEF-HIPE 2026
ACL-Sponsored Event: 
No
Event Notification Type: 
Call for Participation
Location: 
Contact: 
Juri Opitz
Contact Email: 
opitz.sci [at] gmail.com
City: 
State: 
Country: 
Description/Details: 

HIPE-2026 Shared Task - CALL FOR PARTICIPATION (apologies for cross-postings)

====
HIPE: Identifying Historical People, Places and other Entities.
Website: https://hipe-eval.github.io/HIPE-2026/
Tasks: Person-Location Relation Extraction from Multilingual Historical Texts.
Registration: https://clef-labs-registration.dipintra.it/ (until 23 April 2026)
Training data releases: 19 Dec 2025 (partial); 19 Jan 2026 (full)
Evaluation period: 5-7 May 2026
Workshop venue: during CLEF conference, 21-24 September 2026, Jena, Germany.
LinkedIn: @ImpressoProject / #HIPE2026 / @clef_initiative / #clef2026
====

## **"Who was where when?"**

We invite participation in the third edition of the HIPE shared task, dedicated to the extraction of person–place relations in multilingual historical documents. Building on the success of HIPE-2020 and HIPE-2022, which focused on entity recognition and linking, HIPE-2026 aims to enable finer-grained analysis of entities and support the accurate reconstruction of individuals’ geographical and temporal trajectories.

The objective of HIPE-2026 is to build systems capable of determining whether a relation holds between a person and a location (place) mentioned in a document, and classify its temporal scope. Participants are asked to develop systems that determine, for each (person, location) pair associated with a historical document, whether the text implies that the person is at that location within the document’s temporal horizon (isAt relation), or that the person was there at some earlier moment in their life (a more general At relation), or that no such link can be established.

Can large language models take up the challenges? Simple co-occurences of entity mentions in a text are not sufficient to uncover the implicit and explicit, temporally anchored relations between person and locations. Addressing this challenge requires temporal reasoning, geographical inference, and the interpretation of noisy historical texts (often with only fragmentary contextual cues) to classify person–location relations with varying degrees of certainty.

The task is designed to be tackled by generative AI systems/LLMs as well as by more traditional classification approaches.

## HIPE-2026 features three evaluation profiles:

- *Accuracy Profile*: Focusing on system performance in relation classification.
- *Efficiency Profile*: Rewarding scalable, lightweight approaches considering model size and compute cost.
- *Generalization Profile*: An unseen dataset from a different domain will be included to evaluate systems’ ability to generalise beyond the newspaper domain data.

For the accuracy and efficiency profile, training and test data originate from historical newspapers in English, German, French and Luxembourgish.

Entity pairs will be provided.

## For further information on data, tasks, and evaluation settings please refer to:

- The HIPE-2026 website: https://hipe-eval.github.io/HIPE-2026/
- The Participation Guidelines: https://doi.org/10.5281/zenodo.17800136
- The HIPE-2026-data GitHub repository: https://github.com/hipe-eval/HIPE-2026-data

## On HIPE shared tasks:

HIPE evaluation lab series is part of the ongoing efforts of the natural language processing and digital humanities communities to adapt and develop technologies to efficiently retrieving and exploring information from historical texts.

## Important dates:

- 17 Nov 2025: Lab registration opens.
- 03 Dec 2025: Release of example data.
- 19 Dec 2025: Release of partial training data.
- 19 Jan 2026: Release of final training data.
- 23 Apr 2026: Lab registration closes.
- 05 May 2026: Test data release (10:00 CEST).
- 07 May 2026: Participant run submission deadline.
- 13 May 2026: Publication of results and release of test data.
- 28 May 2026: Submission of participant notebook paper.
- 10 Jul 2026 / 31 Aug 2026: CLEF conference regular/late registration DL.
- 21 Sep 2026: CLEF 2026 Conference.

Best regards,

HIPE-2026 Shared Task Organizers
https://hipe-eval.github.io/HIPE-2026/

Call for Participation – SemEval-2026 Task 9: Detecting Multilingual, Multicultural, and Multievent Online Polarization

Abbreviated Title: 
POLAR @ SemEval-2026 Task 9
ACL-Sponsored Event: 
No
Event Notification Type: 
Call for Participation
Location: 
Contact: 
Usman Naseem
Contact Email: 
usman.naseem [at] mq.edu.au
City: 
State: 
Country: 
Description/Details: 

Greetings All,

We are delighted to announce SemEval-2026 Task 9: Detecting Multilingual, Multicultural, and Multievent Online Polarization.

Polarization refers to the sharp division of opinions into opposing groups, often accompanied by hostility and exclusion. This shared task aims to advance our understanding of how polarization manifests in text across languages, cultures, and events. Participants will develop systems to detect and interpret polarized content in contexts such as elections, conflicts, protests, and debates.

———————
Languages
———————
We provide data in 20+ languages, including: German, Spanish, English, Arabic, Hausa, Urdu, Amharic, Italian, Russian, Myanmar, Chinese, Nepali, Hindi, Telugu, Persian, Turkish, Bangla, Somali, Emakhuwa, Mozambican Portuguese, Igbo, Khmer, Odia, and Punjabi.

———————
Subtasks
———————

Subtask 1 (LIVE): Polarization Detection – Identify whether a text exhibits polarization.

Subtask 2: Polarization Type Classification – Classify polarized content into specific types (e.g., political, social, cultural).

Subtask 3: Manifestation Identification – Determine how polarization is expressed (e.g., language cues, tone, argumentative structure).

———————
Participation
———————
Website: https://polar-semeval.github.io/

Register & Submit Results (Subtask 1): https://www.codabench.org/competitions/10522/

Discord (community Q&A and discussion): https://discord.com/invite/RF9MzYwKZP

Google Group (official updates & registration): https://groups.google.com/g/polarization-semeval-2026-participants

We warmly invite the community to participate in this exciting shared task and contribute to advancing multilingual and multicultural NLP research.

Best regards,
SemEval-2026 Task 9 Organizers

1st Shared Task on Morphosyntactic Parsing

Abbreviated Title: 
MSP
ACL-Sponsored Event: 
No
Event Notification Type: 
Call for Participation
Location: 
SyntaxFest
Submission Deadline: 
Tuesday, 1 July 2025
Contact: 
Omer Goldman
Leonie Weissweiler
Contact Email: 
omer.goldman [at] gmail.com
weissweiler [at] utexas.edu
City: 
Ljubljana
State: 
Country: 
Slovenia
Description/Details: 

Dear ACL members,

We are happy to announce the 1st Shared task on Morphosyntactic Parsing, to be held jointly with Syntaxfest 2025.

Morphosyntactic parsing redefines the content-function boundary to differentiate 'morphological' from 'syntactic' elements. In our morphosyntactic data structure, content words are represented as separate nodes on a dependency graph, even if they share a whitespace-separated word, and both function words and morphemes contribute morphology-style features to characterise the nodes. This makes our format independent of wordhood decisions which are not well-defined across languages.

Interested parties are invited to join the mailing list to be involved in the competition. Participating teams will be invited to submit a paper describing their work to the workshop at SyntaxFest and to present it in a special session in the workshop.

Key Details
- Website: https://unidive.lisn.upsaclay.fr/doku.php?id=other-events:msp
- Participant Google Group: https://groups.google.com/g/msp-sharedtask-2025-participants/

Key Dates
- March 7th: training data release
- July 1st: system submission deadline
- August 26th: Workshop at SyntaxFest in Ljubljana

We look forward to your contributions and hope to see you at the workshop!

Omer Goldman, Leonie Weissweiler and Reut Tsarfaty

The 2nd Workshop on Practical LLM-assisted Data-to-Text Generation

Abbreviated Title: 
Practical D2T 2024
ACL-Sponsored Event: 
This is an ACL Sponsored Event
Event Notification Type: 
Call for Papers
Location: 
INLG'24
Event Dates: 
Monday, 23 September 2024
Submission Deadline: 
Monday, 22 July 2024
Contact: 
Simone Balloccu
Contact Email: 
d2t2024 [at] googlegroups.com
balloccu [at] ufal.mff.cuni.cz
City: 
Tokyo
State: 
Country: 
Japan
Description/Details: 

While large language models (LLMs) offer to become a viable alternative to traditional rule-based data-to-text (D2T) natural language generation (NLG), they still suffer from well-known neural model issues, such as lack of controllability and risk of producing harmful text. There are many potential solutions to this problem up for discussion.

The Practical D2T workshop at INLG 2024 aims to build a space for researchers to discuss and present innovative work on D2T systems using LLMs. Building upon the 2023 edition’s hackathon, Practical D2T 2024 opens up a broader range of activities, including a special track for neuro-symbolic D2T approaches and a shared task in D2T evaluation focused on semantic accuracy.

Workshop Topic and Content
---------------------------------------------------
Practical D2T 2024 will be a full-day in-person-only event. We welcome contributions from both original unpublished work and non-archival submissions, in the form of long (8 pages) or short (4 pages) papers, on topics including but not limited to:
- Design, implementation and evaluation of LLM-assisted D2T systems
- Cross-domain adaption of LLMs for D2T
- User perceptions and acceptance of LLM-generated text in D2T
- Bias, fairness and red-teaming issues in LLM-assisted D2T systems
- Leveraging LLMs for D2T in low-resource languages and domains
- Error analysis and debugging techniques for LLM-assisted D2T
- Human-in-the-loop approaches for improving LLM-assisted D2T
- Comparison between LLM-assisted D2T and traditional symbolic approaches

Special Track: Neuro-Symbolic D2T
---------------------------------------------------
Research is currently seeing a renewed interest in developing systems combining neural and symbolic approaches to improve explainability and reduce dependence on training data. Practical D2T 2024 will feature a special track on neuro-symbolic approaches to D2T. Submissions for papers in the special track follow the same requirements and procedure as the main workshop submissions.

Shared task: Improving Semantic Accuracy in LLM-assisted D2T
---------------------------------------------------
This year will feature a shared task on improving semantic accuracy of D2T systems. Participants will build an LLM-assisted D2T system to generate textual reports from various domains, such as weather forecasting, product descriptions or sports reports. We will provide testing data obtained from public APIs, to limit potential previous exposure to the used LLMs.

We encourage participants to focus on system robustness and objective evaluation, rather than metrics scores. Because of this, participants will receive an initial evaluation script, that they are encouraged to change/improve. All submitted system’s outputs will be evaluated against every submitted custom evaluation, and correlated with human ratings.

The system reaching the highest correlation with humans will be declared winner of the competition. Results and participants’ system descriptions will be featured in the workshop proceedings.

Important dates (all deadlines are 23:59 UTC-12)
---------------------------------------------------
- Evaluation script and data release for known domains (shared task) 24 June
- Regular paper submission (main & special track, archival & non-archival): 22 July
- Known domains system output submission & surprise domain data release: 29 July
- Surprise domain system outputs submission: 5 August
- System description submission (shared task): 12 August
- Notification of acceptance (main, special track and shared task): 19 August
- Camera-ready (main, special track and shared task): 28 August
- Workshop: 23/24 September (to be announced)

Shared task on Multilingual Grammatical Error Correction 2025

Abbreviated Title: 
MultiGEC-2025
ACL-Sponsored Event: 
No
Event Notification Type: 
Call for Participation
Location: 
Hestia Hotel Europa
Event Dates: 
Wednesday, 5 March 2025
Submission Deadline: 
Wednesday, 20 November 2024
Contact: 
Arianna Masciolini
Elena Volodina
Contact Email: 
arianna.masciolini [at] gu.se
elena.volodina [at] gu.se
City: 
Tallinn
Country: 
Estonia
Description/Details: 

We invite you to participate in the shared task on Multilingual Grammatical Error Correction, MultiGEC-2025, covering 12 languages: Czech, English, Estonian, German, Greek, Icelandic, Italian, Latvian, Russian, Slovene, Swedish and Ukrainian.

The results will be presented on March 5th 2025, at the NLP4CALL workshop, colocated with the NoDaLiDa conference to be held in Estonia, Tallinn, on 2--5 March 2025: https://spraakbanken.gu.se/en/research/themes/icall/nlp4call-workshop-se...
The publication venue for system descriptions will be the proceedings of the NLP4CALL workshop, also co-published in the ACL anthology.
Official system evaluation will be carried out on CodaLab (link comes later).

To register for/express interest in the shared task, please fill in this form (https://forms.gle/nTPfARVqy1XmqT4t6).
Note that you will be prompted to sign Terms of Use for the data at https://forms.gle/VLJ18WbwsxitEBYi7. The data access is personal, please do not forget to fill in the form.

The GitHub page for the shared task is https://github.com/spraakbanken/multigec-2025/; task description and general information are also available on https://spraakbanken.gu.se/en/compsla/multigec-2025

== TASK DESCRIPTION ==
In this shared task, your goal is to rewrite learner-written texts to make them grammatically correct or both grammatically correct and idiomatic, that is either adhering to the "minimal correction" principle or applying fluency edits.

For instance, the text

> My mother became very sad, no food. But my sister better five months later.

can be corrected minimally as

> My mother became very sad, and ate no food. But my sister felt better five months later.

or with fluency edits as

> My mother was very distressed and refused to eat. Luckily, my sister recovered five months later.

For fair evaluation of both approaches to the correction task, we will provide two evaluation metrics, one favoring minimal correction, one suited for fluency-edited output (read more under Evaluation).

We particularly encourage development of multilingual systems that can process all (or several) languages using a single model, but this is not a mandatory requirement to participate in the task.

== DATA ==
We provide training, development and test data for each of the languages. The training and development dataset splits are available through Github. Evaluation will be performed on a separate test set.
See website for more detailed information: https://github.com/spraakbanken/multigec-2025/

Note: The English data is expected a bit later.

== EVALUATION ==
During the shared task, evaluation will be based on cross-lingually applicable automatic metrics:

* reference-based:
- GLEU score
- Precision, Recall, F0.5 score

* reference-free: Scribendi score

After the shared task, we also plan on carrying out a human evaluation experiment on a subset of the submitted results.

== TIMELINE ==
* October 20, 2024 - third call for participation. Training and validation data released, CodaLab opens for team registrations
* October 30, 2024 - reminder. Validation server released online
* November 13, 2024 - test period starts
* November 29, 2024 (extended) - system submission deadline (system output)
* December 2, 2024 - results announced
* December 16, 2024 - paper submission deadline with system descriptions
* January 20, 2025 - paper reviews sent to the authors
* February 3, 2025 - camera-ready deadline
* March 5, 2025 - presentations of the systems at the NLP4CALL workshop

== PUBLICATION ==
We encourage you to submit a paper with your system description to the NLP4CALL workshop special track. We follow the same requirements for paper submissions as the NLP4CALL workshop, i.e. we use the same template and apply the same page limit. All papers will be reviewed by the organizing committee. Upon paper publication, we encourage you to share models, code, fact sheets, extra data, etc. with the community through GitHub or other repositories.

== ORGANIZERS ==
* Arianna Masciolini, University of Gothenburg, Sweden
* Andrew Caines, University of Cambridge, UK
* Orphée De Clercq, Ghent university, Belgium
* Joni Kruijsbergen, Ghent university, Belgium
* Murathan Kurfali, Stockholm University, Sweden
* Ricardo Muñoz Sánchez, University of Gothenburg, Sweden
* Elena Volodina, University of Gothenburg, Sweden
* Robert Östling, Stockholm University, Sweden

== DATA PROVIDERS ==
* Czech:
-- Alexandr Rosen, Charles University, Prague
-- Adrian Jan Zasina, Charles University, Prague
* English:
-- Andrew Caines, University of Cambridge
* Estonian:
-- Mark Fishel, University of Tartu, Estonia
-- Kais Allkivi-Metsoja, Tallinn University, Estonia
-- Kristjan Suluste, Eesti Keele Instituut, Estonia
* German:
-- Andrea Horbach, IPN / CAU Kiel, Germany
-- Josef Ruppenhofer, FernUniversität in Hagen, Germany
-- Katrin Wisniewski, Universität Leipzig
-- Torsten Zesch, FernUniversität in Hagen, Germany
* Greek:
-- Alex Tantos, Aristotle University of Thessaloniki
-- Konstantinos Tsiotskas, Aristotle University of Thessaloniki
-- Vassilis Varsamopoulos, Aristotle University of Thessaloniki
-- Pinelopi Kikilintza, Aristotle University of Thessaloniki
-- Elena Drakonaki, Aristotle University of Thessaloniki
-- Eleni Tsourilla, Aristotle University of Thessaloniki
-- Despoina-Ourania Touriki, Aristotle University of Thessaloniki
* Icelandic:
-- Isidora Glisič, University of Iceland
* Italian:
-- Jennifer-Carmen Frey, Eurac Research Bolzano, Italy
-- Lionel Nicolas, Eurac Research Bolzano, Italy
* Latvian:
-- Roberts Darģis, University of Latvia
-- Ilze Auzina, University of Latvia
* Russian:
-- Alla Rozovskaya, City University of New York (CUNY), USA
* Slovene:
-- Špela Arhar Holdt, University of Ljubljana, Slovenia
-- Aleš Žagar, University of Ljubljana, Slovenia
* Swedish:
-- Arianna Masciolini, University of Gothenburg, Sweden
* Ukrainian:
-- Oleksiy Syvokon, Microsoft
-- Mariana Romanyshyn, Grammarly

== CONTACT ==
Please join the MultiGEC-2025 Google group (https://groups.google.com/g/multigec-2025) in order to ask questions, hold discussions and browse for already answered questions.

Generation Challenges 2024

Abbreviated Title: 
GenChal@INLG 2024
ACL-Sponsored Event: 
This is an ACL Sponsored Event
Event Notification Type: 
Call for Proposals
Location: 
INLG'24
Event Dates: 
Monday, 23 September 2024 to Friday, 27 September 2024
Submission Deadline: 
Monday, 24 June 2024
Contact: 
Miruna Clinciu
Simon Mille
Contact Email: 
mc191 [at] hw.ac.uk
simon.mille [at] adaptcentre.ie
City: 
Tokyo
State: 
Country: 
Japan
Description/Details: 

We invite submissions of papers describing ideas for future shared tasks in the general area of language generation (Generation Challenges 2024). Proposed tasks can be in the area of core NLG, or in other research areas in which language is generated. Examples include, but are not limited to: data-to-text NLG, text-to-text generation (including MT and summarisation), combining core NLG and MT, combining core NLG and text summarisation, NLG quality estimation, NLG evaluation metrics, and/or generating language from heterogeneous data, including image and video.

The Generation Challenges (GenChal) are an umbrella event designed to bring together a variety of shared-task efforts that involve the generation of natural language. This year, Generation Challenges will be held as a workshop at the 17th International Conference on Natural Language Generation, scheduled on September 23-27 2024. The workshop will follow the format of previous GenChal results sessions, with presentations of results by the organisers of the generation challenges that are currently running, a poster session for task participants to present their submissions, as well as presentations of proposals for new shared tasks in the Task Proposals Track, and discussion sessions. You can see some of the previous GenChal tasks in the past GenChal proceedings on the ACL Anthology (see e.g. https://aclanthology.org/volumes/2023.inlg-genchal/) or on the dedicated repository (https://sites.google.com/view/genchalrepository/home).

Submissions should describe possible future tasks in detail, including information regarding organisers, task description, motivating theoretical interest and/or application context, size and state of completion of data to be used, schedule and evaluation plans. Accepted shared tasks will be run in the 2025 iteration of INLG.

Important dates (All deadlines are 11.59 pm UTC -12h - "anywhere on Earth")
-------------------------
Submission deadline: June 24th 2024
Notification: July 15th 2024
Camera-ready submission: August 16th 2024
Workshop at INLG conference: September 23rd-24h 2024

Submissions and format
-------------------------
Submissions in the Shared Task Proposals track should be no more than 4 (four) pages long excluding citations, and should follow the ACLPUB formatting guidelines (https://acl-org.github.io/ACLPUB/formatting.html; you will also find LaTeX style files and Microsoft Word templates under this link).

Proposals should be uploaded to the SoftConf GenChal submission page, using the Submission type New shared task proposal: https://softconf.com/n/inlg2024/user/scmd.cgi?scmd=submitPaperCustom&pag....

Submissions will be peer-reviewed by the program committee. As reviewing will not be blind, there is no need to anonymise papers.
This is not intended to be a selective process, since the aim is to discuss new potential shared tasks with INLG delegates. However, the organisers reserve the right to reject proposals which do not fall within the scope of the GenChal initiative, or which do not follow guidelines. Accepted submissions will be published in separate GenChal 2023 proceedings on the ACL Anthology, as was done in 2022 and 2023.

Pages

Subscribe to RSS - shared task