Recent progress in speech and language technologies, increasingly driven by scale, with larger models, larger datasets, and more compute, has lead to impressive performance gains. However, these systems still have massive performance gaps that are glaringly obvious to human users, yet completely invisible to standard evaluation methods. As a result, evaluation has become a checklist exercise that reports incremental improvements on existing benchmarks rather than providing a meaningful assessment of what systems actually understand. This creates a growing gap between technological capability and our ability to measure, interpret, and ground that progress in principled ways.
The EWSLT workshop aims to address this gap by bringing together researchers and practitioners from across the globe working on speech and language technologies onto a common platform to collectively discuss, debate, and develop innovative, effective, interpretable, and linguistically aware evaluation methodologies. This includes new standards, benchmarks, datasets, metrics, and shared challenges that encourage deeper and more reliable ways of understanding progress in speech and language technologies.
The workshop will include invited talks, paper presentations, panel discussions, and shared evaluation challenges focused on addressing the most pressing bottlenecks in the evaluation of modern speech and language technologies.
We invite submissions describing novel standards, evaluation methods, benchmarks, datasets, and analysis of speech and language technologies. Submissions must follow the ACL formatting guidelines. Accepted papers will be presented at the workshop and included in the proceedings.
Topics of interest include (but are not limited to):
- Evaluation of Automatic Speech Recognition (ASR)
- Evaluation of Text-to-Speech (TTS)
- Speech Translation evaluation
- Evaluation of Speech-to-Speech systems
- Speech LLM evaluation
- Evaluation metrics beyond WER
- Human vs automatic evaluation methods
- Low-resource and multilingual evaluation
- Evaluation of expressive and prosodic speech
- Bias and robustness in speech systems