--- Call for Papers (more details on the Website) ---
We welcome submissions that focus on performance and interpretability analyses or evaluation methods for multimodal, multipurpose, or massive-scale models. Topics include but are not limited to:
- Multimodal or Massive-Scale (Single Task Models): Analyses that examine differences in evaluations of narrow (single modality, single task) and multipurpose AI and the limitations/challenges of existing metrics/benchmarks
- Multipurpose Models or Emergent Behavior (Multiple Task Models): Analyses that evaluate performance or behavior of multipurpose models (e.g., foundation models, neural platforms) that can support multiple tasks across single or multiple modalities. This includes zero-shot, few-shot, and finetuned tasks as well as emergent behavior.
- Nontraditional Evaluation/Interpretability Methods: Nontraditional methods, measures, or metrics targeting limitations of existing evaluation and interpretability of multimodality, multiple tasks (multipurpose), or emergent behavior
Submissions may incorporate the following:
- Evaluations that encompass multiple principles of responsible AI/ML (e.g., robustness, accountability, fairness, transparency, interpretability) for multimodal, multipurpose, or massive-scale models
- Domain-driven evaluations for performance or interpretability needs of different use cases (e.g., commercial, academic) and users (e.g., researchers, domain scientists, practitioners, students)
The workshop will be organized around the complexity (multimodal, multipurpose, or massive-scale) of AI models under analysis or for which nontraditional methods of evaluation are developed to support, or traditional methods of evaluation/interpretability are extended.
-- Special Themes for Position Papers "Multimodal, Multipurpose, or Massive-scale models and Beyond” --
We are delighted to seek submissions for special themes that reflect progress and future directions in this area. These themes will include presentations and panel discussions around open questions, major obstacles, and integration of new techniques for NLP. These themes lend themselves to position papers, however, contributions with empirical evidence are encouraged as well.