We are happy to announce the first workshop on Pragmatic Reasoning in Language Models — PragLM!
OVERVIEW:
We produce language based on our understanding of how context contributes to meaning and deliberate on the choice of utterances and interpretations that helps us collaborate and engage in social interactions. While recent large language models (LLMs) have shown impressive performance on a variety of language-related tasks, could these models be considered as true pragmatic language users?
The 1st Workshop on Pragmatic Reasoning in Language Models (PragLM) aims to stimulate research on LLMs as pragmatically competent language users. We invite contributions that will forward the discussion of understanding and improvement of LLMs' capability to generate natural language flexibly and efficiently across contexts, with relations to research on the cognitive and linguistic processes supporting effective, context-sensitive communication. Our interdisciplinary theme brings together researchers in NLP, computational pragmatics, cognitive science, and other fields.
Workshop website:
https://sites.google.com/berkeley.edu/praglm/
TOPICS OF INTEREST:
We invite papers on research questions including, but not limited to:
- Improving Pragmatic, Contextual Language Use in LLMs: How can LLMs' pragmatic abilities be improved and made more human-like, and how should human-likeness in pragmatic competence be assessed?
- Evaluating Pragmatic Competence of Language Models: How well can LLMs comprehend and/or generate pragmatic language, across task formats and prompting strategies? What are key considerations for designing benchmarks and evaluation frameworks?
- Theory-of-Mind (ToM) and Pragmatics: Is there a correlation between LLM capabilities for Theory-of-Mind reasoning and contextual language generation? If so, what are the mechanisms behind these capabilities?
- Pragmatics across Cultures and Languages: Are LLM abilities in functional language use similar across cultural contexts and languages?
- Application of LLMs for Understanding Human Pragmatic Language Use: How can LLMs be leveraged for better theoretical, experimental and computational understanding of human pragmatic language use?
- Interpretability of LLMs' Pragmatic Competence: What are the mechanisms supporting LLMs' pragmatic competence, e.g., through the lens of mechanistic or representational interpretability?
SUBMISSIONS:
We seek both 4-page extended abstracts and 8-page full papers, excluding references and appendices. All workshop papers are non-archival, and we welcome position papers on topics of interest to the workshop. All submissions will be in the COLM 2025 LaTeX format and submitted via the OpenReview portal. Accepted papers will be invited for poster/oral presentations and will be publicly available on the workshop website.
KEYNOTE SPEAKERS:
Vera Demberg, Saarland University
Michael Franke, University of Tübingen
Daniel Fried, Carnegie Mellon University
Jennifer Hu, Harvard University / Johns Hopkins University
Link to submission:
https://openreview.net/group?id=colmweb.org/COLM/2025/Workshop/PragLM
Submission deadline:
June 23rd
Notification of acceptance: July 24th
Camera-ready due: TBD
Workshop date: October 10th
We look forward to your submissions!
Best,
Organizing committee of PragLM