COLM 2025 Workshop

Pragmatic Reasoning
in Language Models

Language Models as Language Users

October 10, 2025
Montreal, Canada

Register and Join Us!

Main Conference

Overview

Humans are pragmatic language users
We produce language based on our understanding of how context contributes to meaning and deliberate on the choice of utterances and interpretations that helps us collaborate and engage in social interactions. While recent large language models (LLMs) have shown impressive performance on a variety of language-related tasks, could these models be considered as true pragmatic language users?

Towards Language Models as Language Users
The 1st Workshop on Pragmatic Reasoning in Language Models (PragLM) aims to stimulate research on LLMs as pragmatically competent language users. We invite contributions that will forward the discussion of understanding and improvent of LLMs' capability to generate natural language flexibly and efficiently across contexts, with relations to research on the cognitive and linguistic processes supporting effective, context-sensitive communication. Our interdisciplinary theme brings together researchers in NLP, comptuational pragmatics, cognitive science, and other fields:
join us at COLM 2025!

Call for Contributions

We invite researchers to present their published and ongoing works on the topics of, but not limited to:

    Improving Pragmatic, Contextual Language Use in LLMsHow can LLMs' pragmatic abilities be improved and made more human-like, and how should human-likeness in pragmatic competence be assessed?
    Evaluating Pragmatic Competence of Language ModelsHow well can LLMs comprehend and/or generate pragmatic language, across task formats and prompting strategies?
    What are key considerations for designing benchmarks and evaluation frameworks?
    Theory-of-Mind (ToM) and PragmaticsIs there a correlation between LLM capabilities for Theory-of-Mind reasoning and contextual language generation? If so, what are the mechanisms behind these capabilities?
    Pragmatics across Cultures and LanguagesAre LLM abilities in functional language use similar across cultural contexts and languages?
    Application of LLMs for Understanding
    Human Pragmatic Language Use    How can LLMs be leveraged for better theoretical, experimental and computational understanding of human pragmatic language use?
    Interpretability of LLMs' Pragmatic CompetenceWhat are the mechanisms supporting LLMs' pragmatic competence, e.g. through the lens of mechanistic or representational interpretability?


Submission Format

We seek both 4-page extended abstracts and 8-page full papers excluding references and appendices. All workshops papers are non-archival , and we welcome position papers on topics of interest to the workshop.

All submissions will be in the COLM 2025 latex format and submitted via the OpenReview portal. Accepted papers will be invited for poster/oral presentations and will be publicly available on the workshop website.

Submit now!

Important Dates

  • 📅 Submission open: TBD
  • 📅 Submission deadline: June 23rd AoE
  • 📅 Notification of acceptance: July 24th
  • 📅 Camera-ready due: TBD
  • 📅 Workshop date: October 10th

Invited Speakers

Daniel Fried

CMU

Jennifer Hu

Harvard/JHU

Vera Demberg

Saarland University

Michael Franke

University of Tübingen

Tentative Schedule

9:00 - 9:15Opening Remarks
9:15 - 10:00Keynote: TBD
10:00 - 10:45Keynote: TBD
10:45 - 11:00Coffee Break ☕
11:00 - 12:00Contributed Talks Session (Oral)
1:00 - 1:45Keynote: TBD
1:45 - 3:00Coffee Break ☕ and Poster Session
3:00 - 3:45Keynote: TBD
3:45 - 4:30Panel Discussion
4:30 - 4:50Best Paper Awards; Closing Remarks

Organizers

  • • Alane Suhr , University of California at Berkeley
  • • Anya Ji , University of California at Berkeley
  • • Kayo Yin , University of California at Berkeley
  • • Minwoo Kang , University of California at Berkeley
  • • Nicholas Tomlin , University of California at Berkeley
  • • Polina Tsvilodub , University of Tübingen
  • • Robert Hawkins , Stanford University
  • • Seun Eisape , University of California at Berkeley
  • • Téa Wright , University of California at Berkeley

Contact

✉ workshop.pragmatics.llms@gmail.com