Abteilungkolloquium, Mi. 14. 5. 2025: Polina Tsvilodub (Uni Tübingen): Using Language Models for Modeling Language Users
Abstract:
Humans seamlessly rely on contextual information and reasoning for efficiently communicating and cooperating across many different contexts, enabling humans to, e.g., ask good questions, provide relevant answers, understand non-literal language and draw rich inferences about the speakers’ goals. But how do humans efficiently engage in pragmatic reasoning, in the face of potentially open-ended, context-dependent information that is relevant for pragmatic reasoning?
In this talk, I will present a simple framework that builds on extant cognitive models of pragmatic language use, and uses them to scaffold Large Language Model (LLM) modules acting as reasonable stand-ins for open-ended intuitive language use and world knowledge. I will argue that this framework helps to combine the transparent, computational analysis of human language use provided by the cognitive model with the flexibility of LLMs, allowing to explain more open-ended human language use.
I will present two case studies within this framework: a model of non-literal number interpretation and a model of pragmatic question answering, highlighting different variants of such neuro-symbolic models, along with evidence from human experiments supporting that the computational analysis formalized in the cognitive models both explains human behavior, and provides a principled approach to analyzing LLM performance in pragmatics.