The exceptional language abilities of transformer language models such as ChatGPT has raised questions as to what extent these models use linguistic structures in the same ways that humans do. Recent work has investigated whether we can find traces of elements theorized by linguists, such as constructions from Construction Grammar theory, or whether model predictions with high uncertainty correspond to linguistic elements that humans find complex.

In this project, you will take a concept from linguistic theory and investigate to what extent is is encoded by large language models, examining whether specific models are more human-like than others. Examples of concepts would be a specific type of construction (e.g. the ‘way’-construction as studied in construction grammar), a phenomenon such as processing difficulty in object relative clauses, or a theoretical construct such as island constraints from generative grammar. This type of work is of interest in the context of Explainable AI and understanding how our large language models operate under the hood, and it is of interest to linguists who are interested in examining whether large language models can be plausible models of human language.

You will be working with Jelke Bloem and will join the NLP & Digital Humanities group at the ILLC, where, in addition to the weekly supervision meetings, we have regular academic events such as the Computational Linguistics Seminar. In addition, you will have opportunities to interact with linguists from the linguistics department, for example in the context of the ACLC research group “Computational and Corpus-based Approaches to Language and Literature”. There is no RA funding available for this topic.

This is an academic research project, so we expect a scientific attitude towards the topic. We aim to publish student papers on this topic in high-impact venues, as we have done before with this one: https://doi.org/10.18653/v1/2023.findings-acl.819

An interest in languages or linguistics would make this a great project for you. An interest in Natural Language Processing (NLP) is beneficial and Python programming skills are essential.