Testing syntactic competence of Large Language Models: Syntactic ambiguity in Russian

Edyta Jurkiewicz-Rohrbacher, Slavistische Linguistik, Universität Hamburg.

The rise of generative pre-trained language models has led to the proliferation of machine-generated texts (MGTs) in various languages. The field now grapples with the question of whether established methods can effectively analyze these artificially generated varieties. The talk investigates the applicability of translation methods to assess the syntactic competence of language models, illustrating this with an example of dative ambiguity in Russian.