2026-06-16T05:59:54Zhttps://iris.cnr.it/oai/request

oai:iris.cnr.it:20.500.14243/5706032026-03-04T01:27:56Zcom_20.500.14243_46com_20.500.14243_21col_20.500.14243_47ou_ou239

Direct and indirect interpretations of speech acts: evidence from human judgments and large language models Massimiliano Orsini Dominique Brunato Orsini, Massimiliano Brunato, Dominique Indirectness Speech acts Italian benchmark Large Language Models Human evaluation This paper introduces INDIR-IT (Indirectness for the Italian language), a linguistically informed, manually curated benchmark for evaluating large language models’ (LLMs) understanding of indirect speech acts (ISAs) in Italian. By systematically contrasting conventionalized and non-conventionalized ISAs with literal interpretations, the corpus enables fine-grained assessment of pragmatic competence, an area still relatively underexplored compared to lexical and syntactic understanding. Preliminary results show that LLMs handle conventionalized ISAs relatively well, while performance on non-conventionalized ISAs remains more sensitive to model size and capacity. INDIR-IT offers a foundation for advancing research on pragmatic inference in both humans and LLMs. 2025 info:eu-repo/semantics/conferenceObject https://hdl.handle.net/20.500.14243/570603 info:eu-repo/semantics/altIdentifier/isbn/979-12-243-0587-3 https://aclanthology.org/2025.clicit-1.79 eng ispartofbook:Proceedings of the Eleventh Italian Conference on Computational Linguistics (CLiC-it 2025) Eleventh Italian Conference on Computational Linguistics (CLiC-it 2025), Cagliari, Italy, September 2025 firstpage:837 lastpage:848 numberofpages:12 info:eu-repo/semantics/openAccess CEUR Workshop Proceedings license:Creative commons license uri:http://creativecommons.org/licenses/by-nc-nd/4.0/