2026-06-07T13:42:44Zhttps://iris.cnr.it/oai/request

oai:iris.cnr.it:20.500.14243/5822292026-05-18T15:22:40Zou_ou239

Exploring the Use of Large Language Models in Critical Discourse Analysis: A Consensus-Based Pilot Study emiliano giovannetti francesca cristiano Arturo Montejo-Ráez, Cristina Grisot, Joanna Blochowiak Giovannetti, Emiliano Cristiano, Francesca critical discourse analysis, large language models, interpretive evaluation, consensus-based analysis, social sciences and humanities Large Language Models (LLMs) are increasingly used in the social sciences and humanities (SSH) to support the analysis of complex textual data, raising methodological questions about evaluation and interpretive reliability. This paper explores the use of LLMs in Critical Discourse Analysis (CDA), considered here as a paradigmatic case of interpretive research in SSH, through a preliminary consensus-based evaluation framework. The study reports on a pilot experiment conducted on a small, theory-driven corpus of opinion articles addressing the October 7, 2023 attack and its aftermath. An LLM is asked to answer analytically motivated questions targeting different levels of discourse structure. Its responses are compared with annotations produced by multiple human analysts and aggregated through a consensus-based procedure. The results reveal an asymmetry in model performance: while LLMs align well with human consensus on macro- and superstructural features, they struggle with microstructural phenomena involving implicit meaning. These findings support the view of LLMs as epistemic support tools rather than replacements for human interpretation. 2026 info:eu-repo/semantics/conferenceObject https://hdl.handle.net/20.500.14243/582229 info:eu-repo/semantics/altIdentifier/isbn/978-2-493814-85-2 http://lrec-conf.org/proceedings/lrec2026/workshops/llms4ssh/2026.llms4ssh-1.0.pdf eng ispartofbook:Proceedings of Shaping Multilingual, Multimodal AI for the Social Sciences and Humanities (LLMs4SSH) @ LREC 2026 Shaping Multilingual, Multimodal AI for the Social Sciences and Humanities (LLMs4SSH) @ LREC 2026 firstpage:18 lastpage:22 numberofpages:5 ELETTRONICO ELRA Language Resources Association (ELRA)