2026-06-15T02:27:09Zhttps://iris.cnr.it/oai/request

oai:iris.cnr.it:20.500.14243/5704612026-03-04T01:28:12Zcom_20.500.14243_46com_20.500.14243_21col_20.500.14243_47ou_ou239

Beyond the Spelling Miracle: Investigating Substring Awareness in Character-Blind Language Models Ciaccio C. Sartor M. Miaschi A. Dell'Orletta F. Ciaccio, C. Sartor, M. Miaschi, A. Dell'Orletta, F. Large Language Models (LLMs) Interpretability Correctly identifying characters and substrings of words should be a basic but essential ability of any Language Model that aims to proficiently understand and produce language. Despite so, the majority of Pre-trained Language Models (PLMs) are "character-blind" and struggle in spelling tasks, although they still seem to acquire some character knowledge during pre-training, a phenomenon dubbed Spelling Miracle. To shed light on this phenomenon, we systematically evaluate a range of PLMs with different parameter sizes using a controlled binary substring identification task. Through a series of experiments, we propose the first comprehensive investigation on where, when, and how PLMs develop awareness of characters and substrings, with a particular linguistic focus on morphemic units such as prefixes, suffixes, and roots. 2025 info:eu-repo/semantics/conferenceObject https://hdl.handle.net/20.500.14243/570461 10.18653/v1/2025.findings-acl.593 info:eu-repo/semantics/altIdentifier/scopus/2-s2.0-105028561206 eng ispartofbook:Proceedings of the Annual Meeting of the Association for Computational Linguistics 63rd Annual Meeting of the Association for Computational Linguistics, ACL 2025 firstpage:11361 lastpage:11372 numberofpages:12 serie:PROCEEDINGS OF THE CONFERENCE - ASSOCIATION FOR COMPUTATIONAL LINGUISTICS. MEETING info:eu-repo/semantics/openAccess Association for Computational Linguistics (ACL) license:Creative commons license uri:http://creativecommons.org/licenses/by-nc-nd/4.0/