Assessing computational thinking through integrated approaches: the case of ModeLLer

Digital technologies and Artificial Intelligence increasingly shape workplace environments, making Computational Thinking (CT) relevant beyond traditional educational settings. This paper addresses the challenge of CT assessment through the evaluation of ModeLLer, a web-based modelling tool designed to support end users, i.e. users with no prior knowledge of semi-formal modelling languages, to formalise domain knowledge. Grounded in End-User Development (EUD) principles, ModeLLer features a block-based editor to represent domain entities and relationships and automatically translates user-generated blocks into standard diagrams in UML,BPMN, and iStar. To evaluate the tool, we conducted a user study involving 30 participants, based on a scenario of agricultural digitalisation. The outcomes were evaluated using a multi-approach strategy combining expert inspection of the tool, artefact-based analysis of user-generated models, CT profiles, workload measures, and log-based analysis of user interactions. An integrated analysis highlights the mutual relationship between tool capabilities, user skills, and artefacts, as well as informing the design of adaptive features. Overall, the study offers empirical insights into CT assessment within EUD, contributing to evaluation practices and the design of AI-powered EUD tools.