This paper presents a study based on the linguistic profiling methodology to explore the relationship between the linguistic structure of a text and how it is perceived in terms of writing quality byhumans. The approach is tested on a selection of Italian L1 learners essays, which were taken from a larger longitudinal corpus of essays written by Italian L1 students enrolled in the first and secondyear of lower secondary school. Human ratings of writing quality by Italian native speakers were collected through a crowdsourcing task, in which annotators were asked to read pairs of essays andrated which one they believed to be better written. By analyzing these ratings, the study identifies a variety of linguistic phenomena spanning across distinct levels of linguistic description thatdistinguish the essays considered as 'winners' and evaluates the impact of students' errors on the human perception of writing quality.
Linguistic Profile of a Text and Human Ratings of Writing Quality: a Case Study on Italian L1 Learner Essays
Dominique Brunato;Felice Dell'Orletta
2023
Abstract
This paper presents a study based on the linguistic profiling methodology to explore the relationship between the linguistic structure of a text and how it is perceived in terms of writing quality byhumans. The approach is tested on a selection of Italian L1 learners essays, which were taken from a larger longitudinal corpus of essays written by Italian L1 students enrolled in the first and secondyear of lower secondary school. Human ratings of writing quality by Italian native speakers were collected through a crowdsourcing task, in which annotators were asked to read pairs of essays andrated which one they believed to be better written. By analyzing these ratings, the study identifies a variety of linguistic phenomena spanning across distinct levels of linguistic description thatdistinguish the essays considered as 'winners' and evaluates the impact of students' errors on the human perception of writing quality.| Campo DC | Valore | Lingua |
|---|---|---|
| dc.authority.ancejournal | IJCOL | en |
| dc.authority.orgunit | Istituto di linguistica computazionale "Antonio Zampolli" - ILC | en |
| dc.authority.people | Aldo Cerulli | en |
| dc.authority.people | Dominique Brunato | en |
| dc.authority.people | Felice Dell'Orletta | en |
| dc.collection.id.s | b3f88f24-048a-4e43-8ab1-6697b90e068e | * |
| dc.collection.name | 01.01 Articolo in rivista | * |
| dc.contributor.appartenenza | Istituto di linguistica computazionale "Antonio Zampolli" - ILC | * |
| dc.contributor.appartenenza.mi | 918 | * |
| dc.date.accessioned | 2024/02/20 18:13:02 | - |
| dc.date.available | 2024/02/20 18:13:02 | - |
| dc.date.firstsubmission | 2025/01/24 13:06:26 | * |
| dc.date.issued | 2023 | - |
| dc.date.submission | 2025/01/24 13:06:26 | * |
| dc.description.abstracteng | This paper presents a study based on the linguistic profiling methodology to explore the relationship between the linguistic structure of a text and how it is perceived in terms of writing quality byhumans. The approach is tested on a selection of Italian L1 learners essays, which were taken from a larger longitudinal corpus of essays written by Italian L1 students enrolled in the first and secondyear of lower secondary school. Human ratings of writing quality by Italian native speakers were collected through a crowdsourcing task, in which annotators were asked to read pairs of essays andrated which one they believed to be better written. By analyzing these ratings, the study identifies a variety of linguistic phenomena spanning across distinct levels of linguistic description thatdistinguish the essays considered as 'winners' and evaluates the impact of students' errors on the human perception of writing quality. | - |
| dc.description.affiliations | Università di Pisa; Istituto di Linguistica Computazionale "Antonio Zampolli" (CNR-ILC) ItaliaNLP Lab; Istituto di Linguistica Computazionale "Antonio Zampolli" (CNR-ILC) ItaliaNLP Lab | - |
| dc.description.allpeople | Cerulli, Aldo; Brunato, Dominique; Dell'Orletta, Felice | - |
| dc.description.allpeopleoriginal | Aldo Cerulli; Dominique Brunato; Felice Dell'Orletta | en |
| dc.description.fulltext | open | en |
| dc.description.numberofauthors | 3 | - |
| dc.identifier.doi | 10.4000/ijcol.1104 | en |
| dc.identifier.scopus | 2-s2.0-85206206582 | - |
| dc.identifier.uri | https://hdl.handle.net/20.500.14243/455146 | - |
| dc.identifier.url | https://www.ai-lc.it/wp-content/uploads/2023/09/IJCOL_9_1_1_cerulli_et_al.pdf | en |
| dc.language.iso | eng | en |
| dc.relation.firstpage | 7 | en |
| dc.relation.issue | 9 | en |
| dc.relation.lastpage | 34 | en |
| dc.relation.numberofpages | 28 | en |
| dc.relation.volume | 1 | en |
| dc.subject.keywords | text quality | - |
| dc.subject.keywords | human ratings | - |
| dc.subject.keywords | Natural Language Processing | - |
| dc.subject.keywords | learner corpus | - |
| dc.subject.singlekeyword | text quality | * |
| dc.subject.singlekeyword | human ratings | * |
| dc.subject.singlekeyword | Natural Language Processing | * |
| dc.subject.singlekeyword | learner corpus | * |
| dc.title | Linguistic Profile of a Text and Human Ratings of Writing Quality: a Case Study on Italian L1 Learner Essays | en |
| dc.type.driver | info:eu-repo/semantics/article | - |
| dc.type.full | 01 Contributo su Rivista::01.01 Articolo in rivista | it |
| dc.type.impactfactor | si | en |
| dc.type.miur | 262 | - |
| dc.type.referee | Sì, ma tipo non specificato | en |
| dc.ugov.descaux1 | 491082 | - |
| iris.mediafilter.data | 2025/04/12 03:33:56 | * |
| iris.orcid.lastModifiedDate | 2025/01/25 09:43:16 | * |
| iris.orcid.lastModifiedMillisecond | 1737794596540 | * |
| iris.scopus.extIssued | 2023 | - |
| iris.scopus.extTitle | Linguistic Profile of a Text and Human Ratings of Writing Quality: a Case Study on Italian L1 Learner Essays | - |
| iris.sitodocente.maxattempts | 5 | - |
| iris.unpaywall.bestoahost | publisher | * |
| iris.unpaywall.bestoaversion | publishedVersion | * |
| iris.unpaywall.doi | 10.4000/ijcol.1104 | * |
| iris.unpaywall.hosttype | publisher | * |
| iris.unpaywall.isoa | true | * |
| iris.unpaywall.journalisindoaj | true | * |
| iris.unpaywall.landingpage | https://doi.org/10.4000/ijcol.1104 | * |
| iris.unpaywall.license | cc-by-nc-nd | * |
| iris.unpaywall.metadataCallLastModified | 17/06/2025 05:24:34 | - |
| iris.unpaywall.metadataCallLastModifiedMillisecond | 1750130675003 | - |
| iris.unpaywall.oastatus | gold | * |
| iris.unpaywall.pdfurl | https://journals.openedition.org/ijcol/pdf/1104 | * |
| scopus.authority.ancejournal | IJCOL###2499-4553 | * |
| scopus.category | 1709 | * |
| scopus.category | 3310 | * |
| scopus.category | 1703 | * |
| scopus.category | 1702 | * |
| scopus.contributor.affiliation | Università di Pisa | - |
| scopus.contributor.affiliation | ItaliaNLP Lab | - |
| scopus.contributor.affiliation | ItaliaNLP Lab | - |
| scopus.contributor.afid | 60028868 | - |
| scopus.contributor.afid | 60008941 | - |
| scopus.contributor.afid | 60008941 | - |
| scopus.contributor.auid | 57373425700 | - |
| scopus.contributor.auid | 55237740200 | - |
| scopus.contributor.auid | 57540567000 | - |
| scopus.contributor.country | Italy | - |
| scopus.contributor.country | Italy | - |
| scopus.contributor.country | Italy | - |
| scopus.contributor.dptid | 115497562 | - |
| scopus.contributor.dptid | 114087935 | - |
| scopus.contributor.dptid | 114087935 | - |
| scopus.contributor.name | Aldo | - |
| scopus.contributor.name | Dominique | - |
| scopus.contributor.name | Felice | - |
| scopus.contributor.subaffiliation | Dipartimento di Filologia Letteratura Linguistica; | - |
| scopus.contributor.subaffiliation | Istituto di Linguistica Computazionale “Antonio Zampolli” (CNR–ILC); | - |
| scopus.contributor.subaffiliation | Istituto di Linguistica Computazionale “Antonio Zampolli” (CNR–ILC); | - |
| scopus.contributor.surname | Cerulli | - |
| scopus.contributor.surname | Brunato | - |
| scopus.contributor.surname | Dell’orletta | - |
| scopus.date.issued | 2023 | * |
| scopus.description.abstracteng | This paper presents a study based on the linguistic profiling methodology to explore the relationship between the linguistic structure of a text and how it is perceived in terms of writing quality by humans. The approach is tested on a selection of Italian L1 learners essays, which were taken from a larger longitudinal corpus of essays written by Italian L1 students enrolled in the first and second year of lower secondary school. Human ratings of writing quality by Italian native speakers were collected through a crowdsourcing task, in which annotators were asked to read pairs of essays and rated which one they believed to be better written. By analyzing these ratings, the study identifies a variety of linguistic phenomena spanning across distinct levels of linguistic description that distinguish the essays considered as ‘winners’ and evaluates the impact of students’ errors on the human perception of writing quality. | * |
| scopus.description.allpeopleoriginal | Cerulli A.; Brunato D.; Dell'orletta F. | * |
| scopus.differences | scopus.description.allpeopleoriginal | * |
| scopus.differences | scopus.description.abstracteng | * |
| scopus.differences | scopus.relation.issue | * |
| scopus.differences | scopus.relation.volume | * |
| scopus.document.type | ar | * |
| scopus.document.types | ar | * |
| scopus.identifier.doi | 10.4000/ijcol.1104 | * |
| scopus.identifier.eissn | 2499-4553 | * |
| scopus.identifier.pui | 2031364970 | * |
| scopus.identifier.scopus | 2-s2.0-85206206582 | * |
| scopus.journal.sourceid | 21101252471 | * |
| scopus.language.iso | eng | * |
| scopus.publisher.name | Accademia University Press | * |
| scopus.relation.firstpage | 7 | * |
| scopus.relation.issue | 1 | * |
| scopus.relation.lastpage | 34 | * |
| scopus.relation.volume | 9 | * |
| scopus.title | Linguistic Profile of a Text and Human Ratings of Writing Quality: a Case Study on Italian L1 Learner Essays | * |
| scopus.titleeng | Linguistic Profile of a Text and Human Ratings of Writing Quality: a Case Study on Italian L1 Learner Essays | * |
| Appare nelle tipologie: | 01.01 Articolo in rivista | |
File in questo prodotto:
| File | Dimensione | Formato | |
|---|---|---|---|
|
prod_491082-doc_204687.pdf
accesso aperto
Descrizione: Linguistic Profile of a Text and Human Ratings of Writing Quality: a Case Study on Italian L1 Learner Essays
Tipologia:
Versione Editoriale (PDF)
Licenza:
Creative commons
Dimensione
654.57 kB
Formato
Adobe PDF
|
654.57 kB | Adobe PDF | Visualizza/Apri |
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
