We take a collection of short texts, some of which are human-written, while others are automatically generated, and ask subjects, who are unaware of the texts' source, whether they perceive them as human-produced. We use this data to fine-tune a GPT-2 model to push it to generate more human-like texts, and observe that the production of this fine-tuned model is indeed perceived as more human-like than that of the original model. Contextually, we show that our automatic evaluation strategy correlates well with human judgements. We also run a linguistic analysis to unveil the characteristics of human- vs machine-perceived language.
Human Perception in Natural Language Generation
Dell'Orletta F;
2021
Abstract
We take a collection of short texts, some of which are human-written, while others are automatically generated, and ask subjects, who are unaware of the texts' source, whether they perceive them as human-produced. We use this data to fine-tune a GPT-2 model to push it to generate more human-like texts, and observe that the production of this fine-tuned model is indeed perceived as more human-like than that of the original model. Contextually, we show that our automatic evaluation strategy correlates well with human judgements. We also run a linguistic analysis to unveil the characteristics of human- vs machine-perceived language.| Campo DC | Valore | Lingua |
|---|---|---|
| dc.authority.orgunit | Istituto di linguistica computazionale "Antonio Zampolli" - ILC | - |
| dc.authority.people | De Mattei L | it |
| dc.authority.people | Lai H | it |
| dc.authority.people | Dell'Orletta F | it |
| dc.authority.people | Nissim M | it |
| dc.collection.id.s | 71c7200a-7c5f-4e83-8d57-d3d2ba88f40d | * |
| dc.collection.name | 04.01 Contributo in Atti di convegno | * |
| dc.contributor.appartenenza | Istituto di linguistica computazionale "Antonio Zampolli" - ILC | * |
| dc.contributor.appartenenza.mi | 918 | * |
| dc.date.accessioned | 2024/02/19 11:10:11 | - |
| dc.date.available | 2024/02/19 11:10:11 | - |
| dc.date.issued | 2021 | - |
| dc.description.abstracteng | We take a collection of short texts, some of which are human-written, while others are automatically generated, and ask subjects, who are unaware of the texts' source, whether they perceive them as human-produced. We use this data to fine-tune a GPT-2 model to push it to generate more human-like texts, and observe that the production of this fine-tuned model is indeed perceived as more human-like than that of the original model. Contextually, we show that our automatic evaluation strategy correlates well with human judgements. We also run a linguistic analysis to unveil the characteristics of human- vs machine-perceived language. | - |
| dc.description.affiliations | Department of Computer Science, University of Pisa, Department of Computer Science, University of Pisa / Italy, , Italy; ItaliaNLP Lab, Istituto di Linguistica Computazionale "Antonio Zampolli", Pisa, ItaliaNLP Lab, Istituto di Linguistica Computazionale "Antonio Zampolli", Pisa / Italy, , Italy; CLCG, University of Groningen, CLCG, University of Groningen / The Netherlands, , Netherlands; Aptus.AI, Pisa, Aptus.AI / Pisa, Italy, , Italy | - |
| dc.description.allpeople | De Mattei L.; Lai H.; Dell'Orletta F.; Nissim M. | - |
| dc.description.allpeopleoriginal | De Mattei L.; Lai H.; Dell'Orletta F.; Nissim M. | - |
| dc.description.fulltext | none | en |
| dc.description.numberofauthors | 1 | - |
| dc.identifier.doi | 10.18653/v1/2021.gem-1.2 | - |
| dc.identifier.isbn | 978-1-954085-67-1 | - |
| dc.identifier.scopus | 2-s2.0-85123713456 | - |
| dc.identifier.uri | https://hdl.handle.net/20.500.14243/445812 | - |
| dc.identifier.url | http://www.scopus.com/record/display.url?eid=2-s2.0-85123713456&origin=inward | - |
| dc.language.iso | eng | - |
| dc.relation.conferencedate | 05/08/2021 | - |
| dc.relation.conferencename | First Workshop on Generation Evaluation and Metrics (GEM 2021) | - |
| dc.relation.conferenceplace | Online | - |
| dc.relation.firstpage | 15 | - |
| dc.relation.ispartofbook | Proceedings of the First Workshop on Generation Evaluation and Metrics (GEM 2021) | - |
| dc.relation.lastpage | 23 | - |
| dc.subject.keywords | Natural Language Generation | - |
| dc.subject.keywords | Neural Language Models | - |
| dc.subject.keywords | Evaluation | - |
| dc.subject.singlekeyword | Natural Language Generation | * |
| dc.subject.singlekeyword | Neural Language Models | * |
| dc.subject.singlekeyword | Evaluation | * |
| dc.title | Human Perception in Natural Language Generation | en |
| dc.type.driver | info:eu-repo/semantics/conferenceObject | - |
| dc.type.full | 04 Contributo in convegno::04.01 Contributo in Atti di convegno | it |
| dc.type.miur | 273 | - |
| dc.type.referee | Sì, ma tipo non specificato | - |
| dc.ugov.descaux1 | 472158 | - |
| iris.orcid.lastModifiedDate | 2024/03/17 09:19:41 | * |
| iris.orcid.lastModifiedMillisecond | 1710663581475 | * |
| iris.scopus.extIssued | 2021 | - |
| iris.scopus.extTitle | Human Perception in Natural Language Generation | - |
| iris.sitodocente.maxattempts | 1 | - |
| iris.unpaywall.bestoahost | publisher | * |
| iris.unpaywall.bestoaversion | publishedVersion | * |
| iris.unpaywall.doi | 10.18653/v1/2021.gem-1.2 | * |
| iris.unpaywall.hosttype | publisher | * |
| iris.unpaywall.isoa | true | * |
| iris.unpaywall.landingpage | https://doi.org/10.18653/v1/2021.gem-1.2 | * |
| iris.unpaywall.license | cc-by | * |
| iris.unpaywall.metadataCallLastModified | 01/07/2025 05:13:10 | - |
| iris.unpaywall.metadataCallLastModifiedMillisecond | 1751339590153 | - |
| iris.unpaywall.oastatus | hybrid | * |
| iris.unpaywall.pdfurl | https://aclanthology.org/2021.gem-1.2.pdf | * |
| scopus.category | 1710 | * |
| scopus.category | 1706 | * |
| scopus.category | 1703 | * |
| scopus.contributor.affiliation | Aptus.AI | - |
| scopus.contributor.affiliation | University of Groningen | - |
| scopus.contributor.affiliation | Istituto di Linguistica Computazionale “Antonio Zampolli” | - |
| scopus.contributor.affiliation | University of Groningen | - |
| scopus.contributor.afid | 125281904 | - |
| scopus.contributor.afid | 60010023 | - |
| scopus.contributor.afid | 60008941 | - |
| scopus.contributor.afid | 60010023 | - |
| scopus.contributor.auid | 57204921228 | - |
| scopus.contributor.auid | 57222015938 | - |
| scopus.contributor.auid | 57540567000 | - |
| scopus.contributor.auid | 8281949100 | - |
| scopus.contributor.country | Italy | - |
| scopus.contributor.country | Netherlands | - |
| scopus.contributor.country | Italy | - |
| scopus.contributor.country | Netherlands | - |
| scopus.contributor.dptid | - | |
| scopus.contributor.dptid | - | |
| scopus.contributor.dptid | 114087935 | - |
| scopus.contributor.dptid | - | |
| scopus.contributor.name | Lorenzo | - |
| scopus.contributor.name | Huiyuan | - |
| scopus.contributor.name | Felice | - |
| scopus.contributor.name | Malvina | - |
| scopus.contributor.subaffiliation | - | |
| scopus.contributor.subaffiliation | CLCG; | - |
| scopus.contributor.subaffiliation | ItaliaNLP Lab; | - |
| scopus.contributor.subaffiliation | CLCG; | - |
| scopus.contributor.surname | De Mattei | - |
| scopus.contributor.surname | Lai | - |
| scopus.contributor.surname | Dell’Orletta | - |
| scopus.contributor.surname | Nissim | - |
| scopus.date.issued | 2021 | * |
| scopus.description.abstracteng | We take a collection of short texts, some of which are human-written, while others are automatically generated, and ask subjects, who are unaware of the texts’ source, whether they perceive them as human-produced. We use this data to fine-tune a GPT-2 model to push it to generate more human-like texts, and observe that the production of this fine-tuned model is indeed perceived as more human-like than that of the original model. Contextually, we show that our automatic evaluation strategy correlates well with human judgements. We also run a linguistic analysis to unveil the characteristics of human- vs machine-perceived language. | * |
| scopus.description.allpeopleoriginal | De Mattei L.; Lai H.; Dell'Orletta F.; Nissim M. | * |
| scopus.differences | scopus.relation.conferencename | * |
| scopus.differences | scopus.publisher.name | * |
| scopus.differences | scopus.relation.conferencedate | * |
| scopus.differences | scopus.identifier.isbn | * |
| scopus.differences | scopus.description.abstracteng | * |
| scopus.differences | scopus.relation.conferenceplace | * |
| scopus.document.type | cp | * |
| scopus.document.types | cp | * |
| scopus.identifier.isbn | 9781954085671 | * |
| scopus.identifier.pui | 637101245 | * |
| scopus.identifier.scopus | 2-s2.0-85123713456 | * |
| scopus.journal.sourceid | 21101074571 | * |
| scopus.language.iso | eng | * |
| scopus.publisher.name | Association for Computational Linguistics (ACL) | * |
| scopus.relation.conferencedate | 2021 | * |
| scopus.relation.conferencename | 1st Workshop on Natural Language Generation, Evaluation, and Metrics, GEM 2021 | * |
| scopus.relation.conferenceplace | tha | * |
| scopus.relation.firstpage | 15 | * |
| scopus.relation.lastpage | 23 | * |
| scopus.title | Human Perception in Natural Language Generation | * |
| scopus.titleeng | Human Perception in Natural Language Generation | * |
| Appare nelle tipologie: | 04.01 Contributo in Atti di convegno | |
File in questo prodotto:
Non ci sono file associati a questo prodotto.
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
