Probing tasks are frequently used to evaluate whether the representations of Neural Language Models (NLMs) encode linguistic information. However, it is still questioned if probing classification tasks really enable such investigation or they simply hint for surface patterns in the data. We present a method to investigate this question by comparing the accuracies of a set of probing tasks on gold and automatically generated control datasets. Our results suggest that probing tasks can be used as reliable diagnostic methods to investigate the linguistic information encoded in NLMs representations.
Probing tasks under pressure
Miaschi A;Alzetta C;Brunato D;Dell'Orletta F;Venturi G
2021
Abstract
Probing tasks are frequently used to evaluate whether the representations of Neural Language Models (NLMs) encode linguistic information. However, it is still questioned if probing classification tasks really enable such investigation or they simply hint for surface patterns in the data. We present a method to investigate this question by comparing the accuracies of a set of probing tasks on gold and automatically generated control datasets. Our results suggest that probing tasks can be used as reliable diagnostic methods to investigate the linguistic information encoded in NLMs representations.| Campo DC | Valore | Lingua |
|---|---|---|
| dc.authority.anceserie | CEUR WORKSHOP PROCEEDINGS | - |
| dc.authority.anceserie | CEUR Workshop Proceedings | - |
| dc.authority.orgunit | Istituto di linguistica computazionale "Antonio Zampolli" - ILC | - |
| dc.authority.people | Miaschi A | it |
| dc.authority.people | Alzetta C | it |
| dc.authority.people | Brunato D | it |
| dc.authority.people | Dell'Orletta F | it |
| dc.authority.people | Venturi G | it |
| dc.collection.id.s | 71c7200a-7c5f-4e83-8d57-d3d2ba88f40d | * |
| dc.collection.name | 04.01 Contributo in Atti di convegno | * |
| dc.contributor.appartenenza | Istituto di linguistica computazionale "Antonio Zampolli" - ILC | * |
| dc.contributor.appartenenza.mi | 918 | * |
| dc.date.accessioned | 2024/02/19 11:54:22 | - |
| dc.date.available | 2024/02/19 11:54:22 | - |
| dc.date.issued | 2021 | - |
| dc.description.abstracteng | Probing tasks are frequently used to evaluate whether the representations of Neural Language Models (NLMs) encode linguistic information. However, it is still questioned if probing classification tasks really enable such investigation or they simply hint for surface patterns in the data. We present a method to investigate this question by comparing the accuracies of a set of probing tasks on gold and automatically generated control datasets. Our results suggest that probing tasks can be used as reliable diagnostic methods to investigate the linguistic information encoded in NLMs representations. | - |
| dc.description.affiliations | Istituto di Linguistica Computazionale (ILC-CNR); Dipartimento di Informatica, Università di Pisa | - |
| dc.description.allpeople | Miaschi A.; Alzetta C.; Brunato D.; Dell'Orletta F.; Venturi G. | - |
| dc.description.allpeopleoriginal | Miaschi A.; Alzetta C.; Brunato D.; Dell'Orletta F.; Venturi G. | - |
| dc.description.fulltext | none | en |
| dc.description.numberofauthors | 5 | - |
| dc.identifier.scopus | 2-s2.0-85121280147 | - |
| dc.identifier.uri | https://hdl.handle.net/20.500.14243/446048 | - |
| dc.identifier.url | http://ceur-ws.org/Vol-3033/paper29.pdf | - |
| dc.language.iso | eng | - |
| dc.miur.last.status.update | 2024-12-20T09:04:17Z | * |
| dc.relation.conferencedate | 29/06-01/07/2022 | - |
| dc.relation.conferencename | 8th Italian Conference on Computational Linguistics (CLIC-it 2021) | - |
| dc.relation.conferenceplace | Milano | - |
| dc.relation.firstpage | 1 | - |
| dc.relation.lastpage | 7 | - |
| dc.relation.numberofpages | 7 | - |
| dc.relation.volume | 3033 | - |
| dc.subject.keywords | Neural Language Models | - |
| dc.subject.keywords | Linguistic probing | - |
| dc.subject.keywords | Treebanks | - |
| dc.subject.singlekeyword | Neural Language Models | * |
| dc.subject.singlekeyword | Linguistic probing | * |
| dc.subject.singlekeyword | Treebanks | * |
| dc.title | Probing tasks under pressure | en |
| dc.type.driver | info:eu-repo/semantics/conferenceObject | - |
| dc.type.full | 04 Contributo in convegno::04.01 Contributo in Atti di convegno | it |
| dc.type.miur | 273 | - |
| dc.ugov.descaux1 | 463833 | - |
| iris.orcid.lastModifiedDate | 2024/03/15 08:58:44 | * |
| iris.orcid.lastModifiedMillisecond | 1710489524615 | * |
| iris.scopus.extIssued | 2021 | - |
| iris.scopus.extTitle | Probing tasks under pressure | - |
| iris.sitodocente.maxattempts | 1 | - |
| scopus.authority.anceserie | CEUR WORKSHOP PROCEEDINGS###1613-0073 | * |
| scopus.category | 1700 | * |
| scopus.contributor.affiliation | University of Pisa | - |
| scopus.contributor.affiliation | Pisa ItaliaNLP Lab | - |
| scopus.contributor.affiliation | Pisa ItaliaNLP Lab | - |
| scopus.contributor.affiliation | Pisa ItaliaNLP Lab | - |
| scopus.contributor.affiliation | Pisa ItaliaNLP Lab | - |
| scopus.contributor.afid | 60028868 | - |
| scopus.contributor.afid | 60008941 | - |
| scopus.contributor.afid | 60008941 | - |
| scopus.contributor.afid | 60008941 | - |
| scopus.contributor.afid | 60008941 | - |
| scopus.contributor.auid | 57211678681 | - |
| scopus.contributor.auid | 57192938832 | - |
| scopus.contributor.auid | 55237740200 | - |
| scopus.contributor.auid | 57540567000 | - |
| scopus.contributor.auid | 27568199800 | - |
| scopus.contributor.country | Italy | - |
| scopus.contributor.country | - | |
| scopus.contributor.country | - | |
| scopus.contributor.country | - | |
| scopus.contributor.country | - | |
| scopus.contributor.dptid | 109696702 | - |
| scopus.contributor.dptid | - | |
| scopus.contributor.dptid | - | |
| scopus.contributor.dptid | - | |
| scopus.contributor.dptid | - | |
| scopus.contributor.name | Alessio | - |
| scopus.contributor.name | Chiara | - |
| scopus.contributor.name | Dominique | - |
| scopus.contributor.name | Felice | - |
| scopus.contributor.name | Giulia | - |
| scopus.contributor.subaffiliation | Department of Computer Science; | - |
| scopus.contributor.subaffiliation | Istituto di Linguistica Computazionale Antonio Zampolli; | - |
| scopus.contributor.subaffiliation | Istituto di Linguistica Computazionale Antonio Zampolli; | - |
| scopus.contributor.subaffiliation | Istituto di Linguistica Computazionale Antonio Zampolli; | - |
| scopus.contributor.subaffiliation | Istituto di Linguistica Computazionale Antonio Zampolli; | - |
| scopus.contributor.surname | Miaschi | - |
| scopus.contributor.surname | Alzetta | - |
| scopus.contributor.surname | Brunato | - |
| scopus.contributor.surname | Dell'Orletta | - |
| scopus.contributor.surname | Venturi | - |
| scopus.date.issued | 2021 | * |
| scopus.description.abstracteng | Probing tasks are frequently used to evaluate whether the representations of Neural Language Models (NLMs) encode linguistic information. However, it is still questioned if probing classification tasks really enable such investigation or they simply hint for surface patterns in the data. We present a method to investigate this question by comparing the accuracies of a set of probing tasks on gold and automatically generated control datasets. Our results suggest that probing tasks can be used as reliable diagnostic methods to investigate the linguistic information encoded in NLMs representations. | * |
| scopus.description.allpeopleoriginal | Miaschi A.; Alzetta C.; Brunato D.; Dell'Orletta F.; Venturi G. | * |
| scopus.differences | scopus.relation.conferencename | * |
| scopus.differences | scopus.authority.anceserie | * |
| scopus.differences | scopus.publisher.name | * |
| scopus.differences | scopus.relation.conferencedate | * |
| scopus.differences | scopus.relation.conferenceplace | * |
| scopus.document.type | cp | * |
| scopus.document.types | cp | * |
| scopus.identifier.pui | 636650900 | * |
| scopus.identifier.scopus | 2-s2.0-85121280147 | * |
| scopus.journal.sourceid | 21100218356 | * |
| scopus.language.iso | eng | * |
| scopus.publisher.name | CEUR-WS | * |
| scopus.relation.conferencedate | 2022 | * |
| scopus.relation.conferencename | 8th Italian Conference on Computational Linguistics, CLiC-it 2021 | * |
| scopus.relation.conferenceplace | Universita degli Studi di Milano-Bicocca, ita | * |
| scopus.relation.volume | 3033 | * |
| scopus.title | Probing tasks under pressure | * |
| scopus.titleeng | Probing tasks under pressure | * |
| Appare nelle tipologie: | 04.01 Contributo in Atti di convegno | |
File in questo prodotto:
Non ci sono file associati a questo prodotto.
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
