The Hate Speech Detection (HaSpeeDe) task is a shared task on Italian social media (Facebook and Twitter) for the detection of hateful content, and it has been proposed for the first time at EVALITA 2018. Providing two datasets from two different online social platforms differently featured from the linguistic and communicative point of view, we organized the task in three tasks where systems must be trained and tested on the same resource or using one in training and the other in testing: HaSpeeDe-FB, HaSpeeDe-TW and Cross-HaSpeeDe (further subdivided into Cross-HaSpeeDe FB and Cross-HaSpeeDe TW sub-tasks). Overall, 9 teams participated in the task, and the best system achieved a macro F1-score of 0.8288 for HaSpeeDe-FB, 0.7993 for HaSpeeDe-TW, 0.6541 for Cross-HaSpeeDe FB and 0.6985 for Cross-HaSpeeDe TW. In this report, we describe the datasets released and the evaluation measures, and we discuss results.
Overview of the EVALITA 2018 hate speech detection task
Dell'Orletta F;Tesconi M
2018
Abstract
The Hate Speech Detection (HaSpeeDe) task is a shared task on Italian social media (Facebook and Twitter) for the detection of hateful content, and it has been proposed for the first time at EVALITA 2018. Providing two datasets from two different online social platforms differently featured from the linguistic and communicative point of view, we organized the task in three tasks where systems must be trained and tested on the same resource or using one in training and the other in testing: HaSpeeDe-FB, HaSpeeDe-TW and Cross-HaSpeeDe (further subdivided into Cross-HaSpeeDe FB and Cross-HaSpeeDe TW sub-tasks). Overall, 9 teams participated in the task, and the best system achieved a macro F1-score of 0.8288 for HaSpeeDe-FB, 0.7993 for HaSpeeDe-TW, 0.6541 for Cross-HaSpeeDe FB and 0.6985 for Cross-HaSpeeDe TW. In this report, we describe the datasets released and the evaluation measures, and we discuss results.| Campo DC | Valore | Lingua |
|---|---|---|
| dc.authority.anceserie | CEUR WORKSHOP PROCEEDINGS | - |
| dc.authority.anceserie | CEUR Workshop Proceedings | - |
| dc.authority.orgunit | Istituto di informatica e telematica - IIT | - |
| dc.authority.orgunit | Istituto di linguistica computazionale "Antonio Zampolli" - ILC | - |
| dc.authority.people | Bosco C | it |
| dc.authority.people | Sanguinetti M | it |
| dc.authority.people | Dell'Orletta F | it |
| dc.authority.people | Poletto F | it |
| dc.authority.people | Tesconi M | it |
| dc.collection.id.s | 71c7200a-7c5f-4e83-8d57-d3d2ba88f40d | * |
| dc.collection.name | 04.01 Contributo in Atti di convegno | * |
| dc.contributor.appartenenza | Istituto di informatica e telematica - IIT | * |
| dc.contributor.appartenenza | Istituto di linguistica computazionale "Antonio Zampolli" - ILC | * |
| dc.contributor.appartenenza.mi | 912 | * |
| dc.contributor.appartenenza.mi | 918 | * |
| dc.date.accessioned | 2024/02/15 22:53:32 | - |
| dc.date.available | 2024/02/15 22:53:32 | - |
| dc.date.issued | 2018 | - |
| dc.description.abstracteng | The Hate Speech Detection (HaSpeeDe) task is a shared task on Italian social media (Facebook and Twitter) for the detection of hateful content, and it has been proposed for the first time at EVALITA 2018. Providing two datasets from two different online social platforms differently featured from the linguistic and communicative point of view, we organized the task in three tasks where systems must be trained and tested on the same resource or using one in training and the other in testing: HaSpeeDe-FB, HaSpeeDe-TW and Cross-HaSpeeDe (further subdivided into Cross-HaSpeeDe FB and Cross-HaSpeeDe TW sub-tasks). Overall, 9 teams participated in the task, and the best system achieved a macro F1-score of 0.8288 for HaSpeeDe-FB, 0.7993 for HaSpeeDe-TW, 0.6541 for Cross-HaSpeeDe FB and 0.6985 for Cross-HaSpeeDe TW. In this report, we describe the datasets released and the evaluation measures, and we discuss results. | - |
| dc.description.affiliations | University of Torino, Italy (1); ILC-CNR, Pisa, Italy (2); Acmos, Torino, Italy (3); IIT-CNR, Pisa, Italy (4) | - |
| dc.description.allpeople | Bosco C. ; Sanguinetti M. ; Dell'Orletta F. ; Poletto F. ; Tesconi M. | - |
| dc.description.allpeopleoriginal | Bosco C. (1), Sanguinetti M. (1), Dell'Orletta F. (2), Poletto F. (3), Tesconi M. (4) | - |
| dc.description.fulltext | none | en |
| dc.description.numberofauthors | 2 | - |
| dc.identifier.scopus | 2-s2.0-85058647605 | - |
| dc.identifier.uri | https://hdl.handle.net/20.500.14243/351854 | - |
| dc.identifier.url | http://www.scopus.com/inward/record.url?eid=2-s2.0-85058647605&partnerID=q2rCbXpz | - |
| dc.language.iso | eng | - |
| dc.relation.conferencedate | 10-12/12/2018 | - |
| dc.relation.conferencename | EVALITA 2018-Sixth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian | - |
| dc.relation.conferenceplace | Torino, Italia | - |
| dc.relation.numberofpages | 9 | - |
| dc.relation.volume | 2263 | - |
| dc.subject.keywords | Hate Speech Detection | - |
| dc.subject.keywords | Social Media Analysis | - |
| dc.subject.singlekeyword | Hate Speech Detection | * |
| dc.subject.singlekeyword | Social Media Analysis | * |
| dc.title | Overview of the EVALITA 2018 hate speech detection task | en |
| dc.type.driver | info:eu-repo/semantics/conferenceObject | - |
| dc.type.full | 04 Contributo in convegno::04.01 Contributo in Atti di convegno | it |
| dc.type.miur | 273 | - |
| dc.type.referee | Sì, ma tipo non specificato | - |
| dc.ugov.descaux1 | 398987 | - |
| iris.orcid.lastModifiedDate | 2024/03/19 10:30:51 | * |
| iris.orcid.lastModifiedMillisecond | 1710840651394 | * |
| iris.scopus.extIssued | 2018 | - |
| iris.scopus.extTitle | Overview of the EVALITA 2018 hate speech detection task | - |
| iris.scopus.metadataErrorDescription | 400 Bad Request: " Document ID is not valid |
- |
| iris.scopus.metadataErrorType | APPLICATION | - |
| iris.scopus.metadataStatus | ERROR | - |
| iris.sitodocente.maxattempts | 1 | - |
| scopus.authority.anceserie | CEUR WORKSHOP PROCEEDINGS###1613-0073 | * |
| scopus.category | 1700 | * |
| scopus.contributor.affiliation | University of Torino | - |
| scopus.contributor.affiliation | University of Torino | - |
| scopus.contributor.affiliation | ILC-CNR | - |
| scopus.contributor.affiliation | Acmos | - |
| scopus.contributor.affiliation | IIT-CNR | - |
| scopus.contributor.afid | 60012259 | - |
| scopus.contributor.afid | 60012259 | - |
| scopus.contributor.afid | 60021199 | - |
| scopus.contributor.afid | 119875779 | - |
| scopus.contributor.afid | 60021199 | - |
| scopus.contributor.auid | 7004550793 | - |
| scopus.contributor.auid | 56587551200 | - |
| scopus.contributor.auid | 57540567000 | - |
| scopus.contributor.auid | 57198886384 | - |
| scopus.contributor.auid | 55884637000 | - |
| scopus.contributor.country | Italy | - |
| scopus.contributor.country | Italy | - |
| scopus.contributor.country | Italy | - |
| scopus.contributor.country | Italy | - |
| scopus.contributor.country | Italy | - |
| scopus.contributor.dptid | - | |
| scopus.contributor.dptid | - | |
| scopus.contributor.dptid | - | |
| scopus.contributor.dptid | - | |
| scopus.contributor.dptid | - | |
| scopus.contributor.name | Cristina | - |
| scopus.contributor.name | Manuela | - |
| scopus.contributor.name | Felice | - |
| scopus.contributor.name | Fabio | - |
| scopus.contributor.name | Maurizio | - |
| scopus.contributor.subaffiliation | - | |
| scopus.contributor.subaffiliation | - | |
| scopus.contributor.subaffiliation | - | |
| scopus.contributor.subaffiliation | - | |
| scopus.contributor.subaffiliation | - | |
| scopus.contributor.surname | Bosco | - |
| scopus.contributor.surname | Sanguinetti | - |
| scopus.contributor.surname | Dell’Orletta | - |
| scopus.contributor.surname | Poletto | - |
| scopus.contributor.surname | Tesconi | - |
| scopus.date.issued | 2018 | * |
| scopus.description.abstract | The Hate Speech Detection (HaSpeeDe) task is a shared task on Italian social media (Facebook and Twitter) for the detection of hateful content, and it has been proposed for the first time at EVALITA 2018. Providing two datasets from two different online social platforms differently featured from the linguistic and communicative point of view, we organized the task in three tasks where systems must be trained and tested on the same resource or using one in training and the other in testing: HaSpeeDe-FB, HaSpeeDe-TW and Cross-HaSpeeDe (further subdivided into Cross-HaSpeeDe FB and Cross-HaSpeeDe TW sub-tasks). Overall, 9 teams participated in the task, and the best system achieved a macro F1-score of 0.8288 for HaSpeeDe-FB, 0.7993 for HaSpeeDe-TW, 0.6541 for Cross-HaSpeeDe FB and 0.6985 for Cross-HaSpeeDe TW. In this report, we describe the datasets released and the evaluation measures, and we discuss results. | * |
| scopus.description.allpeopleoriginal | Bosco C.; Sanguinetti M.; Dell'Orletta F.; Poletto F.; Tesconi M. | * |
| scopus.differences | scopus.relation.conferencename | * |
| scopus.differences | scopus.authority.anceserie | * |
| scopus.differences | scopus.publisher.name | * |
| scopus.differences | scopus.relation.conferencedate | * |
| scopus.differences | scopus.description.allpeopleoriginal | * |
| scopus.differences | scopus.relation.conferenceplace | * |
| scopus.document.type | cp | * |
| scopus.document.types | cp | * |
| scopus.funding.funders | 100011201 - FP7 Science in Society; | * |
| scopus.funding.ids | S1618 L2 BOSC 01; | * |
| scopus.identifier.pui | 625515661 | * |
| scopus.identifier.scopus | 2-s2.0-85058647605 | * |
| scopus.journal.sourceid | 21100218356 | * |
| scopus.language.iso | eng | * |
| scopus.publisher.name | CEUR-WS | * |
| scopus.relation.conferencedate | 2018 | * |
| scopus.relation.conferencename | 6th Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop, EVALITA 2018 | * |
| scopus.relation.conferenceplace | ita | * |
| scopus.relation.volume | 2263 | * |
| scopus.title | Overview of the EVALITA 2018 hate speech detection task | * |
| scopus.titleeng | Overview of the EVALITA 2018 hate speech detection task | * |
| Appare nelle tipologie: | 04.01 Contributo in Atti di convegno | |
File in questo prodotto:
Non ci sono file associati a questo prodotto.
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
