We explore competitive Hebbian learning strategies to train feature detectors in Convolutional Neural Networks (CNNs), without supervision. We consider variants of the Winner-Takes-All (WTA) strategy explored in previous works, i.e. k-WTA, e-soft-WTA and p-soft-WTA, performing experiments on different object recognition datasets. Results suggest that the Hebbian approaches are effective to train early feature extraction layers, or to re-train higher layers of a pre-trained network, with soft competition generally performing better than other Hebbian approaches explored in this work. Our findings encourage a path of cooperation between neuroscience and computer science towards a deeper investigation of biologically inspired learning principles.
Training convolutional neural networks with competitive hebbian learning approaches
Falchi F;Gennaro C;Amato G
2022
Abstract
We explore competitive Hebbian learning strategies to train feature detectors in Convolutional Neural Networks (CNNs), without supervision. We consider variants of the Winner-Takes-All (WTA) strategy explored in previous works, i.e. k-WTA, e-soft-WTA and p-soft-WTA, performing experiments on different object recognition datasets. Results suggest that the Hebbian approaches are effective to train early feature extraction layers, or to re-train higher layers of a pre-trained network, with soft competition generally performing better than other Hebbian approaches explored in this work. Our findings encourage a path of cooperation between neuroscience and computer science towards a deeper investigation of biologically inspired learning principles.File | Dimensione | Formato | |
---|---|---|---|
prod_465267-doc_182672.pdf
Open Access dal 03/02/2023
Descrizione: Preprint - Training convolutional neural networks with competitive hebbian learning approaches
Tipologia:
Versione Editoriale (PDF)
Dimensione
501.8 kB
Formato
Adobe PDF
|
501.8 kB | Adobe PDF | Visualizza/Apri |
prod_465267-doc_182717.pdf
Open Access dal 03/02/2023
Descrizione: Training convolutional neural networks with competitive hebbian learning approaches
Tipologia:
Versione Editoriale (PDF)
Dimensione
4.49 MB
Formato
Adobe PDF
|
4.49 MB | Adobe PDF | Visualizza/Apri |
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.