This paper investigates Stochastic Local Search (SLS) algorithms for training neural networks with threshold activation functions. and proposes a novel technique, called Binary Learning Machine (BLM). BLM acts by changing individual bits in the binary representation of each weight and picking improving moves. While brute-force implementations of SLS lead to enormous CPU times, due to the limited extent of each move, the use of incremental neighborhood evaluation, first-improving strategies, and Gray encoding accelerates SLS by a huge factor, opening the way to affordable CPU times for a wide range of problems. Comparisons with alternative methods demonstrate the effectiveness of the approach. In details, BLM outperforms state-of-the-art techniques either by achieving better generalization properties, or by allowing for more compact networks, suitable for the type of applications for which threshold neural networks were originally introduced.
Titolo: | Stochastic Local Search for direct training of threshold networks |
Autori: | Brunato, Mauro; Battiti, Roberto |
Autori Unitn: | |
Titolo del volume contenente il saggio: | Proceedings of the International Joint Conference on Neural Networks |
Luogo di edizione: | Killarney |
Casa editrice: | Institute of Electrical and Electronics Engineers Inc. |
Anno di pubblicazione: | 2015 |
Codice identificativo Scopus: | 2-s2.0-84950996840 |
Codice identificativo ISI: | WOS:000370730603068 |
ISBN: | 9781479919604 9781479919598 |
Handle: | http://hdl.handle.net/11572/126438 |
Appare nelle tipologie: | 04.1 Saggio in atti di convegno (Paper in proceedings) |
File in questo prodotto:
File | Descrizione | Tipologia | Licenza | |
---|---|---|---|---|
07280770.pdf | Articolo nella versione pubblicata, scaricato da IEEE Xplore, anche disponibile su DVD in possesso degli autori | Versione editoriale (Publisher’s layout) | Tutti i diritti riservati (All rights reserved) | Administrator |