This paper investigates Stochastic Local Search (SLS) algorithms for training neural networks with threshold activation functions. and proposes a novel technique, called Binary Learning Machine (BLM). BLM acts by changing individual bits in the binary representation of each weight and picking improving moves. While brute-force implementations of SLS lead to enormous CPU times, due to the limited extent of each move, the use of incremental neighborhood evaluation, first-improving strategies, and Gray encoding accelerates SLS by a huge factor, opening the way to affordable CPU times for a wide range of problems. Comparisons with alternative methods demonstrate the effectiveness of the approach. In details, BLM outperforms state-of-the-art techniques either by achieving better generalization properties, or by allowing for more compact networks, suitable for the type of applications for which threshold neural networks were originally introduced.

Stochastic Local Search for direct training of threshold networks

Brunato, Mauro;Battiti, Roberto
2015-01-01

Abstract

This paper investigates Stochastic Local Search (SLS) algorithms for training neural networks with threshold activation functions. and proposes a novel technique, called Binary Learning Machine (BLM). BLM acts by changing individual bits in the binary representation of each weight and picking improving moves. While brute-force implementations of SLS lead to enormous CPU times, due to the limited extent of each move, the use of incremental neighborhood evaluation, first-improving strategies, and Gray encoding accelerates SLS by a huge factor, opening the way to affordable CPU times for a wide range of problems. Comparisons with alternative methods demonstrate the effectiveness of the approach. In details, BLM outperforms state-of-the-art techniques either by achieving better generalization properties, or by allowing for more compact networks, suitable for the type of applications for which threshold neural networks were originally introduced.
2015
Proceedings of the International Joint Conference on Neural Networks
Killarney
Institute of Electrical and Electronics Engineers Inc.
9781479919604
9781479919598
Brunato, Mauro; Battiti, Roberto
File in questo prodotto:
File Dimensione Formato  
07280770.pdf

Solo gestori archivio

Descrizione: Articolo nella versione pubblicata, scaricato da IEEE Xplore, anche disponibile su DVD in possesso degli autori
Tipologia: Versione editoriale (Publisher’s layout)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 1.52 MB
Formato Adobe PDF
1.52 MB Adobe PDF   Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11572/126438
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 3
  • ???jsp.display-item.citation.isi??? 1
social impact