Backing up the intermediate results of hardware-accelerated deep inference is crucial to ensure the progress of execution on batteryless computing platforms. However, hardware accelerators in low-power AI platforms only support the one-shot atomic execution of one neural network inference without any backups. This paper introduces a new toolchain for MAX78000, which is a brand-new microcontroller with a hardware-based convolutional neural network (CNN) accelerator. Our toolchain converts any MAX78000-compatible neural network into an intermittently executable form. The toolchain enables finer checkpoint granularity on the MAX78000 CNN accelerator, allowing for backups of any intermediate neural network layer output. Based on the layer-by-layer CNN execution, we propose a new backup technique that performs only necessary (urgent) checkpoints. The method involves the batteryless system switching to ultra-low-power mode while charging, saving intermediate results only when input power is lower than ultra-low-power mode energy consumption. By avoiding unnecessary memory transfer, the proposed solution increases the inference throughput by 1.9x for simulation and by 1.2x for real-world setup compared to the coarse-grained baseline execution.

Fine-grained Hardware Acceleration for Efficient Batteryless Intermittent Inference on the Edge / Caronti, Luca; Akhunov, Khakim; Nardello, Matteo; Yildirim, Kasim Sinan; Brunelli, Davide. - In: ACM TRANSACTIONS ON EMBEDDED COMPUTING SYSTEMS. - ISSN 1539-9087. - 22:5(2023), pp. 8201-8219. [10.1145/3608475]

Fine-grained Hardware Acceleration for Efficient Batteryless Intermittent Inference on the Edge

LUCA CARONTI;KHAKIM AKHUNOV;MATTEO NARDELLO;KASIM SINAN YILDIRIM;DAVIDE BRUNELLI
2023-01-01

Abstract

Backing up the intermediate results of hardware-accelerated deep inference is crucial to ensure the progress of execution on batteryless computing platforms. However, hardware accelerators in low-power AI platforms only support the one-shot atomic execution of one neural network inference without any backups. This paper introduces a new toolchain for MAX78000, which is a brand-new microcontroller with a hardware-based convolutional neural network (CNN) accelerator. Our toolchain converts any MAX78000-compatible neural network into an intermittently executable form. The toolchain enables finer checkpoint granularity on the MAX78000 CNN accelerator, allowing for backups of any intermediate neural network layer output. Based on the layer-by-layer CNN execution, we propose a new backup technique that performs only necessary (urgent) checkpoints. The method involves the batteryless system switching to ultra-low-power mode while charging, saving intermediate results only when input power is lower than ultra-low-power mode energy consumption. By avoiding unnecessary memory transfer, the proposed solution increases the inference throughput by 1.9x for simulation and by 1.2x for real-world setup compared to the coarse-grained baseline execution.
2023
5
Caronti, Luca; Akhunov, Khakim; Nardello, Matteo; Yildirim, Kasim Sinan; Brunelli, Davide
Fine-grained Hardware Acceleration for Efficient Batteryless Intermittent Inference on the Edge / Caronti, Luca; Akhunov, Khakim; Nardello, Matteo; Yildirim, Kasim Sinan; Brunelli, Davide. - In: ACM TRANSACTIONS ON EMBEDDED COMPUTING SYSTEMS. - ISSN 1539-9087. - 22:5(2023), pp. 8201-8219. [10.1145/3608475]
File in questo prodotto:
File Dimensione Formato  
FINAL_ACM_TECS___Fine_grained_Hardware_Acceleration_for_Efficient_Batteryless_Intermittent_Inference_on_the_Edge.pdf

accesso aperto

Tipologia: Pre-print non referato (Non-refereed preprint)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 1.47 MB
Formato Adobe PDF
1.47 MB Adobe PDF Visualizza/Apri
EarlyAccess.pdf

Solo gestori archivio

Tipologia: Post-print referato (Refereed author’s manuscript)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 978.38 kB
Formato Adobe PDF
978.38 kB Adobe PDF   Visualizza/Apri
3608475.pdf

accesso aperto

Tipologia: Versione editoriale (Publisher’s layout)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 2.37 MB
Formato Adobe PDF
2.37 MB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11572/383629
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 3
  • ???jsp.display-item.citation.isi??? 3
  • OpenAlex ND
social impact