Graph attention models (A-GNNs), a type of Graph Neural Networks (GNNs), have been shown to be more powerful than simpler convolutional GNNs (C-GNNs). However, A-GNNs are more complex to program and difficult to scale. To address this, we develop a novel mathematical formulation, based on tensors that group all the feature vectors, targeting both training and inference of A-GNNs. The formulation enables straightforward adoption of communication-minimizing routines, it fosters optimizations such as vectorization, and it enables seamless integration with established linear algebra DSLs or libraries such as GraphBLAS. Our implementation uses a data redistribution scheme explicitly developed for sparse-dense tensor operations used heavily in GNNs, and fusing optimizations that further minimize memory usage and communication cost. We ensure theoretical asymptotic reductions in communicated data compared to the established message-passing GNN paradigm. Finally, we provide excellent scalability and speedups of even 4 - 5x over modern libraries such as Deep Graph Library.

High-Performance and Programmable Attentional Graph Neural Networks with Global Tensor Formulations / Besta, M.; Renc, P.; Gerstenberger, R.; Sylos Labini, P.; Ziogas, A.; Chen, T.; Gianinazzi, L.; Scheidl, F.; Szenes, K.; Carigiet, A.; Iff, P.; Kwasniewski, G.; Kanakagiri, R.; Ge, C.; Jaeger, S.; Was, J.; Vella, F.; Hoefler, T.. - (2023), pp. -16. (Intervento presentato al convegno 2023 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2023 tenutosi a Stati Uniti d'America nel 2023) [10.1145/3581784.3607067].

High-Performance and Programmable Attentional Graph Neural Networks with Global Tensor Formulations

Vella F.;
2023-01-01

Abstract

Graph attention models (A-GNNs), a type of Graph Neural Networks (GNNs), have been shown to be more powerful than simpler convolutional GNNs (C-GNNs). However, A-GNNs are more complex to program and difficult to scale. To address this, we develop a novel mathematical formulation, based on tensors that group all the feature vectors, targeting both training and inference of A-GNNs. The formulation enables straightforward adoption of communication-minimizing routines, it fosters optimizations such as vectorization, and it enables seamless integration with established linear algebra DSLs or libraries such as GraphBLAS. Our implementation uses a data redistribution scheme explicitly developed for sparse-dense tensor operations used heavily in GNNs, and fusing optimizations that further minimize memory usage and communication cost. We ensure theoretical asymptotic reductions in communicated data compared to the established message-passing GNN paradigm. Finally, we provide excellent scalability and speedups of even 4 - 5x over modern libraries such as Deep Graph Library.
2023
Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2023
New York, USA
Association for Computing Machinery, Inc
9798400701092
Besta, M.; Renc, P.; Gerstenberger, R.; Sylos Labini, P.; Ziogas, A.; Chen, T.; Gianinazzi, L.; Scheidl, F.; Szenes, K.; Carigiet, A.; Iff, P.; Kwasni...espandi
High-Performance and Programmable Attentional Graph Neural Networks with Global Tensor Formulations / Besta, M.; Renc, P.; Gerstenberger, R.; Sylos Labini, P.; Ziogas, A.; Chen, T.; Gianinazzi, L.; Scheidl, F.; Szenes, K.; Carigiet, A.; Iff, P.; Kwasniewski, G.; Kanakagiri, R.; Ge, C.; Jaeger, S.; Was, J.; Vella, F.; Hoefler, T.. - (2023), pp. -16. (Intervento presentato al convegno 2023 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2023 tenutosi a Stati Uniti d'America nel 2023) [10.1145/3581784.3607067].
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11572/401009
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 0
  • ???jsp.display-item.citation.isi??? ND
social impact