The increasing size of scientific datasets has caused a deep transformation in the way scientific research is currently carried out. In multiple domains, Big Data challenges have called for novel software solutions capable of exploiting fast computing resources, workflows technologies and parallel paradigms at scale, by providing a high-level of abstraction while hiding, at the same time, the underlying infrastructural and software complexity from scientists. This paper describes PyOphidia, an open-source Python module for High Performance Data Analytics on multi-dimensional scientific datasets. PyOphidia aims to simplify the execution of parallel data analysis over scientific datacubes on High Performance Computing infrastructures. It can be easily integrated with other existing Python libraries and tools, within multiple data science environments.

PyOphidia: A Python library for High Performance Data Analytics at scale / Elia, Donatello; Palazzo, Cosimo; Fiore, Sandro Luigi; D’Anca, Alessandro; Mariello, Andrea; Aloisio, Giovanni. - In: SOFTWAREX. - ISSN 2352-7110. - 24:(2023), pp. 1015381-1015386. [10.1016/j.softx.2023.101538]

PyOphidia: A Python library for High Performance Data Analytics at scale

Fiore, Sandro Luigi;Mariello, Andrea;
2023-01-01

Abstract

The increasing size of scientific datasets has caused a deep transformation in the way scientific research is currently carried out. In multiple domains, Big Data challenges have called for novel software solutions capable of exploiting fast computing resources, workflows technologies and parallel paradigms at scale, by providing a high-level of abstraction while hiding, at the same time, the underlying infrastructural and software complexity from scientists. This paper describes PyOphidia, an open-source Python module for High Performance Data Analytics on multi-dimensional scientific datasets. PyOphidia aims to simplify the execution of parallel data analysis over scientific datacubes on High Performance Computing infrastructures. It can be easily integrated with other existing Python libraries and tools, within multiple data science environments.
2023
Elia, Donatello; Palazzo, Cosimo; Fiore, Sandro Luigi; D’Anca, Alessandro; Mariello, Andrea; Aloisio, Giovanni
PyOphidia: A Python library for High Performance Data Analytics at scale / Elia, Donatello; Palazzo, Cosimo; Fiore, Sandro Luigi; D’Anca, Alessandro; Mariello, Andrea; Aloisio, Giovanni. - In: SOFTWAREX. - ISSN 2352-7110. - 24:(2023), pp. 1015381-1015386. [10.1016/j.softx.2023.101538]
File in questo prodotto:
File Dimensione Formato  
1-s2.0-S2352711023002340-main.pdf

accesso aperto

Tipologia: Versione editoriale (Publisher’s layout)
Licenza: Creative commons
Dimensione 1.16 MB
Formato Adobe PDF
1.16 MB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11572/401049
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 3
  • ???jsp.display-item.citation.isi??? 1
social impact