The increasing size of scientific datasets has caused a deep transformation in the way scientific research is currently carried out. In multiple domains, Big Data challenges have called for novel software solutions capable of exploiting fast computing resources, workflows technologies and parallel paradigms at scale, by providing a high-level of abstraction while hiding, at the same time, the underlying infrastructural and software complexity from scientists. This paper describes PyOphidia, an open-source Python module for High Performance Data Analytics on multi-dimensional scientific datasets. PyOphidia aims to simplify the execution of parallel data analysis over scientific datacubes on High Performance Computing infrastructures. It can be easily integrated with other existing Python libraries and tools, within multiple data science environments.
PyOphidia: A Python library for High Performance Data Analytics at scale / Elia, Donatello; Palazzo, Cosimo; Fiore, Sandro Luigi; D’Anca, Alessandro; Mariello, Andrea; Aloisio, Giovanni. - In: SOFTWAREX. - ISSN 2352-7110. - 24:(2023), pp. 1015381-1015386. [10.1016/j.softx.2023.101538]
PyOphidia: A Python library for High Performance Data Analytics at scale
Fiore, Sandro Luigi;Mariello, Andrea;
2023-01-01
Abstract
The increasing size of scientific datasets has caused a deep transformation in the way scientific research is currently carried out. In multiple domains, Big Data challenges have called for novel software solutions capable of exploiting fast computing resources, workflows technologies and parallel paradigms at scale, by providing a high-level of abstraction while hiding, at the same time, the underlying infrastructural and software complexity from scientists. This paper describes PyOphidia, an open-source Python module for High Performance Data Analytics on multi-dimensional scientific datasets. PyOphidia aims to simplify the execution of parallel data analysis over scientific datacubes on High Performance Computing infrastructures. It can be easily integrated with other existing Python libraries and tools, within multiple data science environments.File | Dimensione | Formato | |
---|---|---|---|
1-s2.0-S2352711023002340-main.pdf
accesso aperto
Tipologia:
Versione editoriale (Publisher’s layout)
Licenza:
Creative commons
Dimensione
1.16 MB
Formato
Adobe PDF
|
1.16 MB | Adobe PDF | Visualizza/Apri |
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione