Scientific workflows and provenance are two faces of the same medal. While the former addresses the coordinated execution of multiple tasks over a set of computational resources, the latter relates to the historical record of data from its original sources. This paper highlights the importance of tracking multi-level provenance metadata in complex, AI-based scientific workflows as a way to (i) foster and (ii) expand documentation of experiments, (iii) enable reproducibility, (iv) address interpretability of the results, (v) facilitate performance bottlenecks diagnosis, and (vi) advance provenance exploration and analysis opportunities.
Scientific workflows and provenance are two faces of the same medal. While the former addresses the coordinated execution of multiple tasks over a set of computational resources, the latter relates to the historical record of data from its original sources. This paper highlights the importance of tracking multi-level provenance metadata in complex, AIbased scientific workflows as a way to (i) foster and (ii) expand documentation of experiments, (iii) enable reproducibility, (iv) address interpretability of the results, (v) facilitate performance bottlenecks diagnosis, and (vi) advance provenance exploration and analysis opportunities.
A software ecosystem for multi-level provenance management in large-scale scientific workflows for AI applications / Padovani, Gabriele; Anantharaj, Valentine; Sacco, Ludovica; Kurihana, Takuya; Bunino, Matteo; Tsolaki, Kalliopi; Girone, Maria; Antonio, Fabrizio; Sopranzetti, Carolina; Fronza, Massimiliano; Fiore, Sandro L.. - (2024), pp. 2024-2031. ( 2024 Workshops of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC Workshops 2024 Atlanta, Georgia 17-22 November 2024 (Workshop 18 November 2024)) [10.1109/SCW63240.2024.00253].
A software ecosystem for multi-level provenance management in large-scale scientific workflows for AI applications
Padovani, Gabriele
;Sacco, Ludovica;Fiore, Sandro L.
2024-01-01
Abstract
Scientific workflows and provenance are two faces of the same medal. While the former addresses the coordinated execution of multiple tasks over a set of computational resources, the latter relates to the historical record of data from its original sources. This paper highlights the importance of tracking multi-level provenance metadata in complex, AI-based scientific workflows as a way to (i) foster and (ii) expand documentation of experiments, (iii) enable reproducibility, (iv) address interpretability of the results, (v) facilitate performance bottlenecks diagnosis, and (vi) advance provenance exploration and analysis opportunities.| File | Dimensione | Formato | |
|---|---|---|---|
|
A_software_ecosystem_for_multi-level_provenance_management_in_large-scale_scientific_workflows_for_AI_applications.pdf
accesso aperto
Descrizione: Accepted Manuscript
Tipologia:
Post-print referato (Refereed author’s manuscript)
Licenza:
Tutti i diritti riservati (All rights reserved)
Dimensione
500.49 kB
Formato
Adobe PDF
|
500.49 kB | Adobe PDF | Visualizza/Apri |
|
A_software_ecosystem_for_multi-level_provenance_management_in_large-scale_scientific_workflows_for_AI_applications.pdf
Solo gestori archivio
Tipologia:
Versione editoriale (Publisher’s layout)
Licenza:
Tutti i diritti riservati (All rights reserved)
Dimensione
649.18 kB
Formato
Adobe PDF
|
649.18 kB | Adobe PDF | Visualizza/Apri |
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione
