Part-prototype Networks (ProtoPNets) are concept-based classifiers designed to achieve the same performance as black-box models without compromising transparency. ProtoPNets compute predictions based on similarity to class-specific part-prototypes learned to recognize parts of training examples, making it easy to faithfully determine what examples are responsible for any target prediction and why. However, like other models, they are prone to picking up confounders and shortcuts from the data, thus suffering from compromised prediction accuracy and limited generalization. We propose ProtoPDebug, an effective concept-level debugger for ProtoPNets in which a human supervisor, guided by the model's explanations, supplies feedback in the form of what part-prototypes must be forgotten or kept, and the model is fine-tuned to align with this supervision. Our experimental evaluation shows that ProtoPDebug outperforms state-of-the-art debuggers for a fraction of the annotation cost. An online experiment with laypeople confirms the simplicity of the feedback requested to the users and the effectiveness of the collected feedback for learning confounder-free part-prototypes. ProtoPDebug is a promising tool for trustworthy interactive learning in critical applications, as suggested by a preliminary evaluation on a medical decision making task.

Concept-level debugging of part-prototype networks / Bontempelli, Andrea; Teso, Stefano; Tentori, Katya; Giunchiglia, Fausto; Passerini, Andrea. - (2023), pp. 1-21. (Intervento presentato al convegno ICLR 2023 tenutosi a Kigali, Rwanda nel 1st -5th Maym, 2023).

Concept-level debugging of part-prototype networks

Bontempelli, Andrea
;
Teso, Stefano;Tentori, Katya;Giunchiglia, Fausto;Passerini, Andrea
2023-01-01

Abstract

Part-prototype Networks (ProtoPNets) are concept-based classifiers designed to achieve the same performance as black-box models without compromising transparency. ProtoPNets compute predictions based on similarity to class-specific part-prototypes learned to recognize parts of training examples, making it easy to faithfully determine what examples are responsible for any target prediction and why. However, like other models, they are prone to picking up confounders and shortcuts from the data, thus suffering from compromised prediction accuracy and limited generalization. We propose ProtoPDebug, an effective concept-level debugger for ProtoPNets in which a human supervisor, guided by the model's explanations, supplies feedback in the form of what part-prototypes must be forgotten or kept, and the model is fine-tuned to align with this supervision. Our experimental evaluation shows that ProtoPDebug outperforms state-of-the-art debuggers for a fraction of the annotation cost. An online experiment with laypeople confirms the simplicity of the feedback requested to the users and the effectiveness of the collected feedback for learning confounder-free part-prototypes. ProtoPDebug is a promising tool for trustworthy interactive learning in critical applications, as suggested by a preliminary evaluation on a medical decision making task.
2023
Proceedings of the The Eleventh International Conference on Learning Representations (ICLR 23)
Appleton, WI USA
ICLR
Bontempelli, Andrea; Teso, Stefano; Tentori, Katya; Giunchiglia, Fausto; Passerini, Andrea
Concept-level debugging of part-prototype networks / Bontempelli, Andrea; Teso, Stefano; Tentori, Katya; Giunchiglia, Fausto; Passerini, Andrea. - (2023), pp. 1-21. (Intervento presentato al convegno ICLR 2023 tenutosi a Kigali, Rwanda nel 1st -5th Maym, 2023).
File in questo prodotto:
File Dimensione Formato  
protopdebug___iclr23.pdf

accesso aperto

Tipologia: Post-print referato (Refereed author’s manuscript)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 6.39 MB
Formato Adobe PDF
6.39 MB Adobe PDF Visualizza/Apri
6296_concept_level_debugging_of_par.pdf

accesso aperto

Tipologia: Versione editoriale (Publisher’s layout)
Licenza: Creative commons
Dimensione 6.39 MB
Formato Adobe PDF
6.39 MB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11572/365047
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus ND
  • ???jsp.display-item.citation.isi??? ND
social impact