Recent works in computer vision and multimedia have shown that image memorability can be automatically inferred exploiting powerful deep-learning models. This article advances the state of the art in this area by addressing a novel and more challenging issue: “Given an arbitrary input image, can we make it more memorable?” To tackle this problem, we introduce an approach based on an editing-by-applying-filters paradigm:given an input image, we propose to automatically retrieve a set of “style seeds,” i.e., a set of style images that, applied to the input image through a neural style transfer algorithm, provide the highest increase in memorability. We show the effectiveness of the proposed approach with experiments on the publicly available LaMem dataset, performing both a quantitative evaluation and a user study. To demonstrate the flexibility of the proposed framework, we also analyze the impact of different implementation choices, such as using different state-of-the-art neural style transfer methods. Finally, we show several qualitative results to provide additional insights on the link between image style and memorability.

Increasing image memorability with neural style transfer / Siarohin, A.; Zen, G.; Majtanovic, C.; Alameda-Pineda, X.; Ricci, E.; Sebe, N.. - In: ACM TRANSACTIONS ON MULTIMEDIA COMPUTING, COMMUNICATIONS AND APPLICATIONS. - ISSN 1551-6857. - 15:2(2019), pp. 4201-4222. [10.1145/3311781]

Increasing image memorability with neural style transfer

A. Siarohin;G. Zen;C. Majtanovic;X. Alameda-Pineda;E. Ricci;N. Sebe
2019-01-01

Abstract

Recent works in computer vision and multimedia have shown that image memorability can be automatically inferred exploiting powerful deep-learning models. This article advances the state of the art in this area by addressing a novel and more challenging issue: “Given an arbitrary input image, can we make it more memorable?” To tackle this problem, we introduce an approach based on an editing-by-applying-filters paradigm:given an input image, we propose to automatically retrieve a set of “style seeds,” i.e., a set of style images that, applied to the input image through a neural style transfer algorithm, provide the highest increase in memorability. We show the effectiveness of the proposed approach with experiments on the publicly available LaMem dataset, performing both a quantitative evaluation and a user study. To demonstrate the flexibility of the proposed framework, we also analyze the impact of different implementation choices, such as using different state-of-the-art neural style transfer methods. Finally, we show several qualitative results to provide additional insights on the link between image style and memorability.
2019
2
Siarohin, A.; Zen, G.; Majtanovic, C.; Alameda-Pineda, X.; Ricci, E.; Sebe, N.
Increasing image memorability with neural style transfer / Siarohin, A.; Zen, G.; Majtanovic, C.; Alameda-Pineda, X.; Ricci, E.; Sebe, N.. - In: ACM TRANSACTIONS ON MULTIMEDIA COMPUTING, COMMUNICATIONS AND APPLICATIONS. - ISSN 1551-6857. - 15:2(2019), pp. 4201-4222. [10.1145/3311781]
File in questo prodotto:
File Dimensione Formato  
331178.pdf

Solo gestori archivio

Tipologia: Versione editoriale (Publisher’s layout)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 8.33 MB
Formato Adobe PDF
8.33 MB Adobe PDF   Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11572/250718
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 14
  • ???jsp.display-item.citation.isi??? 5
social impact