Recently, there has been an increasing interest in image editing methods that employ pre-trained unconditional image generators (e.g., StyleGAN). However, applying these methods to translate images to multiple visual domains remains challenging. Existing works do not often preserve the domain-invariant part of the image (e.g., the identity in human face translations), or they do not usually handle multiple domains or allow for multi-modal translations. This work proposes an implicit style function (ISF) to straightforwardly achieve multi-modal and multi-domain image-to-image translation from pre-trained unconditional generators. The ISF manipulates the semantics of a latent code to ensure that the image generated from the manipulated code lies in the desired visual domain. Our human faces and animal image manipulations show significantly improved results over the baselines. Our model enables cost-effective multi-modal unsupervised image-to-image translations at high resolution using pre-trained unconditional GANs. The code and data are available at: https://github.com/yhlleo/stylegan-mmuit.

ISF-GAN: An Implicit Style Function for High-Resolution Image-to-Image Translation / Liu, Y.; Chen, Y.; Bao, L.; Sebe, N.; Lepri, B.; De Nadai, M.. - In: IEEE TRANSACTIONS ON MULTIMEDIA. - ISSN 1520-9210. - 25:(2023), pp. 3343-3353. [10.1109/TMM.2022.3159115]

ISF-GAN: An Implicit Style Function for High-Resolution Image-to-Image Translation

Liu, Y.;Chen, Y.;Sebe, N.;Lepri, B.;De Nadai, M.
2023-01-01

Abstract

Recently, there has been an increasing interest in image editing methods that employ pre-trained unconditional image generators (e.g., StyleGAN). However, applying these methods to translate images to multiple visual domains remains challenging. Existing works do not often preserve the domain-invariant part of the image (e.g., the identity in human face translations), or they do not usually handle multiple domains or allow for multi-modal translations. This work proposes an implicit style function (ISF) to straightforwardly achieve multi-modal and multi-domain image-to-image translation from pre-trained unconditional generators. The ISF manipulates the semantics of a latent code to ensure that the image generated from the manipulated code lies in the desired visual domain. Our human faces and animal image manipulations show significantly improved results over the baselines. Our model enables cost-effective multi-modal unsupervised image-to-image translations at high resolution using pre-trained unconditional GANs. The code and data are available at: https://github.com/yhlleo/stylegan-mmuit.
2023
Liu, Y.; Chen, Y.; Bao, L.; Sebe, N.; Lepri, B.; De Nadai, M.
ISF-GAN: An Implicit Style Function for High-Resolution Image-to-Image Translation / Liu, Y.; Chen, Y.; Bao, L.; Sebe, N.; Lepri, B.; De Nadai, M.. - In: IEEE TRANSACTIONS ON MULTIMEDIA. - ISSN 1520-9210. - 25:(2023), pp. 3343-3353. [10.1109/TMM.2022.3159115]
File in questo prodotto:
File Dimensione Formato  
ISF-GAN_TMM22-compressed.pdf

Solo gestori archivio

Tipologia: Post-print referato (Refereed author’s manuscript)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 589.82 kB
Formato Adobe PDF
589.82 kB Adobe PDF   Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11572/388192
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 4
  • ???jsp.display-item.citation.isi??? 1
social impact