Traditional guided image translation methods, based on encoder–decoder or U-Net structures, often struggle with complex or contrasting images. To address this, we introduce a novel dual-stage strategy. First, we use a cascaded cross-gating MLP-Mixer to merge image and semantic guidance codes, generating intermediate results influenced by these cues. Second, we implement a refined pixel-level loss function to handle semantic guidance noise, along with a new cross-attention gating mechanism for detail refinement. Additionally, our framework utilizes an MLP-Mixer-based discriminator, ensuring that the entire system is built on the MLP-Mixer architecture. Our results in cross-view image translation and person image synthesis outperform current benchmarks, demonstrating the effectiveness of our method.

A pure MLP-Mixer-based GAN framework for guided image translation / Tang, H.; Ren, B.; Sebe, N.. - In: PATTERN RECOGNITION. - ISSN 0031-3203. - 157:(2025). [10.1016/j.patcog.2024.110894]

A pure MLP-Mixer-based GAN framework for guided image translation

Tang H.;Ren B.;Sebe N.
2025-01-01

Abstract

Traditional guided image translation methods, based on encoder–decoder or U-Net structures, often struggle with complex or contrasting images. To address this, we introduce a novel dual-stage strategy. First, we use a cascaded cross-gating MLP-Mixer to merge image and semantic guidance codes, generating intermediate results influenced by these cues. Second, we implement a refined pixel-level loss function to handle semantic guidance noise, along with a new cross-attention gating mechanism for detail refinement. Additionally, our framework utilizes an MLP-Mixer-based discriminator, ensuring that the entire system is built on the MLP-Mixer architecture. Our results in cross-view image translation and person image synthesis outperform current benchmarks, demonstrating the effectiveness of our method.
2025
Tang, H.; Ren, B.; Sebe, N.
A pure MLP-Mixer-based GAN framework for guided image translation / Tang, H.; Ren, B.; Sebe, N.. - In: PATTERN RECOGNITION. - ISSN 0031-3203. - 157:(2025). [10.1016/j.patcog.2024.110894]
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11572/427291
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 0
  • ???jsp.display-item.citation.isi??? ND
social impact