With the strength of deep generative models, 3D pose transfer regains intensive research interests in recent years. Existing methods mainly rely on a variety of constraints to achieve the pose transfer over 3D meshes, e.g., the need for manually encoding for shape and pose disentanglement. In this paper, we present an unsupervised approach to conduct the pose transfer between any arbitrate given 3D meshes. Specifically, a novel Intrinsic-Extrinsic Preserved Generative Adversarial Network (IEP-GAN) is presented for both intrinsic (i.e., shape) and extrinsic (i.e., pose) information preservation. Extrinsically, we propose a co-occurrence discriminator to capture the structural/pose invariance from distinct Laplacians of the mesh. Meanwhile, intrinsically, a local intrinsic-preserved loss is introduced to preserve the geodesic priors while avoiding heavy computations. At last, we show the possibility of using IEP-GAN to manipulate 3D human meshes in various ways, including pose transfer, identity swapping and pose interpolation with latent code vector arithmetic. The extensive experiments on various 3D datasets of humans, animals and hands qualitatively and quantitatively demonstrate the generality of our approach. Our proposed model produces better results and is substantially more efficient compared to recent state-of-the-art methods. Code is available: https://github.com/mikecheninoulu/Unsupervised_IEPGAN

Intrinsic-Extrinsic Preserved GANs for Unsupervised 3D Pose Transfer / Chen, Haoyu; Tang, Hao; Shi, Henglin; Peng, Wei; Sebe, Nicu; Zhao, Guoying. - (2021), pp. 8610-8619. (Intervento presentato al convegno ICCV’21 tenutosi a Virtual event nel 11th-17th October 2021) [10.1109/ICCV48922.2021.00851].

Intrinsic-Extrinsic Preserved GANs for Unsupervised 3D Pose Transfer

Tang, Hao;Sebe, Nicu;
2021-01-01

Abstract

With the strength of deep generative models, 3D pose transfer regains intensive research interests in recent years. Existing methods mainly rely on a variety of constraints to achieve the pose transfer over 3D meshes, e.g., the need for manually encoding for shape and pose disentanglement. In this paper, we present an unsupervised approach to conduct the pose transfer between any arbitrate given 3D meshes. Specifically, a novel Intrinsic-Extrinsic Preserved Generative Adversarial Network (IEP-GAN) is presented for both intrinsic (i.e., shape) and extrinsic (i.e., pose) information preservation. Extrinsically, we propose a co-occurrence discriminator to capture the structural/pose invariance from distinct Laplacians of the mesh. Meanwhile, intrinsically, a local intrinsic-preserved loss is introduced to preserve the geodesic priors while avoiding heavy computations. At last, we show the possibility of using IEP-GAN to manipulate 3D human meshes in various ways, including pose transfer, identity swapping and pose interpolation with latent code vector arithmetic. The extensive experiments on various 3D datasets of humans, animals and hands qualitatively and quantitatively demonstrate the generality of our approach. Our proposed model produces better results and is substantially more efficient compared to recent state-of-the-art methods. Code is available: https://github.com/mikecheninoulu/Unsupervised_IEPGAN
2021
Proceedings: 2021 IEEE/CVF International Conference on Computer Vision
Piscataway, NJ
IEEE
978-1-6654-2812-5
Chen, Haoyu; Tang, Hao; Shi, Henglin; Peng, Wei; Sebe, Nicu; Zhao, Guoying
Intrinsic-Extrinsic Preserved GANs for Unsupervised 3D Pose Transfer / Chen, Haoyu; Tang, Hao; Shi, Henglin; Peng, Wei; Sebe, Nicu; Zhao, Guoying. - (2021), pp. 8610-8619. (Intervento presentato al convegno ICCV’21 tenutosi a Virtual event nel 11th-17th October 2021) [10.1109/ICCV48922.2021.00851].
File in questo prodotto:
File Dimensione Formato  
Chen_Intrinsic-Extrinsic_Preserved_GANs_for_Unsupervised_3D_Pose_Transfer_ICCV_2021_paper (1).pdf

accesso aperto

Tipologia: Post-print referato (Refereed author’s manuscript)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 3.35 MB
Formato Adobe PDF
3.35 MB Adobe PDF Visualizza/Apri
Intrinsic-Extrinsic_Preserved_GANs_for_Unsupervised_3D_Pose_Transfer.pdf

Solo gestori archivio

Tipologia: Versione editoriale (Publisher’s layout)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 3.95 MB
Formato Adobe PDF
3.95 MB Adobe PDF   Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11572/326206
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 14
  • ???jsp.display-item.citation.isi??? 6
social impact