Inspired by the success of adversarial learning, we propose a new end-to-end unsupervised deep learning framework for monocular depth estimation consisting of two Generative Adversarial Networks (GAN), deeply coupled with a structured Conditional Random Field (CRF) model. The two GANs aim at generating distinct and complementary disparity maps and at improving the generation quality via exploiting the adversarial learning strategy. The deep CRF coupling model is proposed to fuse the generative and discriminative outputs from the dual GAN nets. As such, the model implicitly constructs mutual constraints on the two network branches and between the generator and discriminator. This facilitates the optimization of the whole network for better disparity generation. Extensive experiments on the KITTI, Cityscapes, and Make3D datasets clearly demonstrate the effectiveness of the proposed approach and show superior performance compared to state of the art methods. The code and models are available at https://github.com/mihaipuscas/3dv-coupled-crf-disparity.

Structured Coupled Generative Adversarial Networks for Unsupervised Monocular Depth Estimation / Puscas, M. M.; Xu, D.; Pilzer, A.; Sebe, N.. - (2019), pp. 18-26. (Intervento presentato al convegno 7th International Conference on 3D Vision, 3DV 2019 tenutosi a Quebec City, Canada nel 16-19 September, 2019) [10.1109/3DV.2019.00012].

Structured Coupled Generative Adversarial Networks for Unsupervised Monocular Depth Estimation

Puscas M. M.;Xu D.;Pilzer A.;Sebe N.
2019-01-01

Abstract

Inspired by the success of adversarial learning, we propose a new end-to-end unsupervised deep learning framework for monocular depth estimation consisting of two Generative Adversarial Networks (GAN), deeply coupled with a structured Conditional Random Field (CRF) model. The two GANs aim at generating distinct and complementary disparity maps and at improving the generation quality via exploiting the adversarial learning strategy. The deep CRF coupling model is proposed to fuse the generative and discriminative outputs from the dual GAN nets. As such, the model implicitly constructs mutual constraints on the two network branches and between the generator and discriminator. This facilitates the optimization of the whole network for better disparity generation. Extensive experiments on the KITTI, Cityscapes, and Make3D datasets clearly demonstrate the effectiveness of the proposed approach and show superior performance compared to state of the art methods. The code and models are available at https://github.com/mihaipuscas/3dv-coupled-crf-disparity.
2019
Proceedings - 2019 International Conference on 3D Vision, 3DV 2019
New York
Institute of Electrical and Electronics Engineers Inc.
978-1-7281-3131-3
Puscas, M. M.; Xu, D.; Pilzer, A.; Sebe, N.
Structured Coupled Generative Adversarial Networks for Unsupervised Monocular Depth Estimation / Puscas, M. M.; Xu, D.; Pilzer, A.; Sebe, N.. - (2019), pp. 18-26. (Intervento presentato al convegno 7th International Conference on 3D Vision, 3DV 2019 tenutosi a Quebec City, Canada nel 16-19 September, 2019) [10.1109/3DV.2019.00012].
File in questo prodotto:
File Dimensione Formato  
08885951.pdf

Solo gestori archivio

Tipologia: Versione editoriale (Publisher’s layout)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 3.56 MB
Formato Adobe PDF
3.56 MB Adobe PDF   Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11572/247606
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 17
  • ???jsp.display-item.citation.isi??? 14
  • OpenAlex ND
social impact