In this paper, we propose a novel Pattern-Affinitive Propagation (PAP) framework to jointly predict depth, surface normal and semantic segmentation. The motivation behind it comes from the statistic observation that pattern-affinitive pairs recur much frequently across different tasks as well as within a task. Thus, we can conduct two types of propagations, cross-task propagation and task-specific propagation, to adaptively diffuse those similar patterns. The former integrates cross-task affinity patterns to adapt to each task therein through the calculation on non-local relationships. Next the latter performs an iterative diffusion in the feature space so that the cross-task affinity patterns can be widely-spread within the task. Accordingly, the learning of each task can be regularized and boosted by the complementary task-level affinities. Extensive experiments demonstrate the effectiveness and the superiority of our method on the joint three tasks. Meanwhile, we achieve the state-of-the-art or competitive results on the three related datasets, NYUD-v2, SUN-RGBD and KITTI.

Pattern-Affinitive Propagation Across Depth, Surface Normal and Semantic Segmentation / Zhang, Zhenyu; Cui, Zhen; Xu, Chunyan; Yan, Yan; Sebe, Nicu; Yang, Jian. - (2019), pp. 4101-4110. (Intervento presentato al convegno IEEE Comference on Computer Vision and Pattern Recognition (CVPR'19) tenutosi a Long Beach nel June 16-20, 2019) [10.1109/CVPR.2019.00423].

Pattern-Affinitive Propagation Across Depth, Surface Normal and Semantic Segmentation

Yan, Yan;Sebe, Nicu;
2019-01-01

Abstract

In this paper, we propose a novel Pattern-Affinitive Propagation (PAP) framework to jointly predict depth, surface normal and semantic segmentation. The motivation behind it comes from the statistic observation that pattern-affinitive pairs recur much frequently across different tasks as well as within a task. Thus, we can conduct two types of propagations, cross-task propagation and task-specific propagation, to adaptively diffuse those similar patterns. The former integrates cross-task affinity patterns to adapt to each task therein through the calculation on non-local relationships. Next the latter performs an iterative diffusion in the feature space so that the cross-task affinity patterns can be widely-spread within the task. Accordingly, the learning of each task can be regularized and boosted by the complementary task-level affinities. Extensive experiments demonstrate the effectiveness and the superiority of our method on the joint three tasks. Meanwhile, we achieve the state-of-the-art or competitive results on the three related datasets, NYUD-v2, SUN-RGBD and KITTI.
2019
IEEE Comference on Computer Vision and Pattern Recognition (CVPR'19)
New York
IEEE
978-1-7281-3293-8
Zhang, Zhenyu; Cui, Zhen; Xu, Chunyan; Yan, Yan; Sebe, Nicu; Yang, Jian
Pattern-Affinitive Propagation Across Depth, Surface Normal and Semantic Segmentation / Zhang, Zhenyu; Cui, Zhen; Xu, Chunyan; Yan, Yan; Sebe, Nicu; Yang, Jian. - (2019), pp. 4101-4110. (Intervento presentato al convegno IEEE Comference on Computer Vision and Pattern Recognition (CVPR'19) tenutosi a Long Beach nel June 16-20, 2019) [10.1109/CVPR.2019.00423].
File in questo prodotto:
File Dimensione Formato  
Zhang_Pattern-Affinitive_Propagation_Across_Depth_Surface_Normal_and_Semantic_Segmentation_CVPR_2019_paper.pdf

accesso aperto

Tipologia: Post-print referato (Refereed author’s manuscript)
Licenza: Altra licenza (Other type of license)
Dimensione 2.17 MB
Formato Adobe PDF
2.17 MB Adobe PDF Visualizza/Apri
08953513.pdf

Solo gestori archivio

Tipologia: Versione editoriale (Publisher’s layout)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 1.36 MB
Formato Adobe PDF
1.36 MB Adobe PDF   Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11572/250765
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 212
  • ???jsp.display-item.citation.isi??? 152
social impact