We address the challenging Voice Activity Detection (VAD) problem, which determines “Who is Speaking and When?” in audiovisual recordings. The typical audiobased VAD systems can be ineffective in the presence of ambient noise or noise variations. Moreover, due to technical or privacy reasons, audio might not be always available. In such cases, the use of video modality to perform VAD is desirable. Almost all existing visual VAD methods rely on body part detection, e.g., face, lips, or hands. In contrast, we propose a novel visual VAD method operating directly on the entire video frame, without the explicit need of detecting a person or his/her body parts. Our method, named S-VVAD, learns body motion cues associated with speech activity within a weakly supervised segmentation framework. Therefore, it not only detects the speakers/not-speakers but simultaneously localizes the image positions of them. It is an end-to-end pipeline, personindependent and it does not require any prior knowledge nor pre-processing. S-VVAD performs well in various challenging conditions and demonstrates the state-of-the-art results on multiple datasets. Moreover, the better generalization capability of S-VVAD is confirmed for cross-dataset and person-independent scenarios.
S-VVAD: Visual Voice Activity Detection by Motion Segmentation / Shahid, Muhammad; Beyan, Cigdem; Murino, Vittorio. - (2021), pp. 2331-2340. ((Intervento presentato al convegno WACV tenutosi a Online nel 5-9 January [10.1109/WACV48630.2021.00238].
Scheda prodotto non validato
|Titolo:||S-VVAD: Visual Voice Activity Detection by Motion Segmentation|
|Autori:||Shahid, Muhammad; Beyan, Cigdem; Murino, Vittorio|
|Titolo del volume contenente il saggio:||IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2021|
|Luogo di edizione:||New York|
|Anno di pubblicazione:||2021|
|Citazione:||S-VVAD: Visual Voice Activity Detection by Motion Segmentation / Shahid, Muhammad; Beyan, Cigdem; Murino, Vittorio. - (2021), pp. 2331-2340. ((Intervento presentato al convegno WACV tenutosi a Online nel 5-9 January [10.1109/WACV48630.2021.00238].|
|Appare nelle tipologie:||04.1 Saggio in atti di convegno (Paper in proceedings)|