The Mystery of Faces: Investigating Face Contribution for Multimedia Event Detection

Multimedia event detection (MED) is a retrieval task with the goal of finding videos of a particular event in a large scale internet video archive, given example videos and text de- scriptions. Nowadays, different multimodal fusion schemes of low-level and high-level features are extensively investigated and evaluated for MED. For most of events in MED, people are usually the central subjects in videos. The face of a person can be considered as the most important fac- tor which brings a lot of information describing the video events. However, face information has not been systemati- cally investigated in the previous research for MED. In this paper, we investigate the possibility of using the high-level face information to assist multimedia event detection. More- over, since the labeled data in TRECVID MED dataset are limited, we propose a semi-supervised kernel ridge regres- sion which works well in practice to explore the useful in- formation from unlabeled data to assist the event d...