You Talkin’ to Me? Recognizing Complex Human Interactions in Unconstrained Videos

Nowadays, due to the exponential growth of the user generated videos and the prevailing videos sharing communities such as YouTube and Hulu, recognizing complex human activities in the wild becomes increasingly important in the research community. These videos are hard to study due to the frequent changes of camera viewpoint, multiple people moving in the scene, fast body movements, and varied lengths of video clips. In this paper, we propose a novel framework to analyze human interactions in TV shows. Firstly, we exploit the motion interchange pattern (MIP) to detect camera viewpoint changes in a video, and extract the salient motion points in the bounding box that covers the region of interest (ROI) in each frame. Then, we compute the large displacement optical flow for the salient pixels in the bounding box, and build the histogram of oriented optical flow as the motion feature vector for each frame. Finally, the self-similarity matrix (SSM) is adopted to capture the global temporal...