Integrating Computer Vision Algorithms and Ontologies for Spectator Crowd Behavior Analysis

Capturing and understanding crowd dynamics is an important issue under diverse perspectives. From social, psychological, and political sciences to safety management, studying, modeling, and predicting the presence, behavior, and dynamics of crowds, possibly preventing dangerous activities, is absolutely crucial. In the literature, crowds have been classified under different categories depending on their size and focus of attention. This chapter focuses on spectator crowds, namely crowds formed by people whose behavior is constrained by a structured environment, whose focus of attention is mainly shared, directed to a specific event. We first propose the backbone of an ontology of spectator crowd behavior based on a foundational analysis of both related literature and S-Hock, a massive annotated video dataset on crowd behavior during hockey events. Then, we present a new methodological approach integrating ontological reasoning, performed with a new description logic-based temporal formalism, with computer vision algorithms, allowing for automatic recognition of events happening in the playground, based on the behavior of the crowd in the stands.