Ski Pose Estimation

Human Pose Estimation (HPE) aims at retrieving the position of human joints from images or videos. In many winter sports, the human pose captures only part of the action, as practitioners use skis and poles for propulsion and balance on the snow. In this work we propose a novel architecture that estimates the position of skis and poles as an integral part of the human skeleton. We leverage available algorithms to estimate the human body pose, which is then used as a prior for a novel self-supervised estimation approach for ski poses to retrieve the position of skis and poles using a lightweight transformer. The architecture learns to reconstruct randomly missing joints, that are masked at training time. During testing, the network the whole skeleton, which now includes also skis and poles. To overcome the lack of annotated data of ski position, we employ a variety of techniques to help our architecture establishing a link between the human body pose prior and the ski position, such as domain adaptation, data annotation and synthetic data augmentation. Our results show how our method can successfully retrieve the ski position for ski-jumping scenarios.