A Hierarchical Framework for Motion Trajectory Forecasting Based on Modality Sampling

In this paper, we present a hierarchical framework for multi-modal trajectory forecasting, which can provide for each pedestrian in the scene the distributions for the next moves at every time step. The overall architecture adopts a standard encoder-decoder paradigm, where the encoder is based on a self-attention mechanism to extract the temporal features of motion histories, while the decoder is built upon a stack of LSTMs to generate the future path sequentially. The model is learned in a discriminative manner, with the purpose of differentiating among varied motion modalities. To this end, we propose a clustering strategy to construct the so-called transformation set. The transformation set collaborates with the hierarchical LSTMs in the decoder, in order to approximate the real distributions in the training data. Experimental results demonstrate that the proposed framework can not only predict the future trajectory accurately, but also provide multi-modal trajectory distributions explicitly.