Cross-modality average precision optimization for visible thermal person re-identification

Metric learning has emerged as a popular approach for addressing the challenges of visible thermal person re-identification (VT-ReID), such as the cross-modality discrepancy and intra-class variations. However, existing metric learning-based methods often focus on optimizing the model for hard positive samples, neglecting the importance of high-ranking ones, due to failing to consider the overall ranking order within a batch. To overcome this limitation, we propose a novel approach called Cross-modality Average Precision (CAP) that directly optimizes the cross-modality overall ranking order in VT-ReID. Unlike the recently introduced Smooth Average Precision (Smooth-AP), which primarily corrects misordered samples at high ranks, CAP specifically targets the main challenge of cross-modality discrepancy in VT-ReID. Our method involves setting a query instance from one modality and calculating the CAP using galleries from another modality. CAP encompasses two complementary aspects: CAP with Visible queries (CAPV) and CAP with Thermal queries (CAPT). By jointly optimizing these two aspects, we can effectively improve the cross-modality overall ranking order. Additionally, to enhance the effectiveness of CAP, we introduce two techniques. The first technique is Dynamic Modality Alignment (DMA), which reduces the cross-modality discrepancy by adaptively adjusting the weights of modality alignment. The second technique involves implementing CAP and DMA on the Global and Local Features (GLF), enabling us to optimize the model at both global and local levels, further enhancing the advantages of CAP and DMA. We conducted extensive experiments on two VT-ReID datasets, and the results demonstrate the effectiveness of our proposed method, which achieves state-of-the-art performance.