Inter-rater reliability of MRI Neck Imaging Reporting and Data System (NI-RADS) in the follow-up of oropharyngeal squamous cell carcinoma

Purpose: To assess the agreement of Neck Imaging Reporting and Data System (NI-RADS) using magnetic resonance imaging (MRI) among readers with different experience in the evaluation of oropharyngeal squamous cell carcinoma (OPSCC) patients. Material and methods: We conducted an observational retrospective study and collected post-treatment follow-up MRIs in patients treated for OPSCC. Each scan was scored according to NI-RADS by 1 general radiologist, 2 radiology residents, and 2 seasoned radiologists. Percentage of agreement (POA) and kappa values (κ) were calculated for the assignment of NI-RADS and its individual MRI features (lymph node, primary tumor size, primary site signal on T2-weighted, contrast-enhanced and diffusion-weighted images). Inter-reader agreement was calculated for all post-treatment MRIs and separately for the first post-treatment MRI (using pre-treatment MRI as reference) and subsequent follow-ups. Results: Ninety-one patients were included (a total of 218 MRIs per rater). The agreement among all readers for NI-RADS (κ = 0.53, POA = 89%) and for each individual MRI feature (κ = 0.42–0.52, POA = 84–93%) assessment was moderate. Lower reliability emerged between the expert radiologist and the radiologists not specialized in head and neck imaging in the first follow-up MRI scan for both primary site contrast enhancement (κ = 0.38-0.41, POA = 72%-88%) and lymph node (κ = 0.25-0.36, POA = 77%-90%) assessment. Conclusion: MRI NI-RADS showed moderate inter-rater agreement in OPSCC patients, with greater interpretative challenges in the evaluation of the first post-treatment MRI. Regular application of the NI-RADS in clinical settings may help enhance consistency and reliability in imaging evaluations.