Inter-reader reliability of MRI and CT Node Reporting and Data System 1.0 (Node-RADS) for mesorectal lymph nodes in rectal cancer

BackgroundNode Reporting and Data System 1.0 (Node-RADS) classifies the risk of cancer metastases in lymph nodes at any anatomical site, using computed tomography (CT) and/or magnetic resonance imaging (MRI). This new RADS needs clinical validation through agreement studies.PurposeTo evaluate the inter-reader reliability of Node-RADS between radiologists with different levels of experience, focusing on rectal cancer CT and MRI.Material and MethodsWe designed a single-center observational retrospective study to identify staging CT or MRI of newly diagnosed individuals with rectal cancer. Four readers (two residents in radiology and two radiologists) scored each mesorectal node with Node-RADS on a per-lymph node basis. Fleiss' κ was calculated to assess the level of agreement both for the score and for each individual category of the Node-RADS. A subgroup analysis was performed on a per-patient basis to evaluate the correlation between the Node-RADS assigned by the most experienced reader and the histopathological diagnosis of node involvement.ResultsA total of 20 patients were included in the study. The number of Node-RADS scored per reader was 242, corresponding to 968 values to be compared for agreement assessment. Fleiss' κ for Node-RADS score among all readers was 0.65 (substantial agreement). Analysis of the individual categories of the Node-RADS demonstrated a good agreement between all readers (κ > 0.61). A moderate positive correlation was documented between the Node-RADS and the likelihood of nodes being metastatic on histology.ConclusionThe Node-RADS score exhibited good inter-reader reliability between experienced radiologists and those still in training.