Implementing AI to predict interval cancers in a national screening program

Purpose

To evaluate the performance of two different versions of an artificial intelligence (AI) system for predicting the risk of developing interval cancer (IC) within 6 to 24 months after a negative screening exam.

Materials and Method

This retrospective study was performed with a data cohort between 2016-2019 derived from women between 40-69 who attended the Turkish breast cancer screening program. During this period, the recall rate was on average 5.3%. The negative screening exams of 323 women who developed IC before the next screening round and 446 women with normal follow-up were collected. The pathological outcome and time-to-diagnosis were retrieved from the national cancer registry. All mammograms were processed by Transpara (ScreenPoint Medical v1.6.0 and v1.7.1), assigning a score between 1-10 to the exam, representing an increasing likelihood of malignancy. The performance of Transpara for detection of IC on negative screening exams was estimated in terms of the area under the receiver operating characteristic curve (AUC), and sensitivity at 90% and 95.0% specificity, with 95% confidence intervals (CI).

Results
More than half of all IC (56.6% and 65.9% respectively for versions 1.6.0 and 1.7.1) were flagged by the highest AI score 10. The AUCs of AI to detect signs of IC on negative screening exams was 0.86 (95% CI [0.8284; 0.8842]) and 0.81 (95% CI [0.7808; 0.8433]) for versions 1.7.1 and 1.6.0 respectively. The sensitivity was 65.9% and 56.6% at a specificity of 90.0% respectively for v1.7.1 and v1.6.0. The latest version performed significantly better than version 1.6.0. The highest performance of Transpara was found for cases that were diagnosed within six months after screening (AUC: 0.82 (95% CI =0.78-0.86), compared to cases diagnosed 6 to 24 months after screening (AUC: 0.57 (95% CI = 0.50-0.65).

Conclusion
Transpara has the potential to reduce the stable high rate of interval cancer in case it is applied as a second or third independent reader within a national breast cancer screening program. Moreover, further developments of AI promise increasing performance towards the prediction of interval cancer.