The Low Rate of Adherence to Checklist for Artificial Intelligence in Medical Imaging Criteria Among Published Prostate MRI Artificial Intelligence Algorithms.

To determine the rigor, generalizability, and reproducibility of published classification and detection artificial intelligence (AI) models for prostate cancer (PCa) on MRI using the Checklist for Artificial Intelligence in Medical Imaging (CLAIM) guidelines, a 42-item checklist that is considered a measure of best practice for presenting and reviewing medical imaging AI research. This review searched English literature for studies proposing PCa AI detection and classification models on MRI. Each study was evaluated with the CLAIM checklist. The additional outcomes for which data were sought included measures of AI model performance (eg, area under the curve [AUC], sensitivity, specificity, free-response operating characteristic curves), training and validation and testing group sample size, AI approach, detection versus classification AI, public data set utilization, MRI sequences used, and definition of gold standard for ground truth. The percentage of CLAIM checklist fulfillment was used to stratify studies into quartiles. Wilcoxon's rank-sum test was used for pair-wise comparisons. In all, 75 studies were identified, and 53 studies qualified for analysis. The original CLAIM items that most studies did not fulfill includes item 12 (77% no): de-identification methods; item 13 (68% no): handling missing data; item 15 (47% no): rationale for choosing ground truth reference standard; item 18 (55% no): measurements of inter- and intrareader variability; item 31 (60% no): inclusion of validated interpretability maps; item 37 (92% no): inclusion of failure analysis to elucidate AI model weaknesses. An AUC score versus percentage CLAIM fulfillment quartile revealed a significant difference of the mean AUC scores between quartile 1 versus quartile 2 (0.78 versus 0.86, P = .034) and quartile 1 versus quartile 4 (0.78 versus 0.89, P = .003) scores. Based on additional information and outcome metrics gathered in this study, additional measures of best practice are defined. These new items include disclosure of public dataset usage, ground truth definition in comparison to other referenced works in the defined task, and sample size power calculation. A large proportion of AI studies do not fulfill key items in CLAIM guidelines within their methods and results sections. The percentage of CLAIM checklist fulfillment is weakly associated with improved AI model performance. Additions or supplementations to CLAIM are recommended to improve publishing standards and aid reviewers in determining study rigor.

Belue MJ ，Harmon SA ，Lay NS ，Daryanani A ，Phelps TE ，Choyke PL ，Turkbey B ... -  《-》

Artificial Intelligence in Magnetic Resonance Imaging-based Prostate Cancer Diagnosis: Where Do We Stand in 2021?

Men suspected of harboring prostate cancer (PCa) increasingly undergo multiparametric magnetic resonance imaging (mpMRI) and mpMRI-guided biopsy. The potential of mpMRI coupled to artificial intelligence (AI) methods to detect and classify PCa before decision-making requires investigation. To review the literature for studies addressing the diagnostic performance of combined mpMRI and AI approaches to detect and classify PCa, and to provide selection criteria for relevant articles having clinical significance. We performed a nonsystematic search of the English language literature using the PubMed-MEDLINE database up to October 30, 2020. We included all original studies addressing the diagnostic accuracy of mpMRI and AI to detect and classify PCa with histopathological analysis as a reference standard. Eleven studies assessed AI and mpMRI approaches for PCa detection and classification based on a ground truth that referred to the entire prostate either with radical prostatectomy specimens (RPS) or relocalization of positive systematic and/or targeted biopsy. Seven studies retrospectively annotated cancerous lesions onto mpMRI identified in whole-mount sections from RPS, three studies used a backward projection of histological prostate biopsy information, and one study used a combined cohort of both approaches. All studies cross-validated their data sets; only four used a test set and one a multisite validation scheme. Performance metrics for lesion detection ranged from 87.9% to 92% at a threshold specificity of 50%. The lesion classification accuracy of the algorithms was comparable to that of the Prostate Imaging-Reporting and Data System. For an algorithm to be implemented into radiological workflows and to be clinically applicable, it must be trained with a ground truth labeling that reflects histopathological information for the entire prostate and it must be externally validated. Lesion detection and classification performance metrics are promising but require prospective implementation and external validation for clinical significance. We reviewed the literature for studies on prostate cancer detection and classification using magnetic resonance imaging (MRI) and artificial intelligence algorithms. The main application is in supporting radiologists in interpreting MRI scans and improving the diagnostic performance, so that fewer unnecessary biopsies are carried out.

Suarez-Ibarrola R ，Sigle A ，Eklund M ，Eberli D ，Miernik A ，Benndorf M ，Bamberg F ，Gratzke C ... -  《European Urology Focus》

Assessment of artificial intelligence (AI) reporting methodology in glioma MRI studies using the Checklist for AI in Medical Imaging (CLAIM).

The Checklist for Artificial Intelligence in Medical Imaging (CLAIM) is a recently released guideline designed for the optimal reporting methodology of artificial intelligence (AI) studies. Gliomas are the most common form of primary malignant brain tumour and numerous outcomes derived from AI algorithms such as grading, survival, treatment-related effects and molecular status have been reported. The aim of the study is to evaluate the AI reporting methodology for outcomes relating to gliomas in magnetic resonance imaging (MRI) using the CLAIM criteria. A literature search was performed on three databases pertaining to AI augmentation of glioma MRI, published between the start of 2018 and the end of 2021 RESULTS: A total of 4308 articles were identified and 138 articles remained after screening. These articles were categorised into four main AI tasks: grading (n= 44), predicting molecular status (n= 50), predicting survival (n= 25) and distinguishing true tumour progression from treatment-related effects (n= 10). The average CLAIM score was 20/42 (range: 10-31). Studies most consistently reported the scientific background and clinical role of their AI approach. Areas of improvement were identified in the reporting of data collection, data management, ground truth and validation of AI performance. AI may be a means of producing high-accuracy results for certain tasks in glioma MRI; however, there remain issues with reporting quality. AI reporting guidelines may aid in a more reproducible and standardised approach to reporting and will aid in clinical integration.

Bhandari A ，Scott L ，Weilbach M ，Marwah R ，Lasocki A ... -  《-》

Reporting Quality of Research Studies on AI Applications in Medical Images According to the CLAIM Guidelines in a Radiology Journal With a Strong Prominence in Asia.

We aimed to evaluate the reporting quality of research articles that applied deep learning to medical imaging. Using the Checklist for Artificial Intelligence in Medical Imaging (CLAIM) guidelines and a journal with prominence in Asia as a sample, we intended to provide an insight into reporting quality in the Asian region and establish a journal-specific audit. A total of 38 articles published in the Korean Journal of Radiology between June 2018 and January 2023 were analyzed. The analysis included calculating the percentage of studies that adhered to each CLAIM item and identifying items that were met by ≤ 50% of the studies. The article review was initially conducted independently by two reviewers, and the consensus results were used for the final analysis. We also compared adherence rates to CLAIM before and after December 2020. Of the 42 items in the CLAIM guidelines, 12 items (29%) were satisfied by ≤ 50% of the included articles. None of the studies reported handling missing data (item #13). Only one study respectively presented the use of de-identification methods (#12), intended sample size (#19), robustness or sensitivity analysis (#30), and full study protocol (#41). Of the studies, 35% reported the selection of data subsets (#10), 40% reported registration information (#40), and 50% measured inter and intrarater variability (#18). No significant changes were observed in the rates of adherence to these 12 items before and after December 2020. The reporting quality of artificial intelligence studies according to CLAIM guidelines, in our study sample, showed room for improvement. We recommend that the authors and reviewers have a solid understanding of the relevant reporting guidelines and ensure that the essential elements are adequately reported when writing and reviewing the manuscripts for publication.

Kim DY ，Oh HW ，Suh CH 《-》

Checklist for Artificial Intelligence in Medical Imaging Reporting Adherence in Peer-Reviewed and Preprint Manuscripts With the Highest Altmetric Attention Scores: A Meta-Research Study.

Sivanesan U ，Wu K ，McInnes MDF ，Dhindsa K ，Salehi F ，van der Pol CB ... -  《-》