Refining dataset curation methods for deep learning-based automated tuberculosis screening.

The study objective was to determine whether unlabeled datasets can be used to further train and improve the accuracy of a deep learning system (DLS) for the detection of tuberculosis (TB) on chest radiographs (CXRs) using a two-stage semi-supervised approach. A total of 111,622 CXRs from the National Institute of Health ChestX-ray14 database were collected. A cardiothoracic radiologist reviewed a subset of 11,000 CXRs and dichotomously labeled each for the presence or absence of potential TB findings; these interpretations were used to train a deep convolutional neural network (DCNN) to identify CXRs with possible TB (Phase I). The best performing algorithm was then used to label the remaining database consisting of 100,622 radiographs; subsequently, these newly-labeled images were used to train a second DCNN (phase II). The best-performing algorithm from phase II (TBNet) was then tested against CXRs obtained from 3 separate sites (2 from the USA, 1 from China) with clinically confirmed cases of TB. Receiver operating characteristic (ROC) curves were generated with area under the curve (AUC) calculated. The phase I algorithm trained using 11,000 expert-labelled radiographs achieved an AUC of 0.88. The phase II algorithm trained on images labeled by the phase I algorithm achieved an AUC of 0.91 testing against a TB dataset obtained from Shenzhen, China and Montgomery County, USA. The algorithm generalized well to radiographs obtained from a tertiary care hospital, achieving an AUC of 0.87; TBNet's sensitivity, specificity, positive predictive value, and negative predictive value were 85%, 76%, 0.64, and 0.9, respectively. When TBNet was used to arbitrate discrepancies between 2 radiologists, the overall sensitivity reached 94% and negative predictive value reached 0.96, demonstrating a synergistic effect between the algorithm's output and radiologists' interpretations. Using semi-supervised learning, we trained a deep learning algorithm that detected TB at a high accuracy and demonstrated value as a CAD tool by identifying relevant CXR findings, especially in cases that were misinterpreted by radiologists. When dataset labels are noisy or absent, the described methods can significantly reduce the required amount of curated data to build clinically-relevant deep learning models, which will play an important role in the era of precision medicine.

Kim TK ，Yi PH ，Hager GD ，Lin CT ... -  《-》

Comparison of radiologist versus natural language processing-based image annotations for deep learning system for tuberculosis screening on chest radiographs.

Although natural language processing (NLP) can rapidly extract disease labels from radiology reports to create datasets for deep learning models, this may be less accurate than having radiologists manually review the images. In this study, we compared agreement between natural language processing (NLP) and radiologist-curated labels for possible tuberculosis (TB) on chest radiographs (CXR) and evaluated the performance of deep convolutional neural networks (DCNN) trained to identify TB using the preceding two sets of labels. We collected 10,951 CXRs from the NIH ChestX-ray14 dataset and labeled them as positive or negative for possible TB based on two methods: 1) NLP-derived disease labels and 2) radiologist-review of images. These images were used to train DCNNs on varying dataset sizes for possible TB and tested on an external dataset of 800 CXRs. Area under the ROC curve (AUC) was used to evaluate DCNNs. There was poor agreement between NLP and radiologist-curated labels for potential TB (Kappa coefficient 0.34). DCNNs trained using radiologist-curated labels had higher performance than the algorithm trained using the NLP-labels, regardless of the number of images used for training. The best-performing DCNN had an AUC of 0.88, which was trained on 10,951 images using the radiologist-annotated sets. DCNNs trained on CXRs labeled by a radiologist consistently outperformed those trained on the same CXRs labeled by NLP, highlighting the benefit of radiologists' determining groundtruth for machine learning dataset curation.

Yi PH ，Kim TK ，Lin CT 《-》

Deep Learning Method for Automated Classification of Anteroposterior and Posteroanterior Chest Radiographs.

Kim TK ，Yi PH ，Wei J ，Shin JW ，Hager G ，Hui FK ，Sair HI ，Lin CT ... -  《-》

Deep Learning-based Diagnosis of Pulmonary Tuberculosis on Chest X-ray in the Emergency Department: A Retrospective Study.

Prompt and correct detection of pulmonary tuberculosis (PTB) is critical in preventing its spread. We aimed to develop a deep learning-based algorithm for detecting PTB on chest X-ray (CXRs) in the emergency department. This retrospective study included 3498 CXRs acquired from the National Taiwan University Hospital (NTUH). The images were chronologically split into a training dataset, NTUH-1519 (images acquired during the years 2015 to 2019; n = 2144), and a testing dataset, NTUH-20 (images acquired during the year 2020; n = 1354). Public databases, including the NIH ChestX-ray14 dataset (model training; 112,120 images), Montgomery County (model testing; 138 images), and Shenzhen (model testing; 662 images), were also used in model development. EfficientNetV2 was the basic architecture of the algorithm. Images from ChestX-ray14 were employed for pseudo-labelling to perform semi-supervised learning. The algorithm demonstrated excellent performance in detecting PTB (area under the receiver operating characteristic curve [AUC] 0.878, 95% confidence interval [CI] 0.854-0.900) in NTUH-20. The algorithm showed significantly better performance in posterior-anterior (PA) CXR (AUC 0.940, 95% CI 0.912-0.965, p-value < 0.001) compared with anterior-posterior (AUC 0.782, 95% CI 0.644-0.897) or portable anterior-posterior (AUC 0.869, 95% CI 0.814-0.918) CXR. The algorithm accurately detected cases of bacteriologically confirmed PTB (AUC 0.854, 95% CI 0.823-0.883). Finally, the algorithm tested favourably in Montgomery County (AUC 0.838, 95% CI 0.765-0.904) and Shenzhen (AUC 0.806, 95% CI 0.771-0.839). A deep learning-based algorithm could detect PTB on CXR with excellent performance, which may help shorten the interval between detection and airborne isolation for patients with PTB.

Wang CH ，Chang W ，Lee MR ，Tay J ，Wu CY ，Wu MC ，Roth HR ，Yang D ，Zhao C ，Wang W ，Huang CH ... -  《-》

Comparison of Chest Radiograph Interpretations by Artificial Intelligence Algorithm vs Radiology Residents.

Wu JT ，Wong KCL ，Gur Y ，Ansari N ，Karargyris A ，Sharma A ，Morris M ，Saboury B ，Ahmad H ，Boyko O ，Syed A ，Jadhav A ，Wang H ，Pillai A ，Kashyap S ，Moradi M ，Syeda-Mahmood T ... -  《JAMA Network Open》