Artificial Intelligence and Its Effect on Dermatologists' Accuracy in Dermoscopic Melanoma Image Classification: Web-Based Survey Study.

Early detection of melanoma can be lifesaving but this remains a challenge. Recent diagnostic studies have revealed the superiority of artificial intelligence (AI) in classifying dermoscopic images of melanoma and nevi, concluding that these algorithms should assist a dermatologist's diagnoses. The aim of this study was to investigate whether AI support improves the accuracy and overall diagnostic performance of dermatologists in the dichotomous image-based discrimination between melanoma and nevus. Twelve board-certified dermatologists were presented disjoint sets of 100 unique dermoscopic images of melanomas and nevi (total of 1200 unique images), and they had to classify the images based on personal experience alone (part I) and with the support of a trained convolutional neural network (CNN, part II). Additionally, dermatologists were asked to rate their confidence in their final decision for each image. While the mean specificity of the dermatologists based on personal experience alone remained almost unchanged (70.6% vs 72.4%; P=.54) with AI support, the mean sensitivity and mean accuracy increased significantly (59.4% vs 74.6%; P=.003 and 65.0% vs 73.6%; P=.002, respectively) with AI support. Out of the 10% (10/94; 95% CI 8.4%-11.8%) of cases where dermatologists were correct and AI was incorrect, dermatologists on average changed to the incorrect answer for 39% (4/10; 95% CI 23.2%-55.6%) of cases. When dermatologists were incorrect and AI was correct (25/94, 27%; 95% CI 24.0%-30.1%), dermatologists changed their answers to the correct answer for 46% (11/25; 95% CI 33.1%-58.4%) of cases. Additionally, the dermatologists' average confidence in their decisions increased when the CNN confirmed their decision and decreased when the CNN disagreed, even when the dermatologists were correct. Reported values are based on the mean of all participants. Whenever absolute values are shown, the denominator and numerator are approximations as every dermatologist ended up rating a varying number of images due to a quality control step. The findings of our study show that AI support can improve the overall accuracy of the dermatologists in the dichotomous image-based discrimination between melanoma and nevus. This supports the argument for AI-based tools to aid clinicians in skin lesion classification and provides a rationale for studies of such classifiers in real-life settings, wherein clinicians can integrate additional information such as patient age and medical history into their decisions.

Maron RC ，Utikal JS ，Hekler A ，Hauschild A ，Sattler E ，Sondermann W ，Haferkamp S ，Schilling B ，Heppt MV ，Jansen P ，Reinholz M ，Franklin C ，Schmitt L ，Hartmann D ，Krieghoff-Henning E ，Schmitt M ，Weichenthal M ，von Kalle C ，Fröhling S ，Brinker TJ ... -  《JOURNAL OF MEDICAL INTERNET RESEARCH》

Deep neural networks are superior to dermatologists in melanoma image classification.

Melanoma is the most dangerous type of skin cancer but is curable if detected early. Recent publications demonstrated that artificial intelligence is capable in classifying images of benign nevi and melanoma with dermatologist-level precision. However, a statistically significant improvement compared with dermatologist classification has not been reported to date. For this comparative study, 4204 biopsy-proven images of melanoma and nevi (1:1) were used for the training of a convolutional neural network (CNN). New techniques of deep learning were integrated. For the experiment, an additional 804 biopsy-proven dermoscopic images of melanoma and nevi (1:1) were randomly presented to dermatologists of nine German university hospitals, who evaluated the quality of each image and stated their recommended treatment (19,296 recommendations in total). Three McNemar's tests comparing the results of the CNN's test runs in terms of sensitivity, specificity and overall correctness were predefined as the main outcomes. The respective sensitivity and specificity of lesion classification by the dermatologists were 67.2% (95% confidence interval [CI]: 62.6%-71.7%) and 62.2% (95% CI: 57.6%-66.9%). In comparison, the trained CNN achieved a higher sensitivity of 82.3% (95% CI: 78.3%-85.7%) and a higher specificity of 77.9% (95% CI: 73.8%-81.8%). The three McNemar's tests in 2 × 2 tables all reached a significance level of p < 0.001. This significance level was sustained for both subgroups. For the first time, automated dermoscopic melanoma image classification was shown to be significantly superior to both junior and board-certified dermatologists (p < 0.001).

Brinker TJ ，Hekler A ，Enk AH ，Berking C ，Haferkamp S ，Hauschild A ，Weichenthal M ，Klode J ，Schadendorf D ，Holland-Letz T ，von Kalle C ，Fröhling S ，Schilling B ，Utikal JS ... -  《-》

Systematic outperformance of 112 dermatologists in multiclass skin cancer image classification by convolutional neural networks.

Recently, convolutional neural networks (CNNs) systematically outperformed dermatologists in distinguishing dermoscopic melanoma and nevi images. However, such a binary classification does not reflect the clinical reality of skin cancer screenings in which multiple diagnoses need to be taken into account. Using 11,444 dermoscopic images, which covered dermatologic diagnoses comprising the majority of commonly pigmented skin lesions commonly faced in skin cancer screenings, a CNN was trained through novel deep learning techniques. A test set of 300 biopsy-verified images was used to compare the classifier's performance with that of 112 dermatologists from 13 German university hospitals. The primary end-point was the correct classification of the different lesions into benign and malignant. The secondary end-point was the correct classification of the images into one of the five diagnostic categories. Sensitivity and specificity of dermatologists for the primary end-point were 74.4% (95% confidence interval [CI]: 67.0-81.8%) and 59.8% (95% CI: 49.8-69.8%), respectively. At equal sensitivity, the algorithm achieved a specificity of 91.3% (95% CI: 85.5-97.1%). For the secondary end-point, the mean sensitivity and specificity of the dermatologists were at 56.5% (95% CI: 42.8-70.2%) and 89.2% (95% CI: 85.0-93.3%), respectively. At equal sensitivity, the algorithm achieved a specificity of 98.8%. Two-sided McNemar tests revealed significance for the primary end-point (p < 0.001). For the secondary end-point, outperformance (p < 0.001) was achieved except for basal cell carcinoma (on-par performance). Our findings show that automated classification of dermoscopic melanoma and nevi images is extendable to a multiclass classification problem, thus better reflecting clinical differential diagnoses, while still outperforming dermatologists at a significant level (p < 0.001).

Maron RC ，Weichenthal M ，Utikal JS ，Hekler A ，Berking C ，Hauschild A ，Enk AH ，Haferkamp S ，Klode J ，Schadendorf D ，Jansen P ，Holland-Letz T ，Schilling B ，von Kalle C ，Fröhling S ，Gaiser MR ，Hartmann D ，Gesierich A ，Kähler KC ，Wehkamp U ，Karoglan A ，Bär C ，Brinker TJ ，Collabrators ... -  《-》

Man against machine reloaded: performance of a market-approved convolutional neural network in classifying a broad spectrum of skin lesions in comparison with 96 dermatologists working under less artificial conditions.

Convolutional neural networks (CNNs) efficiently differentiate skin lesions by image analysis. Studies comparing a market-approved CNN in a broad range of diagnoses to dermatologists working under less artificial conditions are lacking. One hundred cases of pigmented/non-pigmented skin cancers and benign lesions were used for a two-level reader study in 96 dermatologists (level I: dermoscopy only; level II: clinical close-up images, dermoscopy, and textual information). Additionally, dermoscopic images were classified by a CNN approved for the European market as a medical device (Moleanalyzer Pro, FotoFinder Systems, Bad Birnbach, Germany). Primary endpoints were the sensitivity and specificity of the CNN's dichotomous classification in comparison with the dermatologists' management decisions. Secondary endpoints included the dermatologists' diagnostic decisions, their performance according to their level of experience, and the CNN's area under the curve (AUC) of receiver operating characteristics (ROC). The CNN revealed a sensitivity, specificity, and ROC AUC with corresponding 95% confidence intervals (CI) of 95.0% (95% CI 83.5% to 98.6%), 76.7% (95% CI 64.6% to 85.6%), and 0.918 (95% CI 0.866-0.970), respectively. In level I, the dermatologists' management decisions showed a mean sensitivity and specificity of 89.0% (95% CI 87.4% to 90.6%) and 80.7% (95% CI 78.8% to 82.6%). With level II information, the sensitivity significantly improved to 94.1% (95% CI 93.1% to 95.1%; P < 0.001), while the specificity remained unchanged at 80.4% (95% CI 78.4% to 82.4%; P = 0.97). When fixing the CNN's specificity at the mean specificity of the dermatologists' management decision in level II (80.4%), the CNN's sensitivity was almost equal to that of human raters, at 95% (95% CI 83.5% to 98.6%) versus 94.1% (95% CI 93.1% to 95.1%); P = 0.1. In contrast, dermatologists were outperformed by the CNN in their level I management decisions and level I and II diagnostic decisions. More experienced dermatologists frequently surpassed the CNN's performance. Under less artificial conditions and in a broader spectrum of diagnoses, the CNN and most dermatologists performed on the same level. Dermatologists are trained to integrate information from a range of sources rendering comparative studies that are solely based on one single case image inadequate.

Haenssle HA ，Fink C ，Toberer F ，Winkler J ，Stolz W ，Deinlein T ，Hofmann-Wellenhof R ，Lallas A ，Emmert S ，Buhl T ，Zutt M ，Blum A ，Abassi MS ，Thomas L ，Tromme I ，Tschandl P ，Enk A ，Rosenberger A ，Reader Study Level I and Level II Groups ... -  《-》

Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists.

Haenssle HA ，Fink C ，Schneiderbauer R ，Toberer F ，Buhl T ，Blum A ，Kalloo A ，Hassen ABH ，Thomas L ，Enk A ，Uhlmann L ，Reader study level-I and level-II Groups ，Alt C ，Arenbergerova M ，Bakos R ，Baltzer A ，Bertlich I ，Blum A ，Bokor-Billmann T ，Bowling J ，Braghiroli N ，Braun R ，Buder-Bakhaya K ，Buhl T ，Cabo H ，Cabrijan L ，Cevic N ，Classen A ，Deltgen D ，Fink C ，Georgieva I ，Hakim-Meibodi LE ，Hanner S ，Hartmann F ，Hartmann J ，Haus G ，Hoxha E ，Karls R ，Koga H ，Kreusch J ，Lallas A ，Majenka P ，Marghoob A ，Massone C ，Mekokishvili L ，Mestel D ，Meyer V ，Neuberger A ，Nielsen K ，Oliviero M ，Pampena R ，Paoli J ，Pawlik E ，Rao B ，Rendon A ，Russo T ，Sadek A ，Samhaber K ，Schneiderbauer R ，Schweizer A ，Toberer F ，Trennheuser L ，Vlahova L ，Wald A ，Winkler J ，Wölbing P ，Zalaudek I ... -  《-》