Error detection model developed using a multi-task convolutional neural network in patient-specific quality assurance for volumetric-modulated arc therapy.

In patient-specific quality assurance (QA) for static beam intensity-modulated radiation therapy (IMRT), machine-learning-based dose analysis methods have been developed to identify the cause of an error as an alternative to gamma analysis. Although these new methods have revealed that the cause of the error can be identified by analyzing the dose distribution obtained from the two-dimensional detector, they have not been extended to the analysis of volumetric-modulated arc therapy (VMAT) QA. In this study, we propose a deep learning approach to detect various types of errors in patient-specific VMAT QA. A total of 161 beams from 104 prostate VMAT plans were analyzed. All beams were measured using a cylindrical detector (Delta4; ScandiDos, Uppsala, Sweden), and predicted dose distributions in a cylindrical phantom were calculated using a treatment planning system (TPS). In addition to the error-free plan, we simulated 12 types of errors: two types of multileaf collimator positional errors (systematic or random leaf offset of 2 mm), two types of monitor unit (MU) scaling errors (±3%), two types of gantry rotation errors (±2° in clockwise and counterclockwise direction), and six types of phantom setup errors (±1 mm in lateral, longitudinal, and vertical directions). The error-introduced predicted dose distributions were created by editing the calculated dose distributions using a TPS with in-house software. Those 13 types of dose difference maps, consisting of an error-free map and 12 error maps, were created from the measured and predicted dose distributions and were used to train the convolutional neural network (CNN) model. Our model was a multi-task model that individually detected each of the 12 types of errors. Two datasets, Test sets 1 and 2, were prepared to evaluate the performance of the model. Test set 1 consisted of 13 types of dose maps used for training, whereas Test set 2 included the dose maps with 25 types of errors in addition to the error-free dose map. The dose map, which introduced 25 types of errors, was generated by combining two of the 12 types of simulated errors. For comparison with the performance of our model, gamma analysis was performed for Test sets 1 and 2 with the criteria set to 3%/2 mm and 2%/1 mm (dose difference/distance to agreement). For Test set 1, the overall accuracy of our CNN model, gamma analysis with the criteria set to 3%/2 mm, and gamma analysis with the criteria set to 2%/1 mm was 0.92, 0.19, and 0.81, respectively. Similarly, for Test set 2, the overall accuracy was 0.44, 0.42, and 0.95, respectively. Our model outperformed gamma analysis in the classification of dose maps containing a single type error, and the performance of our model was inferior in the classification of dose maps containing compound errors. A multi-task CNN model for detecting errors in patient-specific VMAT QA using a cylindrical measuring device was constructed, and its performance was evaluated. Our results demonstrate that our model was effective in identifying the error type in the dose map for VMAT QA.

Kimura Y ，Kadoya N ，Oku Y ，Kajikawa T ，Tomori S ，Jingu K ... -  《-》

Error detection using a convolutional neural network with dose difference maps in patient-specific quality assurance for volumetric modulated arc therapy.

The aim of this study was to evaluate the use of dose difference maps with a convolutional neural network (CNN) to detect multi-leaf collimator (MLC) positional errors in patient-specific quality assurance for volumetric modulated radiation therapy (VMAT). A cylindrical three-dimensional detector (Delta4, ScandiDos, Uppsala, Sweden) was used to measure 161 beams from 104 clinical prostate VMAT plans. For the simulation used error-free plans plus plans with two types of MLC error were introduced: systematic error and random error. A total of 483 dose distributions in a virtual cylindrical phantom were calculated with a treatment planning system. Dose difference maps were created from two planar dose distributions from the measured and calculated dose distributions, and these were used as the input for the CNN, with 375 datasets assigned for training and 108 datasets assigned for testing. The CNN model had three convolution layers and was trained with five-fold cross-validation. The CNN model classified the error types of the plans as "error-free," "systematic error," or "random error," with an overall accuracy of 0.944. The sensitivity values for the "error-free," "systematic error," and "random error" classifications were 0.889, 1.000, and 0.944, respectively, and the specificity values were 0.986, 0.986, and 0.944, respectively. This approach was superior to those based on gamma analysis. Using dose difference maps with a CNN model may provide an effective solution for detecting MLC errors for patient-specific VMAT quality assurance.

Kimura Y ，Kadoya N ，Tomori S ，Oku Y ，Jingu K ... -  《-》

Development of a deep learning-based error detection system without error dose maps in the patient-specific quality assurance of volumetric modulated arc therapy.

To detect errors in patient-specific quality assurance (QA) for volumetric modulated arc therapy (VMAT), we proposed an error detection method based on dose distribution analysis using unsupervised deep learning approach and analyzed 161 prostate VMAT beams measured with a cylindrical detector. For performing error simulation, in addition to error-free dose distribution, dose distributions containing nine types of error, including multileaf collimator (MLC) positional errors, gantry rotation errors, radiation output errors and phantom setup errors, were generated. Only error-free data were employed for the model training, and error-free and error data were employed for the tests. As a deep learning model, the variational autoencoder (VAE) was adopted. The anomaly of test data was quantified by calculating Mahalanobis distance based on the feature vectors acquired from a trained encoder. Based on this anomaly, test data were classified as 'error-free' or 'any-error.' For comparison with conventional approaches, gamma (γ)-analysis was performed, and supervised learning convolutional neural network (S-CNN) was constructed. Receiver operating characteristic curves were obtained to evaluate their performance with the area under the curve (AUC). For all error types, except systematic MLC positional and radiation output errors, the performance of the methods was in the order of S-CNN ˃ VAE-based ˃ γ-analysis (only S-CNN required error data for model training). For example, in random MLC positional error simulation, the AUC of our method, S-CNN and γ-analysis were 0.699, 0.921 and 0.669, respectively. Our results showed that the VAE-based method has the potential to detect errors in patient-specific VMAT QA.

Kimura Y ，Kadoya N ，Oku Y ，Jingu K ... -  《-》

Error detection and classification in patient-specific IMRT QA with dual neural networks.

Despite being the standard metric in patient-specific quality assurance (QA) for intensity-modulated radiotherapy (IMRT), gamma analysis has two shortcomings: (a) it lacks sensitivity to small but clinically relevant errors (b) it does not provide efficient means to classify the error sources. The purpose of this work is to propose a dual neural network method to achieve simultaneous error detection and classification in patient-specific IMRT QA. For a pair of dose distributions, we extracted the dose difference histogram (DDH) for the low dose gradient region and two signed distance-to-agreement (sDTA) maps (one in x direction and one in y direction) for the high dose gradient region. An artificial neural network (ANN) and a convolutional neural network (CNN) were designed to analyze the DDH and the two sDTA maps, respectively. The ANN was trained to detect and classify six classes of dosimetric errors: incorrect multileaf collimator (MLC) transmission (±1%) and four types of monitor unit (MU) scaling errors (±1% and ±2%). The CNN was trained to detect and classify seven classes of spatial errors: incorrect effective source size, 1 mm MLC leaf bank overtravel or undertravel, 2 mm single MLC leaf overtravel or undertravel, and device misalignment errors (1 mm in x- or y direction). An in-house planar dose calculation software was used to simulate measurements with errors and noise introduced. Both networks were trained and validated with 13 IMRT plans (totaling 88 fields). A fivefold cross-validation technique was used to evaluate their accuracy. Distinct features were found in the DDH and the sDTA maps. The ANN perfectly identified all four types of MU scaling errors and the specific accuracies for the classes of no error, MLC transmission increase, MLC transmission decrease were 98.9%, 96.6%, and 94.3%, respectively. For the CNN, the largest confusion occurred between the 1-mm-MLC bank overtravel class and the 1-mm-device alignment error in x-direction class, which brought the specific accuracies down to 90.9% and 92.0%, respectively. The specific accuracy for the 2-mm-single MLC leaf undertravel class was 93.2% as it misclassified 5.7% of the class as being error free (false negative). Otherwise, the specific accuracy was above 95%. The overall accuracies across the fivefold were 98.3 ± 0.7% and 95.6% ± 1.5% for the ANN and the CNN, respectively. Both the DDH and the sDTA maps are suitable features for error classification in IMRT QA. The proposed dual neural network method achieved simultaneous error detection and classification with excellent accuracy. It could be used in complement with the gamma analysis to potentially shift the IMRT QA paradigm from passive pass/fail analysis to active error detection and root cause identification.

Potter NJ ，Mund K ，Andreozzi JM ，Li JG ，Liu C ，Yan G ... -  《-》

The structural similarity index for IMRT quality assurance: radiomics-based error classification.

The implementation of radiomics and machine learning (ML) techniques on analyzing two-dimensional gamma maps has been demonstrated superior to the conventional gamma analysis for error identification in intensity modulated radiotherapy (IMRT) quality assurance (QA). Recently, the Structural SIMilarity (SSIM) sub-index maps were shown to be able to reveal the error types of the dose distributions. In this study, we aimed to apply radiomics analysis on SSIM sub-index maps and develop ML models to classify delivery errors in patient-specific dynamic IMRT QA. Twenty-one sliding-window IMRT plans of 180 beams for three treatment sites were involved in this study. Four types of machine-related errors of various magnitudes were simulated for each beam at each control point, including the monitor unit (MU) variations, same-directional and opposite-directional shifts of the multileaf collimators (MLCs) and random mispositioning of the MLCs. In the QA process, a total of 1620 portal dose (PD) images were acquired for the beams with and without errors. The predicted PD images of the original beams were set as references. To quantify the agreement between a measured PD image and the corresponding predicted PD image, four difference maps including three SSIM sub-index maps, and one dose difference-derived map were calculated. Then, radiomic features were extracted from the four difference maps of each measured PD image. We tested four typical classifiers including linear discriminant classifier (LDC), two supporting vector machine (SVM) classifiers, and random forest (RF) for this multiclass classification task. A nested cross-validation scheme was used for model evaluations, where the SVM recursive feature elimination method was applied for feature selection. Finally, the performance of the ML model on identifying the error-free and the erroneous cases was compared to that of the conventional gamma analysis. The statistics of the selected features showed that all of the difference maps and the feature categories made balanced contributions to solve this classification task. Best performance was achieved by the Linear-SVM model with average overall classification accuracy of 0.86. Specifically, the average classification accuracies of the shift, opening, and the random errors were around 0.9. Moreover, ~80% of error-free and MU errors were correctly classified. Using gamma analysis, the 3 mm/3% criterion was found insensitive to errors (sensitivity was only 0.33). Although the sensitivity to errors with the 2 mm/2% criterion increased to 0.79, still 8% worse than that of the ML model. We proposed an ML-based method for machine-related error identification in patient-specific dynamic IMRT QA, where radiomic analysis on SSIM sub-index maps were used for feature extraction. With extensive validation to select the best features and classifiers, high accuracies in error classification were achieved. Compared with the conventional gamma threshold method, this approach has great potential in error identification for the patient-specific IMRT QA process.

Ma C ，Wang R ，Zhou S ，Wang M ，Yue H ，Zhang Y ，Wu H ... -  《-》