Assessment of the modulation degrees of intensity-modulated radiation therapy plans

Background To evaluate the modulation indices (MIs) for predicting the plan delivery accuracies of intensity-modulated radiation therapy (IMRT) plans. Methods A total of 100 dynamic IMRT plans that used TrueBeam STx and 102 dynamic IMRT plans that used Trilogy were selected. For each plan, various MIs were calculated, which included the modulation complexity score (MCS), plan-averaged beam area (PA), plan-averaged beam irregularity (PI), plan-averaged beam modulation (PM), MI quantifying multi-leaf collimator (MLC) speeds (MIs), MI quantifying MLC acceleration (MIa), and MI quantifying MLC acceleration and segment aperture irregularity (MIc,IMRT). To determine plan delivery accuracy, global gamma passing rates, MLC errors of log files, and dose-volumetric parameter differences between original and log file-reconstructed IMRT plans were obtained. To assess the ability of each MI for predicting plan delivery accuracy, Spearman’s rank correlation coefficients (rs) between MIs and plan delivery accuracy measures were calculated. Results PI showed moderately strong correlations with gamma passing rates in MapCHECK2 measurements of both TrueBeam STx and Trilogy (rs = − 0.591 with p <  0.001 and − 0.427 with p <  0.001 to with gamma criterion of 2%/2 mm, respectively). For ArcCHECK measurements, PI also showed moderately strong correlations with the gamma passing rates in the ArcCHECK measurements of TrueBeam STx and Trilogy (rs = − 0.545 with p <  0.001 and rs = − 0.581 with p <  0.001 with gamma criterion of 2%/2 mm, respectively). The PI showed the second strongest correlation with MLC errors in both TrueBeam STx and Trilogy (rs = 0.861 with p <  0.001 and rs = 0.767 with p <  0.001, respectively). In general, the PI showed moderately strong correlations with every plan delivery accuracy measure. Conclusions The PI showed moderately strong correlations with every plan delivery accuracy measure and therefore is a useful predictor of IMRT delivery accuracy. Electronic supplementary material The online version of this article (10.1186/s13014-018-1193-9) contains supplementary material, which is available to authorized users.


Background
Intensity-modulated radiation therapy (IMRT) delivers highly conformal radiation dose distributions to target volumes, while sparing normal tissues [1][2][3][4][5][6][7][8]. As the IMRT plan complexity increases to acquire the desired dose distribution, i.e., the modulation degree of IMRT increases, a large number of small and irregularly shaped beam apertures are utilized. It results in increased dose calculation and multi-leaf collimator (MLC) movement uncertainties [4,[9][10][11]. Therefore, the probability that the intended dose distribution is not accurately delivered to the patient during treatment increases. In this respect, verification of IMRT plan delivery accuracy is highly recommended before treatment in a clinical setting.
To verify IMRT plan delivery accuracy, gamma evaluations have been widely adopted in the clinic, as pre-treatment quality assurance (QA) methods [12][13][14]. Although gamma evaluation is a practical method, it has some limitations. It has been reported that the gamma passing rates of individual dosimeters have been inconsistent with respect to each other [15]. Therefore, a thorough gamma evaluation validation of each device that is installed in the clinic should be performed.
Moreover, recent studies have demonstrated the clinical irrelevance of the gamma passing rates [16][17][18]. As an alternative to the gamma evaluation, an analysis of the machine log files that are recorded during delivery has been recommended as a pre-treatment QA by several studies [19][20][21][22]. This method can be expanded to analyze the differences between dose-volume parameters of the original plan and the plans that are reconstructed with machine log files [23][24][25]. However, this method has limitations, as it does not provide an independent verification of linac beam delivery.
On the other hand, to predict IMRT delivery accuracy and calculate the complexity of IMRT plans, several studies have introduced modulation indices (MIs) [4,10,26,27]. Webb et al. measured the modulation degree of IMRT plans by quantifying the changes of adjacent beamlets that exceeded a pre-defined tolerance levels, derived with standard deviations of the fluence map [10]. McNiven et al. designed modulation complexity score (MCS) quantifications of beam aperture and MLC position variations. The MCSs were found to vary from 0 to 1 and decreased as the IMRT plan modulation degrees increased [27]. Du et al. quantified the complexities of both the IMRT and volumetric modulated arc therapy (VMAT) plans, using the beam aperture areas and perimeters, to quantify the usage of small and irregularly shaped beam apertures [4]. For VMAT, Park et al. expanded the concept of Webb's MI to comprehensively assess MLC speeds and accelerations, the variation of gantry rotation speed, the variation of dose-rate, and the usage of small and irregularly shaped beam apertures [26].
Previous studies have investigated the performance of several MIs for predicting IMRT delivery accuracy, through the use of correlation analyses on conventional plan delivery accuracy measures, such as gamma passing rates. However, there are limitations to evaluating MIs with a single type of linac, single photon energy, limited treatment sites, or using a single type of measuring device. Therefore, further studies need to evaluate the performance of various MIs that are used as clinical predictors of IMRT plan deliveries. In this study, we assessed the performances of various MIs that have been suggested by previous studies to predict IMRT plan delivery accuracy with correlation analyses.

Patient selection and IMRT planning
Two machines, Trilogy and TrueBeam STx (Varian Medical Systems, Palo Alto, CA), which have different plan delivery systems and machine log file generation systems, were selected for this study. After approval by the institutional review board (IRB), a total of 181 patients with various treatment sites were retrospectively selected, and a total of 202 dynamic IMRT plans were generated for this study.
With the TrueBeam STx system, a total of 100 dynamic IMRT plans were generated for head and neck (HN) cancer (20 cases), brain tumor (20 cases), liver cancer (20 cases), spine tumor (20 cases), and lung cancer (20 cases). For HN cancer, a simultaneous integrated boost (SIB) technique was applied. For liver cancer, spine tumors, and lung cancer, stereotactic ablative radiotherapy (SABR) was performed using flattening filter free (FFF) photon beams of 6 MV (6 FFF) and 10 FFF. With Trilogy, a total of 102 dynamic IMRT plans were generated for HN cancer (40 cases), prostate cancer (21 primary plans and 21 boost plans), liver cancer (11 cases), and spine tumor (9 cases). The HN IMRT plans were generated with the SIB technique as with TrueBeam STx system. Information of IMRT plans generated in this study is summarized in Additional file 1: Table S1.
Every IMRT plan in this study was generated in Eclipse™ (Varian Medical Systems, Palo Alto, CA). All IMRT plans used in this study were optimized with the dose volume optimizer (DVO, ver. 10, Varian Medical Systems, Palo Alto, CA), and the dose distributions were calculated using the anisotropic analytic algorithm (AAA, ver. 10, Varian Medical Systems, Palo Alto, CA), with a calculation grid of 1 mm. All IMRT plans were normalized to cover 100% of the target volumes with 90% of the prescription doses.

Calculation of IMRT plan modulation indices
For each IMRT plan, the MCS from McNiven et al.; the plan-averaged beam area (PA), plan-averaged beam irregularity (PI), and plan-averaged beam modulation (PM) from Du et al.; the modulation index quantifying the speed of MLC (MI s ); the modulation index for quantifying the MLC acceleration (MI a ); and the modulation index for quantifying the MLC acceleration and aperture irregularity of the IMRT plan (MI c,IMRT ) were calculated. Although MI c,IMRT values were originally designed for VMAT, we modified these indices to consider only MLC movements for application to IMRT plans. In other words, we excluded gantry rotation accuracy and dose-rate variability evaluations from the MI c,IMRT value. To eliminate the influence of various segment numbers on the MCS, MI s , MI a , and MI c,IMRT values, we normalized the values of these MIs with the total segment numbers. Equations of the MIs that were calculated in this study (including modifications) are summarized in Additional file 1: Table S2.

Measures of IMRT plan delivery accuracy
To evaluate plan delivery accuracy, three types of IMRT plan verification methods were used in this study, which included the global gamma evaluation with absolute doses, differences in MLC positions between planned and the actual deliveries utilizing machine log files, and the differences in the dose-volumetric parameters that were calculated from the original IMRT plans and from the IMRT plans that were reconstructed with machine log files.
For global gamma evaluation, reference dose distributions were calculated in the Eclipse system, with a calculation grid size of 1 mm. The calculated dose distributions were compared to the measured dose distributions using MapCHECK2™ and ArcCHECK™ detector arrays (Sun Nuclear Corporation, Melbourne, FL). Global gamma evaluations with gamma criteria of 3%/3 mm, 2%/2 mm, 2%/1 mm, and 1%/2 mm were performed with absolute doses. Percent dose differences in the global gamma evaluation were calculated relative to the maximum dose in the calculated dose distribution. Before the gamma evaluation measurements, MapCHECK2 and ArcCHECK dosimeters were calibrated, according to manufacturer guidelines.
When IMRT plans were delivered to the dosimeters for gamma evaluation, the actual MLC positions and delivered monitor units (MUs) at each segment were obtained from the DynaLog (Trilogy) and Trajectory (TrueBeam STx) files. These log files were reconstructed as DICOM-RT files, using an in-house program that was written in MATLAB (R2016a, Mathworks Inc., Natick, MA). The differences in the MLC positions between the original IMRT plan and the actual delivery were calculated and averaged for each IMRT plan. After the reconstructed DICOM-RT files from the log files were imported in the Eclipse system, dose distributions were calculated with the same dose calculation condition of the original IMRT plan. With the dose distributions calculated from reconstructed IMRT plans, dose-volumetric parameters for the target volumes and organs at risk (OARs) were calculated and compared to those of the original IMRT plans. The total number of analyzed dose volumetric parameters of TrueBeam STx and Trilogy were 156 and 152, respectively.

Correlation analysis
To evaluate the performance of each MI that was calculated in this study as an indicator of IMRT delivery accuracy, Spearman's rho (r s ) and corresponding p values were calculated between the MIs and the IMRT delivery accuracy measures. In the correlation analysis of different dose-volumetric parameters, the sample sizes were different from one another and the numbers of analyzed dose-volumetric parameters were large. For these reasons, we only counted the number of r s values with corresponding p values less than 0.05, which were considered as statistically significant in this study. The correlations with r s values from 0.2 to 0.39 are regarded as weak correlations. Those from 0.4 to 0.59 and 0.6 to 1 are regarded as moderately strong and very strong correlations, respectively [27,28].

The values of modulation indices
The values of MIs are shown in Table 1. The values of MIs showed various tendencies, depending on the type of linac and treatment sites.

Measurement of plan delivery accuracies
The global gamma passing rates with gamma criteria of 3%/3 mm, 2%/2 mm, 2%/1 mm, and 1%/2 mm are shown in Table 2. The mean gamma passing rate for each treatment site was remarkably different from one another at gamma criteria of 2%/1 mm and 1%/2 mm.
The differences between the planned and actual MLC positions are shown in Fig. 1. The average MLC differences of TrueBeam STx and Trilogy were 0.06 mm and 0.25 mm, respectively. For TrueBeam STx, the average MLC difference of the brain IMRT plans was the highest, at 0.13 mm, while for Trilogy, the average MLC difference of the HN IMRT plans was the highest (0.37 mm). The MLC errors of TrueBeam STx were much smaller than those of the Trilogy system.

Correlations between gamma passing rates and modulation indices
The r s values of MIs for the global gamma passing rates, with gamma criteria of 3%/3 mm, 2%/2 mm, 2%/ 1 mm and 1%/2 mm, of the TrueBeam STx and Trilogy systems are shown in Tables 3 and 4, respectively. The results presented the r s values with statistically significant p values less than 0.05.

Correlations between differences in machine parameters and modulation indices
The correlations between MLC errors and various MIs are shown in Table 5

Correlations between differences in dose-volume parameters and modulation indices
The numbers of statistically significant r s values (p < 0.05) of the various MIs to the differences in dose-volumetric parameters between original and reconstructed IMRT plans are shown in Fig. 2. The PI showed statistically significant r s values most frequently in the TrueBeam STx system (26 cases), whereas the PM showed statistically significant r s values most frequently in the Trilogy system (37 cases).

Discussion
In this study, we calculated the various MIs that have been previously suggested to predict IMRT delivery accuracy and then evaluated the performances of the MIs. When reviewing the mean values of MIs comprehensively, they showed inconsistent trends in IMRT plan complexities for the type of linac and treatment sites. This is because that the factors to be considered when calculating the values of MIs were different from one another. The MCS, PA, PI, and PM indices mainly focus on the beam aperture irregularity or the degree of segmentation, whereas the MI s , MI a , and MI c, IMRT indices mainly focus on the degree of mechanical movements. Nevertheless, for both the TrueBeam STx and Trilogy systems, all MIs except for the MI s and PA indices, indicated that the HN IMRT had the first or second highest complexity. The MI s values seem to have a limitation to calculate the plan complexities properly for various treatment sites since only MLC movements were considered for the MI s . The PA values increased as the modulation degree of IMRT increased unlike the  Since the PA regards the use of small-sized apertures as high modulation of a plan, it is not appropriate to evaluate modulation degree of plans with various treatment sites. For example, the target volume sizes of the HN IMRT plans, which were highly modulated, were the largest among all IMRT plans, which resulted in the highest value of PA (the high value of PA means low modulation of a plan by its definition). On the contrary, the target volume sizes of the prostate boost IMRT plans, which were modulated less, were small, which resulted in the lowest value of PA. Therefore, PA increased owing to the target volume size variations not the modulation degree in this study. If we analyzed IMRT plans of a single treatment site with similar target volume sizes, this contradictory tendency to the definition of the PA would not be observed [26,29]. PA values seem to not provide an appropriate modulation index for predicting IMRT delivery accuracy, through our analysis of IMRT plans with various treatment sites. For TrueBeam STx, every gamma passing rate with both the MapCHECK2 and ArcCHECK indicated that HN IMRT plans had the lowest gamma passing rates for all gamma criteria. For Trilogy, every gamma passing rate with both the MapCHECK2 and ArcCHECK indicated that prostate boost plans had the lowest gamma passing rates for all gamma criteria, showing the inconsistency with the tendencies in the values of MIs. It was indicated that high-dose gradients of the field edge with low modulation and small field sizes increased the measurement uncertainty [30,31] and it seems to decrease the gamma passing rates. This may   Note: r s Spearman's rho, MI s Modulation index considering multi-leaf collimator speed, MI a Modulation index considering multi-leaf collimator acceleration, MI c, IMRT Modulation index considering multi-leaf collimator acceleration and field aperture irregularity for intensity modulated radiation therapy, MCS Modulation complexity score, PA Plan-averaged beam area, PI Plan-averaged beam irregularity, PM Plan-averaged beam modulation be a factor in lowering correlations between MIs and gamma passing rates. Even for identical IMRT plan deliveries under identical gamma parameter settings, we acquired different gamma passing rates between the MapCHECK2 and ArcCHECK dosimeters, similar the findings of a previous study [15,31]. Because the Map-CHECK2 dosimeter has a high angular dependency, the gamma passing rates become inaccurate when the weight of delivered MUs increase at gantry angles near 90°or 270°. Jin et al. demonstrated that the Map-CHECK2 measured doses near a gantry angle of 90°w ere inaccurate, and the inaccuracy increased as the photon beam energy decreased (30.6, 8.9, and 2.2% at gantry angles of 90°± 10°for photon beam energies of 6, 10, and 15 MV, respectively) [30,31]. Similarly, differences in the gamma passing rates between two dosimeters were large for lung SABR of the TrueBeam STx system, which used fields close to 90°or 270°. On the other hand, small differences in gamma passing rates between two dosimeters were observed for spine SABR and liver SABR of the TrueBeam STx and liver IMRT plans of Trilogy, which used fields close to 0°a nd 180°. The lower gamma passing rates of the MapCHECK2 dosimeter, compared to those of the ArcCHECK dosimeter, were due to dosimeter characteristics.
The DynaLog file is a record of actual motor values with an update rate of 50 ms. The motor values are converted to MLC position values using a conversion table (mlctable.txt). The Trajectory file is a record of the direct MLC position values, with an update rate of 20 ms [30]. The information of the Trajectory file is more accurate than that of the DynaLog file, owing to the short record time and direct-record method [21,[32][33][34]. With respect to HN IMRT plans, the Trilogy MLC errors were much larger than those of the True-Beam STx system, but the gamma passing rates of the TrueBeam STx system were smaller and as low as  11.5% of those of the Trilogy system, which were contradictory results to each other. Therefore, a direct absolute comparison between the DynaLog and Trajectory files seems inappropriate, since the recording systems differ. At the very least, we could identify the degree of IMRT plan modulation for various treatment sites within a single linac system. For TrueBeam STx with the MapCHECK2, the PI showed the highest r s value for gamma passing rates with 2%/2 mm, 2%/1 mm and 1%/2 mm (r s = − 0.591 with p < 0.001, r s = − 0.550 with p < 0.001 and r s = − 0.474 with p < 0.001, respectively). For the ArcCHECK measurements, the r s values of the PI were − 0.545 and − 0.677 for gamma passing rates with 2%/2 mm and 1%/2 mm (all with p < 0.001), showing the second highest correlations. For the Trilogy system with the MapCHECK2, PI showed the second highest r s values for every passing rate gamma criterion (all with p < 0.001). For ArcCHECK measurements, the moderately strong correlations were observed between PI and the gamma passing rates with 2%/2 mm and 2%/1 mm (r s = − 0.581 with p < 0.001 and r s = − 0.663 with p < 0.001). By reviewing the results of the correlation analysis, PI mostly showed the first or second strongest correlations with every gamma passing rate, regardless of gamma criteria with the MapCHECK2 dosimeter, in both TrueBeam STx and Trilogy results (all with p < 0.001). Although the PI did not show the best performance with the ArcCHECK measurements, it showed moderately strong correlations with the gamma passing rates from the Arc-CHECK measurements.
In the present study, the first attempt was made to evaluate performance of the previously developed MIs for IMRT by analyzing correlations of MIs with various plan delivery accuracy measures. By utilizing a large number of IMRT plans, various treatment sites, multiple photon energies and linac types, and multiple dosimeters, the correlations in this study were not as high as those found in earlier studies [3,14,16,31]. Among various modulation indices, the PI showed moderately strong correlations with every plan delivery accuracy measure. Therefore, the PI seems to be used to verify IMRT plan delivery accuracy at the planning level in the clinical setting. Furthermore, it has a potential to be integrated in the treatment planning system (TPS) to generate IMRT plans with high delivery accuracy. It is expected to reduce resources in the clinical settings.
Although this study was expanded to comprehensively examine the performance of MIs as predictors of IMRT delivery accuracy by utilizing various treatment sites, photon beam energies, and dosimeters, the results were limited to only Varian linacs. This is a limitation of the present study. To acquire more comprehensive results, further study will be conducted by utilizing various types of linac from various manufacturers in the future.

Conclusions
The PI value showed best performance, among the modulation indices that were evaluated in this study, as a predictor of IMRT plan delivery accuracy showing strong correlations with various measures of IMRT delivery accuracy. The PI value could support the verification of IMRT plan delivery accuracies before patient treatment and reduce resource consumption in the clinic, as it can be calculated at the planning level.

Additional file
Additional file 1: Table S1. Summary of intensity modulated radiation therapy plan information. Table S2