Evaluation of multiple institutions’ models for knowledge-based planning of volumetric modulated arc therapy (VMAT) for prostate cancer

Background The aim of this study was to evaluate the performance of a commercial knowledge-based planning system, in volumetric modulated arc therapy for prostate cancer at multiple radiation therapy departments. Methods In each institute, > 20 cases were assessed. For the knowledge-based planning, the estimated dose (ED) based on geometric and dosimetric information of plans was generated in the model. Lower and upper limits of estimated dose were saved as dose volume histograms for each organ at risk. To verify whether the models performed correctly, KBP was compared with manual optimization planning in two cases. The relationships between the EDs in the models and the ratio of the OAR volumes overlapping volume with PTV to the whole organ volume (Voverlap/Vwhole) were investigated. Results There were no significant dosimetric differences in OARs and PTV between manual optimization planning and knowledge-based planning. In knowledge-based planning, the difference in the volume ratio of receiving 90% and 50% of the prescribed dose (V90 and V50) between institutes were more than 5.0% and 10.0%, respectively. The calculated doses with knowledge-based planning were between the upper and lower limits of ED or slightly under the lower limit of ED. The relationships between the lower limit of ED and Voverlap/Vwhole were different among the models. In the V90 and V50 for the rectum, the maximum differences between the lower limit of ED among institutes were 8.2% and 53.5% when Voverlap/Vwhole for the rectum was 10%. In the V90 and V50 for the bladder, the maximum differences of the lower limit of ED among institutes were 15.1% and 33.1% when Voverlap/Vwhole for the bladder was 10%. Conclusion Organs’ upper and lower limits of ED in the models correlated closely with the Voverlap/Vwhole. It is important to determine whether the models in KBP match a different institute’s plan design before the models can be shared.


Background
The plan quality for intensity-modulated radiotherapy (IMRT) and volumetric-modulated arc therapy (VMAT), which are created by inverse planning, depends on the planner's or institution's experience and skills [1][2][3]. Institutional experience substantially influences survival in locally advanced head and neck cancer [4]. Some studies have suggested methods to verify the quality of plans created by inverse planning [5][6][7].
For quality assurance of an inverse planning algorithm, Moore et al. [5] reported that predicting the dose to an organ at risk (OAR) from the volume of the OAR within the planning target volume (PTV) was useful to reduce variations in planning quality. Recently, a new assistance tool for inverse planning, RapidPlan (Varian Medical Systems, Palo Alto CA, USA), which performs knowledge-based planning (KBP), was developed and released for clinical use. Details of the system have been described in a previous study [8]. Some studies have suggested that the performance of KBP be compared with manually-optimized plans for clinical use. They mentioned that KBP is superior to manual planning in reducing OAR dose [9][10][11][12].
The KBP system has the advantage that its model is shared by multiple institutions. Sharing models is considered to be a good method for reducing variability in planning quality among multiple institutions. There has been no report that KBP with the models in multiple institutions was employed for the same CT data. The aim of this study was to evaluate the performance of KBP models in multiple institutions to optimize the model.

Institutes and plan design
In this study, five institutes (A-E) were enrolled. These institutes treated patients with T1-T2c prostate cancer using VMAT. Table 1 shows the definition of gross tumor volume (GTV), margins to define the clinical target volume (CTV) and PTV in each direction. In each institution, the dose constraints are shown in Table 2. The five institutes had different plan designs.

The model for KBP and exporting the estimated dose
In each institute, the model for KBP was created using the VMAT plans for clinical use at each institute before April 2017. The number of registered cases in institute A, institute B, institute C, institute D, and institute E were 123, 53, 20, 60, and 100, respectively.
Users performed three main steps to create models for KBP. In the first step, dose volume histogram (DVH) estimation model configuration, > 20 plans that had been used in clinical settings were registered. The next step was the extraction phase. In each OAR of registered plans, dosimetric and geometric information was imported in the model. The last step was the training step, based on the information from the extraction phase. In this step, in each OAR of registered plans new DVH curves were generated. Upper and lower limits of the estimated doses (ED) were obtained. These dose limits were saved in the form of DVH in the model. To attain the ideal dose distribution, the parameters, except line objects shown in Table 3, were set in some institutes.
These data were read from an .xml file exported to the website of Model Analytics (https://ModelAnalytics.varian.com). The file also contained basic information on the model, such as original and estimated DVH data and OAR volume, and the ratio of an OAR's volume overlapping with PTV to the whole organ volume (V overlap /V whole ). To evaluate the performance in reducing the dose to rectum and bladder in each model, the original DVH, and upper and lower limits of ED, were extracted from the file.

Calculation of dose distributions with manual optimization and KBP
To investigate whether KBP was performed correctly, two sets of CT data and structures of patients at institute B were anonymized and delivered to other institutes. Written informed consent was obtained from all patients, and the Institutional Ethics Committee approved this study (Kindai University review board number: 29-133). The thickness of the CT sections was 2.5 mm and the field of view was 50 cm. The target and OARs were contoured by a physician according to the protocol of institute B. The bladder in one case (case I) had a volume of 83.8 cm 3 , in another case (case II), bladder volume of 181.8 cm 3 . V overlap /V whole of the rectum and bladder were 9.8% and 11.1% in case I and 5.9% and 5.9% in case II, respectively.
At each institute, the planners who participated in this study had experience with inverse planning for IMRT or VMAT with the Eclipse (Varian Medical Systems, Palo Alto CA, USA) treatment planning system (TPS). They attended a special lecture (RapidPlan Clinical Advisory Board) on Rapidplan held by the manufacturer in Tokyo in June 2017. In KBP using Rapidplan, single optimization was performed. Next, in the manual optimization planning, the optimization was repeated until it achieved the institutional ideal dose distribution. In manual optimization, the generalized equivalent uniform dose (gEUD) was not used in all institutes. In KBP and manual optimization planning, the same calculation parameters and beam parameters were used. The photon optimization was used with 2.5 mm grid size. The calculation algorithm was Anisotropic Analytical Algorithm ver. 13.0 (Varian Medical Systems, Palo Alto CA, USA).

Manual optimization planning vs. KBP
In the rectal and bladder doses calculated by KBP and manual optimization planning, V90 and V50 are shown in Fig. 2 (a), (b), (e), and (f ). In the V90 of the rectum, the mean ± SD of difference between KBP and manual optimization planning was 0.4% ± 1.6% and − 0.1% ± 1.5% in cases I and II, respectively. A negative value implies that dosimetric values for KBP were higher than those for manual optimization planning. In the V50 of the rectum, the mean ± SD of difference between KBP and manual optimization planning was 2.2% ± 6.9% and 2.6% Abbreviations: CTV the clinical target volume, PTV the planning target volume, gEUD generalized equivalent uniform dose ± 8.0% in cases I and II, respectively. For the V90 of the bladder, the mean ± SD of difference between KBP and manual optimization planning was 1.3% ± 2.0% and 1.0% ± 0.9% in cases I and II, respectively. For the V50 of the bladder, the mean ± SD of differences between KBP and manual optimization planning was 4.8% ± 5.0% and 3.6% ± 0.9% in cases I and II, respectively. The dose received by at least 95% of the volume (D95) for the OARs is shown in Fig. 2 (c) and (g). For the D95 of the rectum, the mean ± SD of differences between KBP and manual optimization planning was 0.5% ± 1.9% and 0.1% ± 2.7% in cases I and II. For the D95 of the bladder, the mean ± SD of differences between KBP and manual optimization planning was 1.4% ± 2.0% and 1.2% ± 2.0% in cases I and II. There were no significant differences in each dosimetric parameter between the cases.
The dose received by at least 2% of the volume (D2) for the organs is shown in Fig. 2 (d) and (h). In the D2 for the rectum, the mean ± SD of the difference between KBP and manual optimization planning were − 0.5% ± 0.8% and − 0.9% ± 1.8% in cases I and II. In the D2 for the bladder, the mean ± SD of difference between KBP and manual optimization planning were − 0.1% ± 0.8% and − 0.2% ± 1.3% in cases I and II. There were no significant differences in each dosimetric parameter between the cases.
Various dosimetric values were calculated by KBP among institutes even if they used the same dosimetric parameters. Among institutions, the maximum differences in V90 for the rectum were 6.7% and 6.7%, V50 for the rectum were 39.0% and 41.9%, V90 of the bladder were 18.2% and 9.9%, and V50 of the bladder were 12.5% and 6.7% in cases I and II, respectively. These results suggested that each institutional KBP was useful in that particular institute regardless of the number of registered plans in the model, but the performance varied widely among the institutes. Figure 3 shows the relationships between the upper limit and lower limit of ED and V overlap /V whole for the rectum and the bladder, in institutes A and B. Dotted lines are quadratic regression curves between EDs and the V overlap /V whole . The black dots are dosimetric values calculated by KBP in cases I and II. Black dots were compiled with regression curves for each organ. The dosimetric   rectum (a, b, c, d) and V overlap /V whole for the bladder (e, f, g, h). The vertical axis is the V90 to rectum (a, c) or bladder (e, g). The vertical axis is the V50 for the rectum (b, d) or bladder (f, h). Yellow dots represent the upper limit of ED and blue dots, the lower limit of ED. Red dotted lines with coefficients of determination (R 2 ) are quadratic regression curves between each organ dose and V overlap /V whole for organs. Black dots are calculated doses with knowledge-based planning (KBP) in cases I and II values that were calculated by KBP were between curves of the upper and lower limits of ED or slightly lower than the curve of the lower limit of ED. In each organ, coefficients of determination (R 2 ) of each dosimetric value and V overlap /V whole for the rectum and the bladder are shown in Table 4. The R 2 values of V90 were greater than those for V50, except at institution B for the rectum. In the bladder, the R 2 of V90 were more than those for V50 at all institutions.

Estimated vs. calculated dose
Quadratic regression curves between lower limit of ED and V overlap /V whole for the rectum with the formulas for all institutes are shown in Fig. 4 (a), (b). In the V90 of the rectum (Fig. 4 [a]), four institutes except institute B had regression curves that tended to increase with increasing V overlap /V whole for the rectum. In institute B, the regression curve was almost horizontal. The V90 dose in institute E was the highest of all V overlap /V whole for the rectum. When V overlap /V whole for the rectum was about 10%, the difference in the lower limits of ED between institutions D and E was > 8%. In the V50 for the rectum (Fig. 4 [b]), Institute D had the highest lower limit of ED in all V overlap /V whole for the rectum. When the V overlap / V whole for the rectum was 10%, the difference in lower limit of ED between institutes C and D was > 50%.
In the V90 and V50 for the bladder (Fig. 4 [c], [d]), the lower limit of ED curves for all institutes tended to show increases with increasing V overlap /V whole for the bladder. In the V90 for the bladder (Fig. 4 [c]), when V overlap / V whole for the bladder was 10%, the slopes of lower limits of ED for institutes B and C were steeper than those for institutions A, D, and E. In the V50 for the bladder (Fig. 4  [d]), V overlap /V whole for the bladder was approximately 10%, the lower limits of ED were almost the same for institutes A, C, and E. The slope of the curves varied according to facilities. In among institutions, the maximum differences for lower limit of ED of V90 for the rectum were 8.2% and 5.7%, V50 for the rectum 53.5% and 45.0%, V90 for the bladder 15.1% and 9.4%, V50 to the bladder 33.1% and 26.0% when overlap volume with PTV was 10.0% and 6.0%, respectively.

Discussion
In this study, five institutes used KBP for two cases each and the performance of the KBP models was compared among institutions. Some reports have evaluated the utility of KBP with one model [9][10][11][12]. This study uncovered Table 4 Coefficients of determination (R 2 ) of between each dosimetric value (V90 and V50) and ratio of an OAR's volume overlapping with PTV to the whole organ volume (V overlap /V whole ).vb  Kubo et al. [12] described that the dose coverage to the PTV was slightly inferior in KBP plans compared with manual optimization planning, as can be seen in values for D95 and D2. They used predicted priority values for PTV to confirm KBP predicted accuracy; these values might be underestimated to achieve the dose constraint objectives. In this study, the dose to the PTV was slightly inferior in KBP plans compared with manual optimization planning in some institutes, although there was no significant difference in D95 for the rectal and bladder volumes within the PTV. The first priority was reducing OAR dose for the KBP.
Schubert et al. have proven that it is possible to share models among different institutes in a cooperative framework [13]. Institutes in the report had the same plan design. In this study, in the KBP for multiple institutions, the maximum dosimetric differences for the V90 and V50 calculated with KBP among institutions were > 6.0% in cases I and II in both bladder and rectum. These results suggest that values calculated with KBP were influenced by plans registered in the model. Therefore, it depends on plan designs were matching between institutions whether the models made in other institutions can be shared.
Moore et al. found that that an OAR's mean dose strongly correlated with the rectal and bladder volumes within the PTV [5]. In inverse planning, the understanding of geometric displacements of PTV and OARs led to predicting OAR dose and reducing the planner's variations [5][6][7]. In this study, it was suggested that V90 and V50 had also strong correlation with the rectal and bladder volumes within the PTV in almost institutes. It was found that the correlation tendencies were different among institutes. To optimize the model for a case, it was acceptable to verify the relationship between OARs dose and the rectal and bladder volumes within the PTV.
Tol et al. [7] found that there were strong linear correlations (R 2 = 0.94-0.99) between estimated and achieved mean doses in KBP. They derived the estimated mean dose from KBP models. The ED of the model was important for understanding the performance of KBP. In this study, EDs for V50 and V90 were compared between institutes. To reduce the volume, such as V50 and V90 for the OARs, leads to prevent radiation toxicity for the rectum and bladder. Peeters et al. argued that both intermediate and high doses to the anorectal wall volume should be considered to evaluate the risk of late GI toxicity [14]. Harsolia et al. found the volume of the bladder wall receiving ≥30 and ≥ 80 Gy predicted grade ≥ 2 late toxicity and grade 3 late toxicity [15]. In this study, it was indicated that the calculated OAR dose with KBP depended on registered plans in the model and correlated with OARs volumes in the PTV strongly. Thus, predicting OAR dose from the V overlap /V whole for the rectum and bladder will be required to select the optimal model among several models.
In the relationships between OAR dose and the rectal and bladder volumes within the PTV, R 2 values of V90 were higher than those of V50, except the rectum in institute B, because the OAR volume in the PTV affects the high dose region in the DVH curve [8]. In institute B, R 2 values of the rectum were lower than those of other institutes. V90 for the rectum registered in the model was weak correlation with V overlap /V whole although there were strong correlations in other institutions. In plan designs at institutions except institute B, V90 for the rectum depended on the rectal volume within the PTV. The correlation values between the R 2 for EDs and dose for original DVH in the model were strong, 0.793 and 0.783 as Table 4. This result was showed the plan designs of plans registered in the models affected the relationships between ED and V overlap /V whole .

Conclusions
It has been suggested that KBP performs correctly regardless of institutional plan design. KBP was able to reproduce dose distributions based on the experience of institutions. There was very wide variation in the organ dose calculated with KBP among sites. To share models for KBP, it will be necessary to determine whether the registered DVH curves in the models match the plan design. The models for the KBP were characterized with the ratio of OAR's volume overlapping with the PTV to the whole organ volume.
Abbreviations CTV: Clinical target volume; D2: Dose received by at least 2% of the volume; D95: Dose received by at least 95% of the volume; DVH: Dose volume histogram; ED: Estimated doses; gEUD: Generalized equivalent uniform dose; GTV: Gross tumor volume; IMRT: Intensity-modulated radiotherapy; KBP: Knowledge-based planning; OAR: Organ at risk; PTV: Planning target volume; TPS: Treatment planning system; V50: Volume receiving 50% of the prescribed dose; V90: Volume receiving 90% of the prescribed dose; VMAT: Volumetric-modulated arc therapy; V overlap /V whole : Ratio of an OAR's volume overlapping with PTV to the whole organ volume