Impact of beam configuration on VMAT plan quality for Pinnacle3Auto-Planning for head and neck cases

The purpose of this study was to compare automatically generated VMAT plans to find the superior beam configurations for Pinnacle3 Auto-Planning and share “best practices”. VMAT plans for 20 patients with head and neck cancer were generated using Pinnacle3 Auto-Planning Module (Pinnacle3 Version 9.10) with different beam setup parameters. VMAT plans for single (V1) or double arc (V2) and partial or full gantry rotation were optimized. Beam configurations with different collimator positions were defined. Target coverage and sparing of organs at risk were evaluated based on scoring of an evaluation parameter set. Furthermore, dosimetric evaluation was performed based on the composite objective value (COV) and a new cross comparison method was applied using the COVs. The evaluation showed a superior plan quality for double arcs compared to one single arc or two single arcs for all cases. Plan quality was superior if a full gantry rotation was allowed during optimization for unilateral target volumes. A double arc technique with collimator setting of 15° was superior to a double arc with collimator 60° and a two single arcs with collimator setting of 15° and 345°. The evaluation showed that double and full arcs are superior to single and partial arcs in terms of organs at risk sparing even for unilateral target volumes. The collimator position was found as an additional setup parameter, which can further improve the target coverage and sparing of organs at risk.


Background
Today intensity-modulated radiation therapy is a widely used clinical treatment modality in many countries utilized to achieve improved target dose conformity and better sparing of critical structures [1]. During an iterative process, the objective parameters are adjusted to obtain a clinically acceptable dose distribution. The resulting plan quality is user dependent due to individual experience and optimization strategies. Several approaches for automated treatment planning were developed to overcome this limitation like multi-criteria optimization [2,3], knowledge-based treatment planning [4], or automated treatment planning in Pinnacle 3 (Philips Radiation Oncology Systems, Milpitas, CA, USA) [5,6]. Several studies have compared automatically with manually generated plans for different entities using intensity or volumetric modulated arc therapy (VMAT). Most of them concluded that Pinnacle 3 Auto-Planning offers similar target coverage and improved sparing of organs at risk (OARs) [5,[7][8][9][10][11][12][13].
The purpose of this study was to compare automatically generated VMAT plans to determine a superior beam arrangement as preset for Pinnacle 3 Auto-Planning. Two evaluation methods were used, one of them was newly developed for this study.

Methods
For this retrospective planning study, the influence of VMAT beam configuration on plan quality was investigated for 20 patients treated for head and neck cancer. Table 1 gives an overview of patient and treatment characteristics. Bilateral and unilateral planning target volumes (PTV) were chosen to consider different shapes of target volumes.

Beam configuration
The Pinnacle 3 Auto-Planning engine (Version 9.10) was used for optimization of VMAT plans. Treatment was planned for an Elekta Synergy Platform® equipped with Agility Head (Elekta Oncology Systems, Crawley, UK).
The VMAT beam configuration was varied (overview and naming see Table 2). For each patient, the clinically accepted treatment plan was evaluated by a physician and served as reference (V2 C15 ). Different arc types were compared: one single arc (V1 C15 ), two single arcs (2V1 C15 ) or one double arc (V2 C15 ). Two single arcs can be directly created by the user before optimization (2V1 C15 ) while both arcs are treated equally during  optimization. In contrast, one double arc is generated by the system during optimization after only one arc was initially defined by the user (V2 C15 ). The influence of arc length (full or partial arcs) was evaluated for 10 cases with unilateral located targets (see Table 1). Start and stop angles of partial arcs were set depending on the treated side: 300°to 179°for left sided targets and 181°t o 60°for right sided targets. The impact of collimator setting was explored for two single arcs and double arcs separately. For double arc technique, the impact of different collimator positions was additionally investigated (V2 C15 , V2 C40 and V2 C60 ). The chosen collimator angles (15°, 40°, 60°) are restricted to the first quadrant. The second quadrant is covered by the opposed gantry position in a course of a full rotation. The third and fourth quadrant would be covered by switching X1 and X2.
For two single arcs, the combination of different collimator positions was examined. The combination of collimator positions 15°with 345°was compared to 15°w ith 60°.This resulted in the comparisons as listed in Table 2 (right column).

VMAT optimization
The goals for PTVs and OARs were defined based on the evaluation parameter set (see Table 3). Planning structures were created based on desired target coverage and OAR sparing. For simultaneous integrated boost techniques, the difference of target volumes and planning organs at risk was calculated to avoid overlapping structures. The Auto-Planning goals of the clinically accepted treatment plan served as reference for the other techniques. For comparability, target and OAR goals, dose grid and structure definition remained the same for different beam configurations of one patient case. The dose distribution was calculated with collapsed cone algorithm, dose grid resolution was set to 2 mm and a control point spacing of 4°was used. In the Pinnacle 3 Auto-Planning module advanced setting parameters were not changed (tuning balance between PTV and OARs 11%, dose fall-off margin 2,2 cm, hot-spot maximum goal 107%). Once the Auto-Planning run was completed, no further adjustment or re-optimization were performed.
Auto-Planning tries to mimic the decision-making process of an experienced operator. During Auto-Planning, individual optimization objectives, constraints and weights are automatically added and adjusted based on the user defined clinical goals. Structures of hot and cold spots are created to compensate for over and under dosage. The Auto-Planning module adjusts iteratively the objective set to best meet the planning goals. During optimization, a certain combination of cold and warm starts is performed. The resulting objective set represent in detail the desired shape of the dose volume histogram (DVH). Nevertheless, each optimization process for any patient-beam-configuration ends potentially in different sets of objectives, because Auto-Planning identifies weak points of the individual plans and tries to compensate them by appropriate additional objectives. For each plan, the composite objective value (COV) is calculated. The COV describes the weighted sum of quadratic deviations of the objective values and the related points in the dose volume histogram. In other words, the COV expresses how good the requirements were fulfilled (target coverage and OAR sparing). The optimization process minimizes COV.

Dosimetric evaluation
Plan evaluation was based on two approaches: (i) scoring of evaluation parameters and (ii) indirect comparison of the COV using a new cross comparison method. The clinically accepted treatment plan served as reference (V2 C15 ) for both evaluation methods. The comparison was done for a set of treatment plans for the same patient only.
For the scoring method, an evaluation parameter set was defined in consensus by the physicians of our institution and is considered as standard evaluation set (Table 3). For each patient, the dose limits were scaled according to the prescription dose (D 95 ) in Table 1. Each plan was scored depending on how many requirements were met in the evaluation parameter set. All requirements were equally weighted. The sum of all scores was calculated for each plan. If a plan fulfilled the requirement, this plan achieved the score (see Table 4). If both plans met the requirement and the deviation was more than 1%, the better plan achieved the score. If the deviation was less than 1%, no trial achieved the score (tie). If both trials failed the requirement, the better plan achieved the score.
The second approach compared indirectly the COVs of two plans. The principle of cross comparison is illustrated in a flowchart in Fig. 1. During the optimization step, each technique A creates its own plan A and its corresponding set of objectives S A . For the evaluation step, the assessment results are expressed by COV(A, S A ). As illustrated in Fig. 1, plan A (generated using technique A) is subjected to its own objective set S A , resulting in COV(A, S A ). Similarly, plan B creates its own objective set S B , resulting in COV(B, S B ). The values COV(A, S A ) and COV(B, S B ) cannot be compared directly because the COVs were calculated based on different objective sets (S A and S B ). Therefore we propose a cross comparison method; the measures S A and S B are mutually applied: Technique A is allowed to set the conditions S A for assessment of plan B leading to COV(B, S A ). A symmetric situation is effected when also Technique B is allowed to set its valuation standard S B resulting in COV(A, S B ). Finally, each of both plans A and B is evaluated with two parameter sets (S A and S B ). As long as each plan wins only according to its own objective set, no decision can be made on which plan is superior (COV(A, S A ) < COV(B, S A ) and COV(B, S B ) < COV (A, S B )). In contrast, if plan A wins according to both objective sets S A and S B , it can clearly be regarded as the better technique: Plan A is better according to its own measure and it is better even if plan B sets the assessment scale. In other words, plan quality of plan A is superior to plan B, if both values of COV(A, S A ) and COV(A,S B ) are lower than COV(B,S B ) and COV(B,S A ). We denote "plan A is superior to plan B" as proposition "F A > B "; it is true, if Re-arrangement of Eq. 1 leads to: An inversion of the left part of Eq. 2 results in which is a reformulation of F A > B .
The difference of scoring was tested using Wilcoxon matched-pair signed rank test. The level of significance was set at 0,05. Plan A is superior to plan B as soon as one of two evaluation methods shows a significant difference between the techniques.
Additional parameters were assessed to investigate how the beam arrangement affects the plan metrics: monitor units (MU), delivery time (T), Paddick conformity index (CI).

Results
First, the plan quality was compared for single and double arc techniques (V1 C15 vs. V2 C15 ). The results of the Wilcoxon Rank test are listed in Table 5. Plan quality differed significantly between V1 C15 and V2 C15 with p < < 0,05 for both evaluation methods (scoring and COV based). Figure 2a shows the COV ratios, expressing that V2 C15 is preferable to V1, if F V2 > V1 is true: The evaluation of COV showed a superior plan quality of V2 C15 compared to V1 C15 for all cases because the first term in eq. 4 is below 1 (open circles in Fig. 2a) and at the same time the second term greater than 1 (filled circles). Table 6 shows the change of plan metrics for V1 C15 compared to V2 C15 . A decrease in delivery time (24% on average) and monitor units (− 9%) was observed. For V1 C15 the conformity was reduced by 4% on average.
In the second comparison, it was tested whether better plan quality can be achieved by a double arc or two single arcs (V2 C15 vs. 2V1 C15 ). The p-values in Table 5 indicate that V2 C15 and 2V1 C15 plans are significantly different. The results of the COV method are shown in Fig. 2b. Improved plan quality was observed for the double arc technique V2 C15 in 18 out of 20 (90%) cases. If the deviation was less than 1%, no trial achieved the score (tie). If both trials failed the requirement, the better plan achieved the score. If a plan fulfilled the requirement (true), this plan achieved the score If both plans met the requirement and the deviation was more than 1%, the better plan achieved the score Fig. 1 Flow chart of COV method The difference of COV values is minimal for case numbers 1 and 5. Similar plan quality could be achieved for these cases. The monitor units decreased by 5% on average while the delivery time increased by 4% on average. For 2V1 C15 the conformity was reduced by 3% on average.
In the third comparison, the influence of arc length on plan quality was investigated for full and partial arcs (V2 C15 vs. V2 C15_Part ). A significant difference was found between V2 C15 and V2 C15_Part . The graphical illustration of COV ratios is shown in Fig. 2c. In 6 out of 10 cases the plan quality was better if a full gantry rotation was allowed during optimization. Similar plan  Fig. 2 Comparison of plan quality with the COV method. Plan quality was evaluated and COV ratios for comparison #1-3 are shown. Plan quality of V2 C15 is superior if the first COV ratio is below 1 (open circles) and the second COV ratio is greater than 1 (filled circles). a COV ratios for comparison #1: single arc (V1 C15 ) and double arc (V2 C15 ) for 20 patients. b COV ratios for two single arcs with collimator rotation (2V1 C15 ) and double arc (V2 C15 ) for 20 patients. c COV ratios for full (V2 C15 ) and partial arcs (V2 C15_Part ) for 10 patients with unilateral target volumes quality could be achieved for cases numbers 13, 14 and 20 using full and partial arcs. For V2 C15_Part , delivery times were shorter (16% on average) and conformity was reduced by 2,4% on average. The change in monitor units was minimal (− 3%). The fourth and fifth comparison explored how collimator setting affects plan quality. Based on the COV evaluation, a double arc technique with collimator setting of 15°(V2 C15 ) was superior to double arc with collimator 60°( V2 C60 ) in 6 out of 20 (30%) cases. Similar plan quality was achieved in 11 cases. The Wilcoxon test showed that plan quality differed significantly (p = 0,0008 and p = 0,008) for both evaluation methods (scoring and COV based) between V2 C15 and V2 C60 . No significant difference was found comparing V2 C15 with V2 C40 . Plan quality of V2 C15 was superior in 7 out of 20 (35%) cases while V2 C40 was better in 6 out of 20 (30%) cases. As shown in Table 6, larger deviations in delivery time and conformity index were observed for comparison 5 (V2 C15 vs. V2 C60 ). The decrease of conformity was 0.5% on average in comparison 4 (V2 C15 vs. V2 C40 ).
In comparison 6 and 7, the two collimator settings (2V1 C15_60 and 2V1 C15_345 ) were compared to the reference plan (V2 C15 ). Plan quality of V2 C15 differed significantly from 2V1 C15_345 . V2 C15 was superior in 14 of 20 cases. Plan metrics differed by − 1,5, 29% and − 1,5% for monitor units, delivery time and conformity index ( Table 6). The Wilcoxon test showed no significant difference between 2V1 C15_60 and V2 C15 . The change of monitor units was − 1,5% on average. The conformity index deviated by − 0,4%. Similar plan quality could be achieved for 2V1 C15_60 and V2 C15 .
Additionally, the two single arcs were compared with each other in comparison 8 (Table 5). 2V1 C15_60 was superior against 2V1 C15_345 in 14 of 20 cases.
The correlation of scoring and COV method is shown in Fig. 3. The p-values from the Wilcoxon test are plotted in logarithmic scaling. A strong correlation between the two evaluation methods was found with a correlation coefficient of 0,91 and a higher sensitivity of the COV method.

Discussion
In this work, it was investigated which VMAT beam configuration shows superior plan quality in head and neck cancer patients. The double arc technique resulted in an improved plan quality compared to one single arc or two single arcs. This superiority of double arc VMAT plans compared with single arc in terms of PTV coverage and OAR sparing was also confirmed in other planning studies for head and neck cases [14][15][16][17]. Guckenberger et al. showed that multiple arc VMAT improved the plan quality compared to single arc VMAT at cost of increased delivery times, increased monitor units and increased spread of low doses [14]. Tol et al. have quantitatively demonstrated increased plan quality when more than two arcs are used. They concluded that the four arc plans seemed to provide a good trade-off between increased delivery time and improved plan quality [15].
We investigated if there is a difference in plan quality using double arc or two single arcs. Plan quality was superior for the double arc technique for the same collimator setting. This may be due to how the two arcs are handled during optimization and sequencing. As already suggested by Yang et al., the definition of two single arcs before optimization allows the user to configure the collimator position of the second arc different from the first arc [18]. The collimator position is an additional setup parameter which can further improve plan quality. For Fig. 3 Comparison of the scoring method and COV method. The correlation of p-values derived from Wilcoxon test is plotted in logarithmic scaling. Significance level (dashed line) and corresponding quadrants are shown the current investigation, it was found that a collimator setting of 15°and 60°should be preferred to a collimator setting of 15°and 345°when two single arcs are used.
Similar plan quality and smallest deviation of conformity was observed for a double arc technique with collimator setting 40°(V2 C40 ) and for a two single arc technique with collimator setting of 15°and 60°( 2V1 C15_60 ). The superior collimator position potentially depends on the individual target volume shape and complexity. If the relationship between target volume complexity and corresponding superior beam arrangement would be known, the superior beam arrangement could be chosen based on the complexity index of the target volume. This can potentially save time during the treatment planning process.
The current investigation of the arc length showed that full arcs were superior to partial arcs. Contrary to expectation for using partial arcs, the irradiation across the ipsilateral side and sparing of the healthy side would provide benefits; the full rotation is the superior technique. Perhaps, this is due to the additional degree of freedom. In previous studies different arrangements of full and partial arcs were compared for the treatment of head and neck cases [18,19]. Miura et al. found that partial arcs were comparable with the full-arc plans regarding dose homogeneity and conformity in maxillary cancer and provided a statistical decrease in mean dose to OAR, total MU, delivery time and gantry angle error [19]. Yang et al. stated that it will be challenging to generate partial arc plans for complicated cases [18].
Furthermore, a new cross comparison method was introduced in this study to allow a simple comparison for plans that use different sets of assessment scales, derived during the optimization process of each plan. Its results were similar to that of a classical method of scoring of an evaluation parameter set; cross comparison seems to be applicable especially for Auto-Planning settings. The COV method is more sensitive than the scoring method.

Limitations
The current work was limited to an evaluation of single and double arc techniques based on patient cases for head and neck cancer. Further improvement of plan quality would be expected for using a larger number of arcs or non coplanar beam arrangements. Multiple arc VMAT showed improved plan quality compared to single arcs at cost of increased delivery times and monitor units [14,15].
As Teoh et al. already mentioned, direct comparisons between different studies are difficult because of significant differences in target volume definitions, target complexity, dose prescription and treatment schedules [20].
An additionally impact will have the treatment planning system and the implemented algorithms for VMAT optimization and segmentation.

Conclusion
The impact of different beam configurations on plan quality using Pinnacle 3 Auto-Planning module was investigated in this study. Quantitative evaluation showed that double arc and full rotation are superior to single arc and partial rotation techniques in terms of organs at risk sparing even for unilateral target volumes. The collimator position was found as an additional setup parameter which can further improve the target coverage and sparing of organs at risk.