Identification of variables and development of a prediction model for DIBH eligibility in left-sided breast cancer radiotherapy: a prospective cohort study with temporal validation

Abstract

Objective

To identify variables associated with a patients’ ability to reproducibly hold their breath for deep-inspiration breath-hold (DIBH) radiotherapy (RT) and to develop a predictive model for DIBH eligibility.

Methods

This prospective, single-institution, IRB-approved observational study included women with left-sided breast cancer treated between January 2023 and March 2024. Patients underwent multiple breath-hold sessions over 2–3 consecutive days. DIBH waveform metrics and clinical factors were recorded and analysed. Logistic mixed modelling was used to predict DIBH eligibility, and a temporal validation cohort was used to assess model performance.

Results

In total, 253 patients were included, with 206 in the model development cohort and 47 in the temporal validation cohort. The final logistic mixed model identified increasing average breath-hold duration (OR, 95% CI: 0.308, 0.104–0.910. p = 0.033) and lower amplitude (OR, 95% CI: 0.737, 0.641–0.848. p < 0.001) as significant predictors of DIBH eligibility. Increasing age was associated with higher odds of being ineligible for DIBH (OR, 95% CI: 1.040, 1.001–1.081. p = 0.044). The model demonstrated good discriminative performance in the validation cohort with an AUC of 80.9% (95% CI: 73.0-88.8).

Conclusion

The identification of variables associated with DIBH eligibility and development of a predictive model has the potential to serve as a decision-support tool. Further external validation is required before its integration into routine clinical practice.

Introduction

Irradiation of patients with left-sided breast cancer using deep-inspiration breath-hold (DIBH) radiotherapy (RT) technique is now the standard of care. An abundance of dosimetric literature concludes that DIBH with forward Intensity-Modulated Radiation Therapy (IMRT) planning, can achieve a remarkable reduction in whole heart mean dose (D_mean), which is the strongest predictor for late cardiac morbidity [1, 2]. Center-specific protocols for assessing eligibility for DIBH RT technique are infrequently reported, resulting in slow adoption due to limited access to practical knowledge and motivated the development of a recent ESTRO-ACROP guideline on DIBH techniques [3, 4].

At most centers, patients undergo a training session before simulation, during which the procedure is explained and practised. Measures that may influence the decision to proceed with DIBH RT technique include reproducibility (defined as inter-fraction or intra-fraction breath-hold variation), duration of reproducible breath-hold (as short breath-holds will increase the ‘on-couch time’ during treatment delivery) and patient-specific factors (age, BMI, smoking history) [4]. The different equipment used for DIBH (surface or marker-based systems, spirometry-based systems and equipment-free DIBH) also contribute to the variation between different centers’ protocols. While the choice of equipment does not affect the reduction in heart D_mean achieved with DIBH RT technique, the literature on characteristics that influence patient eligibility for DIBH RT technique is sparse [5].

This analysis attempts to identify variables associated with a patient’s ability to reproducibly hold their breath for the DIBH RT technique (PROGRESS Type II study) [6]. Furthermore, we attempted to develop and validate a model (PROGRESS Type III study) to predict whether a patient will be eligible for DIBH RT technique [7].

Materials & methods

Population – model development and temporal validation cohorts

This prospective, single-institution, IRB-approved [protocol ID: RES/SCM/62/2024/01, IRB Approval ID: RGCIRC/IRB-BHR/30/2024] observational study was conducted between January 2023 and March 2024. We developed the model using data from January 2023 to December 2023 and used data from January 2024 to March 2024 for temporal validation (TRIPOD Analysis Type 2b). Only women with left-sided breast cancer were eligible. Inclusion criteria were: (a) age over 18 years with pathological stage I-III disease after surgery (breast conservation or mastectomy), and; (b) requiring adjuvant RT (42.5 Gy/16Fx, 5 days/week) to whole breast or chest wall with/without elective regional nodal irradiation (sequential lumpectomy boost permitted; 10 Gy in 4Fx). Neoadjuvant and adjuvant chemotherapy and/or trastuzumab was permitted.

Exclusion criteria were: (a) metastatic breast cancer or an indication for palliative radiotherapy, or; (b) an Eastern Cooperative Oncology Group performance status of 3 or higher, or; (c) individuals unable to follow or understand the instructions for DIBH due to any reason (language barriers, hearing loss, or cognitive impairment), or; (d) any cardiac or pulmonary condition with NYHA class III-IV or MRC dyspnea grade IV-V, respectively.

All included patients were assessed for treatment in DIBH (RPM system, Varian Medical Systems, USA) and received RT via Field-in-Field IMRT or Volumetric Modulated Arc Therapy on a 6MV LINAC with daily kV-MV verification (Clinac 2100c, Varian Medical Systems, USA). There are no treatment time limits on our LINACs, allowing the accommodation of varying DIBH performance amongst patients.

To achieve our primary objective of identifying variables to predict DIBH eligibility, we used DIBH waveforms and clinical factors associated with reproducible breath-hold in patients undergoing assessment for DIBH RT technique. The identified variables were then used to develop a prediction model.

Study procedure - DIBH assessment

All patients underwent 2–3 consecutive days of DIBH assessment performed by a team of technologists (radiation therapists), medical physicists and radiation oncology registrars. After explaining the DIBH procedure to the patients, they underwent multiple breath-hold sessions each day, interspersed with 20–30 s breaks. The difference between chest versus abdominal breathing was explained to them and they were coached to avoid arching the spine to reach the DIBH threshold. Each day, at least 15 sessions were performed depending on the patient’s comfort and visual feedback was provided via an in-room display. A 40 min slot was kept for each patient on each assessment day. After the day’s assessment was completed, the team reviewed the waveforms and variables were extracted as defined below.

Let n be the total number of breath-hold attempts a patient underwent on a day, then:

• 1. Upper Amplitude: The highest stable amplitude reached in any of the n attempts.

• 2. Lower Amplitude: The lowest stable amplitude reached in any of the n attempts)

• 3. Average Amplitude (in all n attempts): (Amplitude_1 + Amplitude_2 + . + Amplitude_n) ÷ n.

• 4. Average Breath-hold Duration (in all n attempts; only the duration when the amplitude was within the gating window is measured): (Duration_1 + Duration_2 + . + Duration_n) ÷ n.

• 5. Consistent Breath-hold (within ± 3 mm gating window): (Yes/No)

An example calculation is shown in Fig. 1 for two patients.

Fig. 1

Illustrative summary of DIBH assessment workflow for two patients across three consecutive days. The numbers shown above the blue line and below the orange line on the waveform represent the gating window, and the breath-hold trace is in black. The amplitude of the breath-hold is measured using the right y-axis

All patients attempted at least 15 breath-hold each day, following which waveform metrics were calculated (for definitions, see materials and methods). The target amplitude for Day 3 was determined using the average amplitudes recorded on Days 1 and 2 and a nominal gating window of 6 mm (± 0.5 mm) was applied. The decision to proceed with DIBH was based on fulfilment of eligibility criteria (for description, see materials and methods)

Patient 1 illustrates a candidate fit for DIBH technique, with consistent breath holds, indicated by the nearly identical waveforms across multiple attempts. Patient 2 illustrates an unfit candidate, with erratic breath holds and slight exhalation during the breath-hold attempts, as shown by the red stars (indicating the fall in amplitude). The red arrows on the waveform for patient 2 represent the start and end of breath-hold duration for each attempt. The data was collected and analyzed over three consecutive days and the final decision on suitability for DIBH treatment was taken on the third day

On the last day of assessment (Day 3), the team decided whether to treat the patient with DIBH or not. First, a nominal gating window of 6 mm (± 0.5 mm) was chosen based on the average amplitude that the patient could reach on previous assessment days. Next, the patient was asked to reach and stay within the specified gating window for three consecutive breath-holds, with two specific criteria needing to be met:

• 1. The patient must remain within the gating window for at least 80% of the total breath-hold duration.

• 2. The 80% duration within the gating window must amount to at least 15 s long.

The gating window was adjusted based on the patient’s performance (if required), as follows:

• 1. If both criteria (defined above) were met, the patient proceeded to DIBH CT simulation directly.

• 2. If the amplitude exceeded the upper limit of the planned gating window, we adjusted the window to the higher amplitude and if both criteria (defined above) were met, the patient proceeded to DIBH CT simulation.

• 3. If the amplitude fell below the lower limit of the planned gating window or the duration of time within the gating window was less than 15 s, we reviewed the previous performance with the patient, allowed a 10–15 min rest, and asked the patient to try again. If both criteria (defined above) were met, we proceeded with a DIBH CT simulation, otherwise the patient underwent a non-DIBH CT simulation.

Data from the DIBH assessment waveforms was prospectively collected and recorded in our workflow management system (OncFlow™, Dashamlav AI Labs, India; dashamlavlabs.ai) and upon extraction, were assessed by the principal investigators (IA, KSC, AAM). Inconsistencies due to incorrect data entry were resolved by reviewing the original waveform data.

Statistical analysis

Data description and sample size calculation

Baseline characteristics were reported as the median with interquartile range (IQR) (continuous variables) or frequencies and percentages (categorical variables). Our previous institutional experience found that 50–55% of patients assessed over 3 days were suitable for the DIBH RT technique. In this study, only 3% (7/253) of patients started treatment in DIBH but required conversion to free-breathing. These patients were unable to reach the amplitude which was set during assessment in the first three treatment fractions, and for modelling, they were classified as non-DIBH. Guidance on a priori sample size calculation for prediction modelling is emerging but specific guidance for a logistic mixed modelling approach is absent [8].

Model development, variable selection and optimisation

A logistic mixed modelling approach was used to predict whether patients would be eligible for DIBH. This improves parameter estimation accuracy by accounting for the correlation between repeated measurements within individuals and can handle missing data. Including all available data in the analysis reduces bias and optimizes power [9, 10]. The variables were modelled as follows:

• Continuous variables were kept as continuous (without dichotomisation) to avoid information loss.

• Age, BMI, comorbidities and surgery type were modelled as fixed effects.

• DIBH Measurements taken on three consecutive days (upper amplitude, lower amplitude, average amplitude of DIBH, average breath-hold duration and consistent breath-hold) were modelled as fixed effects with random slopes for the day of measurement to account for the possibility that the rate of change over the days may differ. Due to deviation from normality, average breath-hold duration was log-transformed.

• Individual patients were modelled as a random effect to account for correlated measurements from the same patient, thereby allowing each patient to have their own baseline probability of the outcome.

Variable selection was performed via manual stepwise backward selection after fitting a maximal model, which included all variables, their respective fixed and random effects, and random slopes for the day of assessment [11]. Initially, multicollinear variables were removed, and then variable selection was performed by comparing the log-likelihood between models with and without the variable. Goodness-of-fit was assessed using the Akaike Information Criterion (AIC), Bayesian Information Criterion (BIC) and log-likelihood of the final model. The BOBYQA (Bound Optimization BY Quadratic Approximation) optimiser was used with an iteration limit of 200,000 to efficiently locate the minima using a derivative-free method. The results of mixed-effects modelling were reported as odds ratios (OR) with respective 95% confidence intervals (CI), the level of significance was set at 5% and the modelling was performed with R v4.3.2 [R core team (2023), Austria] and lme4 package [12]. Patients and the general public were not involved in this analysis.

Model performance

Logistic mixed models have two components: a fixed effect (which models average trends that persist across the population) and a random effect (which models the extent of variation in fixed effects by a grouping factor). As outlined above, the grouping factor in our model were individual patients (each with a unique ID). An assessment of the model’s performance on the development data would exhibit near-perfect evaluation metrics. This is likely a consequence of overfitting to the development data, utilising unique IDs for precise, individualised predictions. To overcome this issue, we performed a temporal validation of the model’s overall performance. The model’s discriminative performance was assessed by the area under the receiver-operating characteristic curve (AUC) and classification measures (Accuracy, Recall, Precision and F1 score) [13, 14]. This report was prepared in accordance with the TRIPOD + AI and STROBE guidelines (supplemental materials) [15, 16].

Results

A total of 253 patients were included in this analysis, of which 206 were in the development dataset (January 2023 to December 2023), and 47 patients were in the temporal validation dataset (January 2024 to March 2024). Their demographic and assessment details are shown in Tables 1 and 2, respectively. A visual comparison of the evaluated waveform metrics (stratified by patients who were deemed eligible or ineligible for DIBH) is shown in the supplemental materials (Figure S1). Overall, 54% (112/206) of patients in the development dataset and 60% (28/47) in the temporal validation dataset were eligible for the DIBH RT technique.

Table 1 Baseline characteristics of the patient population (model development dataset, n = 206; temporal validation dataset, n = 47)

Table 2 Results of assessments performed during DIBH training on the patient population (model development dataset, top; temporal validation dataset, bottom)

Logistic mixed model

The final variables in the logistic mixed model are shown in Fig. 2. Increasing average breath-hold duration (OR, 95% CI: 0.308, 0.104–0.910. p = 0.033) and lower amplitude (OR, 95% CI: 0.737, 0.641–0.848. p < 0.001) were significantly associated with higher odds of being eligible for DIBH. Increasing age was significantly associated with higher odds of being ineligible for DIBH (OR, 95% CI: 1.040, 1.001–1.081. p = 0.044), while the inability to stay within the defined gating window consistently showed a trend towards significance (OR, 95% CI: 2.741, 0.961–7.818, p = 0.059). The model’s discriminative performance metrics on the development data are reported in supplemental materials (Figure S2).

Fig. 2

Results of logistic mixed modelling of variables influencing DIBH eligibility. Variables marked with red dots on the forest plot are significant

Abbreviations: BCS: Breast Conservation Surgery; BH: Breath-hold; BMI: Body Mass Index; CVD: Cardiovascular Disease

Temporal validation

Figure 3 shows the logistic mixed model’s performance on the temporal validation dataset. The model’s accuracy in predicting patients who will not be eligible for the DIBH RT technique was 76.9 (95% CI: 68.8–83.7), and its AUC was 80.9 (95% CI: 73.0–88.8). The kappa statistic of 0.5191 indicates a moderate agreement beyond chance, and the model significantly surpassed the No Information Rate of 62.69% (p = 0.0003), demonstrating its predictive performance over a naïve classification approach.

Fig. 3

Performance of the logistic mixed model on the temporal validation dataset. (A) Receiver Operating Characteristic (ROC) curve illustrating the model’s performance in distinguishing between DIBH (Deep-inspiration Breath-hold) and non-DIBH cases. (B) Density plot and histogram of predicted probabilities for DIBH and Non-DIBH cases. The blue and red curves represent the density distributions of predicted probabilities for DIBH and Non-DIBH, respectively. Grey bars indicate the histogram of predicted probabilities across the dataset. Blue and red circles along the x-axis denote individual predictions for DIBH and Non-DIBH cases, respectively. The vertical black line represents the decision threshold for classification between DIBH and Non-DIBH

Discussion

To the best of our knowledge, this analysis of DIBH eligibility based on waveforms and clinical factors is the first of its kind reported in the literature. We report several important findings. First, besides age, no other baseline patient characteristics influenced DIBH eligibility. Second, average breath-hold duration and lower amplitude were associated with DIBH eligibility. Finally, our model demonstrated moderate accuracy in predicting an individual patient’s eligibility for DIBH RT technique. The key strengths are the large sample size, assessment performed over three consecutive days, and robust statistical analysis with temporal validation of the developed model. We have also defined variables to be extracted from assessment waveforms, which allows objective evaluation of DIBH eligibility, provides institution-specific data for analysis, and tracks patient performance.

A few differences between our analysis and the literature are worth highlighting. While the literature states that up to 90% of patients are eligible for DIBH, our real-world analysis found that overall, 55% of patients were eligible. Since we did not place any pre-assessment criteria for DIBH eligibility (except for those with ECOG PS 3/4 or significant cardio-pulmonary comorbidity), our results may represent the experience of large-volume centres treating patients with default DIBH approach, outside of a study protocol. A similar experience has been reported by other centres, where up to 29–33% of patients were found ineligible for DIBH after initial assessment [17, 18].

Similar to the definition of a moderately deep breath-hold, which is approximately 70–85% of the maximum BH level (using spirometer-based techniques), our analysis found that using the RPM-based system, our patients’ average breath-hold amplitude (analogous to the moderately deep BH) was also 72.2% (IQR: 61.1–81.8%) when compared to the upper amplitude (analogous to maximum BH) [4]. Interestingly, both upper and average amplitude were not significantly associated with eligibility for the DIBH technique in our analysis and were excluded from the developed model. Instead, the lower amplitude determined eligibility, implying that patients who could adequately hold their breath at a higher level (despite being their worst attempt) were more likely to be eligible for DIBH.

A potential criticism of our assessment protocol is the time and resources expended. We contend that the time spent on DIBH assessment is less resource-intensive than that spent on the treatment machine. Over-zealous utilisation of DIBH in patients with short BH can increase on-couch time, and repeated BH attempts can potentially exhaust their BH capacity faster, creating a vicious cycle. This has potential downstream effects on patient waiting time and satisfaction [19]. Ultimately, the conversion of a patient’s treatment from DIBH to non-DIBH may be more resource-intensive and may increase their anxiety [18]. The relationship between waveform metrics during assessment and on-couch time during treatment, as well as patients’ perception of our assessment protocol are avenues for future research that our group will undertake.

In contrast to our three day assessment protocol for a default DIBH approach, some centres prefer an initial quantification of the magnitude of benefit which will be achieved with DIBH compared to FB [20, 21]. This preference arises from the increased workload associated with assessing all patients for DIBH treatment delivery and the average heart D_mean in high-income countries (HIC) ranging from 2.8 to 3.8 Gy [22]. The problem in lower-middle income countries (LMIC) is that the average heart D_mean is higher (6.2 Gy), the overall population is at higher risk of major cardiovascular disease and less than a third of practicing radiation oncologists utilise any form of respiratory motion management [3, 22, 23]. Therefore, from the LMIC perspective any reduction in heart D_mean is important and since situations where DIBH would lead to higher cardiac exposure are rare, a ‘DIBH for all’ approach has the potential to reach the most patients despite constrained resources [24].

Another potential criticism of our analysis could be the use of a logistic mixed modelling approach, which is more complex and may be less familiar to readers than more straightforward techniques such as repeated measures analysis of variance (RM-ANOVA). The use of RM-ANOVA would have excluded patients with missing assessment data because it handles missing observations via listwise deletion, thereby reducing the sample size with potential implications for the real-world usage of the model [25]. We also acknowledge that our sample size may not have been large enough to detect more subtle associations. Our group will also report alternative modelling approaches using machine learning and deep learning techniques in the near future [26].

The developed model will now be tested at our institution to check the agreement between the team’s decision and model predictions, and it is anticipated that it may serve as a decision-support tool in the future by reducing subjectivity. It may also serve as a tool for trainees to gain confidence in their decision-making abilities. To perform external validation, we intend to release the model’s code to interested researchers pending further testing at our institution. Once completed, we intend to design a R Shiny application for broader use and envision a hybrid DIBH assessment protocol in the future, where model predictions assist, but do not replace clinical judgement.

In conclusion, our analysis identified variables associated with a patient’s ability to consistently maintain DIBH which were then used to develop a prediction model. External validation of the model on a more diverse patient population is required before integration into routine departmental workflows.