Predicting the survival of patients with painful tumours treated with palliative radiotherapy: a secondary analysis using the 3-variable number-of-risk-factors model

Abstract

Background

The 3-variable number-of-risk-factors (NRF) model is a prognostic tool for patients undergoing palliative radiotherapy (PRT). However, there is little research on the NRF model for patients with painful non-bone-metastasis tumours treated with PRT, and the efficacy of the NRF model in predicting survival is unclear to date. Therefore, we aimed to assess the prognostic accuracy of a 3-variable NRF model in patients undergoing PRT for bone and non- bone-metastasis tumours.

Methods

This was a secondary analysis of studies on PRT for bone-metastasis (BM) and PRT for miscellaneous painful tumours (MPTs), including non-BM tumours. Patients were grouped in the NRF model and survival was compared between groups. Discrimination was evaluated using a time-independent C-index and a time-dependent area under the receiver operating characteristic curve (AUROC). A calibration curve was used to assess the agreement between predicted and observed survival.

Results

We analysed 485 patients in the BM group and 302 patients in the MPT group. The median survival times in the BM group for groups I, II, and III were 35.1, 10.1, and 3.3 months, respectively (P < 0.001), while in the MPT group, they were 22.1, 9.5, and 4.6 months, respectively (P < 0.001). The C-index was 0.689 in the BM group and 0.625 in the MPT group. In the BM group, time-dependent AUROCs over 2 to 24 months ranged from 0.738 to 0.765, while in the MPT group, they ranged from 0.650 to 0.689, with both groups showing consistent accuracy over time. The calibration curve showed a reasonable agreement between the predicted and observed survival.

Conclusions

The NRF model predicted survival moderately well in both the BM and MPT groups.

Background

Palliative radiotherapy (PRT) for cancer-related pain due to bone metastases (BMs) has been recognised as beneficial, and there exists a solid consensus regarding its efficacy [1,2,3]. The 3-variable number-of-risk-factors (NRF) model reported by Chow et al. [4]. serves as a prognostic tool for patients undergoing PRT [4]. Numerous validation studies of the NRF model have been conducted, and its utility has been well-documented [5,6,7,8,9,10,11]. In addition, the NRF model is simple and easy to use in clinical practice without requiring detailed imaging or blood tests [4].

While many validation studies of the NRF model have investigated patients receiving PRT for BMs [5, 7,8,9, 11], some validation studies have included patients with tumours other than BM. For instance, Mojica-Márquez et al. included patients who received PRT for brain metastases as well as BMs [6], and Glare et al. included both patients who received PRT and those who did not in the setting of a palliative care outpatient clinic [10] in their validation studies. However, to our knowledge, there have been no validation studies of the NRF model for patients with painful non-BM tumours treated with PRT, and the efficacy of the NRF model in predicting survival is unclear to date.

In the present study, we sought to validate the performance of the NRF model in patients who underwent PRT for painful BM and non-BM tumours.

Methods

Study design and patients

This study was approved by the institutional review board of Kanazawa University Hospital [approval no.: 2022 − 361 (714265)] and was conducted according to the tenets of the Declaration of Helsinki. We performed a secondary analysis using data from two previously published studies: a single-centre retrospective study on PRT for BM (BM group) [11] and a multicentre prospective observational study on PRT for painful BM and non-BM tumours (miscellaneous painful tumours [MPTs] group) [12]. The BM group consisted of 485 patients (including 109 patients excluded based on the availability of information to assess the new Katagiri scoring system [13] in the original retrospective study [11]), and the MPT group consisted of all 302 patients analysed in the original multicentre study [12]. In the MPT group, the irradiated tumours (n = 302) were solid (n = 262) and hematologic (n = 40) tumours. Of the 262 solid tumours, the irradiated tumours were primary tumour lesion (n = 69), lymph node metastasis (n = 30), bone metastases (n = 127), hematogenous metastasis other than bone metastasis (n = 7), pleural dissemination (n = 11), and others (n = 18). Of the 40 hematologic tumours, the irradiated tumours were myeloma (n = 18), plasmacytoma (n = 6), lymphoma (n = 13), and others (n = 3) [12].

The 3-variable NRF model

All patients were grouped according to the total number of the following three risk factors: (1) non-breast cancer, (2) sites of metastases other than bone, and (3) a Karnofsky performance status (KPS) score ≤ 60 (an Eastern Cooperative Oncology Group performance status [ECOG PS] scale score ≥ 2).

Patients with one or no risk factors were classified as group I, those with two risk factors were classified as group II, and those with three risk factors were classified as group III [4]. We evaluated performance status (PS) using the ECOG PS, not the KPS. For the ECOG PS and KPS correspondence, ECOG PS 0 corresponds to KPS 90–100, ECOG PS 1 to KPS 70–80, ECOG PS 2 to KPS 50–60, ECOG PS 3 to KPS 30–40, and ECOG PS 4 to KPS 10–20 [4].

Statistical analysis

Overall survival (OS), defined as the time from the start of PRT until death from any cause, was estimated using the Kaplan–Meier method, and differences between survival curves were tested using the log-rank test. Univariate and multivariate Cox proportional hazards models were used to assess the association between the risk factors in the NRF model and OS.

Model performance was assessed in two aspects: discrimination and calibration. The discrimination performance (i.e. the ability of a model to distinguish between patients who will die earlier and those who will die later) of the NRF model was evaluated using time-independent and time-dependent methods. Time-independent evaluation was performed using Harrell’s C index [14]. The C-index is a discrimination performance assessment (a value between 0 and 1), with a value closer to 1 indicating a more accurate performance [14]. Time-dependent evaluation was performed using the time-dependent receiver operating characteristic (ROC) analysis [15]. The area under the ROC curve (AUROC) was a measure of discrimination.

Cutoff values were set in the NRF model so that the patients were divided into two groups (Group I vs. II and III [cutoff value 1]; I and II vs. III [cutoff value 2]; and Group I, II, and III vs. no patients [cutoff value 3]). Time-dependent cumulative sensitivity, dynamic specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated for predicting death within 2, 4, and 12 months.

In the assessment of calibration, we evaluated the concordance between the observed and predicted survival outcomes using a calibration curve. This process gauges the alignment between the model’s predicted probabilities and the actual observed outcomes [16]. Calibration plots were created for predicting 2-month mortality. Statistical analyses were performed using R (version 4.2.2); the “timeROC” package was used for the time-dependent ROC analysis.

Results

Patients’ clinicopathological characteristics

The characteristics of the 485 patients in the BM group and the 302 patients in the MPT group are shown in Table 1. The most common primary site was the lung in both groups. According to the NRF model, the largest number of patients in both groups belonged to Group II. The next most common group was Group III patients in the BM group and Group I patients in the MPT group (Table 1).

Table 1 Patient characteristics

OS

The median follow-up duration using the reverse Kaplan–Meier method [17] was 25.7 (95% confidence interval [CI]: 21.9–29.4) and 24.9 (95% CI: 20.7–30.9) months in the BM and MPT groups, respectively (Table 1). The OS rates, as grouped according to the NRF model, in the BM and MPT groups are shown in Fig. 1. Using the NRF model, the median survival times in the BM group for groups I, II, and III were 35.1 (95% CI: 21.9–64.0), 10.1 (95% CI: 8.3–13.2), and 3.3 (95% CI: 2.6–4.1) months, respectively (P < 0.001), while in MPT group, they were 22.1 (95% CI: 12.2–30.2), 9.5 (95% CI: 6.5–12.2), and 4.6 (95% CI: 3.3–7.9) months, respectively (P < 0.001).

Fig. 1

The Kaplan–Meier curve of overall survival according to the three-variable number of risk factors model for patients who receive palliative irradiation for bone metastases (a) and patients who receive palliative irradiation for miscellaneous painful tumours (b)

P-value represent values from log-rank test

Association between the risk factors in the NRF model and OS

Table 2 shows the results from the univariate and multivariate Cox proportional hazards models to examine how each factor in the NRF model affected OS. Multivariate analysis revealed that in both groups, the presence of metastases other than in bone and an ECOG PS ≥ 2 had significant associations with survival.

Table 2 Evaluation of prognostic factors by Cox regression analysis

Discrimination performance of the NRF model

Table 3 shows the C-index (the measure of time-independent discrimination performance) and the time-dependent AUROC (the measure of time-dependent discrimination performance) of the NRF model. The estimates of time-dependent AUROC showed that the discrimination performance seemed to be constant over the course of time in both groups.

Table 3 C index and time-dependent area under the receiver operating characteristic curve

Time-dependent performance metrics

Table 4 shows the time-dependent cumulative sensitivity, dynamic specificity, PPV, and NPV when using the NRF model cutoff values of 1–3. The NPVs with a cutoff value of 1 (i.e. probability of not dying within 2 months for patients in group I) showed that almost all of the group I patients (97% in the BM group and 96% in the MPT group) did not die within 2 months (Table 4). On the other hand, when using the cutoff value of 3 (when all patients were predicted to survive > 2 months), the NPVs were 84% and 89%, respectively (i.e. 16% and 11% of non-selected patients died within 2 months, respectively). The calibration curve for predicting 2-month mortality in both groups showed favourable agreement between the predicted and observed survival (Fig. 2).

Table 4 Accuracy of predicting death within 2, 4, and 12 months using three-variable number-of-risk-factors model cutoff values of 1–3

Fig. 2

Calibration curve for patients who receive palliative irradiation for bone metastases (a) and patients who receive palliative irradiation for miscellaneous painful tumours (b). The grey line represents perfect prediction. Calibration is high when the red curve is close to the grey line

Discussion

The NRF model had moderately favourable prognostic performance in the MPT group as well as in the BM group. This finding was supported by two points. First, the survival analysis of the BM and MPT groups showed clear differences between the three groups of the NRF model classification. Second, the results of the discrimination performance evaluation by two methods (C-index and time-dependent AUROC) were acceptable. Considering that the NRF model is simple and easy to use and requires no detailed imaging or blood tests, it is highly useful in daily practice.

This present study appears to have analyse patients with a better prognosis compared to past studies involving patients with various diseases. The NRF model, developed by Chow et al. for patients receiving PRT for miscellaneous tumours [4, 18] included a patient group where 70% had BM, 69% had visceral metastases, 18% were referred to the clinic for brain metastases, and others were referred for symptomatic relief of bleeding, shortness of breath, and tumour mass [18]. The MSTs reported by Chow et al. for groups I, II, and III were 60 weeks (8.6 months), 26 weeks (3.7 months), and 9 weeks (1.3 months), respectively [4]. In a validation study of the NRF model involving patients with miscellaneous tumours, where 32.5% received PRT for brain metastases, 50.1% for bone metastases, and 17.4% for other sites, the MSTs for groups I, II, and III were 15.0, 6.5, and 2.3 months, respectively [6]. Another validation study of the NRF model in an outpatient palliative care clinic, regardless of PRT, showed MSTs for groups I, II, and III were 9.0, 4.6, and 2.1 months, respectively [10]. Our MSTs, which appear longer than those in these studies, suggest that the NRF model may be useful in stratifying the prognosis of patients with a better overall prognosis.

In the evaluation of discrimination performance by time-independent methods, Chow et al. [4]. evaluated the model performance of the NRF model, mainly using the C-index [4]. They reported a model C-index of 0.65 for the training set, 0.66 for the temporal validation set, and 0.63 for the external validation set [4], and our C-index was similar to those values (0.685 for the BM group and 0.625 for the MPT group). Other studies of patients who received irradiation or re-irradiation for spinal metastases reported C-indexes of 0.76 and 0.6, respectively [8, 9]. The external validation set of the RTOG 9714 trial for patients with breast or prostate cancer with BMs reported a C-index of 0.94, indicating very high model performance [5]. On the other hand, in the evaluation of discrimination performance by time-dependent methods, Yap et al. [7]. evaluated the discrimination performance of the NRF model using the Uno C statistic [7]. The Uno C statistic, as well as Harrel’s C-index, when significantly greater than 0.5, indicates favourable model discrimination, demonstrating the model’s ability to predict survival with higher precision as values approach 1 in a 0 to 1 range [19]. The Uno C statistic for the NRF model was 0.58 for 3 months, 0.58 for 6 months, and 0.59 for 12 months [7]. As also shown in the present results of the time-dependent AUROC, the discrimination performance of the NRF model was approximately equally as favourable in the short term (a few months) and in the long term (12–24 months). This finding contrasts with the high short-term discriminative performance of another widely used prognostic model in the palliative care setting, the Palliative Prognostic Index, which showed poorer long-term discriminative ability [20].

Stereotactic body radiation therapy (SBRT) for painful BM increases treatment costs and delays the start of radiation. However, its high complete pain response rate [21, 22] and potential for long-term pain control [23] might make it suitable for patients with a good long-term prognosis. This study supports such decision-making, suggesting that the NRF model may be a useful tool in determining the suitability of SBRT.

As a study limitation, we performed explorative sub-analyses of a single-centre retrospective study (the BM group) and a three-centre prospective observational study (the MPT group). Our results should be tested prospectively, preferably in multicentre settings.

Conclusions

The NRF model predicted survival moderately well in the BM and MPT groups, which was supported by distinct survival differences between the groups in the NRF model classification and evaluations of discrimination performance by the C-index and time-dependent AUROC.