Research | Open | Published:
Delta-radiomics signature predicts treatment outcomes after preoperative chemoradiotherapy and surgery in rectal cancer
Radiation Oncologyvolume 14, Article number: 43 (2019)
To develop and compare delta-radiomics signatures from 2- (2D) and 3-dimensional (3D) features that predict treatment outcomes following preoperative chemoradiotherapy (CCRT) and surgery for locally advanced rectal cancer.
In total, 101 patients (training cohort, n = 67; validation cohort, n = 34) with locally advanced rectal adenocarcinoma between 2008 and 2015 were included. We extracted 55 features from T2-weighted magnetic resonance imaging (MRI) scans. Delta-radiomics feature was defined as the difference in radiomics feature before and after CCRT. Signatures were developed to predict local recurrence (LR), distant metastasis (DM), and disease-free survival (DFS) from 2D and 3D features. The least absolute shrinkage and selection operator regression was used to select features and build signatures. The delta-radiomics signatures and clinical factors were integrated into Cox regression analysis to determine if the signatures were independent prognostic factors.
The radiomics signatures for LR, DM, and DFS were developed and validated using both 2D and 3D features. Outcomes were significantly different in the low- and high-risk patients dichotomized by optimal cutoff in both the training and validation cohorts. In multivariate analysis, the signatures were independent prognostic factors even when considering the clinical parameters. There were no significant differences in C-index from 2D vs. 3D signatures.
This is the first study to develop delta-radiomics signatures for rectal cancer. The signatures successfully predicted the outcomes and were independent prognostic factors. External validation is warranted to ensure their performance.
After the landmark randomized trial , preoperative chemoradiotherapy (CCRT) followed by total mesorectal excision (TME) has been a standard treatment strategy for locoregionally advanced rectal cancer. However, efforts have been continuously made to promote risk-adaptive therapy. One such approach is local excision [2, 3] or even observation [4, 5], rather than TME, in good responders following CCRT to reduce the risk of impaired quality of life. Alternatively, adding chemotherapeutic agents or intensifying radiation doses may be attempted in patients with poor response or prognosis [6,7,8]. These strategies can be implemented with the help of treatment outcome predictors; however, there are still no tools explicitly available for this purpose.
Radiomics provide image features associated with clinical characteristics or outcomes that are extracted from medical images. Numerous studies have been conducted on various cancer types, including lung cancer , glioma , and head and neck cancer  as well as proposed radiomics-based predictors of good performance. Radiomics models for rectal cancer have recently been developed using computed tomography (CT), magnetic resonance imaging (MRI), and positron emission tomography (PET) [12,13,14,15,16,17,18]. Some researchers have built radiomics models to predict pathologic response to CCRT [12, 13, 17, 18]. Meng et al.  reported a radiomics signature to predict disease-free survival (DFS) using pretreatment MRI in which the models could predict the treatment response or prognosis with acceptable predictability. Nonetheless, clinical response to treatment is another important indicator and may improve the performance of the models. Although treatment response is an important prognostic factor, it cannot describe the entire details of response. In this context, some researchers have examined delta-radiomic features, which are the differences in radiomic features before and after treatment. Delta-radiomics deals with serial changes in images, which is one of the major parts of radiologic studies. Delta-radiomics features have been reported to be associated with treatment response or outcome [19, 20]; however, reports on delta-radiomics in rectal cancer are less .
In this paper, we have focused on applying delta-radiomics features extracted from T2-weighted MRI to build prediction signatures for treatment outcomes and compared the performances of 2- (2D) and 3-dimensional (3D) features.
This retrospective study was approved by the institutional review board of our hospital; the requirement of informed consent was waived. The protocol was compliant with the Health Insurance Portability and Accountability Act. We retrospectively enrolled patients with locally advanced (cT3–4 and/or cN1–2) biopsy-proven rectal adenocarcinoma treated at our institution with preoperative CCRT and TME between 2008 and 2015. Patients with the following were excluded: distant metastasis (DM) at the time of diagnosis, MRI with poor quality (e.g., artifact), or slice spacing of MRI not 4 mm (to minimize the influence of different voxel sizes). Included patients were randomly allocated to the training or validation cohort in a 2:1 ratio.
We examined local recurrence (LR), DM, and DFS. LR and DM were defined as recurrences inside and outside the true pelvis, respectively. DFS was calculated as time from beginning of preoperative CCRT to death from any cause or recurrence.
An MRI scan was obtained for each patient before preoperative CCRT (MRI-before) and before TME after completion of CCRT (MRI-after). MRI-after was acquired at 72 days (median; interquartile range, 70–78) after the start of CCRT. MRI was performed using 1 .5T Gyroscan Intera, 3 T Achieva, or 3 T Ingenia MR scanners (Philips Medical Systems, Best, Netherlands). The protocol included T2-weighted sequences using the following parameters: repetition time, 2424–8296 ms; echo time, 92–120 ms; flip angle, 90°; slice thickness, 3 mm; slice spacing, 4 mm; matrix, 512 × 512–576 × 576.
Each region of interest (ROI) was segmented on all T2-weighted axial with reference to diffusion-weighted imaging (DWI) sequences. On MRI-before, the ROI was delineated on the tumor with an area of low to intermediate signal intensity on T2-weighted images, excluding the intestinal lumen. The ROI on MRI-after was defined as residual tumor and/or rectal tissues with abnormal signal intensity on T2-weighted images where tumor preexisted . Bladder urine of approximately 1-cm3 sphere volume was drawn to obtain average pixel value of bladder urine which was used for normalization. Segmentation of all patients was performed manually using the Eclipse system (Varian Medical Systems, Palo Alto, CA, USA) by a radiation oncologist with 12-year experience in gastro-intestinal tumor. Representative examples of tumor segmentations are demonstrated on Fig. 1.
Image preprocessing and feature extraction
Image preprocessing and feature extraction were performed using in-house MATLAB R2017b software (MathWorks, Natick, MA, USA). For preprocessing, Collewet normalization algorithm  was used to reduce the differences between image acquisition protocols. All pixel values were normalized to average intensity of bladder urine  to improve image reproducibility. Pixel values were quantized into 64 levels with bladder urine signal intensity corresponding to the highest level. The 3-dimensional ROIs were isotropically resampled to 1 × 1 × 1-mm3 voxels. For 2-dimensional analysis, the ROI on the axial slice with the largest area was selected and resampled to 1 × 1-mm2 pixels.
Within each ROI, (a) volume (or area in 2-dimensional analysis), (b) 8 first-order features, (c) 15 texture features from gray level co-occurrence matrix, (d) 13 texture features from gray level run length matrix, (e) 13 texture features from gray level size zone matrix, and (f) 5 texture features from neighbor gray tone difference matrix were extracted. The details and list of the extracted features are described in Additional file 1: Appendix A and Table B1, respectively. The delta-radiomics feature was defined as the difference between features on MRI-before and MRI-after and calculated as follows:
Feature selection and statistical analysis
Robustness of each feature was evaluated by generating translated ROIs and calculating their features. The method was modified from the stability test introduced by Bologna et al.  Eight translated ROIs representing inter-observer variability were generated by translating ROI by ±1 mm in lateral and/or ± 1 mm in vertical directions; 0 mm in both directions yields the original ROI and is thus excluded from the robustness test. After extracting the features from the original ROI and 8 translated ROIs, intraclass correlation coefficient (ICC) values were calculated for each feature. Features with ICC > 0.9 in both 3D and 2D extraction in MRI-before and MRI-after were considered robust and selected. This process substituted the comparison of features derived from multiple observers.
The least absolute shrinkage and selection operator (LASSO) method was used to select core features and to develop score-based signatures in the training cohort. The final value of λ, a tuning parameter, was determined by 10-fold cross-validation, which gave minimum cross-validation error. A radiomics score (Rad score) was generated by linearly combining the selected core features and their respective coefficients. Consequently, the optimal cutoff of Rad score, making the greatest difference in outcome between the two groups divided by the cutoff, was determined.
The differences in clinical and treatment parameters between the training and validation cohort were evaluated using the Student’s t-test or chi-squared test, as appropriate. Survival outcomes were compared between these cohorts using the log-rank test. Univariate and multivariate analyses of clinical factors and radiomics scores were performed using the Cox proportional hazards model. Performance of the models were evaluated with area under the ROC curve (AUC) and Hosmer-Lemeshow goodness-of-fit test, and the relationship between radiomics scores was quantified using Pearson correlation coefficient and variance inflation factor (VIF). Variables with a significant association were integrated into multivariate analysis; the association was considered significant when p < 0.05. R software version 3.5.0 was used to perform all statistical analyses (http://www.r-project.org).
Patients and treatment characteristics
A total of 101 patients were included in the analysis, with 67 in the training and 34 in the validation cohort. The median follow-up duration was 49.7 months (range, 9.3–99.4). Clinical characteristics of the two cohorts are summarized in Table 1. There was no significant difference between the two cohorts.
All patients received radiotherapy doses of 50.4 Gy in 28 fractions to the primary tumor and regional lymphatics with the 2D (n = 18) or 3D (n = 83) technique. Additionally, 5-fluorouracil (n = 19) or capecitabine (n = 82) was administered concurrently with radiotherapy. TME-based surgery was performed at a median 48 days (range, 28–90) after the end of CCRT. Adjuvant chemotherapy was administered in 91 patients (90.1%); fluorouracil and leucovorin in 20, capecitabine in 36, uracil and tegafur in 7, and FOLFOX in 28 patients, respectively.
Development and validation of Delta-Radiomics signature
The delta-radiomics signature was developed using the remaining 22 features after the robustness test. The 22 robust features with ICC > 0.9 are listed in Additional file 1: Appendix Table B2. LASSO Cox regression analysis was conducted in the training cohort to select radiomics features with non-zero coefficients. Rad scores predicting LR, DM, and DFS are as follows:
Optimal cutoff values of Rad scores were determined and used to divide the cohort into high- and low-risk groups, and a higher score was correlated to a higher risk. In the training cohort, all Rad scores were significantly associated with respective outcomes (all p < 0.05, log-rank test). The prognostic performance of all Rad scores was validated in a randomly selected cohort. All Rad scores significantly stratified the risk in the 2 groups (all p < 0.05, log-rank test). The Kaplan-Meier survival curves for LR, DM, and DFS according to 3D and 2D Radscores are demonstrated on Fig. 2 and Fig.3, respectively. The AUC and Hosmer-Lemeshow Chi-square values suggest that the predictability of the signatures is acceptable and are detailed on Additional file 1: Appendix Table C.
Integration with clinical features
Univariate analysis followed by multivariate analysis was performed to verify radiomics scores as independent prognostic factors for the respective endpoints in the combined cohort. Detailed results of the analyses are presented in Additional file 1: Appendix Table D1–3. All radiomics scores, regardless of 2D or 3D signature, were significantly associated with the corresponding outcomes on multivariate analysis.
Comparisons of 3D and 2D signatures
Correlations between 3D and 2D Rad scores were investigated in the 101 patients. Pearson correlation coefficients of 3D and 2D Rad scores for LR, DM, and DFS were 0.840 (95% CI = 0.771–0.889, p < 0.0001, VIF = 3.39), 0.641 (95% CI = 0.510–0.743, p < 0.0001, VIF = 1.70), and 0.665 (95% CI = 0.540–0.761, p < 0.0001, VIF = 1.79), respectively. The scatterplot of Rad scores and their correlations are shown in Fig. 4.
The value of C-index was calculated and compared for each Rad score. In terms of all outcomes, the C-indices between the 2D and 3D Rad scores, both as continuous and binary variables divided by cutoff, were not significant (Table 2).
In the present study, we developed and validated the prognostic role of delta-radiomics signatures in locally advanced rectal cancer treated with preoperative CCRT and surgery. Furthermore, we compared the performance of 2D and 3D radiomics features. To our knowledge, this is the first publication to incorporate delta-radiomics features to predict recurrences in rectal cancer.
A good response to preoperative CCRT is consistently associated with improved treatment outcomes of rectal cancer [25, 26]. Although pretreatment radiomics features predict pathologic tumor response, they do not contain all of the information regarding response. Images following preoperative CCRT can reveal indicators of tumor response. Tumor regression grade according to post-treatment T2-weighted MRI is correlated with pathologic tumor regression grade [27, 28]. Thus, we hypothesized that delta-radiomics features on T2-weighted MRI have prognostic power. A recent study showed correlation between delta-radiomics features and clinical response in rectal cancer . Analyzing 16 patients, the study provided an evidence of clinical significance of delta-radiomics features. Since there are no studies with large patients concerning delta-radiomics, however, we strictly limited the number of features included in the investigation. We only included the features that are widely used in radiomics studies, and the image preprocessing step was onefold. Furthermore, we included the features with ICC > 0.9 in all 3D and 2D analyses, leaving 22 features for LASSO regression.
The developed Rad scores were successfully validated in a randomly selected cohort. Some radiomics features may be closely related to clinical factors, thus the signatures should be independent prognostic factors in multivariate analysis to be valuable. Rad scores along with clinical factors consistently reported to be prognostic were independently associated with treatment outcomes. Remarkably, post-treatment pathologic characteristics such as pathologic stage and tumor regression grade are incorporated in the analysis. The results suggest that delta-radiomics features may contain more information than microscopic findings, e.g., tumor genotype or microenvironment.
The developed signatures are believed to be useful in daily practice. There is no consensus regarding the use of adjuvant chemotherapy after preoperative CCRT and surgery. Subgroups that may benefit from adjuvant chemotherapy have been reported [29,30,31]. It is generally hypothesized that patients at high risk of recurrence, usually those with distant metastasis, benefit from adjuvant chemotherapy. Most of the patients (90.1%) in our study received adjuvant chemotherapy, suggesting that, for patients with low Rad scores, adjuvant chemotherapy can be omitted.
We compared 2D and 3D delta-radiomics signatures in predicting outcomes. One major advantage of the radiomics approach is that it can represent properties of the whole tumor. However, in the case of 2D features, only part of the tumor is segmented. Therefore, there are concerns regarding the power of 2D radiomics features. The main advantage of 2D radiomics features is the convenience in investigation and application; investigators or users only need to delineate the tumor on 1 representative slice. In addition, 2D features, particularly slice thickness and spacing, may be less dependent on the image-acquiring protocol. Several authors have utilized 2D radiomics features of rectal cancer and reported their predictive power [32,33,34]. As rectal cancer usually grows along the wall and has an irregular shape, the segmented whole tumor may not represent its actual shape ; hence, 2D radiomics features need to be further studied. Regarding all outcomes, both 2D and 3D Rad scores were independent prognostic factors on multivariate analysis. By comparing C-index, 2D and 3D Rad scores were not statistically different in prognostic power, possibly because of the high correlation between scores. Hence, our data suggest that 2D delta-radiomics features can be investigated as a good surrogate for 3D features of rectal cancer.
One of the drawbacks of our study is the exclusion of functional images such as DWI or other modalities such as CT and PET. Several studies have reported the correlation between DWI parameters and response or outcomes after CCRT [36,37,38]. Recent work by Giannini and colleagues demonstrated the role of PET-derived radiomics features in predicting treatment response . We believe that the performance of delta-radiomics signature would improve with the incorporation of other sequences or modalities. We hope that radiomics features from various images can be used in subsequent delta-radiomics investigations. Another limitation of the study is the different parameters of the analyzed T2-weighted images. We normalized the pixel intensity using Collewet’s method and urine intensity and resampled the voxels or pixels into isometric cubes or squares. Nonetheless, the preprocessing steps cannot fully compensate for the differences. However, we believe that the radiomics signature should be applicable to various image protocols for widespread clinical use. In that context, the wide applicability of our signatures needs to be tested in MRIs from other institutions.
In conclusion, we developed radiomics scores to predict treatment outcomes after preoperative CCRT and surgery. The results support further investigation of delta-radiomics features in rectal cancer. The 2D and 3D delta-radiomics features were similarly informative. External validation of our signatures is necessary to ensure their performance.
Intraclass correlation coefficient
Least absolute shrinkage and selection operator
Region of interest
Total mesorectal excision
Sauer R, Liersch T, Merkel S, et al. Preoperative versus postoperative chemoradiotherapy for locally advanced rectal cancer: results of the German CAO/ARO/AIO-94 randomized phase III trial after a median follow-up of 11 years. J Clin Oncol. 2012;30:1926–33.
Stipa F, Picchio M, Burza A, et al. Long-term outcome of local excision after preoperative chemoradiation for ypT0 rectal cancer. Dis Colon Rectum. 2014;57:1245–52.
Yu CS, Yun HR, Shin EJ, et al. Local excision after neoadjuvant chemoradiation therapy in advanced rectal cancer: a national multicenter analysis. Am J Surg. 2013;206:482–7.
Habr-Gama A, Perez RO, Nadalin W, et al. Operative versus nonoperative treatment for stage 0 distal rectal cancer following chemoradiation therapy: long-term results. Ann Surg. 2004;240:711–7.
Renehan AG, Malcomson L, Emsley R, et al. Watch-and-wait approach versus surgical resection after chemoradiotherapy for patients with rectal cancer (the OnCoRe project): a propensity-score matched cohort analysis. Lancet Oncol. 2016;17:174–83.
Gérard JP, Azria D, Gourgou-Bourgade S, et al. Comparison of two neoadjuvant chemoradiotherapy regimens for locally advanced rectal cancer: results of the phase III trial ACCORD 12/0405-Prodige 2. J Clin Oncol. 2010;28:1638–44.
Habr-Gama A, Perez RO, São Julião GP, et al. Consolidation chemotherapy during neoadjuvant chemoradiation (CRT) for distal rectal cancer leads to sustained decrease in tumor metabolism when compared to standard CRT regimen. Radiat Oncol. 2016;11:24.
Breugom AJ, Swets M, Bosset JF, et al. Adjuvant chemotherapy after preoperative (chemo) radiotherapy and surgery for patients with rectal cancer: a systematic review and meta-analysis of individual patient data. Lancet Oncol. 2015;16:200–7.
Sun W, Jiang M, Dang J, et al. Effect of machine learning methods on predicting NSCLC overall survival time based on Radiomics analysis. Radiat Oncol. 2018;13(1):197.
Li Q, Bai H, Chen Y, et al. A fully-automatic multiparametric Radiomics model: towards reproducible and prognostic imaging signature for prediction of overall survival in glioblastoma Multiforme. Sci Rep. 2017;7:14331.
Zhang B, Tian J, Dong D, et al. Radiomics features of multiparametric MRI as novel prognostic factors in advanced nasopharyngeal carcinoma. Clin Cancer Res. 2017;23:4259–69.
Nie K, Shi L, Chen Q, et al. Rectal Cancer: assessment of neoadjuvant Chemoradiation outcome based on Radiomics of multiparametric MRI. Clin Cancer Res. 2016;22:5256–64.
Liu Z, Zhang XY, Shi YJ, et al. Radiomics analysis for evaluation of pathological complete response to neoadjuvant Chemoradiotherapy in locally advanced rectal Cancer. Clin Cancer Res. 2017;23:7253–62.
Meng Y, Zhang Y, Dong D, et al. Novel radiomic signature as a prognostic biomarker for locally advanced rectal cancer. J Magn Reson Imaging. 2018. https://doi.org/10.1002/jmri.25968 [Epub ahead of print].
Sun Y, Hu P, Wang J, et al. Radiomic features of pretreatment MRI could identify T stage in patients with rectal cancer: preliminary findings. J Magn Reson Imaging. 2018. https://doi.org/10.1002/jmri.25969 [Epub ahead of print].
Cusumano D, Dinapoli N, Boldrini L, et al. Fractal-based radiomic approach to predict complete pathological response after chemo-radiotherapy in rectal cancer. Radiol Med. 2018;123:286–95.
Dinapoli N, Barbaro B, Gatta R, et al. Magnetic resonance, vendor-independent, intensity histogram analysis predicting pathologic complete response after Radiochemotherapy of rectal Cancer. Int J Radiat Oncol Biol Phys. 2018;102:765–74.
Giannini V, Mazzetti S, Bertotto I, et al. Predicting locally advanced rectal cancer response to neoadjuvant therapy with 18F-FDG PET and MRI radiomics features [published online ahead of print January 13, 2019]. Eur J Nucl Med Mol Imaging. 2019. https://doi.org/10.1007/s00259-018-4250-6 [Epub ahead of print].
Fave X, Zhang L, Yang J, et al. Delta-radiomics features for the prediction of patient outcomes in non-small cell lung cancer. Sci Rep. 2017;7:588.
Goh V, Ganeshan B, Nathan P, et al. Assessment of response to tyrosine kinase inhibitors in metastatic renal cell cancer: CT texture as a predictive biomarker. Radiology. 2011;261:165–71.
Boldrini L, Cusumano D, Chiloiro G, et al. Delta radiomics for rectal cancer response prediction with hybrid 0.35 T magnetic resonance-guided radiotherapy (MRgRT): a hypothesis-generating study for an innovative personalized medicine approach. Radiol Med. 2019;124:145–53.
Collewet G, Strzelecki M, Mariette F. Influence of MRI acquisition protocols and image intensity normalization methods on texture classification. Magn Reson Imaging. 2004;22:81–91.
Johnston E, Punwani S. Can we improve the reproducibility of quantitative multiparametric prostate MR imaging metrics? Radiology. 2016;281:652–3.
Bologna M, Corino VDA, Montin E, et al. Assessment of stability and discrimination capacity of Radiomic features on apparent diffusion coefficient images. J Digit Imaging. 2018. https://doi.org/10.1007/s10278-018-0092-9 [Epub ahead of print].
Maas M, Nelemans PJ, Valentini V, et al. Long-term outcome in patients with a pathological complete response after chemoradiation for rectal cancer: a pooled analysis of individual patient data. Lancet Oncol. 2010;11:835–44.
Agarwal A, Chang GJ, Hu CY, et al. Quantified pathologic response assessed as residual tumor burden is a predictor of recurrence-free survival in patients with rectal cancer who undergo resection after neoadjuvant chemoradiotherapy. Cancer. 2013;119:4231–41.
Bhoday J, Smith F, Siddiqui MR, et al. Magnetic resonance tumor regression grade and residual mucosal abnormality as predictors for pathological complete response in rectal Cancer Postneoadjuvant Chemoradiotherapy. Dis Colon Rectum. 2016;59:925–33.
Patel UB, Brown G, Rutten H, et al. Comparison of magnetic resonance imaging and histopathological response to chemoradiotherapy in locally advanced rectal cancer. Ann Surg Oncol. 2012;19:2842–52.
Song C, Chung JH, Kang SB, et al. Impact of tumor regression grade as a major prognostic factor in locally advanced rectal cancer after neoadjuvant chemoradiotherapy: a proposal for a modified staging system. Cancers (Basel). 2018;10:e319.
Maas M, Nelemans PJ, Valentini V, et al. Adjuvant chemotherapy in rectal cancer: defining subgroups who may benefit after neoadjuvant chemoradiation and resection: a pooled analysis of 3,313 patients. Int J Cancer. 2015;137:212–20.
van Erning FN, Rutten HJ, van den Berg HA, et al. Effect of adjuvant chemotherapy on recurrence-free survival varies by neo-adjuvant treatment in patients with stage III rectal cancer. Eur J Surg Oncol. 2015;41:1630–5.
Chee CG, Kim YH, Lee KH, et al. CT texture analysis in patients with locally advanced rectal cancer treated with neoadjuvant chemoradiotherapy: a potential imaging biomarker for treatment response and prognosis. PLoS One. 2017;12:e0182883.
Jalil O, Afaq A, Ganeshan B, et al. Magnetic resonance based texture parameters as potential imaging biomarkers for predicting long-term survival in locally advanced rectal cancer treated by chemoradiotherapy. Color Dis. 2017;19:349–62.
De Cecco CN, Ciolina M, Caruso D, et al. Performance of diffusion-weighted imaging, perfusion imaging, and texture analysis in predicting tumoral response to neoadjuvant chemoradiotherapy in rectal cancer patients studied with 3T MR: initial experience. Abdom Radiol (NY). 2016;41:1728–35.
Liu L, Liu Y, Xu L, et al. Application of texture analysis based on apparent diffusion coefficient maps in discriminating different stages of rectal cancer. J Magn Reson Imaging. 2017;45:1798–808.
Birlik B, Obuz F, Elibol FD, et al. Diffusion-weighted MRI and MR- volumetry--in the evaluation of tumor response after preoperative chemoradiotherapy in patients with locally advanced rectal cancer. Magn Reson Imaging. 2015;33:201–12.
Sun Y, Tong T, Cai S, et al. Apparent diffusion coefficient (ADC) value: a potential imaging biomarker that reflects the biological features of rectal cancer. PLoS One. 2014;9:e109371.
Lambregts DM, Rao SX, Sassen S, et al. MRI and diffusion-weighted MRI Volumetry for identification of complete tumor responders after preoperative Chemoradiotherapy in patients with rectal Cancer: a bi-institutional validation study. Ann Surg. 2015;262:1034–9.
This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (Grant number: NRF-2017R1D1A1B03033892).
Availability of data and materials
The datasets generated and analyzed during the current study are not publicly available due to the Personal Information Protection Act but are available from the corresponding author on reasonable request.
Ethics approval and consent to participate
This retrospective patient study was approved by Institutional Review Board of the Seoul National University Bundang Hospital, and the requirement for written informed consent was waived. This study complies with the standards of the Declaration of Helsinki and current ethics guidelines.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix A. Extracted radiomics features. (DOCX 37 kb)