FDG uptake heterogeneity in FIGO IIb cervical carcinoma does not predict pelvic lymph node involvement
© Brooks and Grigsby; licensee BioMed Central Ltd. 2013
Received: 25 September 2013
Accepted: 13 December 2013
Published: 23 December 2013
Many types of cancer are located and assessed via positron emission tomography (PET) using the 18F-fluorodeoxyglucose (FDG) radiotracer of glucose uptake. There is rapidly increasing interest in exploiting the intra-tumor heterogeneity observed in these FDG-PET images as an indicator of disease outcome. If this image heterogeneity is of genuine prognostic value, then it either correlates to known prognostic factors, such as tumor stage, or it indicates some as yet unknown tumor quality. Therefore, the first step in demonstrating the clinical usefulness of image heterogeneity is to explore the dependence of image heterogeneity metrics upon established prognostic indicators and other clinically interesting factors. If it is shown that image heterogeneity is merely a surrogate for other important tumor properties or variations in patient populations, then the theoretical value of quantified biological heterogeneity may not yet translate into the clinic given current imaging technology.
We explore the relation between pelvic lymph node status at diagnosis and the visually evident uptake heterogeneity often observed in 18F-fluorodeoxyglucose positron emission tomography (FDG-PET) images of cervical carcinomas.
We retrospectively studied the FDG-PET images of 47 node negative and 38 node positive patients, each having FIGO stage IIb tumors with squamous cell histology. Imaged tumors were segmented using 40% of the maximum tumor uptake as the tumor-defining threshold and then converted into sets of three-dimensional coordinates. We employed the sphericity, extent, Shannon entropy (S) and the accrued deviation from smoothest gradients (ζ) as image heterogeneity metrics. We analyze these metrics within tumor volume strata via: the Kolmogorov-Smirnov test, principal component analysis and contingency tables.
We found no statistically significant difference between the positive and negative lymph node groups for any one metric or plausible combinations thereof. Additionally, we observed that S is strongly dependent upon tumor volume and that ζ moderately correlates with mean FDG uptake.
FDG uptake heterogeneity did not indicate patients with differing prognoses. Apparent heterogeneity differences between clinical groups may be an artifact arising from either the dependence of some image metrics upon other factors such as tumor volume or upon the underlying variations in the patient populations compared.
There is evidence that pelvic lymph node status is an important indicator of both disease free and overall survival in patients diagnosed with cervical carcinoma [1, 2]. There is also a growing belief that the uptake heterogeneity observed within tumors assayed via FDG-PET may be of prognostic value [3, 4]. A correlation between increased FDG uptake heterogeneity and worse prognosis has been reported for human sarcomas [5, 6], non-small cell lung cancer , esophageal cancer  and cervical cancer [9, 10]. Although the relationship is not well-understood, it seems that tumor state (e.g., quiescence or rapid proliferation) likely is strongly linked to tumor metabolism . Because FDG uptake is some measure of metabolic activity within the tumor, this connection to tumor state offers one plausible means for uptake heterogeneity to indicate disease outcome. We therefore hypothesize that increased intra-tumor FDG uptake heterogeneity may correlate with lymph node involvement. If such a link were established, it will provide a new avenue of studying why only some of cervical carcinoma patients exhibit pelvic lymph node involvement and perhaps even provide insight as to why that status so profoundly affects prognosis.
The image heterogeneity observed may be thought of as comprising variations in intensity: distribution, arrangement and shape. The grayscale intensity distribution results from some combination of underlying biology, scanner noise and the partial volume effect. These are the shades available to create image detail; less shades means that less variation can be conveyed. The spatial arrangement of the intensities also conveys variation. For example, intuitively, the smooth gradation of the brightest image intensities to the dimmest is less varied than those same intensities randomly juxtaposed. Overall shape is distinct from arrangement because shape defines a priori the bounds of where the intensities may be arranged. For example, a tumor growing near a physical barrier may be unable to achieve the same shape a tumor protruding into a void might achieve. These considerations are crucial when determining if two distinct intensity samples (i.e., tumor regions) differ significantly.
There have been several previous attempts at quantifying uptake heterogeneity seen in FDG-PET images [5–10, 14–16]. While we do not exhaustively review them here, we recapitulate some apposite concerns. One study analyzed almost twenty higher-order texture metrics and found only a few to be statistically reproducible on FDG-PET images . This is consistent with another study which showed that dozens of texture metrics applied to FDG-PET images of cervical carcinomas were not useful indicators of prognosis . In general, texture metrics often are inter-related and thus including additional metrics might not include additional information. For example, the image entropy is (qualitatively) inversely proportional to the image energy . In practice, similar dependences between other metrics may be may not be at all intuitive. For example, it was argued that the slope of the SUV-volume histogram was an intra-tumor heterogeneity metric with prognostic value , however, later that same metric was shown to be a surrogate for tumor volume . An analogous dependence upon tumor volume also was illustrated with virtual tumor images for the local image entropy . Still another metric, the area under the cumulative SUV-volume histogram, also was presented as a prognostic heterogeneity measure [7, 14]. It was argued later that that metric is equivocal in the sense that it is possible for unequal heterogeneity scenarios to achieve identical metric values while unequal metric values can result from identical heterogeneity scenarios [21, 22]. Quantification of image heterogeneity is a difficult problem in general and has only recently been addressed in the specific context of predicting disease outcome from FDG-PET images. Therefore, challenges and refinements to previously proposed metrics (including those we employ) are to be expected as the mathematical sophistication of the problem is revealed via further study.
We analyze our image data via a spatial metric which has been demonstrated to be both independent of tumor volume and consistent with visual ranking of FDG-PET images by human experts . While this metric does provide some measure of the variation in intensity arrangement within pre-established tumor regions, it also is sensitive the overall region shape . We employ the sphericity and extent as distinct, volume-independent shape metrics which may provide complementary information about the contribution of shape to perceived heterogeneity. The sphericity and extent are metrics which can distinguish rounder, more compact regions from highly asymmetrical, porous regions . We employ the well-established Shannon entropy [18, 24] as the measure of informational content within the individual distributions of grayscale intensities. We note however, that the Shannon entropy is predicted to depend non-linearly upon tumor volume and that the exact functional form of that dependence is influenced by both the intensity histogram bin size and image segmentation threshold employed .
Materials and methods
This is a retrospective study of 85 patients with newly diagnosed cervical cancer who underwent FDG-PET or FDG-PET/CT at Washington University in St. Louis between July 2003 and March 2012. This study was approved by the Washington University Human Research Protection Office. All patients were evaluated by history and physical examination, examination under anesthesia, and FDG-PET or integrated PET/CT before initiating treatment. Patients were staged clinically, according to FIGO staging (AJCC 2002, 6th edition). The selection criteria for inclusion into this study were FIGO clinical stage IIb tumors and squamous cell histology. The patients ranged in age from 27 y to 85 y with a median of 49 y. The maximum SUV of the tumors analyzed ranged from 4 to 56 and approximately followed a log-normal distribution with median of 13 and variance of 74.
Identification of primary tumors
For patients (n = 58) examined between July 2003 and June 2008 diagnostic imaging scanning was performed via Siemens Biograph 2 (Munich, Germany) while for those examined (n = 27) between October 2008 and March 2012 were scanned via Siemens Biograph 40 (Munich, Germany). In both cases, images were reconstructed via ordered subset expectation maximization with 8 sets; 2 iterations for the former and 4 iterations for the latter. In both cases, a Gaussian smoothing filter was applied post-reconstruction with 5.3 mm and 4 mm full width at half maximum, respectively. We note that the 9 most recent Biograph 40 image sets underwent an additional point-spread function/time-of-flight correction. The primary tumor evident in each FDG-PET image set was identified and segmented specifically for the present study by a veteran oncologist using MIM version 5.6.3 (MIM Software Inc., Cleveland, OH). Within the manually approximated tumor region, any image voxel with SUV greater than 40% of the maximum region SUV was considered to be part of the tumor . The oncologist then made slight manual adjustments to the ROI to remove any obvious non-tumor pixels such as those comprising bladder or bowel regions. For each patient, these data were exported as a set of (x,y,z) coordinates, each with a single 15-bit grayscale image intensity corresponding to radioactivity density in Bq/mL.
Assessment of FDG uptake heterogeneity
The extent is defined as the ratio of the net object area (or volume) to a bounding area (or volume) . We note one difference from the usual definition of the extent. Instead of a bounding box, we employed a bounding sphere with diameter matching the maximal diameter of the tumor object and centered at the geometric center of the tumor object. For example, a largely hollow but basically round tumor will have a low extent (i.e., few pixels in the bounding sphere) but a high sphericity. Thus, ψ and ξ together form a set of complementary metrics which together feasibly discern heterogeneous from homogeneous tumor geometry. Lastly, we compute the accrued deviation from smoothest gradients (ζ) . Automated ranking of tumor image via increasing ζ has been shown to be consistent with rankings done visually by human experts. As described in Ref. , we computed ζ for our three-dimensional intensity data by using tri-linear interpolation to approximate the 15-bit intensity value whenever a fractional (x,y,z) coordinate was required (recall that only integer coordinates are stored within single FDG-PET images).
Potential differences between the distribution of a heterogeneity statistic for distinct patient subgroups defined by pelvic lymph node status were assessed via the two-sample Kolmogorov-Smirnov test. This test of empirical cumulative distribution functions (CDFs) was chosen because it: does not require modeling of the distributions compared, can compare continuous data sets of unequal size and/or variance and is sensitive to more than just differences in distribution location . The subgroups compared are those with no indication of pelvic lymph node involvement (LN-) to those with (LN+). For our population comprising 47 LN- and 38 LN+ patients, a maximum difference (D) between the CDFs greater than 0.286 is significant at the 95% level . In cases where cross-group ties amongst ranks occur, the p-value given for D is only an upper-bound of the true p-value . Specific cases of additional data stratification with differing significance criteria are detailed in the Results section, as necessary. Where appropriate, we employ Kendall’s τ as a measure of variable association and test other potential variable dependencies via χ2-test applied to contingency tables. In cases where histograms were constructed, the Freedman-Diaconis bin width was employed. All statistical analyses were performed using R version 2.15.2 (R Foundation for Statistical Computing, Vienna, Austria).
Within each volume stratum, the Kolmogorov-Smirnov test was used to discern potential differences between lymph node status groups for each of the heterogeneity metrics
Volume (in cc)
Shannon entropy ( S)
Sphericity ( ψ)
Spherical extent ( ξ)
4.0 ≤ V < 22.4
D = 0.227; p = 0.886
D = 0.355; p = 0.390
D = 0.200; p ≤ 0.985
22.4 ≤ V < 32.8
D = 0.548; p ≤ 0.102
D = 0.471; p ≤ 0.222
D = 0.269; p ≤ 0.865
32.8 ≤ V < 52.8
D = 0.173; p = 0.987
D = 0.500; p ≤ 0.168
D = 0.539; p = 0.078
52.8 ≤ V < 185.1
D = 0.364; p = 0.479
D = 0.182; p ≤ 0.993
D = 0.273; p ≤ 0.808
We now note some important aspects of our uptake heterogeneity analysis. First, our patient population was chosen in order to reduce variation in characteristics which affect FDG-PET image heterogeneity. For example, one might expect image differences between squamous and non-squamous histology precisely because of the differences in tissue type and/or density. Additionally, because FIGO stage is known to have prognostic value, but the relation between stage and FDG-PET uptake is not well understood, stage could affect the image data in unknown ways. Our patients are all of the same FIGO stage and histology. This increases the believability that the statistical distribution of a given uptake heterogeneity metric comes from intra-tumor variability and not from comparison of unlike tumors.
Second, we specifically consider the relation between uptake heterogeneity metrics and tumor volume. The plot of informational entropy versus volume given in Figure 3 is but one ready example of how heterogeneity metrics should be expected to depend upon tumor volume [13, 20]. Consider that the tumor volume is nothing more than the size of the sample chosen from the distribution of FDG-PET intensities possible for a given tumor type. Decreasing the sample size (volume) must decrease the believability that the underlying distribution has been represented adequately and therefore must also decrease the reliability of comparisons to much larger samples. In our analysis, we employed metrics proven to be volume independent or, in the case of the informational entropy, specifically stratified our patient data as to diminish the dependence upon volume. We may therefore be more certain that any observed uptake differences are not the result of the comparison of disparate volumes.
Third, we analyze the original FDG-PET image data unaltered via changes in image bit depth. This is crucial because any heterogeneity metric, by definition, must be sensitive to the image bit depth. Consider that visually evident differences in tumor heterogeneity might be described verbally as differences in image: detail, variation, texture, tone, contrast and/or grain. By reducing the number of grayscale shades available to render those qualities, the entire image is smoothed. That is, reducing bit depth always decreases heterogeneity. However, vast cross-patient differences in tumor shape, tumor size, orientation (relative to the scanner) and maximum FDG uptake imply that a bit depth reduction can affect each data set differently. Thus, while reducing image variation for individuals, population variation actually could be enhanced by bit depth reduction. For example, consider a case where two differently shaped and shaded tumors happen to yield similar heterogeneity metric values. Because bit depth reduction affects shade more than shape, one tumor can be smoothed significantly more than the other, thus yielding tumors with different metric values in the new bit depth. This is another way that the statistical distribution of uptake heterogeneity metric values can depend upon patient population more strongly than upon intra-image variation (which is what ostensibly is being measured).
Our main result is that none of the previously established uptake heterogeneity metrics we employed yielded statistically significant differences between patients with pelvic lymph node involvement and those without. In other words, we found FDG uptake heterogeneity does not predict membership to groups with differing prognoses. This is counter to previous claims that measurements of intra-tumor uptake heterogeneity generally are of prognostic value [5–10, 14]. We posit two main reasons those analyses yielded contradictory results. First, the precise uptake heterogeneity found in studies of other tumor types (such as, e.g., non-squamous small cell lung carcinomas) simply may not be applicable to, or even possible in, the cervical carcinomas we study. Second, and more generally, is the lack of variation in our patient population. In brief, when patient populations vary in ways which plausibly affect FDG-PET image heterogeneity metrics, differing heterogeneity was found; when this population variation was controlled, differing heterogeneity was not found. Indeed, it is possible that all apparent prognostic FDG uptake heterogeneity is a statistical effect arising from the partial volume attenuation of vastly differing tumor types, shapes and sizes.
To objectively isolate tumor from background, we employed the 40% of maximum threshold established for cervical carcinomas . This threshold is conservative in that it almost certainly misses some tumor at the periphery. The point of using a conservative threshold is so that one may be reasonably assured that the data analyzed actually correspond to intra-tumor regions. As the tumor-defining threshold is lowered, so too is the believability that the desired intra-tumor measurements aren’t skewed by inclusion of background pixels. If, instead, the threshold were raised significantly, it is unlikely that heterogeneity even could be measured when so few pixels of so few shades are given as input. In essence, a super-conservative threshold virtually guarantees homogeneity.
Another reasonable concern about our analysis is in our use of image data from two scanners. The sphericity and extent were computed using Euclidean distances in centimeters. While is true that the older scanner cannot give as precise an estimate of size as does the current one, it should not give an inaccurate estimate. Both the conversion of radioactivity to grayscale intensity and the histogram binning of those intensities were uniquely determined for each patient. The deviation from smoothest gradients is derived on a percent of size, percent of intensity scale a priori. For these reasons we expect that use of multiple scanners is not a significant source of data heterogeneity in our analysis.
One weakness of our analysis in the creation of the ζ n statistic. The original, unbounded heterogeneity metric ζ only weakly correlates with the normalized metric ζ n (τ = 0.040). Therefore while ζ is a metric proven to correlate with heterogeneity rankings done visually by human experts, ζ n is unproven. Although it is reasonable that ζ n can distinguish image sets, the exact interpretation of ζ n or its correspondence with particular visually perceived image properties is not established. We may therefore only say that a feasible measure of image variation (ζ n ) did not distinguish patient groups of differing lymph node status and that this lack of distinction is consistent with the results using proven metrics (S and ζ). Another potential weakness is in the use of the sphericity (ψ) and extent (ξ) to assess heterogeneity in tumor shape. In general, ψ measures the closeness to spherical shape and ξ measures the percent filling of that shape. Because neither situation is likely to occur for real tumors, use of ψ and ξ imposes an unrealistic shape constraint. This means that less variation is possible a priori and genuine differences in tumor shape are compressed into a smaller range of variable value. Also, these and similar metrics can exhibit non-linear jumps in value. When segmenting an image, some thresholding process is employed to discern tumor from background. This means, for example, that a ring-like cross-sectional shape could be opened to a horseshoe-like shape simply due to slight signal variations which do not surmount the intensity threshold. The result is that biologically similar tumors could suddenly become mathematically different when assessed by metrics strongly dependent upon the ratio of interior pixels to surface pixels. We visually inspected every image of our entire data set and found holes and/or disconnected tumor regions to occur in 218 of the 1,328 images analyzed. Given the strength of our other results, we do not feel that this particular source of error will change our overall conclusions, however, this is one more example of how the analysis of FDG-PET images is inherently more challenging than the commonality of the metrics employed belie.
We analyzed a patient population homogeneous in qualities which plausibly affect the image heterogeneity observed in FDG-PET assays of some cervical carcinomas. We found that established metrics of image heterogeneity do not indicate involvement of pelvic lymph nodes. Because pelvic lymph node involvement is a demonstrated prognostic indicator, our results seem at odds with studies which suggest intra-tumor FDG uptake heterogeneity has prognostic value. We argue that the potential conflict might be explained by the heterogenous nature of the patient populations in other studies influencing the statistical distribution of some image heterogeneity metrics.
We wish to formally thank Richard Laforest for his advice on the FDG-PET image acquisition process.
This work was supported by the National Institutes of Health under Grant 1R01-CA136931-01A2.
- Stehman FB, Bundy BN, DiSaia PJ, Keys HM, Larson JE, Fowler WC: Carcinoma of the cervix treated with radiation therapy: I: a multi-variate analysis of prognostic variables in the gynecologic oncology group. Cancer 1991,67(11):2776-2785. 10.1002/1097-0142(19910601)67:11<2776::AID-CNCR2820671111>3.0.CO;2-LView ArticlePubMedGoogle Scholar
- Kidd EA, Siegel BA, Dehdashti F, Rader JS, Mutch DG, Powell MA, et al.: Lymph node staging by positron emission tomography in cervical cancer: relationship to prognosis. J Clin Oncol 2010,28(12):2108-2113. 10.1200/JCO.2009.25.4151View ArticlePubMedGoogle Scholar
- Asselin M-C, O’Connor JPB, Boellaard R, Thacker NA, Jackson A: Quantifying heterogeneity in human tumours using MRI and PET. Eur J Cancer 2012,48(4):447-455. 10.1016/j.ejca.2011.12.025View ArticlePubMedGoogle Scholar
- Chicklore S, Goh V, Siddique M, Roy A, Marsden PK, Cook GJR: Quantifying tumour heterogeneity in 18 F-FDG PET/CT imaging by texture analysis. Eur J Nucl Med Mol Imaging 2013, 40: 133-140. 10.1007/s00259-012-2247-0View ArticlePubMedGoogle Scholar
- O’Sullivan F, Roy S, Eary J: A statistical measure of tissue heterogeneity with application to 3D PET sarcoma data. Biostatistics 2003,4(3):433-448. 10.1093/biostatistics/4.3.433View ArticlePubMedGoogle Scholar
- Eary JF, O’Sullivan F, O’Sullivan J, Conrad EU: Spatial heterogeneity in sarcoma 18 F- FDG uptake as a predictor of patient outcome. J Nucl Med 2008,49(12):1973-1979. 10.2967/jnumed.108.053397View ArticlePubMedPubMed CentralGoogle Scholar
- Van Velden FHP, Cheebsumon P, Yaqub M, Smit EF, Hoekstra OS, Lammertsma AA, et al.: Evaluation of a cumulative SUV-volume histogram method for parameterizing heterogeneous intratumoural FDG uptake in non-small cell lung cancer PET studies. Eur J Nucl Med Mol Imaging 2011,38(9):1636-1647. 10.1007/s00259-011-1845-6View ArticlePubMedPubMed CentralGoogle Scholar
- Tixier F, Le Rest CC, Hatt M, Albarghach N, Pradier O, Metges J-P, et al.: Intratumor heterogeneity characterized by textural features on baseline 18 F-FDG PET images predicts response to concomitant radiochemotherapy in esophageal cancer. J Nucl Med 2011,52(3):369-378. 10.2967/jnumed.110.082404View ArticlePubMedPubMed CentralGoogle Scholar
- Kidd EA, Grigsby PW: Intratumoral metabolic heterogeneity of cervical cancer. Clin Cancer Res 2008,14(16):5236-5241. 10.1158/1078-0432.CCR-07-5252View ArticlePubMedGoogle Scholar
- Yang F, Thomas MA, Dehdashti F, Grigsby PW: Temporal analysis of intratumoral metabolic heterogeneity characterized by textural features in cervical cancer. Eur J Nucl Med Mol Imaging. Springer-Verlag 2013,40(5):716-727. Epub 2013 Jan 23 doi:10.1007/s00259-012-2332-4 10.1007/s00259-012-2332-4View ArticleGoogle Scholar
- Vander Heiden MG, Cantley LC, Thompson CB: Understanding the Warburg effect: the metabolic requirements of cell proliferation. Science 2009,324(5930):1029-1033. 10.1126/science.1160809View ArticlePubMedPubMed CentralGoogle Scholar
- Erlandsson K, Buvat I, Pretorius PH, Thomas BA, Hutton BF: A review of partial volume correction techniques for emission tomography and their applications in neurology, cardiology and oncology. Phys Med Biol 2012,57(21):R119-R159. 10.1088/0031-9155/57/21/R119View ArticlePubMedGoogle Scholar
- Brooks FJ: On some misconceptions about tumor heterogeneity quantification. Eur J Nucl Med Mol Imaging 2013,40(8):1292-12924. Epub 2013 Apr 30 doi:10.1007/s00259-013-2430-y 10.1007/s00259-013-2430-yView ArticlePubMedGoogle Scholar
- El Naqa I, Grigsby P, Apte A, Kidd E, Donnelly E, Khullar D, et al.: Exploring feature- based approaches in PET images for predicting cancer treatment outcomes. Pattern Recognit 2009,42(6):1162-1171. 10.1016/j.patcog.2008.08.011View ArticlePubMedPubMed CentralGoogle Scholar
- O’Sullivan F, Wolsztynski E, O’Sullivan J, Richards T, Conrad EU, Eary JF: A statistical modeling approach to the analysis of spatial patterns of FDG-PET uptake in human sarcoma. IEEE Trans Med Imaging 2011,30(12):2059-2071.View ArticlePubMedPubMed CentralGoogle Scholar
- Brooks FJ, Grigsby PW: Quantification of heterogeneity observed in medical images. BMC Med Imaging 2013, 13: 7. 10.1186/1471-2342-13-7View ArticlePubMedPubMed CentralGoogle Scholar
- Tixier F, Hatt M, Le Rest CC, Le Pogam A, Corcos L, Visvikis D: Reproducibility of tumor uptake heterogeneity characterization through textural feature analysis in 18 F-FDG PET. J Nucl Med 2012,53(5):693-700. 10.2967/jnumed.111.099127View ArticlePubMedPubMed CentralGoogle Scholar
- Haidekker MA: Advanced biomedical image analysis. 1st edition. Hoboken NJ: John Wiley and Sons; 2011.Google Scholar
- Brooks FJ, Grigsby PW: Current measures of metabolic heterogeneity within cervical cancer do not predict disease outcome. Radiat Oncol 2011, 6: 69. 10.1186/1748-717X-6-69View ArticlePubMedPubMed CentralGoogle Scholar
- Brooks FJ, Grigsby PW: The effect of small tumor volumes upon metabolic heterogeneity studies. J Nucl Med 2013. [Epub ahead of print]Google Scholar
- Brooks FJ: Area under the cumulative SUV-volume histogram is not a viable metric of intratumoral metabolic heterogeneity. Eur J Nucl Med Mol Imaging 2013,40(6):967-968. 10.1007/s00259-013-2381-3View ArticlePubMedGoogle Scholar
- Brooks FJ: Area under the cumulative SUV-volume histogram is not a viable metric of intratumoral metabolic heterogeneity: further comments. Eur J Nucl Med Mol Imaging 2013,40(12):1926-1927. 10.1007/s00259-013-2572-yView ArticlePubMedGoogle Scholar
- Russ JC: The image processing handbook. 6th edition. Boca Raton: CRC Press; 2011.Google Scholar
- Cover TM, Thomas JA: Elements of information theory. 2nd edition. Hoboken, NJ: Wiley-Interscience; 2006.Google Scholar
- Miller TR, Grigsby PW: Measurement of tumor volume by PET to evaluate prognosis in patients with advanced cervical cancer treated by radiation therapy. Int J Radiat Oncol Biol Phys 2002,53(2):353-359. 10.1016/S0360-3016(02)02705-0View ArticlePubMedGoogle Scholar
- Wadell H: Volume, shape and roundness of rock particles. J Geol 1932,40(5):443-451. 10.1086/623964View ArticleGoogle Scholar
- Lehmann EL, D’Abrera HJM: Nonparametrics: statistical methods based on ranks. San Francisco: Holden-Day; 1975.Google Scholar
- Kim P, Jennrich RI: Tables of the Exact Sampling Distribution of the Two-sample Kolmogorov-Smirnov Criterion Dmn. In Selected Tables in Mathematical Statistics. 1st edition. Edited by: Institute of Mathematical Sciences, Harter HL, Owen DB. American Mathemematical Society; 1973. ISBN 0-8218-1901-1Google Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.