How much will linked deformable registrations decrease the quality of multi-atlas segmentation fusions?
© Sjöberg et al.; licensee BioMed Central. 2014
Received: 7 November 2013
Accepted: 4 November 2014
Published: 20 December 2014
Background and purpose
Multi-atlas segmentation can yield better results than single atlas segmentation, but practical applications are limited by long calculation times for deformable registration. To shorten the calculation time pre-calculated registrations of atlases could be linked via a single atlas registered in runtime to the current patient. The primary purpose of this work is to investigate and quantify segmentation quality changes introduced by such linked registrations. We also determine the optimal parameters for fusing linked multi-atlas labels using probabilistic weighted fusion.
Material and methods
Computed tomography images of 10 head and neck cancer patients were used as atlases, with parotid glands, submandibular glands, the mandible and lymph node levels II-IV segmented by an experienced radiation oncologist following published consensus guidelines. The change in segmentation quality scored by Dice similarity coefficient (DSC) for linking free-form deformable registrations, modeled by B-splines, was investigated for both single- and multi-atlas label fusion by using a leave-one-out approach.
The median decrease of the DSC was in the range 2.8% to 8.4% compared to direct registrations for all structures while reducing the computer calculation time to that of a single deformable registration. Linking several registrations showed a DSC decrease almost linear to the number of links, suggesting that extrapolation to zero links provides an observer independent measure of the inherent precision with which the segmentation guidelines can be applied.
Linking pre-made registrations of multiple atlases via a runtime registration of a single atlas provides a feasible method for reducing computation time in multi-atlas registration.
Image segmentation for outlining targets and risk organs is a tedious and time consuming part of the radiotherapy process. Segmentation uncertainty can also contribute significantly to the total uncertainty in radiotherapy . Atlas-based segmentation, where deformable image registration is used to transform segmentations from pre-segmented images, can be used both to reduce manual labor time and to decrease inter-operator variability ,. Multi-atlas registration with label fusion, where the results from several independently registered atlases are automatically fused to yield a segmentation proposal, has been demonstrated to increase atlas based segmentation performance as compared to using a single atlas -. Several methods have been developed for the label fusion process, see e.g. ,.
A practical problem for implementing multi-atlas methods into clinical routine is the long calculation times needed for the deformable registrations of multiple image series. Several methods have been proposed in the literature to overcome this problem. One is to carefully choose and limit the number of individual segmentation proposals used in the fusion process ,. It has also been proposed to select a subset of images based on their similarity and only register those deemed to most likely give a good final segmentation result ,. The selection could be made initially, or after a fast registration using an affine transformation model with only a few degrees of freedom. Another approach would be to store pre-calculated transformations to a representative atlas that can be registered to the patient and used as a link of the premade registrations to the actual patient images. This would only be feasible if the segmentation quality losses caused by the linking itself are not too large. In a clinical setting, a large number of previously treated patients could potentially be of interest to use as multi-atlas material for new patients. With a very large atlas database available, pre-calculated deformable registrations could be clustered in a hierarchical strategy through strategically selected atlases serving as “hubs” to save computation time. Under such circumstances some atlas structures could be subject to several linked transformations which motivates to quantify the expected change in resulting segmentation quality as function of the number of applied linked transformations.
In this work we investigate the change in segmentation quality by comparing linked atlas registrations to direct registrations for structures relevant to radiotherapy of head and neck cancer. The result is determined for both individual segmentations and fused segmentation, where in the latter case we use a method based on probabilistic averaging of distance functions . We also optimize the parameters of this fusion method for the set of head and neck atlases used in this work.
Methods and materials
In atlas-based segmentation, a moving image M(x, y, z) is registered to a fixed image F(x, y, z) yielding a spatial transformation T M ← F(x, y, z) relating the positions of structures in F with the corresponding positions in M. Applying a transformation on a moving image results in a registered image R M ← F(x, y, z) = M(T M ← F(x, y, z)). To explicitly calculate R, interpolation must be used as in general grid point locations for the voxel positions in one of the images is by T M ← F(x, y, z) related to off-grid points in the other image. The structures in M are represented as a label map L M (x, y, z) which is transformed using T M ← F(x, y, z) such that voxels belonging to the same object are given the same label in the transformed image. In multi-atlas segmentation, the label images from several deformable registrations of different atlas images are combined in a label fusion process to provide the final segmentation proposal, as further described in section Label fusion.
Registrations can be linked by composing several transformations. A resulting transformation is then formed by T M ← I(l) ← F(x, y, z) = T M ← I(T I ← F(x, y, z)), or shorter T M ← I(l) ← F = T M ← I ∘ T I ← F which is calculated by linking registration results from registering M via one or several intermediate images I and where l indicates the number of intermediate transformations that are used in the linking process, e.g. As the transformations T I ← F for any given F and I can be pre-calculated and retrieved from a database, critical computer time savings can be achieved as compared to making the multiple registrations directly.
We will investigate the difference in segmentation quality using a leave-one-out evaluation strategy with a set of atlases for targets and risk organs in head and neck radiotherapy. The atlases consist of planning computed tomography image sets A i , i = 1, …, N for N = 10 atlas patients consistently segmented by an experienced radiation-oncologist following the guidelines of Lengele et al. . The patients used as atlases were randomly selected from a database of radiotherapy patients treated for tumors in the head and neck region, so none of the patients had a completely normal anatomy. To the best of our knowledge, no specific guidelines are published for the node positive neck so we used the guideline for the node-negative neck for all segmentations. The structures used for comparison were medulla, mandible and right and left parotid glands, submandibular glands and lymph node regions II-IV. The lymph node regions were segmented as one contiguous structure per side.
to score the segmentation qualities for the binary image volumes B 1 and B 2. This measure achieves a value of one for identical segmentations and zero for segmentations with no spatial overlap.
Another measure used was a fractional mean absolute distance measure (fMAD), calculated as the fraction of surface voxels for B 1 within a given distance (in 3D) from the closest surface voxel of B 2. A surface voxels for a binary volume is in this case defined as any object voxel sharing one or more sides, i.e. is 6-connected, to a background voxel. This gives an asymmetric distance measure describing how large fraction of a binary volume that is within some distance from a reference binary volume.
Each explicit registration was performed in two parts, first an affine registration which optimized parameters for translation, rotation, scaling and shearing. This transformation was then used as initialization for a deformable registration step modeled by B-splines . This method parameterizes the transformation as a linear combination of compactly supported splines placed on a regular grid. The coefficients for the linear combination were optimized with regard to an image similarity measure. For this work, normalized mutual Information (NMI)  was used as the similarity measure.
All registrations were performed in a multi-resolution fashion using a Gaussian pyramid with four levels to reduce the risk that the optimization converges to a local optimum. A registration starts at the coarsest level, and the resulting transformations from each level are then used as initialization for the next level. This process was repeated until the final image resolution was reached. As the resolutions of the images are anisotropic with a slice thickness larger than the in-slice pixel sides, the down-sampling for the first level was only performed in the in-slice direction, and for the remaining levels the images were down-sampled with an equal factor for all dimensions. The value of the down-sampling scaling factor was 2 for all dimensions and levels. For the deformable registrations, the B-spline grid spacing was 8 mm in all directions for the finest resolution and down-sampled in accordance with the scheme outlined above.
As optimization method we used the adaptive stochastic gradient descent (ASGD) , as implemented in the Elastix package . This method speeds up the registration process by using a sub-sampled set of intensity values for the computation of the image similarity measure and its derivatives. We used 2048 pairs of intensity values as the sub-sampled set, sampled at randomly selected locations within the image volumes. To achieve convergence of the stochastic optimization, the step size was reduced for each iteration with a fixed number of iterations calculated for each registration level. We calculated 500 iterations per level, which gave reasonable computation times for the individual registrations. Linear interpolation was used during registration and 3:rd degree B-spline interpolation for applying the final transformation to the moving image.
This work was approved by the Regional Ethical Review Board in Uppsala (2013/277).
Linking of registrations
up to the maximum possible number of combinations permitted by our atlas material, which yields a maximum of nine transformations. As the number of possible combinations quickly becomes so large that it is not realistic to calculate all of them, we sampled randomly the combinations used for evaluation.
Preliminary investigations of a value of r was performed by varying it from 0 to 50 mm in steps of 10 mm. For the mandible, with a well-defined border, a value of 0 mm was selected whereas for the other structures a value of 10 mm yielded the best results.
Selection of intermediate image for linked registrations
The selection of the image to use as intermediate image for registration linkage might be of importance for the final result. One selection strategy would be to register a larger set of images using fast affine registrations and to use the resulting image similarities for ranking, as more similar images should be more suitable as atlases. By comparing results using this strategy to average results using all available intermediate images we noted that results were slightly improved by this strategy. To retain generality, we present results for decrease in segmentation quality using all available moving images as nodes in a leave-one-out fashion as previously described. For optimization of parameters for the probabilistic weighting algorithm based on image similarity information, the strategy of selecting the node with highest image similarity after affine registration was used.
Median relative change of DSC when using one linked registration compared to direct registration results
Median relative change (%)
−3.2 (-7.8, 1.9)*
−4.3 (-8.3, -0.7)*
−7.0 (-20, 4.4)*
−9.5 (-37, 11)
−6.5 (-10, -3.7)*
−2.8 (-5.4, -1.3)*
Lymph node levels II-IV
−7.6 (-16, -1.0)*
−8.4 (-17, -3.1)*
Average time for full multi-atlas based segmentation including 9 affine and deformable registrations and label fusions
Segmentation time (minutes and seconds)
Full multi-atlas registration
35 minutes 32 s
9 minutes 59 s
4 minutes 18 s
1 minute 43 s
Discussion and conclusions
Transferring segmentations by composition of pre-calculated registration decrease the quality of registrations and atlas-based segmentations. However, our results demonstrate a moderate drop in segmentation quality while saving large amounts of calculation time, indicating that linking registrations is a feasible way of using multi-atlas registrations in a clinical setting. Registration times depend on actual methods and implementations, in this work the mean wall clock time for all registrations were 3 minutes and 50 seconds with a range from 3 minutes and 17 seconds to 4 minutes and 53 seconds. Compared to a full multi-atlas segmentation, the segmentation time was reduced to less than one third by using the linked registration method.
For the structure with the highest resulting DSC, the medulla, a larger drop in segmentation quality suggests that single atlas registration results, as can be seen in Figure 2, could be sufficient for this case.
In Figure 6 it can be noted that the average distance from the segmentation proposal to the reference segmentation is increased by linking registrations. The fMAD value gives an indication of the amount of editing needed before the structure is clinically acceptable. For the parotid glands, the quality decrease introduced by linking registrations, as measured by fMAD, is negligible. For the submandibular glands, there is a decrease of segmentation quality. Since manual re-contouring of the entire structures only takes a few minutes per structure, if only parotid glands and submandibular glands are to be segmented, the calculation time even for segmentations using linked registration will be at the order of complete manual re-contouring. However, if lymph node levels are part of the segmentation task for the creation of a treatment plan, the atlas-based segmentation method will yield segmentation proposals also for the glands with almost no additional calculation time. If the proposals for linked registrations will give shorter manual editing times compared to re-contouring remains to be investigated. For the lymph node levels that require a long manual contouring time, the significant time saving from using linked registrations compared to direct atlas-based segmentation will most likely lead to a reduction in total segmentation time.
The available atlases were based on images of patients with head-and-neck tumors that perturbed the normal anatomy. Since image registration is a more difficult problem when the images to be registered are less similar, selecting an atlas with a lower similarity to the current patient is likely to lead to a lower quality of the resulting segmentation. This will in turn lead to a larger spread in segmentation result based on the magnitude of the perturbation both in the patient selected through the leave-one-out process and the atlas selected as the node in the linking process. This can also explain why sometimes individual segmentations created by linked registrations have higher quality than direct segmentation results, for example for the submandibular glands.
We note from Figure 3 that the segmentation quality decrease is approximately linear to the number of links in the composed transformations. Segmentation quality for multi-atlas segmentation depends both on registration accuracy as well as the precision of the manual atlas segmentations. We hypothesize that extrapolation to zero registrations could provide observer independent information about the quality of the registrations and the limitations in precisions from the consensus protocol. If the registration accuracy is sufficient, the slope of the linear fit would be small and extrapolation to zero would indicate a limit in segmentation precision.
When label fusion is used, a similar reduction in segmentation quality as for the individual segmentations are seen. However, variance of the relative DSC change is decreased, as can be seen in Figure 5. This also means that the increase of segmentation quality that is occasionally seen for the individual results disappears, however to choose these improved segmentations is not trivial.
Time savings using a linked registration method compared to direct registrations are inversely proportional to the number of registrations used in the linking process. The additional overhead is simply the composition of the transforms, which can be implemented in different ways. If the transformations are pre-saved as full deformation fields, a simple linear interpolation per linking is the only additional over-head, which is very small compared to a complete registration. We compared segmentation results for composing registrations through successive linear interpolations with composition through the explicit B-spline transformations, with negligible differences in segmentation qualities.
All the data used in this study were anonymized. The study design was approved by the ethical review board without the need for written informed consent.
Funding for this work was provided by Cancerfonden.
- Weiss E, Hess CF: The impact of Gross Tumor Volume (GTV) and Clinical Target Volume (CTV) definition on the total accuracy in radiotherapy. Strahlenther Onkol. 2003, 179: 21-30. 10.1007/s00066-003-0976-5.View ArticlePubMedGoogle Scholar
- Chao KS, Bhide S, Chen H, Asper J, Bush S, Franklin G, Kavadi V, Liengswangwong V, Gordon W, Raben A, Strasser J, Koprowski C, Frank S, Chronowski G, Ahamad A, Malyapa R, Zhang L, Dong L: Reduce in variation and improve efficiency of target volume delineation by a computer-assisted system using a deformable image registration approach. Int J Radiat Oncol Biol Phys. 2007, 68: 1512-1521. 10.1016/j.ijrobp.2007.04.037.View ArticlePubMedGoogle Scholar
- Stapleford LJ, Lawson JD, Perkins C, Edelman S, Davis L, McDonald MW, Waller A, Schreibmann E, Fox T: Evaluation of automatic atlas-based lymph node segmentation for head-and-neck cancer. Int J Radiat Oncol Biol Phys. 2010, 77: 959-966. 10.1016/j.ijrobp.2009.09.023.View ArticlePubMedGoogle Scholar
- Isgum I, Staring M, Rutten A, Prokop M, Viergever MA, van Ginneken B: Multi-atlas-based segmentation with local decision fusion–application to cardiac and aortic segmentation in CT scans. IEEE Trans Med Imaging. 2009, 28: 1000-1010. 10.1109/TMI.2008.2011480.View ArticlePubMedGoogle Scholar
- Heckemann RA, Hajnal JV, Aljabar P, Rueckert D, Hammers A: Automatic anatomical brain MRI segmentation combining label propagation and decision fusion. Neuroimage. 2006, 33: 115-126. 10.1016/j.neuroimage.2006.05.061.View ArticlePubMedGoogle Scholar
- Rohlfing T, Brandt R, Menzel R, Russakoff DB, Maurer J, Calvin R: Quo Vadis, Atlas-Based Segmentation? The Handbook of Medical Image Analysis - Volume III: Registration Models. 2005, Kluwer Academic / Plenum Publishers, New York, NYGoogle Scholar
- Warfield SK, Zou KH, Wells WM: Simultaneous truth and performance level estimation (STAPLE): an algorithm for the validation of image segmentation. IEEE Trans Med Imaging. 2004, 23: 903-921. 10.1109/TMI.2004.828354.PubMed CentralView ArticlePubMedGoogle Scholar
- Aljabar P, Heckemann RA, Hammers A, Hajnal JV, Rueckert D: Multi-atlas based segmentation of brain images: atlas selection and its effect on accuracy. Neuroimage. 2009, 46: 726-738. 10.1016/j.neuroimage.2009.02.018.View ArticlePubMedGoogle Scholar
- van Rikxoort EM, Isgum I, Arzhaeva Y, Staring M, Klein S, Viergever MA, Pluim JP, van Ginneken B: Adaptive local multi-atlas segmentation: application to the heart and the caudate nucleus. Med Image Anal. 2010, 14: 39-49. 10.1016/j.media.2009.10.001.View ArticlePubMedGoogle Scholar
- Sjöberg C, Ahnesjö A: Multi-atlas based segmentation using probabilistic label fusion with adaptive weighting of image similarity measures. Comput Methods Programs Biomed. 2013, 110: 308-319. 10.1016/j.cmpb.2012.12.006.View ArticlePubMedGoogle Scholar
- Lengele B, Hamoir M, Scalliet P, Gregoire V: Anatomical bases for the radiological delineation of lymph node areas. Major collecting trunks, head and neck. Radiother Oncol. 2007, 85: 146-155. 10.1016/j.radonc.2007.02.009.View ArticlePubMedGoogle Scholar
- Dice LR: Measures of the amount of ecologic association between species. Ecology. 1945, 26: 297-302. 10.2307/1932409.View ArticleGoogle Scholar
- Rueckert D, Sonoda LI, Hayes C, Hill DL, Leach MO, Hawkes DJ: Nonrigid registration using free-form deformations: application to breast MR images. IEEE Trans Med Imaging. 1999, 18: 712-721. 10.1109/42.796284.View ArticlePubMedGoogle Scholar
- Studholme C, Hill DLG, Hawkes DJ: An overlap invariant entropy measure of 3D medical image alignment. Pattern Recogn. 1999, 32: 71-86. 10.1016/S0031-3203(98)00091-0.View ArticleGoogle Scholar
- Klein S, Pluim JP, Staring M, Viergever MA: Adaptive stochastic gradient descent optimisation for image registration. Int J Comput Vision. 2009, 81: 227-239. 10.1007/s11263-008-0168-y.View ArticleGoogle Scholar
- Klein S, Staring M, Murphy K, Viergever MA, Pluim JP: elastix: a toolbox for intensity-based medical image registration. IEEE Trans Med Imaging. 2010, 29: 196-205. 10.1109/TMI.2009.2035616.View ArticlePubMedGoogle Scholar
- Sjöberg C, Lundmark M, Granberg C, Johansson S, Ahnesjö A, Montelius A: Clinical evaluation of multi-atlas based segmentation of lymph node regions in head and neck and prostate cancer patients.Radiat Oncol 2013, 8:229.Google Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.