Comprehensive quantitative evaluation of tumor segmentation technique on large scale clinical data sets is crucial
for routine clinical use of CT based tumor volumetry for cancer diagnosis and treatment response evaluation.
In this paper, we present a systematic validation study of a semi-automatic image segmentation technique for
measuring tumor volume from CT images. The segmentation algorithm was tested using clinical data of 200
tumors in 107 patients with liver, lung, lymphoma and other types of cancer. The performance was evaluated
using both accuracy and reproducibility. The accuracy was assessed using 7 commonly used metrics that can
provide complementary information regarding the quality of the segmentation results. The reproducibility was
measured by the variation of the volume measurements from 10 independent segmentations. The effect of
disease type, lesion size and slice thickness of image data on the accuracy measures were also analyzed. Our
results demonstrate that the tumor segmentation algorithm showed good correlation with ground truth for all
four lesion types (r = 0.97, 0.99, 0.97, 0.98, p < 0.0001 for liver, lung, lymphoma and other respectively). The
segmentation algorithm can produce relatively reproducible volume measurements on all lesion types (coefficient
of variation in the range of 10-20%). Our results show that the algorithm is insensitive to lesion size (coefficient
of determination close to 0) and slice thickness of image data(p > 0.90). The validation framework used in this
study has the potential to facilitate the development of new tumor segmentation algorithms and assist large scale
evaluation of segmentation techniques for other clinical applications.
Appropriate validation of the segmentation algorithms is important for clinical acceptance of those methods. Receiver
operating characteristic (ROC) analysis provides the most comprehensive description of the accuracy performance of
image segmentation. Total area under an ROC curve (AUC) is widely used as an index of ROC analysis of performance
test. However, a large part of the ROC curve is in the clinically irrelevant range. The total area can be misleading in
some clinical situation. In this paper, we proposed a partial area index of ROC curves, which measures the segmentation
performance in a clinically relevant range decided by learning from subjective ratings. The boundary of the range is
defined by a linear cost function of false positive fraction (FPF) and true positive fraction (TPF). The cost factors of FPF
and TPF are learned by maximizing the Kendall's coefficient of concordance (KCC) between the partial areas and the
subjective ratings. Experiment results show that our method gives a large cost factor on FPF and a small cost factor on
TPF on a tumor data set. This is consistent with the fact that a large FPF is generally more difficult to be accepted in
tumor segmentation. Our method is able to determine the optimal range for partial area index of ROC analysis, and this
partial area index is more appropriate than AUC for evaluating segmentation performance.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.