We developed and validated a research-only deep learning (DL) based automatic algorithm to detect thoracic and abdominal aortic aneurysms on contrast and non-contrast CT images and compared its performance with assessments obtained from retrospective radiology reports. The DL algorithm was developed using 556 CT scans. Manual annotations of aorta centerlines and cross-sectional aorta boundaries were created to train the algorithm. Aorta segmentation and aneurysm detection performances were evaluated on 2263 retrospective CT scans (154 thoracic and 176 abdominal aneurysms). Evaluation was performed by comparing the automatically detected aneurysm status to the aneurysm status reported in the radiology reports and the AUC was reported. In addition, a quantitative evaluation was performed to compare the automatically measured aortic diameters to manual diameters on a subset of 59 CT scans. Pearson correlation coefficient was used. For aneurysm detection, the AUC was 0.95 for thoracic aneurysm detection (95% confidence region [0.93, 0.97]) and 0.94 for abdominal aneurysm detection (95% confidence region [0.92, 0.96]). For aortic diameter measurement, the Pearson correlation coefficient was 0.973 (p<0.001).
Spine fractures are serious injuries that are often missed on standard chest x-rays, especially the frontal view. Automatic detection using artificial intelligence could help reduce these missed findings. To detect fractures, radiologists compare the morphological characteristics within the ordered vertebrae sequence. To this end, we designed a time-distributed CNN+LSTM model with Res-Net-inspired backbone to classify a sequence of vertebral image patches extracted from a frontal chest x-ray. While the CNN component encapsulates spatial characteristics of vertebrae, the time-distributed aspect ensures the sequential information of each image is explicitly preserved and better exploited by the LSTM. A Dense UNet was trained to automatically detect vertebrae locations for patch extraction. For this retrospective researchonly study, a comparative analysis was performed using six random partitions of 1250 images, and the AUCs for each partition were averaged. Using manual vertebrae detection to extract vertebral images patches from the x-rays, the proposed method achieved an average AUC of 83.17 ± 3.8%. Using the Dense UNet for automatic vertebrae detection in the pipeline, the proposed method achieved an average AUC of 79.3 ± 1.8% in differentiating x-rays with and without vertebral fractures. This is an average improvement of 6.0% relative to the same CNN+LSTM architecture that is not time distributed. Using a model ensemble, an AUC of 82.7 ± 3.7% was achieved. These findings suggest the importance of exploiting both sequential and spatial information for fracture detection.
Lung opacities on CT scans, such as ground-glass opacities (GGOs) and consolidation, can manifest with various conditions, including lung cancers, pulmonary edema and COVID-19. Presentation of these non-specific findings can vary from isolated focal to diffuse opacities in all lobes. Moreover, disease distributions and progressions vary across disease types and patients. This unpredictability can challenge one’s ability to accurately quantify and compare the percentage of infected lung within and across patients. Despite the promise of AI models for image segmentation, the inconsistency of lung opacities, and limited access to large annotated datasets affect generalization performance of models to the cohort of lung diseases. In this paper, we developed a single-stage system to jointly localize the lung and opacifications in CT scans using a diverse real-world dataset with sparse annotations. A multi-class Dense U-Net model was designed to segment the lungs and two classes of opacity regions (GGOs and consolidations) in CT images. The model was trained on 4075 slices from 495 sparsely annotated CT studies and evaluated on 18625 slices from 103 densely annotated studies (37 positive). A comparative analysis of different training data subsets and loss functions was performed to determine optimal model design. Performance was evaluated by comparing manual and automated lung opacity percentages via Pearson Correlation Coefficient. The optimal model achieved a Pearson Correlation Coefficient of 0.99. These findings suggest the potential of developing an accurate method to localize lung opacification, unspecific to a particular disease
A three-dimensional (3-D) convolutional neural network (CNN) trained from scratch is presented for the classification of pulmonary nodule malignancy from low-dose chest CT scans. Recent approval of lung cancer screening in the United States provides motivation for determining the likelihood of malignancy of pulmonary nodules from the initial CT scan finding to minimize the number of follow-up actions. Classifier ensembles of different combinations of the 3-D CNN and traditional machine learning models based on handcrafted 3-D image features are also explored. The dataset consisting of 326 nodules is constructed with balanced size and class distribution with the malignancy status pathologically confirmed. The results show that both the 3-D CNN single model and the ensemble models with 3-D CNN outperform the respective counterparts constructed using only traditional models. Moreover, complementary information can be learned by the 3-D CNN and the conventional models, which together are combined to construct an ensemble model with statistically superior performance compared with the single traditional model. The performance of the 3-D CNN model demonstrates the potential for improving the lung cancer screening follow-up protocol, which currently mainly depends on the nodule size.
With the advent of fully automated image analysis and modern machine learning methods, there is a need for very large image datasets having documented segmentations for both computer algorithm training and evaluation. This paper presents a method and implementation for facilitating such datasets that addresses the critical issue of size scaling for algorithm validation and evaluation; current evaluation methods that are usually used in academic studies do not scale to large datasets. This method includes protocols for the documentation of many regions in very large image datasets; the documentation may be incrementally updated by new image data and by improved algorithm outcomes. This method has been used for 5 years in the context of chest health biomarkers from low-dose chest CT images that are now being used with increasing frequency in lung cancer screening practice. The lung scans are segmented into over 100 different anatomical regions, and the method has been applied to a dataset of over 20,000 chest CT images. Using this framework, the computer algorithms have been developed to achieve over 90% acceptable image segmentation on the complete dataset.
The segmentation and labeling of the individual bones serve as the first step to the fully automated measurement of skeletal characteristics and the detection of abnormalities such as skeletal deformities, osteoporosis, and vertebral fractures. Moreover, the identified landmarks on the segmented bone structures can potentially provide relatively reliable location reference to other non-rigid human organs, such as breast, heart and lung, thereby facilitating the corresponding image analysis and registration. A fully automated anatomy-directed framework for the segmentation and labeling of the individual bone structures from low-dose chest CT is presented in this paper. The proposed system consists of four main stages: First, both clavicles are segmented and labeled by fitting a piecewise cylindrical envelope. Second, the sternum is segmented under the spatial constraints provided by the segmented clavicles. Third, all ribs are segmented and labeled based on 3D region growing within the volume of interest defined with reference to the spinal canal centerline and lungs. Fourth, the individual thoracic vertebrae are segmented and labeled by image intensity based analysis in the spatial region constrained by the previously segmented bone structures. The system performance was validated with 1270 lowdose chest CT scans through visual evaluation. Satisfactory performance was obtained respectively in 97.1% cases for the clavicle segmentation and labeling, in 97.3% cases for the sternum segmentation, in 97.2% cases for the rib segmentation, in 94.2% cases for the rib labeling, in 92.4% cases for vertebra segmentation and in 89.9% cases for the vertebra labeling.
Breast cancer is the most common cancer diagnosed among US women and the second leading cause of cancer death 1 . Breast density is an independent risk factor for breast cancer and more than 25 states mandate its reporting to patients as part of the lay mammogram report 2 . Recent publications have demonstrated that breast density measured from low-dose chest CT (LDCT) correlates well with that measured from mammograms and MRIs 3-4 , thereby providing valuable information for many women who have undergone LDCT but not recent mammograms. A fully automated framework for breast density assessment from LDCT is presented in this paper. The whole breast region is first segmented using an anatomy-orientated novel approach based on the propagation of muscle fronts for separating the fibroglandular tissue from the underlying muscles. The fibroglandular tissue regions are then identified from the segmented whole breast and the percentage density is calculated based on the volume ratio of the fibroglandular tissue to the local whole breast region. The breast region segmentation framework was validated with 1270 LDCT scans, with 96.1% satisfactory outcomes based on visual inspection. The density assessment was evaluated by comparing with BI-RADS density grades established by an experienced radiologist in 100 randomly selected LDCT scans of female subjects. The continuous breast density measurement was shown to be consistent with the reference subjective grading, with the Spearman’s rank correlation 0.91 (p-value < 0.001). After converting the continuous density to categorical grades, the automated density assessment was congruous with the radiologist’s reading in 91% cases.
A fully-automated computer algorithm has been developed to identify early-stage Usual Interstitial Pneumonia (UIP) using features computed from low-dose CT scans. In each scan, the pre-segmented lung region is divided into N subsections (N = 1, 8, 27, 64) by separating the lung from anterior/posterior, left/right and superior/inferior in 3D space. Each subsection has approximately the same volume. In each subsection, a classic density measurement (fractional high-density volume h) is evaluated to characterize the disease severity in that subsection, resulting in a feature vector of length N for each lung. Features are then combined in two different ways: concatenation (2*N features) and taking the maximum in each of the two corresponding subsections in the two lungs (N features).
The algorithm was evaluated on a dataset consisting of 51 UIP and 56 normal cases, a combined feature vector was computed for each case and an SVM classifier (RBF kernel) was used to classify them into UIP or normal using ten-fold cross validation. A receiver operating characteristic (ROC) area under the curve (AUC) was used for evaluation. The highest AUC of 0.95 was achieved by using concatenated features and an N of 27. Using lung partition (N = 27, 64) with concatenated features had significantly better result over not using partitions (N = 1) (p-value < 0.05). Therefore this equal-volume partition fractional high-density volume method is useful in distinguishing early-stage UIP from normal cases.
A fully automated computer algorithm has been developed to evaluate coronary artery calcification (CAC) from lowdose CT scans. CAC is identified and evaluated in three main coronary artery groups: Left Main and Left Anterior Descending Artery (LM + LAD) CAC, Left Circumflex Artery (LCX) CAC, and Right Coronary Artery (RCA) CAC. The artery labeling is achieved by segmenting all CAC candidates in the heart region and applying geometric constraints on the candidates using locally pre-identified anatomy regions. This algorithm was evaluated on 1,359 low-dose ungated CT scans, in which each artery CAC content was categorically visually scored by a radiologist into none, mild, moderate and extensive. The Spearman correlation coefficient R was used to assess the agreement between three automated CAC scores (Agatston-weighted, volume, and mass) and categorical visual scores. For Agatston-weighted automated scores, R was 0.87 for total CAC, 0.82 for LM + LAD CAC, 0.66 for LCX CAC and 0.72 for RCA CAC; results using volume and mass scores were similar. CAC detection sensitivities were: 0.87 for total, 0.82 for LM + LAD, 0.65 for LCX and 0.74 for RCA. To assess the impact of image noise, the dataset was further partitioned into three subsets based on heart region noise level (low<=80HU, medium=(80HU, 110HU], high>110HU). The low and medium noise subsets had higher sensitivities and correlations than the high noise subset. These results indicate that location specific heart risk assessment is possible from low-dose chest CT images.
With the advent of modern machine learning methods and fully automated image analysis there is a need for very large image datasets having documented segmentations for both computer algorithm training and evaluation. Current approaches of visual inspection and manual markings do not scale well to big data. We present a new approach that depends on fully automated algorithm outcomes for segmentation documentation, requires no manual marking, and provides quantitative evaluation for computer algorithms. The documentation of new image segmentations and new algorithm outcomes are achieved by visual inspection. The burden of visual inspection on large datasets is minimized by (a) customized visualizations for rapid review and (b) reducing the number of cases to be reviewed through analysis of quantitative segmentation evaluation. This method has been applied to a dataset of 7,440 whole-lung CT images for 6 different segmentation algorithms designed to fully automatically facilitate the measurement of a number of very important quantitative image biomarkers. The results indicate that we could achieve 93% to 99% successful segmentation for these algorithms on this relatively large image database. The presented evaluation method may be scaled to much larger image databases.
Segmentation of the sternum in medical images is of clinical significance as it frequently serves as a stable reference to image registration and segmentation of other organs in the chest region. In this paper we present a fully automated algorithm to segment the sternum in low-dose chest CT images (LDCT). The proposed algorithm first locates an axial seed slice and then segments the sternum cross section on the seed slice by matching a rectangle model. Furthermore, it tracks and segments the complete sternum in the cranial and caudal direction respectively through sequential axial slices starting from the seed slice. The cross section on each axial slice is segmented using score functions that are designed to have local maxima at the boundaries of the sternum. Finally, the sternal angle is localized. The algorithm is designed to be specifically robust with respect to cartilage calcifications and to accommodate the high noise levels encountered with LDCT images. Segmentation of 351 cases from public datasets was evaluated visually with only 1 failing to produce a usable segmentation. 87.2% of the 351 images have good segmentation and 12.5% have acceptable segmentation. The sternal body segmentation and the localization of the sternal angle and the vertical extents of the sternum were also evaluated quantitatively for 25 good cases and 25 acceptable cases. The overall weighted mean DC of 0.897 and weighted mean distance error of 2.88 mm demonstrate that the algorithm achieves encouraging performance in both segmenting the sternal body and localizing the sternal angle.
A new measurement of the pulmonary artery diameter is obtained where the artery may be robustly segmented between the heart and the artery bifurcation. An automated algorithm is presented that can make this pulmonary artery measurement in low-dose non-contrast chest CT images. The algorithm uses a cylinder matching method following geometric constraints obtained from other adjacent organs that have been previously segmented. This new measurement and the related ratio of pulmonary artery to aortic artery measurement are compared to traditional manual approaches for pulmonary artery characterization. The algorithm was qualitatively evaluated on 124 low-dose and 223 standard-dose non-contrast chest CT scans from two public datasets; 324 out of the 347 cases had good segmentations and in the other 23 cases there was significant boundary inaccuracy. For quantitative evaluation, the comparison was to manually marked pulmonary artery boundary in an axial slice in 45 cases; the resulting average Dice Similarity Coefficient was 0.88 (max 0.95, min 0.74). For the 45 cases with manual markings, the correlation between the automated pulmonary artery to ascending aorta diameter ratio and manual ratio at pulmonary artery bifurcation level was 0.81. Using Bland-Altman analysis, the mean difference of the two ratios was 0.03 and the limits of agreement was (-0.12, 0.18). This automated measurement may have utility as an alternative to the conventional manual measurement of pulmonary artery diameter at the bifurcation level especially in the context of noisy low-dose CT images.
Cardiac visceral fat was segmented from low-dose non-contrast chest CT images using a fully automated method. Cardiac visceral fat is defined as the fatty tissues surrounding the heart region, enclosed by the lungs and posterior to the sternum. It is measured by constraining the heart region with an Anatomy Label Map that contains robust segmentations of the lungs and other major organs and estimating the fatty tissue within this region. The algorithm was evaluated on 124 low-dose and 223 standard-dose non-contrast chest CT scans from two public datasets. Based on visual inspection, 343 cases had good cardiac visceral fat segmentation. For quantitative evaluation, manual markings of cardiac visceral fat regions were made in 3 image slices for 45 low-dose scans and the Dice similarity coefficient (DSC) was computed. The automated algorithm achieved an average DSC of 0.93. Cardiac visceral fat volume (CVFV), heart region volume (HRV) and their ratio were computed for each case. The correlation between cardiac visceral fat measurement and coronary artery and aortic calcification was also evaluated. Results indicated the automated algorithm for measuring cardiac visceral fat volume may be an alternative method to the traditional manual assessment of thoracic region fat content in the assessment of cardiovascular disease risk.
The automated segmentation of the nucleus and cytoplasm regions in 3D optical CT microscope images has been achieved with two methods, a global threshold gradient based approach and a graph-cut approach. For the first method, the first two peaks of a gradient figure of merit curve are selected as the thresholds for cytoplasm and nucleus segmentation. The second method applies a graph-cut segmentation twice: the first identifies the nucleus region and the second identifies the cytoplasm region. Image segmentation of single cells is important for automated disease diagnostic systems. The segmentation methods were evaluated with 200 3D images consisting of 40 samples of 5 different cell types. The cell types consisted of columnar, macrophage, metaplastic and squamous human cells and cultured A549 cancer cells. The segmented cells were compared with both 2D and 3D reference images and the quality of segmentation was determined by the Dice Similarity Coefficient (DSC). In general, the graph-cut method had a superior performance to the gradient-based method. The graph-cut method achieved an average DSC of 86% and 72% for nucleus and cytoplasm segmentations respectively for the 2D reference images and 83% and 75% for the 3D reference images. The gradient method achieved an average DSC of 72% and 51% for nucleus and cytoplasm segmentation for the 2D reference images and 71% and 51% for the 3D reference images. The DSC of cytoplasm segmentation was significantly lower than for the nucleus since the cytoplasm was not differentiated as well by image intensity from the background.
Coronary artery calcification (CAC) measurement from low-dose CT images can be used to assess the risk of coronary artery disease. A fully automatic algorithm to detect and measure CAC from low-dose non-contrast, non-ECG-gated chest CT scans is presented. Based on the automatically detected CAC, the Agatston score (AS), mass score and volume score were computed. These were compared with scores obtained manually from standard-dose ECG-gated scans and low-dose un-gated scans of the same patient. The automatic algorithm segments the heart region based on other pre-segmented organs to provide a coronary region mask. The mitral valve and aortic valve calcification is identified and excluded. All remaining voxels greater than 180HU within the mask region are considered as CAC candidates. The heart segmentation algorithm was evaluated on 400 non-contrast cases with both low-dose and regular dose CT scans. By visual inspection, 371 (92.8%) of the segmentations were acceptable. The automated CAC detection algorithm was evaluated on 41 low-dose non-contrast CT scans. Manual markings were performed on both low-dose and standard-dose scans for these cases. Using linear regression, the correlation of the automatic AS with the standard-dose manual scores was 0.86; with the low-dose manual scores the correlation was 0.91. Standard risk categories were also computed. The automated method risk category agreed with manual markings of gated scans for 24 cases while 15 cases were 1 category off. For low-dose scans, the automatic method agreed with 33 cases while 7 cases were 1 category off.
The extent of aortic calcification has been shown to be a risk indicator for vascular events including cardiac events. We have developed a fully automated computer algorithm to segment and measure aortic calcification in low-dose noncontrast, non-ECG gated, chest CT scans. The algorithm first segments the aorta using a pre-computed Anatomy Label Map (ALM). Then based on the segmented aorta, aortic calcification is detected and measured in terms of the Agatston score, mass score, and volume score. The automated scores are compared with reference scores obtained from manual markings. For aorta segmentation, the aorta is modeled as a series of discrete overlapping cylinders and the aortic centerline is determined using a cylinder-tracking algorithm. Then the aortic surface location is detected using the centerline and a triangular mesh model. The segmented aorta is used as a mask for the detection of aortic calcification. For calcification detection, the image is first filtered, then an elevated threshold of 160 Hounsfield units (HU) is used within the aorta mask region to reduce the effect of noise in low-dose scans, and finally non-aortic calcification voxels (bony structures, calcification in other organs) are eliminated. The remaining candidates are considered as true aortic calcification. The computer algorithm was evaluated on 45 low-dose non-contrast CT scans. Using linear regression, the automated Agatston score is 98.42% correlated with the reference Agatston score. The automated mass and volume score is respectively 98.46% and 98.28% correlated with the reference mass and volume score.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.