Prostate cancer is a significant contributor to cancer-related deaths in men. Detecting prostate cancer early can greatly increase the likelihood of successful treatment. However, detecting and assessing prostate lesions from multiparametric magnetic resonance images (MRI) is time-consuming and variable across radiologists with different levels of experience. We present an integrated framework for segmenting and classifying prostate lesions from MRI. The proposed approach is in contrast with most existing automated prostate analysis approaches, which treat segmentation and classification of prostate lesions as two separate tasks with no interactions between them. In the proposed framework, preliminary lesion boundaries were first segmented from T2-weighted (T2W) and diffusion-weighted images (DWI) by a three-stream network. The region of interest (ROI) enclosing the segmented lesion was fed to a weakly supervised classification network, which predicted the Gleason grade of the lesion and provided the class activation maps (CAMs) corresponding to multiple MRI modalities. Finally, MR images of different modalities with the corresponding CAMs were fed to a six-stream network to generate an enhanced lesion mask. Our experiments showed that CAMs generated by the proposed weakly supervised classifier improved segmentation performance. Our proposed method has a great potential to improve the accuracy and efficiency of prostate MRI interpretation workflow.
Stroke is a leading cause of morbidity and mortality throughout the world. Three-dimensional ultrasound (3DUS) imaging was shown to be more sensitive to treatment effect and more accurate in stratifying stroke risk than two-dimensional ultrasound (2DUS) imaging. Point-of-care ultrasound screening (POCUS) is important for patients with limited mobility and at times when the patients have limited access to the ultrasound scanning room, such as in the COVID-19 era. We used an optical tracking system to track the 3D position and orientation of the 2DUS frames acquired by a commercial wireless ultrasound system and subsequently reconstructed a 3DUS image from these frames. The tracking requires spatial and temporal calibrations. Spatial calibration is required to determine the spatial relationship between the 2DUS machine and the tracking system. Spatial calibration was achieved by localizing the landmarks with known coordinates in a custom-designed Z-fiducial phantom in an 2DUS image. Temporal calibration is needed to synchronize the clock of the wireless ultrasound system and the optical tracking system so that position and orientation detected by the optical tracking system can be registered to the corresponding 2DUS frame. Temporal calibration was achieved by initiating the scanning by an abrupt motion that can be readily detected in both systems. This abrupt motion establishes a common reference time point, thereby synchronizing the clock in both systems. We demonstrated that the system can be used to visualize the three-dimensional structure of a carotid phantom. The error rate of the measurements is 2.3%. Upon in-vivo validation, this system will allow POCUS carotid scanning in clinical research and practices.
KEYWORDS: Image segmentation, Arteries, Magnetic resonance imaging, 3D image processing, Data modeling, 3D modeling, Atherosclerosis, Algorithm development
Vessel wall volume (VWV) for the femoral arteries is a sensitive indicator of coexistent generalized atherosclerosis. Measuring VWV requires the segmentation of lumen and outer wall boundaries from 3D MR images. The main challenge for vessel wall segmentation is the small size of femoral artery in a 3D MR image and the existence of objects mimicking arteries. Besides, due to the long span of the femoral artery and time-consuming manual segmentation, a large number of image slices are not manually segmented, and therefore, cannot be used to train fully supervised methods. We proposed a semi-supervised end-to-end artery localization and segmentation model that improves segmentation performance through the use of axial image slices that are not manually segmented (unlabeled slices). The method localizes femoral arteries with bounding boxes and performs segmentation over the selected regions. A mean teacher framework was trained to generate high-quality segmentation for unlabeled slices, serving as pseudo-labels to improve the student model’s performance in arterial detection and vessel wall segmentation. A new continuity score was developed to further improve the quality of the vessel wall segmentation on unlabeled image slices. Our experiments show that the semi-supervised approach and the proposed continuity score independently improve the femoral vessel wall segmentation.
Stroke is among the leading causes of death and disability throughout the world. Carotid atherosclerosis is a focal disease predominantly occurring at bifurcations, and for this reason, local progression/regression measurements of atherosclerosis allow for more sensitive detection of treatment effect than global measurements, such as total vessel wall volume (VWV). Vessel-wall-plus-plaque thickness (VWT) change has been developed to characterize local changes and has shown to be sensitive to treatment effect, but is unable to isolate changes in individual plaque components. In this work, we propose to quantify longitudinal voxel-by-voxel plaque-and-vessel-wall volume change (ΔVVol) and represent the ΔVVol distribution on a 3D standardized atlas. Such representation allows for quantitative comparison across patients and of the measurements obtained for the same patient at different time points. We introduced a 3D non-rigid registration framework to register the carotid ultrasound images acquired at baseline and a follow-up imaging session for each patient. A 3D volume equipped with voxel-by-voxel ΔVVol was obtained by taking the divergence of the displacement field obtained in non-rigid registration. This 3D volume was uniformly sampled in the vessel wall, and the ΔVVol distribution for each patient was represented in a 3D standardized map. The proposed 3D standardized ΔVVol map allows for the characterization of feature changes on a voxel-by-voxel basis that are masked in VWT quantification. This tool has the potential to further improve the sensitivity in treatment evaluation already attained by VWT quantification.
Vessel wall volume (VWV) and local vessel-wall-plus-plaque thickness (VWT) measured from 3D ultrasound (3DUS) are sensitive to change of plaque burden over time and are useful in evaluating treatment effect. Segmentation of the media-adventitia (MAB) and lumen-intima boundaries (LIB) was required in VWV and VWT quantification. Manual segmentation of these boundaries is time-consuming and prone to observer variability. In this work, we developed and validated a method to segment MAB and LIB from axial images re-sliced from 3DUS images using a light-weight coarse-to-fine network. The proposed network is computationally efficient with only 0.59M parameters (compared to 31M parameters in U-Net). The boundaries segmented by the proposed algorithm were compared with manually segmented boundaries. The proposed algorithm attained Dice similarity coefficients (DSC) of 92:5±3:09% and 85:4±6:04% for MAB and LIB respectively, which are higher than those attained by U-Net family networks, including U-Net++, scaled U-Net and attention U-Net. This segmentation tool will facilitate efficient quantification of VWV and VWT, thereby making it more feasible for them to be measured in clinical trials evaluating treatment effect or for stroke risk stratification.
Sensitive and cost-effective biomarkers for carotid atherosclerosis are required to evaluate the efficacy of dietary and medical treatments. Carotid atherosclerosis is a focal disease predominantly occurring in bends and bifurcations. For this reason, we have previously developed a method to measure local vessel-wall-plus-plaque thickness (VWT); a biomarker based on a weighted average of the point-wise ΔVWT was also developed and validated to be sensitive to treatment effect. However, the weight determined on a point-by-point basis did not take into account the spatial correlation of ΔVWT in neighboring points. In this paper, we developed a biomarker that is able to characterize the correlation within each local patch of the VWT map. The deep autoencoder (DAE) initialized by the stacked restricted Boltzmann machines (RBMs) was introduced to learn a compact feature representation of each patch in the 2D VWT map. The patch-based feature change was obtained by taking the difference between the features obtained at baseline and a follow-up imaging session. The new biomarker, denoted by ∆VWTpatch, was computed by taking a weighted average of the patch-based feature change. The sensitivity of the patch-based average was compared with that of the point-wise average (∆VWTpoint) in 40 subjects involved in a placebo-controlled clinical trial of the efficacy of pomegranate. ∆VWTpatch detected a significant difference between the pomegranate and placebo groups (p = 0.017), but not ∆VWTpoint (p = 0.056). The sample size required by ∆VWTpatch to establish significance was 37% smaller than that by ∆VWTpoint.
Prostatic adenocarcinoma is one of the most commonly occurring cancers among men in the world, and it also the most curable cancer when it is detected early. Multiparametric MRI (mpMRI) combines anatomic and functional prostate imaging techniques, which have been shown to produce high sensitivity and specificity in cancer localization, which is important in planning biopsies and focal therapies. However, in previous investigations, lesion localization was achieved mainly by manual segmentation, which is time-consuming and prone to observer variability. Here, we developed an algorithm based on locality alignment discriminant analysis (LADA) technique, which can be considered as a version of linear discriminant analysis (LDA) localized to patches in the feature space. Sensitivity, specificity and accuracy generated by the proposed algorithm in five prostates by LADA were 52.2%, 89.1% and 85.1% respectively, compared to 31.3%, 85.3% and 80.9% generated by LDA. The delineation accuracy attainable by this tool has a potential in increasing the cancer detection rate in biopsies and in minimizing collateral damage of surrounding tissues in focal therapies.
Image matting is a method that separates foreground and background objects in an image, and has been widely used in medical image segmentation. Previous work has shown that matting can be formulated as a graph Laplacian matrix. In this paper, we derived matting from a local regression and global alignment view, as an attempt to provide a more intuitive solution to the segmentation problem. In addition, we improved the matting algorithm by adding a weight extension and refer to the proposed approach as Adaptive Weight Matting (AWM), where an adaptive weight was added to each local regression term to reduce the bias caused by outliers. We compared the segmentation results generated by the proposed method and several state-of-the-art segmentation methods, including conventional matting, graph-cuts and random walker, on medical images of different organs acquired using different imaging modalities. Experimental results demonstrated the advantages of AWM on medical image segmentation.
KEYWORDS: Image registration, Ultrasonography, Arteries, 3D image processing, Rigid registration, Image segmentation, Image processing algorithms and systems, Independent component analysis, Simulation of CCA and DLA aggregates, Motion models
Registration of longitudinally acquired 3D ultrasound (US) images plays an important role in monitoring and quantifying progression/regression of carotid atherosclerosis. We introduce an image-based non-rigid registration algorithm to align the baseline 3D carotid US with longitudinal images acquired over several follow-up time points. This algorithm minimizes the sum of absolute intensity differences (SAD) under a variational optical-flow perspective within a multi-scale optimization framework to capture local and global deformations. Outer wall and lumen were segmented manually on each image, and the performance of the registration algorithm was quantified by Dice similarity coefficient (DSC) and mean absolute distance (MAD) of the outer wall and lumen surfaces after registration. In this study, images for 5 subjects were registered initially by rigid registration, followed by the proposed algorithm. Mean DSC generated by the proposed algorithm was 79:3±3:8% for lumen and 85:9±4:0% for outer wall, compared to 73:9±3:4% and 84:7±3:2% generated by rigid registration. Mean MAD of 0:46±0:08mm and 0:52±0:13mm were generated for lumen and outer wall respectively by the proposed algorithm, compared to 0:55±0:08mm and 0:54±0:11mm generated by rigid registration. The mean registration time of our method per image pair was 143±23s.
Intraventricular hemorrhage (IVH) or bleed within the brain is a common condition among pre-term infants that occurs in very low birth weight preterm neonates. The prognosis is further worsened by the development of progressive ventricular dilatation, i.e., post-hemorrhagic ventricle dilation (PHVD), which occurs in 10-30% of IVH patients. In practice, predicting PHVD accurately and determining if that specific patient with ventricular dilatation requires the ability to measure accurately ventricular volume. While monitoring of PHVD in infants is typically done by repeated US and not MRI, once the patient has been treated, the follow-up over the lifetime of the patient is done by MRI. While manual segmentation is still seen as a gold standard, it is extremely time consuming, and therefore not feasible in a clinical context, and it also has a large inter- and intra-observer variability. This paper proposes a segmentation algorithm to extract the cerebral ventricles from 3D T1- weighted MR images of pre-term infants with PHVD. The proposed segmentation algorithm makes use of the convex optimization technique combined with the learned priors of image intensities and label probabilistic map, which is built from a multi-atlas registration scheme. The leave-one-out cross validation using 7 PHVD patient T1 weighted MR images showed that the proposed method yielded a mean DSC of 89.7% ± 4.2%, a MAD of 2.6 ± 1.1 mm, a MAXD of 17.8 ± 6.2 mm, and a VD of 11.6% ± 5.9%, suggesting a good agreement with manual segmentations.
KEYWORDS: 3D image processing, Image segmentation, Ultrasonography, 3D metrology, Brain, 3D acquisition, Traumatic brain injury, Brain mapping, 3D image reconstruction, 3D applications
Intraventricular hemorrhage (IVH) is a major cause of brain injury in preterm neonates. Quantitative measurement of ventricular dilation or shrinkage is important for monitoring patients and in evaluation of treatment options. 3D ultrasound (US) has been used to monitor the ventricle volume as a biomarker for ventricular dilation. However, volumetric quantification does not provide information as to where dilation occurs. The location where dilation occurs may be related to specific neurological problems later in life. For example, posterior horn enlargement, with thinning of the corpus callosum and parietal white matter fibres, could be linked to poor visuo-spatial abilities seen in hydrocephalic children. In this work, we report on the development and application of a method used to analyze local surface change of the ventricles of preterm neonates with IVH from 3D US images. The technique is evaluated using manual segmentations from 3D US images acquired in two imaging sessions. The surfaces from baseline and follow-up were registered and then matched on a point-by-point basis. The distance between each pair of corresponding points served as an estimate of local surface change of the brain ventricle at each vertex. The measurements of local surface change were then superimposed on the ventricle surface to produce the 3D local surface change map that provide information on the spatio-temporal dilation pattern of brain ventricles following IVH. This tool can be used to monitor responses to different treatment options, and may provide important information for elucidating the deficiencies a patient will have later in life.
KEYWORDS: 3D metrology, 3D image processing, Image segmentation, Arteries, Magnetic resonance imaging, 3D acquisition, Simulation of CCA and DLA aggregates, Independent component analysis, Anisotropy, 3D magnetic resonance imaging
Quantification of vessel wall thickness is important in longitudinal monitoring of atherosclerosis. Black-blood
MRI has been useful in measuring vessel wall thickness. Studies using two-dimensional (2D) imaging protocols
measured wall thickness by matching the arterial wall and lumen boundaries on an acquisition plane. If the
acquisition plane is oblique to the artery, the wall thickness would be overestimated by a factor that is dependent
on the obliqueness angle. This problem can be understood as a three-dimensional (3D) surface mismatch problem,
and we evaluated the effect of this problem by comparing the thickness measurements obtained using a 2D contour
matching method and a 3D surface matching method. In addition to the surface mismatch problem, two other
parameters may affect the wall thickness estimation: reslicing angle and slice thickness. We measured the wall
thickness using images resliced perpendicular to the centerline of the vessel and quantified the difference between
the thickness measurements obtained from parallel and centerline-based resliced images. Images obtained from
a 2D MRI protocol typically have a slice thickness of 2mm, while the 3D MRI technique applied in this study
produced images with sub-millimeter isotropic voxel size. To investigate the effect of slice thickness, we simulated
2mm-thick images by averaging the 3D black-blood image. Our results show that the wall thickness measured
from 2mm-thick images was overestimated, especially in the carotid artery, which is associated with a larger
obliqueness angle. This result underscores the advantage of the 3D isotropic acquisition technique in wall
thickness measurement, especially in more tortuous vessels.
Local hemodynamic forces in atherosclerotic carotid arteries are thought to trigger cellular and molecular mechanisms
that determine plaque vulnerability. Magnetic Resonance Imaging (MRI) has emerged as a powerful tool to characterize
human carotid atherosclerotic plaque composition and morphology, and to identify plaque features shown to be key
determinants of plaque vulnerability. Image-based computational fluid dynamics (CFD) has allowed researchers to
obtain time-resolved wall shear stress (WSS) information for atherosclerotic carotid arteries. A deeper understanding of
the mechanisms of initiation and progression of atherosclerosis can be obtained through the comparison of WSS and
plaque composition. The aim of this study was to explore the hypothesis that intra-plaque hemorrhage, a feature
associated with adverse outcomes and plaque progression, is more likely to occur in plaques with elevated WSS levels.
We compared 2D representations of the WSS distribution and the amount of intra-plaque hemorrhage to determine
relationships between WSS patterns and plaque vulnerability. We extracted WSS data to compare patterns between cases
with and without hemorrhage. We found elevated values of WSS at regions where intra-plaque hemorrhage was
detected, suggesting that WSS might be used as a marker for the risk of intra-plaque hemorrhage and subsequent
complications.
KEYWORDS: 3D image processing, Arteries, 3D metrology, Ultrasonography, Simulation of CCA and DLA aggregates, Image segmentation, Independent component analysis, Medical research, 3D acquisition, Biomedical engineering
Quantitative measurements of the progression (or regression) of carotid plaque burden are important in monitoring
patients and evaluating new treatment options. 3D ultrasound (US) has been used to monitor the progression
of carotid artery plaques in symptomatic and asymptomatic patients. Different methods of measuring various
ultrasound phenotypes of atherosclerosis have been developed. In this work, we extended concepts used in
intima-media thickness (IMT) measurements based on 2D images and introduced a metric called 3D vessel-wall-plus-plaque thickness (3D VWT), which was obtained by computing the distance between the carotid wall and
lumen surfaces on a point-by-point basis in a 3D image of the carotid arteries. The VWT measurements were
then superimposed on the arterial wall to produce the VWT map. Since the progression of plaque thickness is
important in monitoring patients who are at risk for stroke, we also computed the change of VWT by comparing
the VWT maps obtained for a patient at two different time points. In order to facilitate the visualization and
interpretation of the 3D VWT and VWT-Change maps, we proposed a technique to flatten these maps in an
area-preserving manner.
KEYWORDS: Image registration, 3D modeling, 3D image processing, Magnetic resonance imaging, Image segmentation, 3D acquisition, Rigid registration, Arteries, Ultrasonography, Neck
Atherosclerosis at the carotid bifurcation resulting in cerebral emboli is a major cause of ischemic stroke. Most
strokes associated with carotid atherosclerosis can be prevented by lifestyle/dietary changes and pharmacological
treatments if identified early by monitoring carotid plaque changes. Plaque composition information from
magnetic resonance (MR) carotid images and dynamic characteristics information from 3D ultrasound (US) are
necessary for developing and validating US imaging tools to identify vulnerable carotid plaques. Combining these
images requires nonrigid registration to correct the non-linear miss-alignments caused by relative twisting and
bending in the neck due to different head positions during the two image acquisitions sessions.
The high degree of freedom and large number of parameters associated with existing nonrigid image registration
methods causes several problems including unnatural plaque morphology alteration, computational
complexity, and low reliability. Our approach was to model the normal movement of the neck using a "twisting
and bending model" with only six parameters for nonrigid registration. We evaluated our registration technique
using intra-subject in-vivo 3D US and 3D MR carotid images acquired on the same day. We calculated the Mean
Registration Error (MRE) between the segmented vessel surfaces in the target image and the registered image
using a distance-based error metric after applying our "twisting bending model" based nonrigid registration
algorithm. We achieved an average registration error of 1.33±0.41mm using our nonrigid registration technique.
Visual inspection of segmented vessel surfaces also showed a substantial improvement of alignment with our
non-rigid registration technique.
KEYWORDS: Image segmentation, 3D image processing, Arteries, Independent component analysis, Simulation of CCA and DLA aggregates, Statistical analysis, 3D metrology, Ultrasonography, Medical research, 3D displays
Atherosclerosis is characterized by the development of plaques in the arterial wall, which ultimately leads to heart attacks and stroke. 3D ultrasound (US) has been used to screen patients' carotid arteries. Plaque measurements obtained from these images may aid in the management and monitoring of patients, and in evaluating the effect of new treatment options. Different types of measures for ultrasound phenotypes of atherosclerosis have been proposed. Here, we report on the development and application of a method used to analyze changes in carotid plaque morphology from 3D US images obtained at two different time points. We evaluated our technique using manual segmentations of the wall and lumen of the carotid artery from images acquired in two US scanning sessions. To incorporate the effect of intraobserver variability in our evaluation, manual segmentation was performed five times each for the arterial wall and lumen. From this set of five segmentations, the mean wall and lumen surfaces were reconstructed, with the standard deviation at each point mapped onto the surfaces. A correspondence map between the mean wall and lumen surfaces was then established, and the thickness of the atherosclerotic plaque at each point in the vessel was estimated to be the distance between each correspondence pairs. The two-sample Student's t-test was used to judge whether the difference between the thickness values at each pair corresponding points of the arteries in the two 3D US images was statistically significant.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.