PurposeDeformable image registration establishes non-linear spatial correspondences between fixed and moving images. Deep learning–based deformable registration methods have been widely studied in recent years due to their speed advantage over traditional algorithms as well as their better accuracy. Most existing deep learning–based methods require neural networks to encode location information in their feature maps and predict displacement or deformation fields through convolutional or fully connected layers from these high-dimensional feature maps. We present vector field attention (VFA), a novel framework that enhances the efficiency of the existing network design by enabling direct retrieval of location correspondences.ApproachVFA uses neural networks to extract multi-resolution feature maps from the fixed and moving images and then retrieves pixel-level correspondences based on feature similarity. The retrieval is achieved with a novel attention module without the need for learnable parameters. VFA is trained end-to-end in either a supervised or unsupervised manner.ResultsWe evaluated VFA for intra- and inter-modality registration and unsupervised and semi-supervised registration using public datasets as well as the Learn2Reg challenge. VFA demonstrated comparable or superior registration accuracy compared with several state-of-the-art methods.ConclusionsVFA offers a novel approach to deformable image registration by directly retrieving spatial correspondences from feature maps, leading to improved performance in registration tasks. It holds potential for broader applications.
Managing patients with hydrocephalus and cerebrospinal fluid disorders requires repeated head imaging. In adults, this is typically done with computed tomography (CT) or less commonly magnetic resonance imaging (MRI). However, CT poses cumulative radiation risks and MRI is costly. Transcranial ultrasound is a radiation-free, relatively inexpensive, and optionally point-of-care alternative. The initial use of this modality has involved measuring gross brain ventricle size by manual annotation. In this work, we explore the use of deep learning to automate the segmentation of brain right ventricle from transcranial ultrasound images. We found that the vanilla U-Net architecture encountered difficulties in accurately identifying the right ventricle, which can be attributed to challenges such as limited resolution, artifacts, and noise inherent in ultrasound images. We further explore the use of coordinate convolution to augment the U-Net model, which allows us to take advantage of the established acquisition protocol. This enhancement yielded a statistically significant improvement in performance, as measured by the Dice similarity coefficient. This study presents, for the first time, the potential capabilities of deep learning in automating hydrocephalus assessment from ultrasound imaging.
Deep learning (DL) has led to significant improvements in medical image synthesis, enabling advanced image-toimage translation to generate synthetic images. However, DL methods face challenges such as domain shift and high demands for training data, limiting their generalizability and applicability. Historically, image synthesis was also carried out using deformable image registration (DIR), a method that warps moving images of a desired modality to match the anatomy of a fixed image. However, concerns about its speed and accuracy led to its decline in popularity. With the recent advances of DL-based DIR, we now revisit and reinvigorate this line of research. In this paper, we propose a fast and accurate synthesis method based on DIR. We use the task of synthesizing a rare magnetic resonance (MR) sequence, white matter nulled (WMn) T1-weighted (T1-w) images, to demonstrate the potential of our approach. During training, our method learns a DIR model based on the widely available MPRAGE sequence, which is a cerebrospinal fluid nulled (CSFn) T1-w inversion recovery gradient echo pulse sequence. During testing, the trained DIR model is first applied to estimate the deformation between moving and fixed CSFn images. Subsequently, this estimated deformation is applied to align the paired WMn counterpart of the moving CSFn image, yielding a synthetic WMn image for the fixed CSFn image. Our experiments demonstrate promising results for unsupervised image synthesis using DIR. These findings highlight the potential of our technique in contexts where supervised synthesis methods are constrained by limited training data.
Deep learning algorithms using Magnetic Resonance (MR) images have demonstrated state-of-the-art performance in the automated segmentation of Multiple Sclerosis (MS) lesions. Despite their success, these algorithms may fail to generalize across sites or scanners, leading to domain generalization errors. Few-shot or one-shot domain adaptation is an option to reduce the generalization error using limited labeled data from the target domain. However, this approach may not yield satisfactory performance due to the limited data available for adaptation. In this paper, we aim to address this issue by integrating one-shot adaptation data with harmonized training data that includes labels. Our method synthesizes new training data with a contrast similar to that of the test domain, through a process referred to as “contrast harmonization” in MRI. Our experiments show that combining one-shot adaptation data with harmonized training data outperformed the use of either one of the data sources alone. Domain adaptation using only harmonized training data achieved comparable or even better performance compared to one-shot adaptation. In addition, all adaptations only required light fine-tuning of two to five epochs for convergence.
Traumatic brain injuries (TBIs) are a major health risk that increases with age. Natural brain aging results in cerebral atrophy and the enlargement of the ventricular regions. The objective of this study is to investigate the effect of cerebral atrophy on brain biomechanics with subject-specific models to determine the risk of traumatic brain injury (TBI). Utilizing subjects from a longitudinal study of aging in healthy volunteers, we created subject-specific brain models of a small cohort with progressive age-related cerebral atrophy. We then simulate concussive loading conditions to study changes in brain deformation, a correlate to risk of TBI. The results display differing trends with increasing ventricle volume, with some subjects exhibiting increases and others showing decreasing strain. Additional subject simulations are needed to clarify these the causes of these trends.
Linear registration to a standard space is a crucial early step in the processing of magnetic resonance images (MRIs) of the human brain. Thus an accurate registration is essential for subsequent image processing steps, as well as downstream analyses. Registration failures are not uncommon due to poor image quality, irregular head shapes, and bad initialization. Traditional quality assurance (QA) for registration requires a substantial manual assessment of the registration results. In this paper, we propose an automatic quality assurance method for the rigid registration of brain MRIs. Without using any manual annotations in the model training, our proposed QA method achieved 99.1% sensitivity and 86.7% specificity in a pilot study on 537 T1-weighted scans acquired from multiple imaging centers.
KEYWORDS: Image segmentation, Magnetic resonance imaging, Voxels, Deep learning, Thalamus, White matter, Data modeling, Visualization, Realistic image synthesis
T1-weighted (T1w) magnetic resonance (MR) neuroimages are usually acquired with an inversion time that nulls the cerebrospinal fluid—i.e., CSFn MPRAGE images—but are rarely acquired with the white matter nulled—i.e., WMn images. Since WMn images can be useful in highlighting thalamic nuclei, we develop a method to synthesize these images from other images that are often acquired. We propose a two-part model, with a deep learning based encoder and a decoder based on an imaging equation which governs the acquisition of our T1w images. This model can be trained on a subset of the dataset where the WMn MPRAGE images are available. Our model takes image contrasts that are often acquired (e.g., CSFn MPRAGE) as input, and generates WMn MPRAGE images as output, along with two quantitative parameter maps as intermediate results. After training, our model is able to generate a synthetic WMn MPRAGE image for any given subject. Our model results have high signal-to-noise ratio and are visually almost identical to the ground truth images. Furthermore, downstream thalamic nuclei segmentation on synthetic WMn MPRAGE images are consistent with ground truth WMn MPRAGE images.
Image quality control (IQC) can be used in automated magnetic resonance (MR) image analysis to exclude erroneous results caused by poorly acquired or artifact-laden images. Existing IQC methods for MR imaging generally require human effort to craft meaningful features or label large datasets for supervised training. The involvement of human labor can be burdensome and biased, as labeling MR images based on their quality is a subjective task. In this paper, we propose an automatic IQC method that evaluates the extent of artifacts in MR images without supervision. In particular, we design an artifact encoding network that learns representations of artifacts based on contrastive learning. We then use a normalizing flow to estimate the density of learned representations for unsupervised classification. Our experiments on large-scale multi-cohort MR datasets show that the proposed method accurately detects images with high levels of artifacts, which can inform downstream analysis tasks about potentially flawed data.
The cranial meninges are membranes enveloping the brain. The space between these membranes contains mainly cerebrospinal fluid. It is of interest to study how the volumes of this space change with respect to normal aging. In this work, we propose to combine convolutional neural networks (CNNs) with nested topology-preserving geometric deformable models (NTGDMs) to reconstruct meningeal surfaces from magnetic resonance (MR) images. We first use CNNs to predict implicit representations of these surfaces then refine them with NTGDMs to achieve sub-voxel accuracy while maintaining spherical topology and the correct anatomical ordering. MR contrast harmonization is used to match the contrasts between training and testing images. We applied our algorithm to a subset of healthy subjects from the Baltimore Longitudinal Study of Aging for demonstration purposes and conducted longitudinal statistical analysis of the intracranial volume (ICV) and subarachnoid space (SAS) volume. We found a statistically significant decrease in the ICV and an increase in the SAS volume with respect to normal aging.
The development of automatic whole brain segmentation algorithms has greatly facilitated large-scale multi cohort magnetic resonance (MR) image analyses in recent years. However, the performance of these segmentation algorithms is often affected by image contrast due to the variations in pulse sequences, acquisitions parameters, and manufacturers. Quantitatively evaluating segmentation algorithms on different image contrasts is challenging because manual delineations of the human brain are usually limited. In this study, we tackle the problem by synthesizing new contrast MR images from a small set of images with manual delineations. We quantitatively evaluate the current state-of-the-art whole brain segmentation algorithm, SLANT, on various MR image contrasts. Based on 50 manually delineated T1-weighted MR images acquired from a single site, we synthesize new contrast images using a deep learning-based harmonization algorithm. Two types of contrast synthesis were conducted to simulate both intra- and inter-site contrast variability in MR imaging. SLANT performance is measured using the Dice similarity coefficient (DSC). Experiments show that the average DSC of SLANT varies with image contrast. We also demonstrate the preferred and the least preferred contrast of SLANT based on 11 real MR imaging sites.
Medical image segmentation is one of the core tasks of medical image analysis. Automatic segmentation of brain magnetic resonance images (MRIs) can be used to visualize and track changes of the brain’s anatomical structures that may occur due to normal aging or disease. Machine learning techniques are widely used in automatic structure segmentation. However, the contrast variation between the training and testing data makes it difficult for segmentation algorithms to generate consistent results. To address this problem, an image–to– image translation technique called MR image harmonization can be used to match the contrast between different data sets. It is important for the harmonization to transform image intensity while maintaining the underlying anatomy. In this paper, we present a 3D U-Net algorithm to segment the thalamus from multiple MR image modalities and investigate the impact of harmonization on the segmentation algorithm. Manual delineations of thalamic nuclei on two data sets are available. However, we aim to analyze the thalamus in another large data set where ground truth labels are lacking. We trained two segmentation networks, one with unharmonized images and the other with harmonized images, on one data set with manual labels, and compared their performances on the other data set with manual labels. These two data groups were diagnosed with two brain disorders and were acquired with similar imaging protocols. The harmonization target is the large data set without manual labels, which also has a different imaging protocol. The networks trained on unharmonized and harmonized data showed no significant difference when evaluating on the other data set; demonstrating that image harmonization can maintain the anatomy and does not affect the segmentation task. The two networks were evaluated on the harmonization target data set and the network trained on harmonized data showed significant improvement over the network trained on unharmonized data. Therefore, the network trained on harmonized data provides the potential to process large amounts of data from other sites, even in the absence of site-specific training data.
Deep learning approaches have been used extensively for medical image segmentation tasks. Training deep networks for segmentation, however, typically requires manually delineated examples which provide a ground truth for optimization of the network. In this work, we present a neural network architecture that segments vascular structures in retinal OCTA images without the need of direct supervision. Instead, we propose a variational intensity cross channel encoder that finds vessel masks by exploiting the common underlying structure shared by two OCTA images of the the same region but acquired on different devices. Experimental results demonstrate significant improvement over three existing methods that are commonly used.
Automatic and accurate cerebellum parcellation has long been a challenging task due to the relative surface complexity and large anatomical variation of the human cerebellum. An inaccurate segmentation will inevitably bias further studies. In this paper we present an automatic approach for the quality control of cerebellum parcellation based on shape analysis in a hierarchical structure. We assume that the overall shape variation of a segmented structure comes from both population and segmentation variation. In this hierarchical structure, the higher level shape mainly captures the population variation of the human cerebellum, while the lower level shape captures both population and segmentation variation. We use a partial least squares regression to combine the lower level and higher level shape information. By compensating for population variation, we show that the estimated segmentation variation is highly correlated with the accuracy of the cerebellum parcellation results, which not only provides a confidence measurement of the cerebellum parcellation, but also gives some clues about when a segmentation software may fail in real scenarios.
The cerebellum plays an important role in both motor control and cognitive functions. Several methods to automatically segment different regions of the cerebellum have been recently proposed. Usually, the performance of the segmentation algorithms is evaluated by comparing with expert delineations. However, this is a laboratory approach and is not applicable in real scenarios where expert delineations are not available. In this paper, we propose a method that can automatically detect cerebellar lobule segmentation outliers. Instead of only evaluating the final segmentation result, the intermediate output of each segmentation step is evaluated and considered using a Hidden Markov Model (HMM) to produce a global segmentation assessment. For each intermediate step, a state-of-the-art image classification model Bag-of-Words" (BoW) is applied to quantize features of segmentation results, which then serves as observations of the trained HMM. Experiments show that the proposed method achieves both a high accuracy on predicting Dice of upcoming segmentation steps, and a high sensitivity to outlier detection.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.