PurposeDeformable image registration establishes non-linear spatial correspondences between fixed and moving images. Deep learning–based deformable registration methods have been widely studied in recent years due to their speed advantage over traditional algorithms as well as their better accuracy. Most existing deep learning–based methods require neural networks to encode location information in their feature maps and predict displacement or deformation fields through convolutional or fully connected layers from these high-dimensional feature maps. We present vector field attention (VFA), a novel framework that enhances the efficiency of the existing network design by enabling direct retrieval of location correspondences.ApproachVFA uses neural networks to extract multi-resolution feature maps from the fixed and moving images and then retrieves pixel-level correspondences based on feature similarity. The retrieval is achieved with a novel attention module without the need for learnable parameters. VFA is trained end-to-end in either a supervised or unsupervised manner.ResultsWe evaluated VFA for intra- and inter-modality registration and unsupervised and semi-supervised registration using public datasets as well as the Learn2Reg challenge. VFA demonstrated comparable or superior registration accuracy compared with several state-of-the-art methods.ConclusionsVFA offers a novel approach to deformable image registration by directly retrieving spatial correspondences from feature maps, leading to improved performance in registration tasks. It holds potential for broader applications.
Managing patients with hydrocephalus and cerebrospinal fluid disorders requires repeated head imaging. In adults, this is typically done with computed tomography (CT) or less commonly magnetic resonance imaging (MRI). However, CT poses cumulative radiation risks and MRI is costly. Transcranial ultrasound is a radiation-free, relatively inexpensive, and optionally point-of-care alternative. The initial use of this modality has involved measuring gross brain ventricle size by manual annotation. In this work, we explore the use of deep learning to automate the segmentation of brain right ventricle from transcranial ultrasound images. We found that the vanilla U-Net architecture encountered difficulties in accurately identifying the right ventricle, which can be attributed to challenges such as limited resolution, artifacts, and noise inherent in ultrasound images. We further explore the use of coordinate convolution to augment the U-Net model, which allows us to take advantage of the established acquisition protocol. This enhancement yielded a statistically significant improvement in performance, as measured by the Dice similarity coefficient. This study presents, for the first time, the potential capabilities of deep learning in automating hydrocephalus assessment from ultrasound imaging.
Deep learning (DL) has led to significant improvements in medical image synthesis, enabling advanced image-toimage translation to generate synthetic images. However, DL methods face challenges such as domain shift and high demands for training data, limiting their generalizability and applicability. Historically, image synthesis was also carried out using deformable image registration (DIR), a method that warps moving images of a desired modality to match the anatomy of a fixed image. However, concerns about its speed and accuracy led to its decline in popularity. With the recent advances of DL-based DIR, we now revisit and reinvigorate this line of research. In this paper, we propose a fast and accurate synthesis method based on DIR. We use the task of synthesizing a rare magnetic resonance (MR) sequence, white matter nulled (WMn) T1-weighted (T1-w) images, to demonstrate the potential of our approach. During training, our method learns a DIR model based on the widely available MPRAGE sequence, which is a cerebrospinal fluid nulled (CSFn) T1-w inversion recovery gradient echo pulse sequence. During testing, the trained DIR model is first applied to estimate the deformation between moving and fixed CSFn images. Subsequently, this estimated deformation is applied to align the paired WMn counterpart of the moving CSFn image, yielding a synthetic WMn image for the fixed CSFn image. Our experiments demonstrate promising results for unsupervised image synthesis using DIR. These findings highlight the potential of our technique in contexts where supervised synthesis methods are constrained by limited training data.
The Segment Anything Model (SAM) has drawn significant attention from researchers who work on medical image segmentation because of its generalizability. However, researchers have found that SAM may have limited performance on medical images compared to state-of-the-art non-foundation models. Regardless, the community sees potential in extending, fine-tuning, modifying, and evaluating SAM for analysis of medical imaging. An increasing number of works have been published focusing on the mentioned four directions, where variants of SAM are proposed. To this end, a unified platform helps push the boundary of the foundation model for medical images, facilitating the use, modification, and validation of SAM and its variants in medical image segmentation. In this work, we introduce SAMM Extended (SAMME), a platform that integrates new SAM variant models, adopts faster communication protocols, accommodates new interactive modes, and allows for fine-tuning of subcomponents of the models. These features can expand the potential of foundation models like SAM, and the results can be translated to applications such as image-guided therapy, mixed reality interaction, robotic navigation, and data augmentation.
Linear registration to a standard space is a crucial early step in the processing of magnetic resonance images (MRIs) of the human brain. Thus an accurate registration is essential for subsequent image processing steps, as well as downstream analyses. Registration failures are not uncommon due to poor image quality, irregular head shapes, and bad initialization. Traditional quality assurance (QA) for registration requires a substantial manual assessment of the registration results. In this paper, we propose an automatic quality assurance method for the rigid registration of brain MRIs. Without using any manual annotations in the model training, our proposed QA method achieved 99.1% sensitivity and 86.7% specificity in a pilot study on 537 T1-weighted scans acquired from multiple imaging centers.
Image quality control (IQC) can be used in automated magnetic resonance (MR) image analysis to exclude erroneous results caused by poorly acquired or artifact-laden images. Existing IQC methods for MR imaging generally require human effort to craft meaningful features or label large datasets for supervised training. The involvement of human labor can be burdensome and biased, as labeling MR images based on their quality is a subjective task. In this paper, we propose an automatic IQC method that evaluates the extent of artifacts in MR images without supervision. In particular, we design an artifact encoding network that learns representations of artifacts based on contrastive learning. We then use a normalizing flow to estimate the density of learned representations for unsupervised classification. Our experiments on large-scale multi-cohort MR datasets show that the proposed method accurately detects images with high levels of artifacts, which can inform downstream analysis tasks about potentially flawed data.
The cranial meninges are membranes enveloping the brain. The space between these membranes contains mainly cerebrospinal fluid. It is of interest to study how the volumes of this space change with respect to normal aging. In this work, we propose to combine convolutional neural networks (CNNs) with nested topology-preserving geometric deformable models (NTGDMs) to reconstruct meningeal surfaces from magnetic resonance (MR) images. We first use CNNs to predict implicit representations of these surfaces then refine them with NTGDMs to achieve sub-voxel accuracy while maintaining spherical topology and the correct anatomical ordering. MR contrast harmonization is used to match the contrasts between training and testing images. We applied our algorithm to a subset of healthy subjects from the Baltimore Longitudinal Study of Aging for demonstration purposes and conducted longitudinal statistical analysis of the intracranial volume (ICV) and subarachnoid space (SAS) volume. We found a statistically significant decrease in the ICV and an increase in the SAS volume with respect to normal aging.
Transcranial Magnetic Stimulation (TMS) is a neurostimulation technique based on the principle of electromagnetic induction of an electric field in the brain with both research and clinical applications. To produce an optimal neuro-modulatory effect, TMS coil must be placed on the head and oriented accurately with respect to the region of interest within the brain. A robotic method can enhance the accuracy and facilitate the procedure for TMS coil placement. This work presents two system improvements for robot-assisted TMS (RA-TMS) application. Previous systems have used outside-in tracking method where a stationary external infrared (IR) tracker is used as a reference point to track the head and TMS coil positions. This method is prone to losing track of the coil or the head if the IR camera is blocked by the robotic arm during its motion. To address this issue, we implemented an inside-out tracking method by mounting a portable IR camera on the robot end-effector. This method guarantees that the line of sight of the IR camera is not obscured by the robotic arm at any time during its motion. We also integrated a portable projection mapping device (PPMD) into the RA-TMS system to provide visual guidance during TMS application. PPMD can track the head via an IR tracker, and can project a planned contact point of TMS coil on the head or overlay the underlying brain anatomy in real-time.
KEYWORDS: Super resolution, Magnetism, Magnetic resonance imaging, Lawrencium, Image resolution, Image segmentation, Signal to noise ratio, Performance modeling, Medical imaging
Robust and accurate segmentation results from high resolution (HR) 3D Magnetic Resonance (MR) images are desirable in many clinical applications. State-of-the-art deep learning methods for image segmentation require external HR atlas image and label pairs for training. However, the availability of such HR labels is limited due to the annotation accuracy and the time required to manually label. In this paper, we propose a 3D label super resolution (LSR) method which does not use an external image or label from a HR atlas data and can reconstruct HR annotation labels only reliant on a LR image and corresponding label pairs. In our method, we present a Deformable U-net, which uses synthetic data with multiple deformation for training and an iterative topology check during testing, to learn a label slice evolving process. This network requires no external HR data because a deformed version of the input label slice acquired from the LR data itself is used for training. The trained Deformable U-net is then applied to through-plane slices to estimate HR label slices. The estimated HR label slices are further combined by label a fusion method to obtain the 3D HR label. Our results show significant improvement compared to competing methods, in both 2D and 3D scenarios with real data.
Deep learning approaches have been used extensively for medical image segmentation tasks. Training deep networks for segmentation, however, typically requires manually delineated examples which provide a ground truth for optimization of the network. In this work, we present a neural network architecture that segments vascular structures in retinal OCTA images without the need of direct supervision. Instead, we propose a variational intensity cross channel encoder that finds vessel masks by exploiting the common underlying structure shared by two OCTA images of the the same region but acquired on different devices. Experimental results demonstrate significant improvement over three existing methods that are commonly used.
Monitoring retinal thickness of persons with multiple sclerosis (MS) provides important bio-markers for disease progression. However, changes in retinal thickness can be small and concealed by noise in the acquired data. Consistent longitudinal retinal layer segmentation methods for optical coherence tomography (OCT) images are crucial for identifying the real longitudinal retinal changes of individuals with MS. In this paper, we propose an iterative registration and deep learning based segmentation method for longitudinal 3D OCT scans. Since 3D OCT scans are usually anisotropic with large slice separation, we extract B-scan features using 2D deep networks and utilize inter-B-scan context with convolutional long-short-term memory (LSTM). To incorporate longitudinal information, we perform fundus registration and interpolate the smooth retinal surfaces of the previous visit to use as a prior on the current visit.
Deep networks provide excellent image segmentation results given copious amounts of supervised training data (source data). However, when a trained network is applied to data acquired at a different clinical center or on a different imaging device (target data), a significant drop in performance can occur due to the domain shift between the test data and the network training data. To solve this problem, unsupervised domain adaptation methods retrain the model with labeled source data and unlabeled target data. In real practice, retraining the model is time consuming and the labeled source data may not be available for people deploying the model. In this paper, we propose a straightforward unsupervised domain adaptation method for multi-device retinal OCT image segmentation which does not require labeled source data and does not require retraining of the segmentation model. The segmentation network is trained with labeled Spectralis images and tested on Cirrus images. The core idea is to use a domain adaptor to convert target domain images (Cirrus) to a domain that can be segmented well by the already trained segmentation network. Unlabeled Spectralis and Cirrus images are used to train this domain adaptor. The domain adaptation block is used before the trained network and a discriminator is used to differentiate the segmentation results from Spectralis and Cirrus. The domain adaptation portion of our network is fully unsupervised and does not change the previously trained segmentation network.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.