The rapid evolution of deep generative models (DGMs) has highlighted their great potential in medical imaging research. Recently, it has been claimed that a diffusion generative model: denoising diffusion probabilistic model (DDPM), performs better at image synthesis than the previously popular DGMs: generative adversarial networks (GANs). However, this claim is based on evaluations employing measures intended for natural images, and thus, does not resolve questions about their relevance to medical imaging tasks. To partially address this problem, we performed a series of assessments to evaluate the ability of a DDPM to reproduce diagnostically relevant spatial context. Our findings show that in all our studies, although context was generally well replicated in DDPM-generated ensembles, it was never perfectly reproduced in the entire ensemble.
In domains such as biomedical imaging, the evaluation of deep generative models (DGMs) for image-to-image translation tasks is additionally challenged by the need for substantial domain expertise, even for visual evaluation. To partially circumvent this problem, we propose a data-driven, human interpretable method to evaluate image-conditioned DGMs for the reproducibility of domain-relevant spatial context before the DGMs are considered for diagnostic tasks and real-world deployment.
KEYWORDS: Signal detection, Image quality, Data modeling, Stochastic processes, Medical imaging, Medical statistics, Mammography, Image analysis, Realistic image synthesis
Modern generative models, such as generative adversarial networks (GANs), hold tremendous promise for several applications in medical imaging that include unconditional medical image synthesis, image translation, and optimization of imaging systems. However, the extent to which a GAN learns image statistics that are relevant to a diagnostic task is unknown. In this work, canonical stochastic image models (SIMs) that simulate realistic mammographic textures are employed to evaluate GAN-based SIMs with respect to detection, detection-localization, and detection-estimation tasks. It is shown that the specific GAN architecture considered has higher propensity to generate statistics that confound the observers performing the three considered tasks. This work highlights the need for continued development of objective metrics for evaluating GANs.
KEYWORDS: Machine learning, Medical imaging, Aneurysms, Computer simulations, Data modeling, Stochastic processes, Angiography, Visualization, Visual process modeling, Systems modeling
Modern generative models, such as generative adversarial networks (GANs), hold tremendous promise for several areas of medical imaging, such as unconditional medical image synthesis, image restoration, reconstruction and translation, and optimization of imaging systems. However, procedures for establishing stochastic image models (SIMs) using GANs remain generic and do not address specific issues relevant to medical imaging. In this work, canonical SIMs that simulate realistic vessels in angiography images are employed to evaluate procedures for establishing SIMs using GANs. The GAN-based SIM is compared to the canonical SIM based on its ability to reproduce those statistics that are meaningful to the particular medically realistic SIM considered. It is shown that evaluating GANs using classical metrics and medically relevant metrics may lead to different conclusions about the fidelity of the trained GANs. This work highlights the need for the development of objective metrics for evaluating GANs.
Given the recent interest in the role of deep generative models (DGM) in medical imaging pipelines, it is imperative to evaluate the capacity of such models to generate medically accurate images. Popular methods of evaluation of natural images generated using generative adversarial networks (GANs), a type of DGM, are often applied to medical data. Such methods are insufficient to evaluate anatomical realism, representations of which include high-order spatial information. To our knowledge, no test exists for the faithful replication of spatial statistics beyond the second-order. In this work, purposefully designed stochastic object models (SOMs) are proposed to encode predetermined rules governing the prevalence of features within single images, thus encoding known high-order spatial information within each realization. These SOMs are independent of the network architecture being tested and can also be applied to any new architecture that may be proposed. Two popular GANs are trained on these SOM datasets and the generated images are tested for the encoded statistics. It is observed that although ensemble statistics might be well replicated, this is not necessarily true for realization i.e., per-image statistics. Thus, GAN-generated images might not be ready for clinical use. With the proposed SOMs, the rate of image errors and the rate of feature malformation can be quantified for any architecture, while providing one measure of GAN utility in a diagnostic scenario.
KEYWORDS: Breast, Tissue optics, 3D image processing, Signal attenuation, Optoacoustics, Blood, 3D image reconstruction, Blood vessels, Imaging systems, Breast imaging
Significance: In three-dimensional (3D) functional optoacoustic tomography (OAT), wavelength-dependent optical attenuation and nonuniform incident optical fluence limit imaging depth and field of view and can hinder accurate estimation of functional quantities, such as the vascular blood oxygenation. These limitations hinder OAT of large objects, such as a human female breast.
Aim: We aim to develop a measurement-data-driven method for normalization of the optical fluence distribution and to investigate blood vasculature detectability and accuracy for estimating vascular blood oxygenation.
Approach: The proposed method is based on reasonable assumptions regarding breast anatomy and optical properties. The nonuniform incident optical fluence is estimated based on the illumination geometry in the OAT system, and the depth-dependent optical attenuation is approximated using Beer–Lambert law.
Results: Numerical studies demonstrated that the proposed method significantly enhanced blood vessel detectability and improved estimation accuracy of the vascular blood oxygenation from multiwavelength OAT measurements, compared with direct application of spectral linear unmixing without optical fluence compensation. Experimental results showed that the proposed method revealed previously invisible structures in regions deeper than 15 mm and/or near the chest wall.
Conclusions: The proposed method provides a straightforward and computationally inexpensive approximation of wavelength-dependent effective optical attenuation and, thus, enables mitigation of the spectral coloring effect in functional 3D OAT imaging.
KEYWORDS: Imaging systems, Medical imaging, Gallium nitride, Stochastic processes, Magnetic resonance imaging, Signal to noise ratio, Statistical modeling, Data acquisition, Systems modeling, Visualization
Purpose: To objectively assess new medical imaging technologies via computer-simulations, it is important to account for the variability in the ensemble of objects to be imaged. This source of variability can be described by stochastic object models (SOMs). It is generally desirable to establish SOMs from experimental imaging measurements acquired by use of a well-characterized imaging system, but this task has remained challenging.
Approach: A generative adversarial network (GAN)-based method that employs AmbientGANs with modern progressive or multiresolution training approaches is proposed. AmbientGANs established using the proposed training procedure are systematically validated in a controlled way using computer-simulated magnetic resonance imaging (MRI) data corresponding to a stylized imaging system. Emulated single-coil experimental MRI data are also employed to demonstrate the methods under less stylized conditions.
Results: The proposed AmbientGAN method can generate clean images when the imaging measurements are contaminated by measurement noise. When the imaging measurement data are incomplete, the proposed AmbientGAN can reliably learn the distribution of the measurement components of the objects.
Conclusions: Both visual examinations and quantitative analyses, including task-specific validations using the Hotelling observer, demonstrated that the proposed AmbientGAN method holds promise to establish realistic SOMs from imaging measurements.
The goal of quantitative optoacoustic tomography (qOAT) is to reconstruct a distribution of absolute chromophore concentrations and/or functional properties from measurements of the optically induced pressure (ultrasound signals) acquired at multiple excitation wavelengths. Estimating the distribution of hemoglobin, an endogenous OAT chromophore, is important because the oxygen saturation distribution of the blood vessels is a well-known indicator of aggressive growth of a cancerous tumor. In a number of studies, a spectral linear unmixing method has been applied to two-dimensional slices of tissue acquired with OAT at multiple wavelengths, leading to promising results at moderate penetration depths of ≤ 2 cm. In the three-dimensional (3D) OAT of the breast, such functional images cannot be accurately reconstructed via the spectral linear unmixing method due to unknown spatial distribution of the optical fluence in a relatively large size of the volume of interest (≥ 4 cm). Optical attenuation in biological tissue depends on the optical wavelength, and the optical fluence is exponentially attenuated with increasing imaging depth. Thus, the accuracy of the estimated distribution decreases with depth. To overcome this challenge, we investigated a spectral linear unmixing method with a simplified optical fluence normalization based on measurements of background absorbed optical energy in the breast. We compare estimates of blood oxygen saturations from two-wavelength clinical OAT breast images and demonstrate acceptable accuracy of ~10% while lack of compensation for the optical fluence distribution can lead to values outside the physiological range. We also quantitatively compare the accuracy of oxygen saturation estimates using numerical simulation of photon transport in realistic 3D OAT breast phantoms at dual wavelengths of 757 and 850 nm with inverse ratio of the optical absorption by deoxy- (Hb) and oxy-hemoglobin (HbO2) and three wavelengths of 757, 800, and 850 nm with inclusion of isosbestic point of the optical absorption in Hb/HbO2.
KEYWORDS: Systems modeling, Stochastic processes, Imaging systems, 3D image processing, Gallium nitride, 3D metrology, Stereoscopy, Network architectures, Medical imaging, Magnetic resonance imaging
Medical imaging systems are commonly assessed and optimized by use of objective-measures of image quality (IQ) that quantify the performance of an observer at specific tasks. Variation in the objects to-be-imaged is an important source of variability that can significantly limit observer performance. This object variability can be described by stochastic object models (SOMs). In order to establish SOMs that can accurately model realistic object variability, it is desirable to use experimental data. To achieve this, an augmented generative adversarial network (GAN) architecture called AmbientGAN has been developed and investigated. However, AmbientGANs cannot be immediately trained by use of advanced GAN training methods such as the progressive growing of GANs (ProGANs). Therefore, the ability of AmbientGANs to establish realistic object models is limited. To circumvent this, a progressively-growing AmbientGAN (ProAmGAN) has been proposed. However, ProAmGANs are designed for generating two-dimensional (2D) images while medical imaging modalities are commonly employed for imaging three-dimensional (3D) objects. Moreover, ProAmGANs that employ traditional generator architectures lack the ability to control specific image features such as fine-scale textures that are frequently considered when optimizing imaging systems. In this study, we address these limitations by proposing two advanced AmbientGAN architectures: 3D ProAmGANs and Style-AmbientGANs (StyAmGANs). Stylized numerical studies involving magnetic resonance (MR) imaging systems are conducted. The ability of 3D ProAmGANs to learn 3D SOMs from imaging measurements and the ability of StyAmGANs to control fine-scale textures of synthesized objects are demonstrated.
Tomographic imaging is an ill-posed linear inverse problem, and is often regularized using prior knowledge of the sought-after object property. However, typical hand-crafted priors such as sparsity-promoting penalties may be insufficient to comprehensively describe the prior knowledge of the object to-be-imaged. In order to utilize more detailed prior knowledge, data-driven methods using deep neural networks have recently been explored for learning a prior from existing image data. However, an analysis of the ability of such data-driven methods to generalize to data that may lie outside the training distribution is still under investigation. This is particularly critical for medical imaging applications. In order to address such concerns, in this work we propose to understand the effect of the prior imposed by a reconstruction method by comparing the null components of the sought-after object and its reconstructed estimate, when ground truth objects are available. The concept of a hallucination map is introduced for the purpose of assessing non-data-driven and data-driven regularization for image reconstruction. Numerical studies were conducted using stylized undersampled k-space measurements from publicly available magnetic resonance imaging (MRI) datasets. It is demonstrated that the proposed method can be employed to identify the source of false structures in estimates of the sought-after object for a given reconstruction method.
The objective optimization of medical imaging systems requires full characterization of all sources of randomness in the measured data, which includes the variability within the ensemble of objects to-be-imaged. This can be accomplished by establishing a stochastic object model (SOM) that describes the variability in the class of objects to-be-imaged. Generative adversarial networks (GANs) can be potentially useful to establish SOMs because they hold great promise to learn generative models that describe the variability within an ensemble of training data. However, because medical imaging systems record imaging measurements that are noisy and indirect representations of object properties, GANs cannot be directly applied to establish stochastic models of objects to-be-imaged. To address this issue, an augmented GAN architecture named AmbientGAN was developed to establish SOMs from noisy and indirect measurement data. However, because the adversarial training can be unstable, the applicability of the AmbientGAN can be potentially limited. In this work, we propose a novel training strategy|Progressive Growing of AmbientGANs (ProAGAN)|to stabilize the training of AmbientGANs for establishing SOMs from noisy and indirect imaging measurements. An idealized magnetic resonance (MR) imaging system and clinical MR brain images are considered. The proposed methodology is evaluated by comparing signal detection performance computed by use of ProAGAN-generated synthetic images and images that depict the true object properties.
Due to a demonstrated capability to assess tumor angiogenesis and hypoxia in mammalian systems, there is great interest in applying optoacoustic tomography (OAT) to the study and screening of breast cancer. In order to translate OAT to clinical applications, in silico studies are crucial for studying imaging system parameters that might be impossible to assess via direct experimentation. Previous numerical phantoms have proven to be too unrealistic for rigorous testing of modern image reconstruction methods and clinically relevant signal detection tasks. Recently, the U.S. Food and Drug Administration has released software to generate realistic three-dimensional numerical realizations of the human female breast as part of the Virtual Imaging Clinical Trials for Regulatory Evaluation (VICTRE) project. By careful selection of physical attributes and material coefficients, the VICTRE breast phantom can be customized for particular imaging tasks, but no such customization has been given for OAT. We propose a general framework of in silico studies for OAT breast imaging using the VICTRE breast phantom. We will create an ensemble of OAT breast phantoms, using appropriate optical and acoustic parameters, that have typical sizes and tissue densities. Various lesions will be created and embedded based on clinical scenarios. We will define and perform several signal detection tasks by which the system performance may be compared. Generation of such an ensemble requires substantial computation but once produced, it can be utilized in other numerical simulation studies of the configuration of OAT imaging systems customized for diverse tasks. We will make this ensemble of phantoms publicly available online. The proposed framework will permit standardization of the assessment of 3D OAT data-acquisition parameters and image reconstruction methods.
The objective optimization of image-derived statistics, including the test statistic of an observer for specific decision tasks, requires a characterization of all sources of variability in the measured data. To accomplish this, it is necessary to establish a stochastic object model (SOM) that describes the variability within a group of objects to-be imaged. In order for the SOM to be realistic, it is desirable to establish it by use of experimental image data, as opposed to establishing it in a non-data-driven manner. Deep learning methods that employ generative adversarial networks (GANs) hold promise for learning SOMs that can generate images that match distributions of training image data. However, because experimental data recorded by an imaging system represent noisy and indirect measurements of the object, conventional GANs cannot be directly employed for this task. Recently, an augmented GAN architecture named AmbientGAN was proposed that can characterize a distribution of images from noisy and indirect measurements of them and knowledge of the measurement operator. In this work, for the first time, we investigate AmbientGANs for establishing SOMs by use of noisy imaging measurements. A canonical tomographic imaging system that is described by a two-dimensional Radon transform model is investigated. The AmbientGAN is evaluated by performing binary signal detection tasks that employ the generated images and true images.
X-ray phase-contrast imaging methods exploit variations in an object’s 3D refractive index distribution to form projection or volumetric images of weakly absorbing objects. Such techniques can resolve subtle tissue structures by employing coherent imaging principles, but retain the ability of traditional (incoherent) X-ray methods to image deep into tissue. In this talk, we describe recent advancements in image formation methods for benchtop applications of X-ray phase-contrast imaging and tomography and present applications of pre-clinical in vivo imaging.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.