This paper proposes an intestine segmentation method to segment intestines from CT volumes for helping clinicians diagnose intestine obstruction. For large-scale labeled datasets, fully-supervised methods have shown superior results. However, medical image segmentation is usually difficult to achieve accurate prediction due to the limited number of labeled data available for training. To address this challenge, we introduce a novel multi-view symmetrical network (MVS-Net) for intestine segmentation and incorporate bidirectional teaching to utilize unlabeled datasets. Specifically, we design the MVS-Net, which can use different sizes of convolution kernels instead of a fixed kernel size, enabling the network to capture multi-scale features from images’ different perceptual fields and ensure segmentation accuracy. Additionally, the pseudo-labels are generated by bidirectional teaching, which can make the network captures semantic information from large-scale unlabeled data for increasing the training data. We repeated the experiment five times, and used the averaged result on the intestines dataset to represent the segmentation accuracy of the proposed method. The experimental results showed the average Dice was 78.86%, the average recall 84.50%, and the average precision 75.94%, respectively.
Medical image analysis approaches such as data augmentation and domain adaption need huge amounts of realistic medical images. Generating realistic medical images by machine learning is a feasible approach. We propose L-former, a lightweight Transformer for realistic medical image generation. L-former can generate more reliable and realistic medical images than recent generative adversarial networks (GANs). Meanwhile, L-former does not consume as high computational cost as conventional Transformer-based generative models. L-former uses Transformers to generate low-resolution feature vectors at shallow layers, and uses convolutional neural networks to generate high-resolution realistic medical images at deep layers. Experimental results showed that L-former outperformed conventional GANs by FID scores 33.79 and 76.85 on two datasets, respectively. We further conducted a downstream study by using the images generated by L-former to perform a super-resolution task. A high PSNR score of 27.87 proved L-former’s ability to generate reliable images for super-resolution and showed its potential for applications in medical diagnosis.
Purpose: We propose a super-resolution (SR) method, named SR-CycleGAN, for SR of clinical computed tomography (CT) images to the micro-focus x-ray CT CT (μCT) level. Due to the resolution limitations of clinical CT (about 500 × 500 × 500 μm3 / voxel), it is challenging to obtain enough pathological information. On the other hand, μCT scanning allows the imaging of lung specimens with significantly higher resolution (about 50 × 50 × 50 μm3 / voxel or higher), which allows us to obtain and analyze detailed anatomical information. As a way to obtain detailed information such as cancer invasion and bronchioles from preoperative clinical CT images of lung cancer patients, the SR of clinical CT images to the μCT level is desired.
Approach: Typical SR methods require aligned pairs of low-resolution (LR) and high-resolution images for training, but it is infeasible to obtain precisely aligned paired clinical CT and μCT images. To solve this problem, we propose an unpaired SR approach that can perform SR on clinical CT to the μCT level. We modify a conventional image-to-image translation network named CycleGAN to an inter-modality translation network named SR-CycleGAN. The modifications consist of three parts: (1) an innovative loss function named multi-modality super-resolution loss, (2) optimized SR network structures for enlarging the input LR image to k2-times by width and height to obtain the SR output, and (3) sub-pixel shuffling layers for reducing computing time.
Results: Experimental results demonstrated that our method successfully performed SR of lung clinical CT images. SSIM and PSNR scores of our method were 0.54 and 17.71, higher than the conventional CycleGAN’s scores of 0.05 and 13.64, respectively.
Conclusions: The proposed SR-CycleGAN is usable for the SR of a lung clinical CT into μCT scale, while conventional CycleGAN output images with low qualitative and quantitative values. More lung micro-anatomy information could be observed to aid diagnosis, such as the shape of bronchioles walls.
This paper proposes an intestine segmentation method on CT volume based on a multi-class prediction of intestinal content materials (ICMs). The mechanical intestinal obstruction and the ileus (non-mechanical intestinal obstruction) are diseases which disrupt the movement of ICMs. Although clinicians find the obstruction point that movement of intestinal contents is required on CT volumes, it is difficult for non-expert clinicians to find the obstruction point. We have studied a CADe system which presents obstruction candidates to users by segmentation of the intestines on CT volumes. Generation of incorrect shortcuts in segmentation results was partly reduced in our proposed method by introducing distance maps. However, incorrect shortcuts still remained between the regions filled by air. This paper proposes an improved intestine segmentation method from CT volumes. We introduce a multi-class segmentation of ICMs (air, liquid, and feces). Reduction of incorrect shortcut generation is specifically applied to air regions. Experiments using 110 CT volumes showed that our proposed method reduced incorrect shortcuts. Rates of segmented regions that are analyzed as running through the intestine were 59.6% and 62.4% for the previous and proposed methods, respectively. This result partly implies that our proposed method reduced production of incorrect shortcuts.
This paper proposes a super-resolution (SR) method, for performing SR of medical images training on a newly-built lung clinical CT / micro CT dataset. Conventional SR methods are always trained on bicubic downsampled images (LR) / original images (HR) image pairs. However, registration precision between LR and HR images is not satisfying for SR. Low precision of registration results in conventional SR methods’ unsatisfactory performance in medical imaging. We propose a coarse-to-fine cascade framework for performing SR of medical images. First, we design a coarse SR network to translate LR medical images into coarse SR images. Next, we utilize a fully convolutional network (FCN) to perform fine SR (translate coarse SR images to fine SR images). We conducted experiments using a newly-built clinical / micro CT lung specimen dataset. Experimental results illustrated that our method obtained PSNR of 27.30 and SSIM of 0.75, outperforming conventional method’s PSNR 19.08 and SSIM 0.63.
This paper proposes an intestinal region reconstruction method from CT volumes of ileus cases. Binarized intestine segmentation results often contain incorrect contacts or loops. We utilize the 3D U-Net to estimate the distance map, which is high only at the centerlines of the intestines, to obtain regions around the centerlines. Watershed algorithm is utilized with local maximums of the distance maps as seeds for obtaining “intestine segments”. Those intestine segments are connected as graphs, for removing incorrect contacts and loops and to extract “intestine paths”, which represent how intestines are running. Experimental results using 19 CT volumes showed that our proposed method properly estimated intestine paths. These results were intuitively visualized for understanding the shape of the intestines and finding obstructions.
This paper presents a visualization method of intestine (the small and large intestine) regions and their stenosed parts caused by ileus from CT volumes. Since it is difficult for non-expert clinicians to find stenosed parts, the intestine and its stenosed parts should be visualized intuitively. Furthermore, the intestine regions of ileus cases are quite hard to be segmented. The proposed method segments intestine regions by 3D FCN (3D U-Net). Intestine regions are quite difficult to be segmented in ileus cases since the inside the intestine is filled with liquids. These liquids have similar intensities with intestinal wall on 3D CT volumes. We segment the intestine regions by using 3D U-Net trained by a weak annotation approach. Weak-annotation makes possible to train the 3D U-Net with small manually-traced label images of the intestine. This avoids us to prepare many annotation labels of the intestine that has long and winding shape. Each intestine segment is volume-rendered and colored based on the distance from its endpoint in volume rendering. Stenosed parts (disjoint points of an intestine segment) can be easily identified on such visualization. In the experiments, we showed that stenosed parts were intuitively visualized as endpoints of segmented regions, which are colored by red or blue.
Purpose: High-resolution cardiac imaging and fiber analysis methods are required to understand cardiac anatomy. Although refraction-contrast x-ray CT (RCT) has high soft tissue contrast, it cannot be commonly used because it requires a synchrotron system. Microfocus x-ray CT (μCT) is another commercially available imaging modality.
Approach: We evaluate the usefulness of μCT for analyzing fibers by quantitatively and objectively comparing the results with RCT. To do so, we scanned a rabbit heart by both modalities with our original protocol of prepared materials and compared their image-based analysis results, including fiber orientation estimation and fiber tracking.
Results: Fiber orientations estimated by two modalities were closely resembled under the correlation coefficient of 0.63. Tracked fibers from both modalities matched well the anatomical knowledge that fiber orientations are different inside and outside of the left ventricle. However, the μCT volume caused incorrect tracking around the boundaries caused by stitching scanning.
Conclusions: Our experimental results demonstrated that μCT scanning can be used for cardiac fiber analysis, although further investigation is required in the differences of fiber analysis results on RCT and μCT.
This paper newly introduces multi-modality loss function for GAN-based super-resolution that can maintain image structure and intensity on unpaired training dataset of clinical CT and micro CT volumes. Precise non- invasive diagnosis of lung cancer mainly utilizes 3D multidetector computed-tomography (CT) data. On the other hand, we can take μCT images of resected lung specimen in 50 μm or higher resolution. However, μCT scanning cannot be applied to living human imaging. For obtaining highly detailed information such as cancer invasion area from pre-operative clinical CT volumes of lung cancer patients, super-resolution (SR) of clinical CT volumes to μCT level might be one of substitutive solutions. While most SR methods require paired low- and high-resolution images for training, it is infeasible to obtain precisely paired clinical CT and μCT volumes. We aim to propose unpaired SR approaches for clincial CT using micro CT images based on unpaired image translation methods such as CycleGAN or UNIT. Since clinical CT and μCT are very different in structure and intensity, direct appliation of GAN-based unpaired image translation methods in super-resolution tends to generate arbitrary images. Aiming to solve this problem, we propose new loss function called multi-modality loss function to maintain the similarity of input images and corresponding output images in super-resolution task. Experimental results demonstrated that the newly proposed loss function made CycleGAN and UNIT to successfully perform SR of clinical CT images of lung cancer patients into μCT level resolution, while original CycleGAN and UNIT failed in super-resolution.
Micro-CT is a nondestructive scanning device that is capable of capturing three dimensional structures at _m level. With the spread of this device uses in medical fields, it is expected that this device may bring further understanding of the human anatomy by analyzing three-dimensional micro structure from volume of in vivo specimens captured by micro-CT. In the topic of micro structure analysis of lung, the methods for extracting surface structures including the interlobular septa and the visceral pleura were not commonly studied. In this paper, we introduce a method to extract sheet structure such as the interlobular septa and the visceral pleura from micro-CT volumes. The proposed method consists of two steps: Hessian analysis based method for sheet structure extraction and Radial Structure Tensor combined with roundness evaluation for hollow-tube structure extraction. We adopted the proposed method on complex phantom data and a medical lung micro-CT volume. We confirmed the extraction of the interlobular septa from medical volume from experiments.
KEYWORDS: Image segmentation, 3D image processing, Data modeling, Medical imaging, Computed tomography, Image processing, Arteries, Veins, 3D modeling, Network architectures
Segmentation is one of the most important tasks in medical image analysis. With the development of deep leaning, fully convolutional networks (FCNs) have become the dominant approach for this task and their extension to 3D achieved considerable improvements for automated organ segmentation in volumetric imaging data, such as computed tomography (CT). One popular FCN network architecture for 3D volumes is V-Net, originally proposed for single region segmentation. This network effectively solved the imbalance problem between foreground and background voxels by proposing a loss function based on the Dice similarity metric. In this work, we extend the depth of the original V-Net to obtain better features to model the increased complexity of multi-class segmentation tasks at higher input/output resolutions using modern large-memory GPUs. Furthermore, we markedly improved the training behaviour of V-Net by employing batch normalization layers throughout the network. In this way, we can efficiently improve the stability of the training optimization, achieving faster and more stable convergence. We show that our architectural changes and refinements dramatically improve the segmentation performance on a large abdominal CT dataset and obtain close to 90% average Dice score.
This paper presents a novel unsupervised segmentation method for the 3D microstructure in micro-computed tomography (micro-CT) images. Micro-CT scanning of resected lung cancer specimens can capture detailed and surrounding anatomical structures of them. However, its segmentation is difficult. Recently, many unsupervised learning methods have become greatly improved, especially in their ability to learn generative models such as variational auto-encoders (VAEs) and generative adversarial networks (GANs). Meanwhile, however, most of the recent segmentation methods using deep neural networks continue to rely on supervised learning. Therefore, it is rather difficult for these segmentation methods to cope with the growing number of unlabeled micro-CT images. In this paper, we develop a generative model that can infer segmentation labels by extending α-GAN, a principled combination that iterates variational inference and adversarial learning. Our method consists of two phases. In the first phase, we train our model by iterating two steps: (1) inferring pairs of continuous and discrete latent variables of image patches randomly extracted from an unlabeled image and (2) generating image patches from the inferred pairs of latent variables. In the second phase, our trained model assigns labels to patches from a target image in order to obtain the segmented image. We evaluated our method using three micro-CT images of a lung cancer specimen. The aim was to automatically divide each image into three regions: invasive carcinoma, noninvasive carcinoma, and normal tissue. Our experiments show promising results both quantitatively and qualitatively.
High-resolution cardiac imaging and fiber analysis methods are desired for deeper understanding cardiac anatomy. Although refraction-contrast X-ray CT (RCT) has high contrast for soft tissues, its scanning cost is very high. On the other hand, micro-focus X-ray CT (μCT) is a modality that is commercially available with lower cost, but its contrast for soft tissue is not as high as RCT. To investigate the efficacy of μCT for fiber analysis, we scanned a common rabbit heart with both modalities with our original protocol of preparing materials, and compared their image-based analysis results. Their results were very similar, with correlation coefficient of 0.95. We confirmed that µCT volumes prepared by our protocol are useful for fiber analysis as well as RCT.
The purpose of this study was to develop a lung segmentation based on a deep learning approach for dynamic chest radiography, and to assess the clinical utility for pulmonary function assessment. Maximum inhale and exhale images were selected in dynamic chest radiographs of 214 cases, comprising 150 images during respiration. In total, 534 images (2 to 4 images per case) with annotations were prepared for this study. Three hundred images were fed into a fullyconvolutional neural network (FCNN) architecture to train a deep learning model for lung segmentation, and 234 images were used for testing. To reduce misrecognition of the lung, post processing methods on the basis of time-series information were applied to the resulting images. The change rate of the lung area was calculated throughout all frames and its clinical utility was assessed in patients with pulmonary diseases. The Sorenson-Dice coefficients between the segmentation results and the gold standard were 0.94 in inhale and 0.95 in exhale phases, respectively. There were some false recognitions (214/234), but 163 were eliminated by our post processing. The measurement of the lung area and its respiratory change were useful for the evaluation of lung conditions; prolonged expiration in obstructive pulmonary diseases could be detected as a reduced change rate of the lung area in the exhale phase. Semantic segmentation deep learning approach allows for the sequential lung segmentation of dynamic chest radiographs with high accuracy (94%) and is useful for the evaluation of pulmonary function.
This study was performed to investigate the detection performance of trapped air in dynamic chest radiography using 4D extended cardiac-torso (XCAT) phantom with a user-defined ground truth. An XCAT phantom of an adult male (50th percentile in height and weight) with a normal heart rate, slow-forced breathing, and diaphragm motion was generated. An air sphere was inserted into the right lung to simulate emphysema. An X-ray simulator was used to create sequential chest radiographs of the XCAT phantom over a whole respiratory cycle covering a period of 10 seconds. Respiratory changes in pixel value were measured in each grid-like region translating during respiration, and differences from a fully exhaled image were then depicted as color-mapping images, representing higher X-ray translucency (increased air) as higher color intensities. The detection performance was investigated using various sizes of air spheres, for each lung field and behind the diaphragm. In the results, respiratory changes in pixel value were decreased as the size of air sphere increased, depending on the lung fields. In color-mapping images, air spheres were depicted as color defects, however, those behind the diaphragm were not detectable. Smaller size sampling depicted the air spheres as island color defects, while larger ones yielded a limited signal. We confirmed that dynamic chest radiography was able to detect trapped air as regionally-reduced changes in pixel value during respiration. The reduction rate could be defined as a function of residual normal tissue in front and behind air spheres.
KEYWORDS: Image segmentation, Medical imaging, 3D image processing, Machine learning, Tissues, 3D acquisition, Convolution, Lung cancer, Information science
This paper presents a novel unsupervised segmentation method for 3D medical images. Convolutional neural networks (CNNs) have brought significant advances in image segmentation. However, most of the recent methods rely on supervised learning, which requires large amounts of manually annotated data. Thus, it is challenging for these methods to cope with the growing amount of medical images. This paper proposes a unified approach to unsupervised deep representation learning and clustering for segmentation. Our proposed method consists of two phases. In the first phase, we learn deep feature representations of training patches from a target image using joint unsupervised learning (JULE) that alternately clusters representations generated by a CNN and updates the CNN parameters using cluster labels as supervisory signals. We extend JULE to 3D medical images by utilizing 3D convolutions throughout the CNN architecture. In the second phase, we apply k-means to the deep representations from the trained CNN and then project cluster labels to the target image in order to obtain the fully segmented image. We evaluated our methods on three images of lung cancer specimens scanned with micro-computed tomography (micro-CT). The automatic segmentation of pathological regions in micro-CT could further contribute to the pathological examination process. Hence, we aim to automatically divide each image into the regions of invasive carcinoma, noninvasive carcinoma, and normal tissue. Our experiments show the potential abilities of unsupervised deep representation learning for medical image segmentation.
This paper presents a novel method for unsupervised segmentation of pathology images. Staging of lung cancer is a major factor of prognosis. Measuring the maximum dimensions of the invasive component in a pathology images is an essential task. Therefore, image segmentation methods for visualizing the extent of invasive and noninvasive components on pathology images could support pathological examination. However, it is challenging for most of the recent segmentation methods that rely on supervised learning to cope with unlabeled pathology images. In this paper, we propose a unified approach to unsupervised representation learning and clustering for pathology image segmentation. Our method consists of two phases. In the first phase, we learn feature representations of training patches from a target image using the spherical k-means. The purpose of this phase is to obtain cluster centroids which could be used as filters for feature extraction. In the second phase, we apply conventional k-means to the representations extracted by the centroids and then project cluster labels to the target images. We evaluated our methods on pathology images of lung cancer specimen. Our experiments showed that the proposed method outperforms traditional k-means segmentation and the multithreshold Otsu method both quantitatively and qualitatively with an improved normalized mutual information (NMI) score of 0.626 compared to 0.168 and 0.167, respectively. Furthermore, we found that the centroids can be applied to the segmentation of other slices from the same sample.
Pancreas segmentation in computed tomography imaging has been historically difficult for automated methods because of the large shape and size variations between patients. In this work, we describe a custom-build 3D fully convolutional network (FCN) that can process a 3D image including the whole pancreas and produce an automatic segmentation. We investigate two variations of the 3D FCN architecture; one with concatenation and one with summation skip connections to the decoder part of the network. We evaluate our methods on a dataset from a clinical trial with gastric cancer patients, including 147 contrast enhanced abdominal CT scans acquired in the portal venous phase. Using the summation architecture, we achieve an average Dice score of 89.7 ± 3.8 (range [79.8, 94.8])% in testing, achieving the new state-of-the-art performance in pancreas segmentation on this dataset.
We propose a novel mediastinal lymph node detection and segmentation method from chest CT volumes based on fully convolutional networks (FCNs). Most lymph node detection methods are based on filters for blob-like structures, which are not specific for lymph nodes. The 3D U-Net is a recent example of the state-of-the-art 3D FCNs. The 3D U-Net can be trained to learn appearances of lymph nodes in order to output lymph node likelihood maps on input CT volumes. However, it is prone to oversegmentation of each lymph node due to the strong data imbalance between lymph nodes and the remaining part of the CT volumes. To moderate the balance of sizes between the target classes, we train the 3D U-Net using not only lymph node annotations but also other anatomical structures (lungs, airways, aortic arches, and pulmonary arteries) that can be extracted robustly in an automated fashion. We applied the proposed method to 45 cases of contrast-enhanced chest CT volumes. Experimental results showed that 95.5% of lymph nodes were detected with 16.3 false positives per CT volume. The segmentation results showed that the proposed method can prevent oversegmentation, achieving an average Dice score of 52.3 ± 23.1%, compared to the baseline method with 49.2 ± 23.8%, respectively.
This paper presents a local intensity structure analysis based on an intensity targeted radial structure tensor (ITRST) and the blob-like structure enhancement filter based on it (ITRST filter) for the mediastinal lymph node detection algorithm from chest computed tomography (CT) volumes. Although the filter based on radial structure tensor analysis (RST filter) based on conventional RST analysis can be utilized to detect lymph nodes, some lymph nodes adjacent to regions with extremely high or low intensities cannot be detected. Therefore, we propose the ITRST filter, which integrates the prior knowledge on detection target intensity range into the RST filter. Our lymph node detection algorithm consists of two steps: (1) obtaining candidate regions using the ITRST filter and (2) removing false positives (FPs) using the support vector machine classifier. We evaluated lymph node detection performance of the ITRST filter on 47 contrast-enhanced chest CT volumes and compared it with the RST and Hessian filters. The detection rate of the ITRST filter was 84.2% with 9.1 FPs/volume for lymph nodes whose short axis was at least 10 mm, which outperformed the RST and Hessian filters.
In this paper we propose an estimation method of extracellular matrix directions of the heart. Myofiber are surrounded by the myocardial cell sheets whose directions have strong correspondence between heart failure. Estimation of the myocardial cell sheet directions is difficult since they are very thin. Therefore, we estimate the extracellular matrices which are touching to the sheets as if piled up. First, we perform a segmentation of the extracellular matrices by using the Hessian analysis. Each extracellular matrix region has sheet-like shape. We estimate the direction of each extracellular matrix region by the principal component analysis (PCA). In our experiments, mean inclination angles of two normal canine hearts were 50.6 and 46.2 degrees, while the angle of a failing canine heart was 57.4 degrees. This results well fit the anatomical knowledge that failing hearts tend to have vertical myocardical cell sheets.
In this paper, we propose a novel supervoxel segmentation method designed for mediastinal lymph node by embedding Hessian-based feature extraction. Starting from a popular supervoxel segmentation method, SLIC, which computes supervoxels by minimising differences of intensity and distance, we overcome this method's limitation of merging neighboring regions with similar intensity by introducing Hessian-based feature analysis into the supervoxel formation. We call this structure-oriented voxel clustering, which allows more accurate division into distinct regions having blob-, line- or sheet-like structures. This way, different tissue types in chest CT volumes can be segmented individually, even if neighboring tissues have similar intensity or are of non- spherical extent. We demonstrate the performance of the Hessian-assisted supervoxel technique by applying it to mediastinal lymph node detection in 47 chest CT volumes, resulting in false positive reductions from lymph node candidate regions. 89 % of lymph nodes whose short axis is at least 10 mm could be detected with 5.9 false positives per case using our method, compared to our previous method having 83 % of detection rate with 6.4 false positives per case.
This paper presents a new blob-like enhancement filter based on Intensity Targeted Radial Structure Tensor (ITRST) analysis to improve mediastinal lymph node detection from chest CT volumes. Blob-like structure enhancement filter based on Radial Structure Tensor (RST) analysis can be utilized for initial detection of lymph node candidate regions. However, some of lymph nodes cannot be detected because RST analysis is influenced by neighboring regions whose intensity is very high or low, such as contrast-enhanced blood vessels and air. To overcome the problem, we propose ITRST analysis that integrate the prior knowledge on detection target intensity into RST analysis. Our lymph node detection method consists of two steps. First, candidate regions are obtained by ITRST analysis. Second, false positives (FPs) are removed by the Support Vector Machine (SVM) classifier. We applied the proposed method to 47 cases. Among 19 lymph nodes whose short axis is no less than 10 mm, 100.0 % of them were detected with 247.7 FPs/case by ITRST analysis, while only 80.0 % were detected with 123.0 FPs/case by RST analysis. After the false positive (FP) reduction by SVM, ITRST analysis outperformed RST analysis in lymph node detection performance.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.