In recent years, the elderly population in Japan has been increasing. Expectations for the utilization of welfare equipment are also increasing. Electric wheelchairs are one of equipment and are widely used as a convenient means of transportation. On the other hand, accidents have also occurred, and dangers have been pointed out when driving the electric wheelchair. Therefore, we believe that the development of an autonomous mobile electric wheelchair can improve the causes of accidents. In addition, it can be expected to reduce accidents and improve the convenience of electric wheelchairs. For the development of an autonomous electric wheelchair, environment recognition such as estimation of the current position, recognition of sidewalks and traffic lights, and prediction of movement of objects is indispensable. To solve these problems, we develop an algorithm to recognize the sidewalks, crosswalks, and traffic lights from video images. In recent years, deep learning has been widely applied in the field of image recognition. Therefore, we improve WideSeg, one of the semantic segmentation algorithms that apply CNN (Convolutional Neural Networks), and develop an object recognition method using a new CNN model. In our approach, we perform adding the sidewalk correction and noise removal processing after performing semantic segmentation with the proposed model.
We developed a novel survival prediction model for images, called pix2surv, based on a conditional generative adversarial network (cGAN), and evaluated its performance based on chest CT images of patients with idiopathic pulmonary fibrosis (IPF). The architecture of the pix2surv model has a time-generator network that consists of an encoding convolutional network, a fully connected prediction network, and a discriminator network. The fully connected prediction network is trained to generate survival-time images from the chest CT images of each patient. The discriminator network is a patchbased convolutional network that is trained to differentiate the “fake pair” of a chest CT image and a generated survivaltime image from the “true pair” of an input CT image and the observed survival-time image of a patient. For evaluation, we retrospectively collected 75 IPF patients with high-resolution chest CT and pulmonary function tests. The survival predictions of the pix2surv model on these patients were compared with those of an established clinical prognostic biomarker known as the gender, age, and physiology (GAP) index by use of a two-sided t-test with bootstrapping. Concordance index (C-index) and relative absolute error (RAE) were used as measures of the prediction performance. Preliminary results showed that the survival prediction by the pix2surv model yielded more than 15% higher C-index value and more than 10% lower RAE values than those of the GAP index. The improvement in survival prediction by the pix2surv model was statistically significant (P < 0.0001). Also, the separation between the survival curves for the low- and high-risk groups was larger with pix2surv than that of the GAP index. These results show that the pix2surv model outperforms the GAP index in the prediction of the survival time and risk stratification of patients with IPF, indicating that the pix2surv model can be an effective predictor of the overall survival of patients with IPF.
We developed and evaluated the effect of U-Net-based radiomic features, called U-radiomics, on the prediction of the overall survival of patients with idiopathic pulmonary fibrosis (IPF). To generate the U-radiomics, we retrospectively collected lung CT images of 72 patients with interstitial lung diseases. An experienced observer delineated regions of interest (ROIs) from the lung regions on the CT images, and labeled them into one of four interstitial lung disease patterns (ground-glass opacity, reticulation, consolidation, and honeycombing) or a normal pattern. A U-Net was trained on these images for classifying the ROIs into one of the above five lung tissue patterns. The trained U-Net was applied to the lung CT images of an independent test set of 75 patients with IPF, and a U-radiomics vector for each patient was identified as the average of the bottleneck layer of the U-Net across all the CT images of the patient. The U-radiomics vector was subjected to a Cox proportional hazards model with elastic-net penalty for predicting the survival of the patient. The evaluation was performed by using bootstrapping with 500 replications, where concordance index (C-index) was used as the comparative performance metric. The preliminary results showed the following C-index values for two clinical biomarkers and the U-radiomics: (a) composite physiologic index (CPI): 64.6%, (b) gender, age, and physiology (GAP) index: 65.5%, and (c) U-radiomics: 86.0%. The U-radiomics significantly outperformed the clinical biomarkers in predicting the survival of IPF patients, indicating that the U-radiomics provides a highly accurate prognostic biomarker for patients with IPF.
Colorectal cancer is the second leading cause of cancer deaths worldwide. Computed tomographic colonography (CTC) can detect large colorectal polyps and cancers at a high sensitivity, whereas it can miss some of the smaller but still clinically significant 6 – 9 mm polyps. Dual-energy CTC (DE-CTC) can be used to provide more detailed information about scanned materials than does conventional single-energy CTC. We compared the classification performance of a 3D convolutional neural network (DenseNet) with those of four traditional 3D machine-learning models (AdaBoost, support vector machine, random forest, Bayesian neural network) and their cascade and ensemble classifier variants in the detection of small polyps in DE-CTC. Twenty patients with colonoscopy-confirmed polyps were examined by DE-CTC with a reduced one-day bowel preparation. The traditional machine-learning models were designed to identify polyps based on native radiomic dual-energy features of the DE-CTC image volumes. The performance of the machine-learning models was evaluated by use of the leave-one-patient-out method. The DenseNet was trained with a large independent external dataset of single-energy CTC cases and tested on blended image volumes of the DE-CTC cases. Although the DenseNet yielded the highest detection accuracy for typical polyps, AdaBoost and its cascade classifier variant yielded the highest overall polyp detection performance.
KEYWORDS: 3D modeling, Computer aided diagnosis and therapy, 3D image processing, Medical imaging, Performance modeling, Computed tomography, Virtual colonoscopy, Visualization, Data modeling, Solid modeling
Three-dimensional (3D) convolutional neural networks (CNNs) can process volumetric medical imaging data in their native volumetric input form. However, there is little information about the comparative performance of such models in medical imaging in general and in CT colonography (CTC) in particular. We compared the performance of a 3D densely connected CNN (3D-DenseNet) with those of the popular 3D residual CNN (3D-ResNet) and 3D Visual Geometry Group CNN (3D-VGG) in the reduction of false-positive detections (FPs) in computer-aided detection (CADe) of polyps in CTC. VGG is the earliest CNN design of these three models. ResNet has been used widely as a de-facto standard model for constructing deep CNNs for image classification in medical imaging. DenseNet is the most recent of these models and improves the flow of information and reduces the number of network parameters as compared to those of ResNet and VGG. For the evaluation, we used 403 CTC datasets from 203 patients. The classification performance of the CNNs was evaluated by use of 5-fold cross-validation, where the area under the receiver operating characteristic curve (AUC) was used as the figure of merit. Each training fold was balanced by use of data augmentation of the samples of real polyps. Our preliminary results showed that the AUC value of the 3D-DenseNet (0.951) was statistically significantly higher than those of the reference models (P < 0.005), indicating that the 3D-DenseNet has the potential of substantially outperforming the other models in reducing FPs in CADe for CTC. This improvement was highest for the smallest polyps.
We developed a novel ensemble three-dimensional residual network (E3D-ResNet) for the reduction of false positives (FPs) in computer-aided detection (CADe) of polyps on CT colonography (CTC). To capture the volumetric multiscale information of CTC images, each polyp candidate was represented with three different sizes of volumes of interest (VOIs), which were enlarged to a common size and were individually subjected to three 3D-ResNets. These 3D-ResNets were trained to calculate three polyp-likelihood probabilities, p1, p2 and p3, corresponding to each input VOI. The final polyp likelihood, p, was obtained as the maximum of p1, p2 and p3. We compared the classification performance of the E3D-ResNet with that of a non-ensemble 3D-ResNet, ensemble 2D-ResNet, and ensemble of 2D- and 3D-convolutional neural network (CNN) models. All models were trained and evaluated with 21,021 VOIs of polyps and 19,557 VOIs of FPs that were sampled with data augmentation from the CADe detections on the CTC data of 20 patients. We evaluated the classification performance of the models with receiver operating characteristics (ROC) analysis using cross-validation, where the area under the ROC curve (AUC) was used as the figure of merit. Preliminary results showed that AUC value (0.98) of the E3D-ResNet was significantly higher than that of the reference models (P < 0.001), indicating that the E3D-ResNet has the potential of substantially reducing the FPs in CADe of polyps on CTC.
To obtain an effective interpretation of organic shape using statistical shape models (SSMs), the correspondence of the landmarks through all the training samples is the most challenging part in model building. In this study, a coarse-tofine groupwise correspondence method for 3-D polygonal surfaces is proposed. We manipulate a reference model in advance. Then all the training samples are mapped to a unified spherical parameter space. According to the positions of landmarks of the reference model, the candidate regions for correspondence are chosen. Finally we refine the perceptually correct correspondences between landmarks using particle filter algorithm, where the likelihood of local surface features are introduced as the criterion. The proposed method was performed on the correspondence of 9 cases of left lung training samples. Experimental results show the proposed method is flexible and under-constrained.
A temporal subtraction image, which is obtained by subtraction of a previous image from a current one, can be used for
enhancing interval changes on medical images by removing most of normal structures. One of the important problems in
temporal subtraction is that subtraction images commonly include artifacts created by slight differences in the size, shape,
and/or location of anatomical structures. In this paper, we developed a new registration method with voxel-matching
technique for substantially removing the subtraction artifacts on the temporal subtraction image obtained from multiple-detector
computed tomography (MDCT). With this technique, the voxel value in a warped (or non-warped) previous
image is replaced by a voxel value within a kernel, such as a small cube centered at a given location, which would be
closest (identical or nearly equal) to the voxel value in the corresponding location in the current image. Our new method
was examined on 16 clinical cases with MDCT images. Preliminary results indicated that interval changes on the
subtraction images were enhanced considerably, with a substantial reduction of misregistration artifacts. The temporal
subtraction images obtained by use of the voxel-matching technique would be very useful for radiologists in the
detection of interval changes on MDCT images.
The detection of very subtle lesions and/or lesions overlapped with vessels on CT images is a time consuming and
difficult task for radiologists. In this study, we have developed a 3D temporal subtraction method to enhance interval
changes between previous and current multislice CT images based on a nonlinear image warping technique. Our
method provides a subtraction CT image which is obtained by subtraction of a previous CT image from a current CT
image. Reduction of misregistration artifacts is important in the temporal subtraction method. Therefore, our
computerized method includes global and local image matching techniques for accurate registration of current and
previous CT images. For global image matching, we selected the corresponding previous section image for each
current section image by using 2D cross-correlation between a blurred low-resolution current CT image and a blurred
previous CT image. For local image matching, we applied the 3D template matching technique with translation and
rotation of volumes of interests (VOIs) which were selected in the current and the previous CT images. The local shift
vector for each VOI pair was determined when the cross-correlation value became the maximum in the 3D template
matching. The local shift vectors at all voxels were determined by interpolation of shift vectors of VOIs, and then the
previous CT image was nonlinearly warped according to the shift vector for each voxel. Finally, the warped previous
CT image was subtracted from the current CT image. The 3D temporal subtraction method was applied to 19 clinical
cases. The normal background structures such as vessels, ribs, and heart were removed without large misregistration
artifacts. Thus, interval changes due to lung diseases were clearly enhanced as white shadows on subtraction CT
images.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.