This guest editorial introduces the Special Section on Image-Guided Procedures, Robotic Interventions, and Modeling in JMI Vol. 5 Issue 2.

Cochlear implants (CIs) are neural prostheses that restore hearing using an electrode array implanted in the cochlea. After implantation, the CI processor is programmed by an audiologist. One factor that negatively impacts outcomes and can be addressed by programming is cross-electrode neural stimulation overlap (NSO). We have proposed a system to assist the audiologist in programming the CI that we call image-guided CI programming (IGCIP). IGCIP permits using CT images to detect NSO and recommend deactivation of a subset of electrodes to avoid NSO. We have shown that IGCIP significantly improves hearing outcomes. Most of the IGCIP steps are robustly automated but electrode configuration selection still sometimes requires manual intervention. With expertise, distance-versus-frequency curves, which are a way to visualize the spatial relationship learned from CT between the electrodes and the nerves they stimulate, can be used to select the electrode configuration. We propose an automated technique for electrode configuration selection. A comparison between this approach and one we have previously proposed shows that our method produces results that are as good as those obtained with our previous method while being generic and requiring fewer parameters.

Laparoscopic liver surgery is challenging to perform due to a compromised ability of the surgeon to localize subsurface anatomy in the constrained environment. While image guidance has the potential to address this barrier, intraoperative factors, such as insufflation and variable degrees of organ mobilization from supporting ligaments, may generate substantial deformation. The severity of laparoscopic deformation in humans has not been characterized, and current laparoscopic correction methods do not account for the mechanics of how intraoperative deformation is applied to the liver. We first measure the degree of laparoscopic deformation at two insufflation pressures over the course of laparoscopic-to-open conversion in 25 patients. With this clinical data alongside a mock laparoscopic phantom setup, we report a biomechanical correction approach that leverages anatomically load-bearing support surfaces from ligament attachments to iteratively reconstruct and account for intraoperative deformations. Laparoscopic deformations were significantly larger than deformations associated with open surgery, and our correction approach yielded subsurface target error of 6.7±1.3  mm and surface error of 0.8±0.4  mm using only sparse surface data with realistic surgical extent. Laparoscopic surface data extents were examined and found to impact registration accuracy. Finally, we demonstrate viability of the correction method with clinical data.

Accurate two-dimensional to three-dimensional (2-D/3-D) registration of preoperative 3-D data and intraoperative 2-D x-ray images is a key enabler for image-guided therapy. Recent advances in 2-D/3-D registration formulate the problem as a learning-based approach and exploit the modeling power of convolutional neural networks (CNN) to significantly improve the accuracy and efficiency of 2-D/3-D registration. However, for surgery-related applications, collecting a large clinical dataset with accurate annotations for training can be very challenging or impractical. Therefore, deep learning-based 2-D/3-D registration methods are often trained with synthetically generated data, and a performance gap is often observed when testing the trained model on clinical data. We propose a pairwise domain adaptation (PDA) module to adapt the model trained on source domain (i.e., synthetic data) to target domain (i.e., clinical data) by learning domain invariant features with only a few paired real and synthetic data. The PDA module is designed to be flexible for different deep learning-based 2-D/3-D registration frameworks, and it can be plugged into any pretrained CNN model such as a simple Batch-Norm layer. The proposed PDA module has been quantitatively evaluated on two clinical applications using different frameworks of deep networks, demonstrating its significant advantages of generalizability and flexibility for 2-D/3-D medical image registration when a small number of paired real-synthetic data can be obtained.

Reproducibly achieving proper implant alignment is a critical step in total hip arthroplasty procedures that has been shown to substantially affect patient outcome. In current practice, correct alignment of the acetabular cup is verified in C-arm x-ray images that are acquired in an anterior–posterior (AP) view. Favorable surgical outcome is, therefore, heavily dependent on the surgeon’s experience in understanding the 3-D orientation of a hemispheric implant from 2-D AP projection images. This work proposes an easy to use intraoperative component planning system based on two C-arm x-ray images that are combined with 3-D augmented reality (AR) visualization that simplifies impactor and cup placement according to the planning by providing a real-time RGBD data overlay. We evaluate the feasibility of our system in a user study comprising four orthopedic surgeons at the Johns Hopkins Hospital and report errors in translation, anteversion, and abduction as low as 1.98 mm, 1.10 deg, and 0.53 deg, respectively. The promising performance of this AR solution shows that deploying this system could eliminate the need for excessive radiation, simplify the intervention, and enable reproducibly accurate placement of acetabular implants.

Segmentation of the levator hiatus in ultrasound allows the extraction of biometrics, which are of importance for pelvic floor disorder assessment. We present a fully automatic method using a convolutional neural network (CNN) to outline the levator hiatus in a two-dimensional image extracted from a three-dimensional ultrasound volume. In particular, our method uses a recently developed scaled exponential linear unit (SELU) as a nonlinear self-normalizing activation function, which for the first time has been applied in medical imaging with CNN. SELU has important advantages such as being parameter-free and mini-batch independent, which may help to overcome memory constraints during training. A dataset with 91 images from 35 patients during Valsalva, contraction, and rest, all labeled by three operators, is used for training and evaluation in a leave-one-patient-out cross validation. Results show a median Dice similarity coefficient of 0.90 with an interquartile range of 0.08, with equivalent performance to the three operators (with a Williams’ index of 1.03), and outperforming a U-Net architecture without the need for batch normalization. We conclude that the proposed fully automatic method achieved equivalent accuracy in segmenting the pelvic floor levator hiatus compared to a previous semiautomatic approach.

Use of four-dimensional cone-beam CT (4D-CBCT) and fiducial markers for image guidance during radiation therapy (RT) of mobile tumors is challenging due to the trade-off among image quality, imaging dose, and scanning time. This study aimed to investigate different 4D-CBCT acquisition settings for good visibility of fiducial markers in 4D-CBCT. Using these 4D-CBCTs, the feasibility of marker-based 4D registration for RT setup verification and manual respiration-induced motion quantification was investigated. For this, we applied a dynamic phantom with three different breathing motion amplitudes and included two patients with implanted markers. Irrespective of the motion amplitude, for a medium field of view (FOV), marker visibility was improved by reducing the imaging dose per projection and increasing the number of projection images; however, the scanning time was 4 to 8 min. For a small FOV, the total imaging dose and the scanning time were reduced (62.5% of the dose using a medium FOV, 2.5 min) without losing marker visibility. However, the body contour could be missing for a small FOV, which is not preferred in RT. The marker-based 4D setup verification was feasible for both the phantom and patient data. Moreover, manual marker motion quantification can achieve a high accuracy with a mean error of <1.4  mm.

Automatic segmentation of the prostate on magnetic resonance images (MRI) has many applications in prostate cancer diagnosis and therapy. We proposed a deep fully convolutional neural network (CNN) to segment the prostate automatically. Our deep CNN model is trained end-to-end in a single learning stage, which uses prostate MRI and the corresponding ground truths as inputs. The learned CNN model can be used to make an inference for pixel-wise segmentation. Experiments were performed on three data sets, which contain prostate MRI of 140 patients. The proposed CNN model of prostate segmentation (PSNet) obtained a mean Dice similarity coefficient of 85.0±3.8% as compared to the manually labeled ground truth. Experimental results show that the proposed model could yield satisfactory segmentation of the prostate on MRI.

Fluoroscopic x-ray guidance is a cornerstone for percutaneous orthopedic surgical procedures. However, two-dimensional (2-D) observations of the three-dimensional (3-D) anatomy suffer from the effects of projective simplification. Consequently, many x-ray images from various orientations need to be acquired for the surgeon to accurately assess the spatial relations between the patient’s anatomy and the surgical tools. We present an on-the-fly surgical support system that provides guidance using augmented reality and can be used in quasiunprepared operating rooms. The proposed system builds upon a multimodality marker and simultaneous localization and mapping technique to cocalibrate an optical see-through head mounted display to a C-arm fluoroscopy system. Then, annotations on the 2-D x-ray images can be rendered as virtual objects in 3-D providing surgical guidance. We quantitatively evaluate the components of the proposed system and, finally, design a feasibility study on a semianthropomorphic phantom. The accuracy of our system was comparable to the traditional image-guided technique while substantially reducing the number of acquired x-ray images as well as procedure time. Our promising results encourage further research on the interaction between virtual and real objects that we believe will directly benefit the proposed method. Further, we would like to explore the capabilities of our on-the-fly augmented reality support system in a larger study directed toward common orthopedic interventions.

We present our work investigating the feasibility of combining intraoperative ultrasound for brain shift correction and augmented reality (AR) visualization for intraoperative interpretation of patient-specific models in image-guided neurosurgery (IGNS) of brain tumors. We combine two imaging technologies for image-guided brain tumor neurosurgery. Throughout surgical interventions, AR was used to assess different surgical strategies using three-dimensional (3-D) patient-specific models of the patient’s cortex, vasculature, and lesion. Ultrasound imaging was acquired intraoperatively, and preoperative images and models were registered to the intraoperative data. The quality and reliability of the AR views were evaluated with both qualitative and quantitative metrics. A pilot study of eight patients demonstrates the feasible combination of these two technologies and their complementary features. In each case, the AR visualizations enabled the surgeon to accurately visualize the anatomy and pathology of interest for an extended period of the intervention. Inaccuracies associated with misregistration, brain shift, and AR were improved in all cases. These results demonstrate the potential of combining ultrasound-based registration with AR to become a useful tool for neurosurgeons to improve intraoperative patient-specific planning by improving the understanding of complex 3-D medical imaging data and prolonging the reliable use of IGNS.

Elastography measures tissue strain, which can be interpreted under certain simplifying assumptions to be representative of the underlying stiffness distribution. This is useful in cancer diagnosis where tumors tend to have a different stiffness to healthy tissue and has also shown potential to provide indication of the degree of bonding at tumor–tissue boundaries, which is clinically useful because of its dependence on tumor pathology. We consider the changes in axial strain for the case of a symmetrical model undergoing uniaxial compression, studied by characterizing changes in tumor contrast transfer efficiency (CTE), inclusion to background strain contrast and strain contrast generated by slip motion, as a function of Young’s modulus contrast and applied strain. We present results from a finite element simulation and an evaluation of these results using tissue-mimicking phantoms. The simulation results show that a discontinuity in displacement data at the tumor boundary, caused by the surrounding tissue slipping past the tumor, creates a halo of “pseudostrain” across the tumor boundary. Mobile tumors also appear stiffer on elastograms than adhered tumors, to the extent that tumors that have the same Young’s modulus as the background may in fact be visible as low-strain regions, or those that are softer than the background may appear to be stiffer than the background. Tumor mobility also causes characteristic strain heterogeneity within the tumor, which exhibits low strain close to the slippery boundary and increasing strain toward the center of the tumor. These results were reproduced in phantom experiments. In addition, phantom experiments demonstrated that when fluid lubrication is present at the boundary, these effects become applied strain-dependent as well as modulus-dependent, in a systematic and characteristic manner. The knowledge generated by this study is expected to aid interpretation of clinical strain elastograms by helping to avoid misinterpretation as well as provide additional diagnostic criteria stated in the paper and stimulate further research into the application of elastography to tumor mobility assessment.

A method for x-ray image-guided robotic instrument positioning is reported and evaluated in preclinical studies of spinal pedicle screw placement with the aim of improving delivery of transpedicle K-wires and screws. The known-component (KC) registration algorithm was used to register the three-dimensional patient CT and drill guide surface model to intraoperative two-dimensional radiographs. Resulting transformations, combined with offline hand–eye calibration, drive the robotically held drill guide to target trajectories defined in the preoperative CT. The method was assessed in comparison with a more conventional tracker-based approach, and robustness to clinically realistic errors was tested in phantom and cadaver. Deviations from planned trajectories were analyzed in terms of target registration error (TRE) at the tooltip (mm) and approach angle (deg). In phantom studies, the KC approach resulted in TRE=1.51±0.51  mm and 1.01  deg±0.92  deg, comparable with accuracy in tracker-based approach. In cadaver studies with realistic anatomical deformation, the KC approach yielded TRE=2.31±1.05  mm and 0.66  deg±0.62  deg, with statistically significant improvement versus tracker (TRE=6.09±1.22  mm and 1.06  deg±0.90  deg). Robustness to deformation is attributed to relatively local rigidity of anatomy in radiographic views. X-ray guidance offered accurate robotic positioning and could fit naturally within clinical workflow of fluoroscopically guided procedures.

Hysterectomies (i.e., surgical removal of the uterus) are the prevailing solution to treat medical conditions such as uterine cancer, endometriosis, and uterine prolapse. One complication of hysterectomies is accidental injury to the ureters located within millimeters of the uterine arteries that are severed and cauterized to hinder blood flow and enable full uterus removal. This work explores the feasibility of using photoacoustic imaging to visualize the uterine arteries (and potentially the ureter) when this imaging method is uniquely combined with a da Vinci^® surgical robot that enables teleoperated hysterectomies. We developed a specialized light delivery system to surround a da Vinci^® curved scissor tool, and an ultrasound probe was placed externally, representing a transvaginal approach, to receive the acoustic signals. Photoacoustic images were acquired while sweeping the tool across our custom 3-D uterine vessel model covered in ex vivo bovine tissue that was placed between the 3-D model and the fiber, as well as between the ultrasound probe and the 3-D model. Four tool orientations were explored, and the robot kinematics were used to provide tool position and orientation information simultaneously with each photoacoustic image acquisition. The optimal tool orientation produced images with contrast >10  dB and background signal-to-noise ratios (SNRs) >1.7, indicating minimal acoustic clutter from the tool tip. We achieved similar contrast and SNR measurements with four unique wrist orientations explored with the scissor tool in open and closed configurations. Results indicate that photoacoustic imaging is a promising approach to enable visualization of the uterine arteries to guide hysterectomies (and other gynecological surgeries). These results are additionally applicable to other da Vinci^® surgeries and other surgical instruments with similar tip geometry.

Three-dimensional ultrasound segmentation of mitral valve (MV) at diastole is helpful for duplicating geometry and pathology in a patient-specific dynamic phantom. The major challenge is the signal dropout at leaflet regions in transesophageal echocardiography image data. Conventional segmentation approaches suffer from missing sonographic data leading to inaccurate MV modeling at leaflet regions. This paper proposes a signal dropout correction-based ultrasound segmentation method for diastolic MV modeling. The proposed method combines signal dropout correction, image fusion, continuous max-flow segmentation, and active contour segmentation techniques. The signal dropout correction approach is developed to recover the missing segmentation information. Once the signal dropout regions of TEE image data are recovered, the MV model can be accurately duplicated. Compared with other methods in current literature, the proposed algorithm exhibits lower computational cost. The experimental results show that the proposed algorithm gives competitive results for diastolic MV modeling compared with conventional segmentation algorithms, evaluated in terms of accuracy and efficiency.

Dynamic reference frames (DRFs) are a common component of modern surgical tracking systems; however, the limited number of commercially available DRFs poses a constraint in developing systems, especially for research and education. This work presents the design and validation of a large, open-source library of DRFs compatible with passive, single-face tracking systems, such as Polaris stereoscopic infrared trackers (NDI, Waterloo, Ontario). An algorithm was developed to create new DRF designs consistent with intra- and intertool design constraints and convert to computer-aided design (CAD) files suitable for three-dimensional printing. A library of 10 such groups, each with 6 to 10 DRFs, was produced and tracking performance was validated in comparison to a standard commercially available reference, including pivot calibration, fiducial registration error (FRE), and target registration error (TRE). Pivot tests showed calibration error (mean±std)=0.46±0.1  mm, indistinguishable from the reference. FRE was 0.15±0.03  mm, and TRE in a CT head phantom was 0.96±0.5  mm, both equivalent to the reference. The library of DRFs offers a useful resource for surgical navigation research and could be extended to other tracking systems and alternative design constraints.

A projector-based augmented reality intracorporeal system (PARIS) is presented that includes a miniature tracked projector, tracked marker, and laparoscopic ultrasound (LUS) transducer. PARIS was developed to improve the efficacy and safety of laparoscopic partial nephrectomy (LPN). In particular, it has been demonstrated to effectively assist in the identification of tumor boundaries during surgery and to improve the surgeon’s understanding of the underlying anatomy. PARIS achieves this by displaying the orthographic projection of the cancerous tumor on the kidney’s surface. The performance of PARIS was evaluated in a user study with two surgeons who performed 32 simulated robot-assisted partial nephrectomies. They performed 16 simulated partial nephrectomies with PARIS for guidance and 16 simulated partial nephrectomies with only an LUS transducer for guidance. With PARIS, there was a significant reduction [30% ( p<0.05)] in the amount of healthy tissue excised and a trend toward a more accurate dissection around the tumor and more negative margins. The combined point tracking and reprojection root-mean-square error of PARIS was 0.8 mm. PARIS’ proven ability to improve key metrics of LPN surgery and qualitative feedback from surgeons about PARIS supports the hypothesis that it is an effective surgical navigation tool.

The most effective treatment for twin-to-twin transfusion syndrome is laser photocoagulation of the shared vascular anastomoses in the placenta. Vascular connections are extremely challenging to locate due to their caliber and the reduced field-of-view of the fetoscope. Therefore, mosaicking techniques are beneficial to expand the scene, facilitate navigation, and allow vessel photocoagulation decision-making. Local vision-based mosaicking algorithms inherently drift over time due to the use of pairwise transformations. We propose the use of an electromagnetic tracker (EMT) sensor mounted at the tip of the fetoscope to obtain camera pose measurements, which we incorporate into a probabilistic framework with frame-to-frame visual information to achieve globally consistent sequential mosaics. We parametrize the problem in terms of plane and camera poses constrained by EMT measurements to enforce global consistency while leveraging pairwise image relationships in a sequential fashion through the use of local bundle adjustment. We show that our approach is drift-free and performs similarly to state-of-the-art global alignment techniques like bundle adjustment albeit with much less computational burden. Additionally, we propose a version of bundle adjustment that uses EMT information. We demonstrate the robustness to EMT noise and loss of visual information and evaluate mosaics for synthetic, phantom-based and ex vivo datasets.

In spite of significant efforts to improve image-guided ablation therapy, a large number of patients undergoing ablation therapy to treat cardiac arrhythmic conditions require repeat procedures. The delivery of insufficient thermal dose is a significant contributor to incomplete tissue ablation, in turn leading to the arrhythmia recurrence. Ongoing research efforts aim to better characterize and visualize RF delivery to monitor the induced tissue damage during therapy. Here, we propose a method that entails modeling and visualization of the lesions in real-time. The described image-based ablation model relies on classical heat transfer principles to estimate tissue temperature in response to the ablation parameters, tissue properties, and duration. The ablation lesion quality, geometry, and overall progression are quantified on a voxel-by-voxel basis according to each voxel’s cumulative temperature and time exposure. The model was evaluated both numerically under different parameter conditions, as well as experimentally, using ex vivo bovine tissue samples undergoing ex vivo clinically relevant ablation protocols. The studies demonstrated less than 5°C difference between the model-predicted and experimentally measured end-ablation temperatures. The model predicted lesion patterns were within 0.5 to 1 mm from the observed lesion patterns, suggesting sufficiently accurate modeling of the ablation lesions. Lastly, our proposed method enables therapy delivery feedback with no significant workflow latency. This study suggests that the proposed technique provides reasonably accurate and sufficiently fast visualizations of the delivered ablation lesions.

Standard surgical resection of glioblastoma, mainly guided by the enhancement on postcontrast T1-weighted magnetic resonance imaging (MRI), disregards infiltrating tumor within the peritumoral edema region (ED). Subsequent radiotherapy typically delivers uniform radiation to peritumoral FLAIR-hyperintense regions, without attempting to target areas likely to be infiltrated more heavily. Noninvasive in vivo delineation of the areas of tumor infiltration and prediction of early recurrence in peritumoral ED could assist in targeted intensification of local therapies, thereby potentially delaying recurrence and prolonging survival. This paper presents a method for estimating peritumoral edema infiltration using radiomic signatures determined via machine learning methods, and tests it on 90 patients with de novo glioblastoma. The generalizability of the proposed predictive model was evaluated via cross-validation in a discovery cohort (n  =  31) and was subsequently evaluated in a replication cohort (n  =  59). Spatial maps representing the likelihood of tumor infiltration and future early recurrence were compared with regions of recurrence on postresection follow-up studies with pathology confirmation. The cross-validated accuracy of our predictive infiltration model on the discovery and replication cohorts was 87.51% (odds ratio = 10.22, sensitivity = 80.65, and specificity = 87.63) and 89.54% (odds ratio = 13.66, sensitivity = 97.06, and specificity = 76.73), respectively. The radiomic signature of the recurrent tumor region revealed higher vascularity and cellularity when compared with the nonrecurrent region. The proposed model shows evidence that multiparametric pattern analysis from clinical MRI sequences can assist in in vivo estimation of the spatial extent and pattern of tumor recurrence in peritumoral edema, which may guide supratotal resection and/or intensification of postoperative radiation therapy.

Fluorescence imaging is a well-established optical modality that has been used to localize and track fluorophores in vivo and has demonstrated great potential for surgical guidance. Despite the variety of fluorophores currently being researched, many existing intraoperative fluorescence imaging systems are specifically designed for a limited number of applications. We present a modular wide-field fluorescence overlay tissue imaging system for intraoperative surgical guidance that is comprised of commercially available standardized components. Its modular layout allows for the accommodation of a broad range of fluorophores, fields of view (FOV), and spatial resolutions while maintaining an integrated portable design for intraoperative use. Measurements are automatic and feature a real-time projection overlay technique that intuitively displays fluorescence maps directly onto a 15  ×  15  cm² FOV from a working distance of 35 cm. At a 20-ms exposure time, 10  μM samples of indocyanine green could be measured with high signal-to-noise ratio and was later tested in an in vivo mouse model before finally being demonstrated for intraoperative autofluorescence imaging of human soft tissue sarcoma margins. The system’s modular design and ability to enable naked-eye visualization of wide-field fluorescence allow for the flexibility to adapt to numerous clinical applications and can potentially extend the adoption of fluorescence imaging for intraoperative use.

Image guidance with cone beam computed tomography in radiotherapy can guarantee the precision and accuracy of patient positioning prior to treatment delivery. During the image guidance process, operators need to take great effort to evaluate the image guidance quality before correcting a patient’s position. This work proposes an image registration assessment method based on control chart monitoring to reduce the effort taken by the operator. According to the control chart plotted by daily registration scores of each patient, the proposed method can quickly detect both alignment errors and image quality inconsistency. Therefore, the proposed method can provide a clear guideline for the operators to identify unacceptable image quality and unacceptable image registration with minimal effort. Experimental results demonstrate that by using control charts from a clinical database of 10 patients undergoing prostate radiotherapy, the proposed method can quickly identify out-of-control signals and find special cause of out-of-control registration events.

Minimally invasive mitral valve repair procedures including MitraClip® are becoming increasingly common. For cases of complex or diseased anatomy, clinicians may benefit from using a patient-specific cardiac phantom for training, surgical planning, and the validation of devices or techniques. An imaging compatible cardiac phantom was developed to simulate a MitraClip® procedure. The phantom contained a patient-specific cardiac model manufactured using tissue mimicking materials. To evaluate accuracy, the patient-specific model was imaged using computed tomography (CT), segmented, and the resulting point cloud dataset was compared using absolute distance to the original patient data. The result, when comparing the molded model point cloud to the original dataset, resulted in a maximum Euclidean distance error of 7.7 mm, an average error of 0.98 mm, and a standard deviation of 0.91 mm. The phantom was validated using a MitraClip® device to ensure anatomical features and tools are identifiable under image guidance. Patient-specific cardiac phantoms may allow for surgical complications to be accounted for preoperative planning. The information gained by clinicians involved in planning and performing the procedure should lead to shorter procedural times and better outcomes for patients.

Catheters are commonly used in many procedures and tracking and localizing them is critical to patient safety and surgical success. The standard of care for catheter tracking is with the use of fluoroscopy. Alternatives using conventional tracking technologies such as electromagnetic trackers have been previously explored. This work explores the use of an emerging imaging modality, photoacoustics, as a means for tracking. A piezoelectric (PZT) sensor is placed at the tip of the catheter, allowing it to receive the acoustic signals generated from photoacoustic markers due to the photoacoustic effect. The locations of these photoacoustic markers are determined by a stereo-camera and the received acoustic signals are converted into distances between the PZT element and the photoacoustic markers. The location of the PZT sensor can be uniquely determined following a multilateration process. This work validates this photoacoustic tracking method in phantom, simulation, and in vivo scenarios using metrics including reconstruction precision, relative accuracy, estimated accuracy, and leave-out accuracy. Submillimeter tracking results were achieved in phantom experiments. Simulation studies evaluated various physical parameters relating to the photoacoustic source and the PZT sensor. In vivo results showed feasibility for the eventual deployment of this technology.

Lumbar punctures (LPs) are interventional procedures that are used to collect cerebrospinal fluid. Since the target window is small, physicians have limited success conducting the procedure. The procedure is especially difficult for obese patients due to the increased distance between bone and skin surface. We propose a simple and direct needle insertion platform, enabling image formation by sweeping a needle with a single ultrasound element at the tip. The needle-shaped ultrasound transducer can not only sense the distance between the tip and a potential obstacle, such as bone, but also visually locate the structures by combining transducer location tracking and synthetic aperture focusing. The concept of the system was validated through a simulation that revealed robust image reconstruction under expected errors in tip localization. The initial prototype was built into a 14 G needle and was mounted on a holster equipped with a rotation shaft allowing one degree-of-freedom rotational sweeping and a rotation tracking encoder. We experimentally evaluated the system using a metal-wire phantom mimicking high reflection bone structures and human spinal bone phantom. Images of the phantoms were reconstructed, and the synthetic aperture reconstruction improved the image quality. These results demonstrate the potential of the system to be used as a real-time guidance tool for improving LPs.

It is well known that properly designed image reconstruction methods can facilitate reductions in imaging doses and data-acquisition times in tomographic imaging. The ability to do so is particularly important for emerging modalities, such as differential x-ray phase-contrast tomography (D-XPCT), which are currently limited by these factors. An important application of D-XPCT is high-resolution imaging of biomedical samples. However, reconstructing high-resolution images from few-view tomographic measurements remains a challenging task due to the high-frequency information loss caused by data incompleteness. In this work, a subspace-based reconstruction strategy is proposed and investigated for use in few-view D-XPCT image reconstruction. By adopting a two-step approach, the proposed method can simultaneously recover high-frequency details within a certain region of interest while suppressing noise and/or artifacts globally. The proposed method is investigated by the use of few-view experimental data acquired by an edge-illumination D-XPCT scanner.

Deformable image registration, a key component of motion correction in medical imaging, needs to be efficient and provides plausible spatial transformations that reliably approximate biological aspects of complex human organ motion. Standard approaches, such as Demons registration, mostly use Gaussian regularization for organ motion, which, though computationally efficient, rule out their application to intrinsically more complex organ motions, such as sliding interfaces. We propose regularization of motion based on supervoxels, which provides an integrated discontinuity preserving prior for motions, such as sliding. More precisely, we replace Gaussian smoothing by fast, structure-preserving, guided filtering to provide efficient, locally adaptive regularization of the estimated displacement field. We illustrate the approach by applying it to estimate sliding motions at lung and liver interfaces on challenging four-dimensional computed tomography (CT) and dynamic contrast-enhanced magnetic resonance imaging datasets. The results show that guided filter-based regularization improves the accuracy of lung and liver motion correction as compared to Gaussian smoothing. Furthermore, our framework achieves state-of-the-art results on a publicly available CT liver dataset.

We describe a hybrid method for left ventricle (LV) endocardial and epicardial segmentation on cardiac magnetic resonance (CMR) images requiring minimal operator intervention. Endocardium extraction results from the union of three independent estimations based on adaptive thresholding, region growing, and active contour with Chan—Vese energy function. Epicardium segmentation relies on conditional morphological dilation of the endocardial mask followed by active contour optimization. The proposed method was first evaluated using an open access database of 18 CMR for which expert manual contouring was available. The method was further validated on a retrospective cohort of 29 patients, who underwent CMR with expert manual segmentation. Regarding the open access database, similarity (Dice index) between hybrid and expert segmentations was good for end-diastolic (ED) endocardium (0.92), end-systolic (ES) endocardium (0.88), and ED epicardium (0.92). As for derived LV parameters, concordance (Lin’s coefficient) was good for ED volume (0.91), ES volume (0.93), ejection fraction (EF; 0.89), and fair for myocardial mass (MM; 0.74). Regarding the retrospective patient study, concordance between expert and hybrid estimations was excellent for ED volume (0.95), ES volume (0.96), good for EF (0.86), and fair for MM (0.71). Hybrid segmentation resulted in small biases (−6  mL for ED volume, +4  mL for ES volume, −6  %   for EF, and −6  g for MM) with little clinical relevance and acceptable for routine practice. The quickness and robustness of the proposed hybrid method and its ability to provide LV volumes, functions, and masses highly concordant with those given by expert segmentation support its pertinence for routine clinical use.

Error estimation in nonlinear medical image registration is a nontrivial problem that is important for validation of registration methods. We propose a supervised method for estimation of registration errors in nonlinear registration of three-dimensional (3-D) images. The method is based on a 3-D convolutional neural network that learns to estimate registration errors from a pair of image patches. By applying the network to patches centered around every voxel, we construct registration error maps. The network is trained using a set of representative images that have been synthetically transformed to construct a set of image pairs with known deformations. The method is evaluated on deformable registrations of inhale–exhale pairs of thoracic CT scans. Using ground truth target registration errors on manually annotated landmarks, we evaluate the method’s ability to estimate local registration errors. Estimation of full domain error maps is evaluated using a gold standard approach. The two evaluation approaches show that we can train the network to robustly estimate registration errors in a predetermined range, with subvoxel accuracy. We achieved a root-mean-square deviation of 0.51 mm from gold standard registration errors and of 0.66 mm from ground truth landmark registration errors.

Computed tomography is a standard diagnostic imaging technique for patients with traumatic brain injury (TBI). A limitation is the poor-to-moderate sensitivity for small traumatic hemorrhages. A pilot study using an automatic method to detect hemorrhages <10  mm in diameter in patients with TBI is presented. We have created an average image from 30 normal noncontrast CT scans that were automatically aligned using deformable image registration as implemented in Elastix software. Subsequently, the average image was aligned to the scans of TBI patients, and the hemorrhages were detected by a voxelwise subtraction of the average image from the CT scans of nine TBI patients. An experienced neuroradiologist and a radiologist in training assessed the presence of hemorrhages in the final images and determined the false positives and false negatives. The 9 CT scans contained 67 small haemorrhages, of which 97% was correctly detected by our system. The neuroradiologist detected three false positives, and the radiologist in training found two false positives. For one patient, our method showed a hemorrhagic contusion that was originally missed. Comparing individual CT scans with a computed average may assist the physicians in detecting small traumatic hemorrhages in patients with TBI.

Cell migration is a key feature for living organisms. Image analysis tools are useful in studying cell migration in three-dimensional (3-D) in vitro environments. We consider angiogenic vessels formed in 3-D microfluidic devices (MFDs) and develop an image analysis system to extract cell behaviors from experimental phase-contrast microscopy image sequences. The proposed system initializes tracks with the end-point confocal nuclei coordinates. We apply convolutional neural networks to detect cell candidates and combine backward Kalman filtering with multiple hypothesis tracking to link the cell candidates at each time step. These hypotheses incorporate prior knowledge on vessel formation and cell proliferation rates. The association accuracy reaches 86.4% for the proposed algorithm, indicating that the proposed system is able to associate cells more accurately than existing approaches. Cell culture experiments in 3-D MFDs have shown considerable promise for improving biology research. The proposed system is expected to be a useful quantitative tool for potential microscopy problems of MFDs.

Percutaneous coronary intervention (PCI) uses x-ray images, which may give high radiation dose and high concentrations of contrast media, leading to the risk of radiation-induced injury and nephropathy. These drawbacks can be reduced by using lower doses of x-rays and contrast media, with the disadvantage of noisier PCI images with less contrast. Vessel-edge-preserving convolutional neural networks (CNN) were designed to denoise simulated low x-ray dose PCI images, created by adding artificial noise to high-dose images. Objective functions of the designed CNNs have been optimized to achieve an edge-preserving effect of vessel walls, and the results of the proposed objective functions were evaluated qualitatively and quantitatively. Finally, the proposed CNN-based method was compared with two state-of-the-art denoising methods: K-SVD and block-matching and 3D filtering. The results showed promising performance of the proposed CNN-based method for PCI image enhancement with interesting capabilities of CNNs for real-time denoising and contrast enhancement tasks.

Quantifying the amount of brown adipose tissue (BAT) within white adipose tissue (WAT) in human depots may serve as a target to combat obesity. We aimed to quantify proton density fat fraction (PDFF) of BAT and WAT in relatively pure and in mixed preparation using water–fat imaging. Three ex-vivo experiments were performed at 3 T using excised interscapular BAT and inguinal/subcutaneous WAT from mice. The first two experiments consisted of BAT and WAT in separate tubes, and the third used mixed preparation with graded quantities of BAT and WAT. To investigate the influence of partial volume on PDFF metrics, low (2.66  mm³) and high spatial resolution (0.55  mm³ acquired voxels) in two orthogonal three-dimensional sections were compared. The low-resolution acquisitions are corrected for T2* and multipeak lipid spectrum, thus considered “quantitative,” whereas the high-resolution acquisitions are not corrected but were performed to better spatially segment BAT from WAT zones. As potential BAT metrics, we quantified the average PDFF and the volume of tissue having PDFF ≤50  %   (VOL_{PDFF  ≤  50  %}) based on the PDFF histogram. In the first experiment, the average PDFF of BAT was 23  ±  6  %   and 21  ±  7.6  %   and the average PDFF of WAT was 76  ±  7  %   and 87  ±  7  %   using high- and low-resolution techniques, respectively. A similar trend with excellent reproducibility in average PDFF of BAT and WAT was observed in the second experiment. In the third experiment over the four acquisitions, the BAT-dominant tube demonstrated lower PDFF (mean ± SD) of 55  ±  2  %   than WAT-dominant   (  69  ±  4  %    )   and WAT-only tubes   (  88  ±  4  %    )  . Estimating VOL_{PDFF  ≤  50  %}, the BAT-dominant tube demonstrated higher volume of 0.26  cm³ than WAT-dominant (0.16  cm³) and WAT-only tubes (0.01  cm³). The presence of BAT exhibits a lower PDFF relative to WAT, thus allowing segmentation of low PDFF tissue for quantification of volume representative of BAT. Future studies will determine the clinical relevance of BAT volume within human depots.

Differentiation between benign and malignant nodules is a problem encountered by radiologists when visualizing computed tomography (CT) scans. Adenocarcinomas and granulomas have a characteristic spiculated appearance and may be fluorodeoxyglucose avid, making them difficult to distinguish for human readers. In this retrospective study, we aimed to evaluate whether a combination of radiomic texture and shape features from noncontrast CT scans can enable discrimination between granulomas and adenocarcinomas. Our study is composed of CT scans of 195 patients from two institutions, one cohort for training (N  =  139) and the other (N  =  56) for independent validation. A set of 645 three-dimensional texture and 24 shape features were extracted from CT scans in the training cohort. Feature selection was employed to identify the most informative features using this set. The top ranked features were also assessed in terms of their stability and reproducibility across the training and testing cohorts and between scans of different slice thickness. Three different classifiers were constructed using the top ranked features identified from the training set. These classifiers were then validated on the test set and the best classifier (support vector machine) yielded an area under the receiver operating characteristic curve of 77.8%.

Intratumoral heterogeneity biomarkers derived from positron emission tomography (PET) imaging with fluorodeoxyglucose (FDG) are of interest for a number of cancers, including sarcoma. A range of radiomic texture variables, adapted from general methodologies for image analysis, has shown promise in the setting. In the context of sarcoma, our group introduced an alternative model-based approach to the measurement of heterogeneity. In this approach, the heterogeneity of a tumor is characterized by the extent to which the 3-D FDG uptake pattern deviates from a simple elliptically contoured structure. By using a nonparametric analysis of the uptake profile obtained from this spatial model, a variable assessing the metabolic gradient of the tumor is developed. The work explores the prognostic potential of this new variable in the context of FDG-PET imaging of sarcoma. A mature clinical series involving 197 patients, 88 of whom have complete time-to-death information, is used. Texture variables based on the imaging data are also evaluated in this series and a range of appropriate machine learning methodologies are then used to explore the complementary prognostic roles for structure and texture variables. We conclude that both texture-based and model-based variables can be combined to achieve enhanced prognostic assessments of outcome for patients with sarcoma based on FDG-PET imaging information.

Remote monitoring of vital physiological signs allows for unobtrusive, nonrestrictive, and noncontact assessment of an individual’s health. We demonstrate a simple but robust image photoplethysmography-based heart rate (HR) estimation method for multiple subjects. In contrast to previous studies, a self-learning procedure of tech was developed in our study. We improved compress tracking algorithm to track the regions of interest from video sequences and used support vector machine to filter out potentially false beats caused by variations in the reflected light from the face. The experiment results on 40 subjects show that the absolute value of mean error reduces from 3.6 to 1.3  beats  /  min. We further explore experiments for 10 subjects simultaneously, regardless of the videos at a resolution of 600 by 800, the HR is predicted real-time and the results reveal modest but significant effects on HR prediction.

A principal difference between the channelized Hotelling (CH) and visual-search (VS) model observers is how they respond to noise texture in images. We compared the two observers in lesion-detection studies to evaluate linear and angular sampling parameters for CT. Simulated lung images were generated from a single two-dimensional mathematical torso phantom containing circular lesions of fixed radius and relative contrast. Projection datasets were produced for two detector pixel sizes and from 15 to 128 projections at 15 and 65 M counts per set. Filtered backprojection reconstructions were obtained with dimensions of 128  ×  128 and 256  ×  256. A localization receiver operating characteristic study was conducted with two human observers, three single-feature VS observers, and a feature-adaptive VS observer. The effects of the sampling parameters on performance were similar for all of these observers. The CH observer, applied in location-known studies with and without background variability, was not affected by the variations in angular sampling. The two-stage VS framework was an effective modification of the CH observer for assessing the effects of noise texture on human-observer performance in this study.

Microwave breast imaging is a painless and nonradiation method. This pilot study aimed to evaluate the detective capability and feasibility of a prototype of a portable breast cancer detector using a radar-based imaging system. Five patients with histologically confirmed breast cancers with a minimum diameter of 1 cm were enrolled in this study. The antenna array dome of the device was placed on the breast of the patient in a supine position for 15 min per single examination. The primary endpoint was a detection rate of breast cancers. The secondary endpoints were positional accuracy and adverse event. All five targeted breast tumors were detected and were visualized at the sites confirmed by other diagnostic modalities. Among five tumors, one was not detected via mammography because of heterogeneously dense breast and another was a microinvasive carcinoma of invasive tumor size 0.5 mm. No study-related adverse events occurred. The prototype of a portable breast cancer detector has sufficient detective capability, is safe for clinical use, and might detect an early stage breast cancer, such as noninvasive carcinoma. Future developments should focus on further decreasing the size of the machine and shortening inspection time.

Dilatation of the cerebral ventricles is a common condition in preterm neonates with intraventricular hemorrhage. This posthemorrhagic ventricle dilatation (PHVD) can lead to lifelong neurological impairment through ischemic injury due to increased intracranial pressure, and without treatment can lead to death. Two-dimensional ultrasound (US) through the fontanelles of the patients is serially acquired to monitor the progression of PHVD. These images are used in conjunction with clinical experience and physical exams to determine when interventional therapies such as needle aspiration of the built up cerebrospinal fluid (ventricle tap, VT) might be indicated for a patient; however, quantitative measurements of the ventricles size are often not performed. We describe the potential utility of the quantitative three-dimensional (3-D) US measurements of ventricle volumes (VVs) in 38 preterm neonates to monitor and manage PHVD. Specifically, we determined 3-D US VV thresholds for patients who received VT in comparison to patients with PHVD who resolve without intervention. In addition, since many patients who have an initial VT will receive subsequent interventions, we determined which PHVD patients will receive additional VT after the initial one has been performed.

We designed and generated pulmonary imaging biomarker pipelines to facilitate high-throughput research and point-of-care use in patients with chronic lung disease. Image processing modules and algorithm pipelines were embedded within a graphical user interface (based on the .NET framework) for pulmonary magnetic resonance imaging (MRI) and x-ray computed-tomography (CT) datasets. The software pipelines were generated using C++ and included: (1) inhaled He3  /  Xe129 MRI ventilation and apparent diffusion coefficients, (2) CT-MRI coregistration for lobar and segmental ventilation and perfusion measurements, (3) ultrashort echo-time H1 MRI proton density measurements, (4) free-breathing Fourier-decomposition H1 MRI ventilation/perfusion and free-breathing H1 MRI specific ventilation, (5) multivolume CT and MRI parametric response maps, and (6) MRI and CT texture analysis and radiomics. The image analysis framework was implemented on a desktop workstation/tablet to generate biomarkers of regional lung structure and function related to ventilation, perfusion, lung tissue texture, and integrity as well as multiparametric measures of gas trapping and airspace enlargement. All biomarkers were generated within 10 min with measurement reproducibility consistent with clinical and research requirements. The resultant pulmonary imaging biomarker pipeline provides real-time and automated lung imaging measurements for point-of-care and high-throughput research.

We are interested in examining how our model-based beamforming algorithm, referred to as aperture-domain model image reconstruction (ADMIRE), performs on plane wave sequences in conjunction with synthetic aperture beamforming. We also aim to identify the impact of ADMIRE applied before and after synthetic focusing. We employed simulated phantoms using Field II and tissue-mimicking phantoms to evaluate ADMIRE as applied to synthetic sequencing. We generated plane wave images with and without synthetic aperture focusing (SAF) and measured contrast and contrast-to-noise ratio (CNR). For simulated cyst images formed from single plane waves, the contrast for delay-and-sum (DAS) and ADMIRE are 15.64 and 28.34 dB, respectively, whereas the CNR are 1.76 and 3.90 dB, respectively. We also applied ADMIRE to simulated resolution phantoms having a point target at 3 cm depth on-axis. We simulated the point spread functions from data obtained from 1 plane wave and 75 steered plane waves, along with linear scans with 3 and 4 cm- focal depths. We then compared the outcome of applying ADMIRE before and after SAF using 3 and 11 steered plane waves. Finally, we applied this to an in vivo carotid artery. Based on the findings in this study, ADMIRE can be adapted to full field insonification sequences to improve image quality in plane wave imaging. Additionally, we investigated how robustly ADMIRE performs in the presence of random noise. We then address identified limitations using a conventional envelope detection method with decluttered signals.

Blood smear is a crucial diagnostic aid. Quantification of both solitary and overlapping erythrocytes within these smears, directly from their whole slide images (WSIs), remains a challenge. Existing software designed to accomplish the computationally extensive task of hematological WSI analysis is too expensive and is widely unavailable. We have thereby developed a fully automated software targeted for erythrocyte detection and quantification from WSIs. We define an optimal region within the smear, which contains cells that are neither too scarce/damaged nor too crowded. We detect the optimal regions within the smear and subsequently extract all the cells from these regions, both solitary and overlapped, the latter of which undergoes a clump splitting before extraction. The performance was systematically tested on 28 WSIs of blood smears obtained from 13 different species from three classes of the subphylum vertebrata including birds, mammals, and reptiles. These data pose as an immensely variant erythrocyte database with diversity in size, shape, intensity, and textural features. Our method detected ∼3.02  times more cells than that detected from the traditional monolayer and resulted in a testing accuracy of 99.14% for the classification into their respective class (bird, mammal, or reptile) and a testing accuracy of 84.73% for the classification into their respective species. The results suggest the potential employment of this software for the diagnosis of hematological disorders, such as sickle cell anemia.