The guest editorial introduces the Special Section on Interventional and Surgical Data Science.

Purpose: Data-intensive modeling could provide insight on the broad variability in outcomes in spine surgery. Previous studies were limited to analysis of demographic and clinical characteristics. We report an analytic framework called “SpineCloud” that incorporates quantitative features extracted from perioperative images to predict spine surgery outcome.

Approach: A retrospective study was conducted in which patient demographics, imaging, and outcome data were collected. Image features were automatically computed from perioperative CT. Postoperative 3- and 12-month functional and pain outcomes were analyzed in terms of improvement relative to the preoperative state. A boosted decision tree classifier was trained to predict outcome using demographic and image features as predictor variables. Predictions were computed based on SpineCloud and conventional demographic models, and features associated with poor outcome were identified from weighting terms evident in the boosted tree.

Results: Neither approach was predictive of 3- or 12-month outcomes based on preoperative data alone in the current, preliminary study. However, SpineCloud predictions incorporating image features obtained during and immediately following surgery (i.e., intraoperative and immediate postoperative images) exhibited significant improvement in area under the receiver operating characteristic (AUC): AUC  =  0.72 (CI₉₅  =  0.59 to 0.83) at 3 months and AUC  =  0.69 (CI₉₅  =  0.55 to 0.82) at 12 months.

Conclusions: Predictive modeling of lumbar spine surgery outcomes was improved by incorporation of image-based features compared to analysis based on conventional demographic data. The SpineCloud framework could improve understanding of factors underlying outcome variability and warrants further investigation and validation in a larger patient cohort.

To compensate for the intraoperative brain tissue deformation, computer-assisted intervention methods have been used to register preoperative magnetic resonance images with intraoperative images. In order to model the deformation due to tissue resection, the resection cavity needs to be segmented in intraoperative images. We present an automatic method to segment the resection cavity in intraoperative ultrasound (iUS) images. We trained and evaluated two-dimensional (2-D) and three-dimensional (3-D) U-Net networks on two datasets of 37 and 13 cases that contain images acquired from different ultrasound systems. The best overall performing method was the 3-D network, which resulted in a 0.72 mean and 0.88 median Dice score over the whole dataset. The 2-D network also had good results with less computation time, with a median Dice score over 0.8. We also evaluated the sensitivity of network performance to training and testing with images from different ultrasound systems and image field of view. In this application, we found specialized networks to be more accurate for processing similar images than a general network trained with all the data. Overall, promising results were obtained for both datasets using specialized networks. This motivates further studies with additional clinical data, to enable training and validation of a clinically viable deep-learning model for automated delineation of the tumor resection cavity in iUS images.

Purpose: Cochlear implants (CIs) use an array of electrodes surgically threaded into the cochlea to restore hearing sensation. Techniques for predicting the insertion depth of the array into the cochlea could guide surgeons toward more optimal placement of the array to reduce trauma and preserve the residual hearing. In addition to the electrode array geometry, the base insertion depth (BID) and the cochlear size could impact the overall array insertion depth.

Approach: We investigated using these measurements to develop a linear regression model that can make preoperative or intraoperative predictions of the insertion depth of lateral wall CI electrodes. Computed tomography (CT) images of 86 CI recipients were analyzed. Using previously developed automated algorithms, the relative electrode position inside the cochlea was measured from the CT images.

Results: A linear regression model is proposed for insertion depth prediction based on cochlea size, array geometry, and BID. The model is able to accurately predict angular insertion depths with a standard deviation of 41 deg and absolute deviation error of 32 deg.

Conclusions: Surgeons may use this model for patient-customized selection of electrode array and/or to plan a BID for a given array that minimizes the likelihood of causing trauma to regions of the cochlea where residual hearing exists.

Purpose: Glioblastoma, the most common and aggressive adult brain tumor, is considered noncurative at diagnosis, with 14 to 16 months median survival following treatment. There is increasing evidence that noninvasive integrative analysis of radiomic features can predict overall and progression-free survival, using advanced multiparametric magnetic resonance imaging (Adv-mpMRI). If successfully applicable, such noninvasive markers can considerably influence patient management. However, most patients prior to initiation of therapy typically undergo only basic structural mpMRI (Bas-mpMRI, i.e., T1, T1-Gd, T2, and T2-fluid-attenuated inversion recovery) preoperatively, rather than Adv-mpMRI that provides additional vascularization (dynamic susceptibility contrast-MRI) and cell-density (diffusion tensor imaging) related information.

Approach: We assess a retrospective cohort of 101 glioblastoma patients with available Adv-mpMRI from a previous study, which has shown that an initial feature panel (IFP, i.e., intensity, volume, location, and growth model parameters) extracted from Adv-mpMRI can yield accurate overall survival stratification. We focus on demonstrating that equally accurate prediction models can be constructed using augmented radiomic feature panels (ARFPs, i.e., integrating morphology and textural descriptors) extracted solely from widely available Bas-mpMRI, obviating the need for using Adv-mpMRI. We extracted 1612 radiomic features from distinct tumor subregions to build multivariate models that stratified patients as long-, intermediate-, or short-survivors.

Results: The classification accuracy of the model utilizing Adv-mpMRI protocols and the IFP was 72.77% and degraded to 60.89% when using only Bas-mpMRI. However, utilizing the ARFP on Bas-mpMRI improved the accuracy to 74.26%. Furthermore, Kaplan–Meier analysis demonstrated superior classification of subjects into short-, intermediate-, and long-survivor classes when using ARFP extracted from Bas-mpMRI.

Conclusions: This quantitative evaluation indicates that accurate survival prediction in glioblastoma patients is feasible using solely Bas-mpMRI and integrative advanced radiomic features, which can compensate for the lack of Adv-mpMRI. Our finding holds promise for generalization across multiple institutions that may not have access to Adv-mpMRI and to better inform clinical decision-making about aggressive interventions and clinical trials.

Purpose: For many patients with intracranial tumors, accurate surgical resection is a mainstay of their treatment paradigm. During surgical resection, image guidance is used to aid in localization and resection. Intraoperative brain shift can invalidate these guidance systems. One cause of intraoperative brain shift is cavity collapse due to tumor resection, which will be referred to as “debulking.” We developed an imaging-driven finite element model of debulking to create a comprehensive simulation data set to reflect possible intraoperative changes. The objective was to create a method to account for brain shift due to debulking for applications in image-guided neurosurgery. We hypothesized that accounting for tumor debulking in a deformation atlas data framework would improve brain shift predictions, which would enhance image-based surgical guidance.

Approach: This was evaluated in a six-patient intracranial tumor resection intraoperative data set. The brain shift deformation atlas data framework consisted of n  =  756 simulated deformations to account for effects due to gravity-induced and hyperosmotic drug-induced brain shift, which reflects previous developments. An additional complement of n  =  84 deformations involving simulated tumor growth followed by debulking was created to capture observed intraoperative effects not previously included.

Results: In five of six patient cases evaluated, inclusion of debulking mechanics improved brain shift correction by capturing global mass effects resulting from the resected tumor.

Conclusions: These findings suggest imaging-driven brain shift models used to create a deformation simulation data framework of observed intraoperative events can be used to assist in more accurate image-guided surgical navigation in the brain.

Purpose: Our paper contributes to the burgeoning field of surgical data science. Specifically, multimodal integration of relevant patient data is used to determine who should undergo a complex pancreatic resection. Intraductal papillary mucinous neoplasms (IPMNs) represent cystic precursor lesions of pancreatic cancer with varying risk for malignancy. We combine previously defined individual models of radiomic analysis of diagnostic computed tomography (CT) with protein markers extracted from the cyst fluid to create a unified prediction model to identify high-risk IPMNs. Patients with high-risk IPMN would be sent for resection, whereas patients with low-risk cystic lesions would be spared an invasive procedure.

Approach: Retrospective analysis of prospectively acquired cyst fluid and CT scans was undertaken for this study. A predictive model combining clinical features with a cyst fluid inflammatory marker (CFIM) was applied to patient data. Quantitative imaging (QI) features describing radiomic patterns predictive of risk were extracted from scans. The CFIM model and QI model were combined into a single predictive model. An additional model was created with tumor-associated neutrophils (TANs) assessed by a pathologist at the time of resection.

Results: Thirty-three patients were analyzed (7 high risk and 26 low risk). The CFIM model yielded an area under the curve (AUC) of 0.74. Adding the QI model improved performance with an AUC of 0.88. Combining the CFIM, QI, and TAN models further increased performance to an AUC of 0.98.

Conclusions: Quantitative analysis of routinely acquired CT scans combined with CFIMs provides accurate prediction of risk of pancreatic cancer progression. Although a larger cohort is needed for validation, this model represents a promising tool for preoperative assessment of IPMN.

Guest editors Scott D. Metzler, Samuel Matej, and J. Webster Stayman provide an introduction to the Special Section on Three-Dimensional Image Reconstruction in Nuclear Medicine, PET, and CT.

Purpose: Bayesian theory provides a sound framework for ultralow-dose computed tomography (ULdCT) image reconstruction with two terms for modeling the data statistical property and incorporating a priori knowledge for the image that is to be reconstructed. We investigate the feasibility of using a machine learning (ML) strategy, particularly the convolutional neural network (CNN), to construct a tissue-specific texture prior from previous full-dose computed tomography.

Approach: Our study constructs four tissue-specific texture priors, corresponding with lung, bone, fat, and muscle, and integrates the prior with the prelog shift Poisson (SP) data property for Bayesian reconstruction of ULdCT images. The Bayesian reconstruction was implemented by an algorithm called SP-CNN-T and compared with our previous Markov random field (MRF)-based tissue-specific texture prior algorithm called SP-MRF-T.

Results: In addition to conventional quantitative measures, mean squared error and peak signal-to-noise ratio, structure similarity index, feature similarity, and texture Haralick features were used to measure the performance difference between SP-CNN-T and SP-MRF-T algorithms in terms of the structure and tissue texture preservation, demonstrating the feasibility and the potential of the investigated ML approach.

Conclusions: Both training performance and image reconstruction results showed the feasibility of constructing CNN texture prior model and the potential of improving the structure preservation of the nodule comparing to our previous regional tissue-specific MRF texture prior model.

Purpose: Neural network image reconstruction directly from measurement data is a relatively new field of research, which until now has been limited to producing small single-slice images (e.g., 1  ×  128  ×  128). We proposed a more efficient network design for positron emission tomography called DirectPET, which is capable of reconstructing multislice image volumes (i.e., 16  ×  400  ×  400) from sinograms.

Approach: Large-scale direct neural network reconstruction is accomplished by addressing the associated memory space challenge through the introduction of a specially designed Radon inversion layer. Using patient data, we compare the proposed method to the benchmark ordered subsets expectation maximization (OSEM) algorithm using signal-to-noise ratio, bias, mean absolute error, and structural similarity measures. In addition, line profiles and full-width half-maximum measurements are provided for a sample of lesions.

Results: DirectPET is shown capable of producing images that are quantitatively and qualitatively similar to the OSEM target images in a fraction of the time. We also report on an experiment where DirectPET is trained to map low-count raw data to normal count target images, demonstrating the method’s ability to maintain image quality under a low-dose scenario.

Conclusion: The ability of DirectPET to quickly reconstruct high-quality, multislice image volumes suggests potential clinical viability of the method. However, design parameters and performance boundaries need to be fully established before adoption can be considered.

Purpose: The paper addresses exact inversion of the integral transform, called the Compton (or cone) transform, that maps a three-dimensional (3-D) function to its integrals over conical surfaces in R³. Compton transform arises in passive detection of gamma-ray sources with a Compton camera, which has promising applications in medical and industrial imaging as well as in homeland security imaging and astronomy.

Approach: A generalized identity relating the Compton and the Radon transforms was formulated. The proposed relation can be used to devise a method for converting the Compton transform data of a function into its Radon projections. The function can then be recovered using well-known inversion techniques for the Radon transform.

Results: We derived a two-step method that uses the full set of available projections to invert the Compton transform: first, the recovery of the Radon transform from the Compton transform, and then the Radon transform inversion. The proposed technique is independent of the geometry of detectors as long as a generous admissibility condition is met.

Conclusions: We proposed an exact inversion formula for the 3-D Compton transform. The stability of the inversion algorithm was demonstrated via numerical simulations.

Purpose: In spite of the general acceptance of iterative reconstruction for clinical use, analytic algorithms provide an important alternative tool due to their linearity, unbiased performance, and predictability for quantitative imaging and quality control studies. On modern time-of-flight (TOF) positron emission tomography scanners with excellent timing resolution, substantial angular compression of (histoprojection) data is possible without loss of resolution, but this also brings challenges for analytical algorithms. We propose TOF and non-TOF Fourier-based analytic approaches that appropriately handle the data sparsity on modern TOF systems.

Approach: The proposed TOF algorithm (3D-DIFTOF—direct inversion Fourier transform for TOF) works directly on histoprojection data. The proposed Fourier-based approaches for histoprojection data are further extended to include non-TOF reconstruction (TOF-binned 3D-DIFT), which is particularly useful in time calibration procedures due to its insensitivity to time calibration errors. TOF information is used here to extend available histoprojection data to a larger number of views, essential for artifact-free non-TOF reconstruction. The proposed algorithms are compared with standard analytic techniques on Siemens scanners—space-based confidence-weighted TOF FBP and non-TOF DIFT.

Results: 3D-DIFTOF reconstruction demonstrates both improved NEMA-based resolution and contrast versus background variability trade-offs. Similarly, the TOF-binned 3D-DIFT approach shows improved contrast-noise trade-offs over the standard non-TOF approach and is well suited for timing calibration.

Conclusions: Our results demonstrate that the proposed 3D-DIFTOF technique provides an improved and more faithful characterization of image resolution compared with standard space-based analytic reconstructions. The proposed tools also provide accurate translation of sparse TOF data available on clinical scanners to upsampled data for non-TOF algorithms.

Purpose: Monte Carlo simulations were used to evaluate the imaging properties of a composite direct–indirect active matrix flat-panel imager (AMFPI) with potentially more favorable tradeoffs between x-ray quantum efficiency and spatial resolution than direct or indirect AMFPIs alone. This configuration, referred to as a hybrid AMFPI, comprises a scintillator that is optically coupled to an a-Se direct AMFPI through a transparent electrode and hole blocking layer, such that a-Se acts as both a direct x-ray converter and an optical sensor.

Approach: GEANT4 was used to simulate x-ray energy deposition, optical transport, and charge signal generation processes in various hybrid AMPFI configurations under RQA5 and RQA9 x-ray beam conditions. The Fujita–Lubberts–Swank method was used to quantify the impact of irradiation geometry, x-ray converter thicknesses, conversion gain of each layer, and x-ray cross talk between layers on detective quantum efficiency (DQE).

Results: Each hybrid configuration had a greater DQE than its direct AMFPI layer alone. The DQE improvement was largest at low spatial frequencies in both front- and back-irradiation (BI) geometries due to increased x-ray quantum efficiency provided by the scintillator. DQE improvements persisted at higher frequencies in BI geometry due to preferential x-ray absorption in a-Se. Matching the x-ray-to-charge conversion gains of a hybrid AMFPI’s direct and indirect detection layers affects its Swank factor and, thus, DQE(0). X-ray cross talk has a negligible impact on the DQE  (  f  )   of hybrid AMFPIs with sufficiently high optical quantum efficiency.

Conclusion: An optimized hybrid AMFPI can achieve greater DQE performance than current direct or indirect AMFPIs.

Purpose: We present photon-counting computed tomography (PCCT) imaging of contrast agent triplets similar in atomic number (Z) achieved with a high-flux cadmium zinc telluride (CZT) detector.

Approach: The table-top PCCT imaging system included a 330-μm-pitch CZT detector of size 8  mm  ×  24  mm² capable of using six energy bins. Four 3D-printed 3-cm-diameter phantoms each contained seven 6-mm-diameter vials with water and low and high concentration solutions of various contrast agents. Lanthanum (Z  =  57), gadolinium (Gd) (Z  =  64), and lutetium (Z  =  71) were imaged together and so were iodine (Z  =  53), Gd, and holmium (Z  =  67). Each phantom was imaged with 1-mm aluminum-filtered 120-kVp cone beam x rays to produce six energy-binned computed tomography (CT) images.

Results: K-edge images were reconstructed using a weighted sum of six CT images, which distinguished each contrast agent with a root-mean-square error (RMSE) of <0.29  %   and 0.51% for the 0.5% and 5% concentrations, respectively. Minimal cross-contamination in each K-edge image was seen, with RMSE values <0.27  %   in vials with no contrast.

Conclusion: This is the first preliminary demonstration of simultaneously imaging three similar Z contrast agents with a difference in Z as low as 3.

Purpose: Impaired insulin-induced microvascular recruitment in skeletal muscle contributes to insulin resistance in type 2 diabetic disease. Previously, quantification of microvascular recruitment at the capillary level has been performed with either the full image or manually selected region-of-interests. These subjective approaches are imprecise, time-consuming, and unsuitable for automated processes. Here, an automated multiscale image processing approach was performed by defining a vessel diameter threshold for an objective and reproducible analysis at the microvascular level.

Approach: A population of C57BL/6J male mice fed standard chow and studied at age 13 to 16 weeks comprised the lean group and 24- to 31-week-old mice who received a high-fat diet were designated the obese group. A clinical ultrasound scanner (Acuson Sequoia 512) equipped with an 15L8-S linear array transducer was used in a nonlinear imaging mode for sensitive detection of an intravascular microbubble contrast agent.

Results: By eliminating large vessels from the dynamic contrast-enhanced ultrasound (DCE-US) images (above 300  μm in diameter), obesity-related changes in perfusion and morphology parameters were readily detected in the smaller vessels, which are known to have a greater impact on skeletal muscle glucose disposal. The results from the DCE-US images including all of the vessels were compared for three different-sized vessel groups, namely, vessels smaller than 300, 200, and 150  μm in diameter.

Conclusions: Our automated image processing provides objective and reproducible results by focusing on a particular size of vessel, thereby allowing for a selective evaluation of longitudinal changes in microvascular recruitment for a specific-sized vessel group between diseased and healthy microvascular networks.

Purpose: Quality assurance (QA) of dose homogeneity in total skin electron therapy (TSET) is challenging since each patient is positioned in six standing poses with two beam angles. Our study tested the feasibility of a unique approach for TSET QA through computational display of the cumulative dose, constructed and synthesized by computer animation methods.

Approach: Dose distributions from Cherenkov emission images were projected onto a scanned 3D body model. Topographically mapped surfaces of the patient were recorded in each of six different delivery positions, while a Cherenkov camera acquired images. Computer animation methods allowed a fitted 3D human body model of the patient to be created with deformation of the limbs and torso to each position. A two-dimensional skin map was extracted from the 3D model of the full surface of the patient. This allowed the dose mapping to be additively accumulated independent of body position, with the total dose summed in a 2D map and reinterpreted on the 3D body display.

Results: For the body model, the mean Hausdorff error distance was below 2 cm, setting the spatial accuracy limit. The dose distribution over the patient’s 3D model generally matched the Cherenkov/dose images. The dose distribution mapping was estimated to be near 1.5 cm accuracy based upon a phantom study. The body model must most closely match at the edges of the mesh to ensure that high dose gradients are not projected onto the wrong location. Otherwise 2 to 3 cm level errors in positioning in the mesh do not appear to cause larger than 5% dose errors. The cumulative dose images showed regions of overlap laterally and regions of low intensity in the posterior arms.

Conclusions: The proposed modeling and animation can be used to visualize and analyze the accumulated dose in TSET via display of the summed dose/Cherenkov images on a single body surface.

Purpose: Diabetic retinopathy is the leading cause of blindness, affecting over 93 million people. An automated clinical retinal screening process would be highly beneficial and provide a valuable second opinion for doctors worldwide. A computer-aided system to detect and grade the retinal images would enhance the workflow of endocrinologists.

Approach: For this research, we make use of a publicly available dataset comprised of 3662 images. We present a hybrid machine learning architecture to detect and grade the level of diabetic retinopathy (DR) severity. We also present and compare simple transfer learning-based approaches using established networks such as AlexNet, VGG16, ResNet, Inception-v3, NASNet, DenseNet, and GoogLeNet for DR detection. For the grading stage (mild, moderate, proliferative, or severe), we present an approach of combining various convolutional neural networks with principal component analysis for dimensionality reduction and a support vector machine classifier. We study the performance of these networks under different preprocessing conditions.

Results: We compare these results with various existing state-of-the-art approaches, which include single-stage architectures. We demonstrate that this architecture is more robust to limited training data and class imbalance. We achieve an accuracy of 98.4% for DR detection and an accuracy of 96.3% for distinguishing severity of DR, thereby setting a benchmark for future research efforts using a limited set of training images.

Conclusions: Results obtained using the proposed approach serve as a benchmark for future research efforts. We demonstrate as a proof-of-concept that an automated detection and grading system could be developed with a limited set of images and labels. This type of independent architecture for detection and grading could be used in areas with a scarcity of trained clinicians based on the necessity.

Purpose: Measurement of global spinal alignment (GSA) is an important aspect of diagnosis and treatment evaluation for spinal deformity but is subject to a high level of inter-reader variability.

Approach: Two methods for automatic GSA measurement are proposed to mitigate such variability and reduce the burden of manual measurements. Both approaches use vertebral labels in spine computed tomography (CT) as input: the first (EndSeg) segments vertebral endplates using input labels as seed points; and the second (SpNorm) computes a two-dimensional curvilinear fit to the input labels. Studies were performed to characterize the performance of EndSeg and SpNorm in comparison to manual GSA measurement by five clinicians, including measurements of proximal thoracic kyphosis, main thoracic kyphosis, and lumbar lordosis.

Results: For the automatic methods, 93.8% of endplate angle estimates were within the inter-reader 95% confidence interval (CI₉₅). All GSA measurements for the automatic methods were within the inter-reader CI₉₅, and there was no statistically significant difference between automatic and manual methods. The SpNorm method appears particularly robust as it operates without segmentation.

Conclusions: Such methods could improve the reproducibility and reliability of GSA measurements and are potentially suitable to applications in large datasets—e.g., for outcome assessment in surgical data science.

Purpose: This technology evaluation study assesses the limits of agreement between the mammographic density (MD) measurement of Quantra™ from different breasts and mammographic views and its agreement with the Royal Australian and New Zealand College of Radiologists (RANZCR) synoptic scale.

Approach: MD of 800 women was assessed by Quantra™ and seven radiologists using the RANZCR synoptic scale. Bland–Altman analysis was used to assess the limits of agreement between Quantra™ MD measures from both breasts and mammographic views. The agreement between Quantra™ and the RANZCR synoptic scale was assessed using weighted kappa (K_w). The receiver operating characteristics area under the curve (AUC) was used to assess the performance of Quantra™ in reproducing RANZCR MD ratings.

Results: There was no significant bias in the mean MD of Quantra™ from both breasts: left versus right craniocaudal (CC) views (B  =    −  0.14; p  =  0.36) and right versus left mediolateral oblique (MLO) (B  =    −  0.021; p  =  0.18). However, MD measures from the same breast but different views showed significant bias: right CC versus right MLO (B  =  0.064; p  <  0.0001) and left CC versus left MLO (B  =  0.56; p  <  0.0001). Quantra™ demonstrated substantial agreement with the RANZCR synoptic scale on four- and two-category scales (K_w  =  0.62; 0.59 to 0.66 and 0.76; 0.72 to 0.81, respectively). Quantra™ better reproduced the RANZCR synoptic scale on a two-category scale (AUC  =  0.88; 0.84 to 0.91) than a four-category scale (AUC  =  0.62; 0.58 to 0.67 to 0.78; 0.74 to 0.82).

Conclusions: Quantra™ reproduces MD classification using the RANZCR synoptic scale on a two-category scale and should help in identification of women with dense breasts who may need adjunctive imaging for early detection of breast cancer.