Dr. Patrick A. Helm Profile

Dr. Patrick A. Helm

SPIE Involvement:

Author

Publications (19)

Proceedings Article | 1 April 2024 Presentation + Paper

Presentation + Paper

One-shot estimation of epistemic uncertainty in deep learning image formation with application to high-quality cone-beam CT reconstruction

Stephen Liu, Prasad Vagdargi, Craig Jones, Mark Luciano, William Anderson, Patrick Helm, Ali Uneri, Jeffrey Siewerdsen, Wojciech Zbijewski, Alejandro Sisniega

Proceedings Volume 12925, 129251E (2024) https://doi.org/10.1117/12.3006934

KEYWORDS: Medical image reconstruction, Cone beam computed tomography, Education and training, Deep learning, Brain, Computed tomography, Error analysis, Data modeling, 3D modeling, Monte Carlo methods

Read Abstract +

Deep Learning (DL) image synthesis has gained increasing popularity for the reconstruction of CT and cone-beam CT (CBCT) images, especially in combination with physically-principled reconstruction algorithms. However, DL synthesis is challenged by the generalizability of training data and noise in the trained model. Epistemic uncertainty has proven as an efficient way of quantifying erroneous synthesis in the presence of out-of-domain features, but its estimation with Monte Carlo (MC) dropout requires a large number of inference runs, variable as a function of the particular uncertain feature. We propose a single-pass method–the Moment Propagation Model–which approximates the MC dropout by analytically propagating the statistical moments through the network layers, removing the need for multiple inferences and removing errors in estimations from insufficient dropout realizations. The proposed approach jointly computes the change of the expectation and the variance of the input (first two statistical moments) through each network layer, where each moment undergoes a different numerical transformation. The expectation is initialized as the network input; the variance is solely introduced at dropout layers, modeled as a Bernoulli process. The method was evaluated using a 3D Bayesian conditional generative adversarial network (GAN) for synthesis of high-quality head MDCT from low-quality intraoperative CBCT reconstructions. 20 pairs of measured MDCT volumes (120kV, 400 to 550mAs) depicting normal head anatomy, and simulated CBCT volumes (100 to120kV, 32 to 200mAs) were used for training. Scatter, beam-hardening, detector lag and glare were added to the simulated CBCT and were corrected (assuming unknown) prior to reconstruction. Epistemic uncertainty was estimated for 30 heads (outside of the training set) containing simulated brain lesions using the proposed single-pass propagation model, and results were compared to the standard 200-pass dropout approach. Image quality and quantitative accuracy of the estimated uncertainty of lesions and other anatomical sites were further evaluated. The proposed propagation model captured >2HU increase in epistemic uncertainty caused by various hyper- and hypo-density lesions, with <0.31HU error over the brain compared to the reference MC dropout result at 200 inferences and <0.1HU difference to a converged MC dropout estimate at 100 inference passes. These findings indicate a potential 100-fold increase in computational efficiency of neural network uncertainty estimation. The proposed moment propagation model is able to achieve accurate quantification of epistemic uncertainty in a single network pass and is an efficient alternative to conventional MC dropout.

Proceedings Article | 29 March 2024 Presentation + Paper

Presentation + Paper

End-to-end 3D neuroendoscopic video reconstruction for robot-assisted ventriculostomy

P. Vagdargi, A. Uneri, S. Liu, C. Jones, A. Sisniega, J. Lee, P. Helm, W. Anderson, M. Luciano, G. Hager, J. Siewerdsen

Proceedings Volume 12928, 129280M (2024) https://doi.org/10.1117/12.3008758

KEYWORDS: Video, Education and training, Deformation, 3D acquisition, Voxels, 3D image processing, Imaging systems, Endoscopy, 3D image reconstruction, Visualization

Read Abstract +

Proceedings Article | 29 March 2024 Presentation + Paper

Presentation + Paper

Probe positioning for robot-assisted intraoperative ultrasound imaging using deep reinforcement learning

Y. Hu, Y. Huang, A. Song, C. Jones, J. Siewerdsen, B. Basar, P. Helm, A. Uneri

Proceedings Volume 12928, 1292803 (2024) https://doi.org/10.1117/12.3006918

KEYWORDS: Education and training, Computed tomography, Ultrasonography, Kidney, Image segmentation, 3D modeling, Motion models, Detection and tracking algorithms, Biomedical optics, Biological imaging

Read Abstract +

Proceedings Article | 3 April 2023 Presentation + Paper

Presentation + Paper

Real-time 3D neuroendoscopic guidance using SLAM: first clinical studies

P. Vagdargi, A. Uneri, C. Jones, X. Zhang, P. Wu, R. Han, A. Sisniega, J. Lee, P. Helm, M. Luciano, W. Anderson, G. Hager, J. Siewerdsen

Proceedings Volume 12466, 1246604 (2023) https://doi.org/10.1117/12.2654595

KEYWORDS: Calibration, Distortion, Deformation, Video, Cameras, Point clouds, Endoscopy, Visualization, Neurosurgery, Vision based navigation, Navigation systems

Read Abstract +

Neurosurgical techniques often require accurate targeting of deep-brain structures even in the presence of deformation due intervention and egress of Cerebrospinal Fluid (CSF) during surgical access. Prior work reported Simultaneous Localization and Mapping (SLAM) methods for endoscopic guidance using 3D reconstruction. In this work, methods for correcting the geometric distortion of a neuroendoscope are reported in a form that have been translated intraoperative use in first clinical studies. Furthermore, SLAM methods are evaluated in first clinical studies for real-time 3D endoscopic navigation with near real-time registration in the presence of deep-brain tissue deformation. A custom calibration jig with swivel mounts was designed and manufactured for neuroendoscope calibration in the operating room. The process is potentially suitable to intraoperative use while maintaining sterility of the endoscope, although the current calibration system was used in the Operating Room (OR) immediately following the case for offline analysis. A six by seven checkerboard pattern was used to obtain corner locations for calibration, and the method was evaluated in terms of Reprojection Error (RPE). Neuroendoscopic video was acquired under an IRB-approved clinical study, demonstrating rich vascular features and other structures on the interior walls of the lateral ventricles for 3D point-cloud reconstruction. Geometric accuracy was evaluated in terms of Projected Error (PE) on a ground truth surface defined from MR or cone-beam CT (CBCT) images. Intraoperative neuroendoscope calibration was achieved with sub-pixel [0.61 ± 0.20 px] error. The calibration yielded a focal length of 816.42 px and 822.71 px in X and Y directions respectively, along with radial distortion coefficients of -0.432 (first order term [𝑘1]) and 0.158 (second order term [𝑘2]). The 3D reconstruction was performed successfully with a PE of 0.23 ± 0.15 mm compared to the ground truth surface. The system for neuroendoscopic guidance based on SLAM 3D point-cloud reconstruction provided a promising platform for the development of 3D neuroendoscopy. The studies reported in this work presented an important means of neuroendoscope calibration in the OR and provided preliminary evidence for accurate 3D video reconstruction in first clinical studies. Future work aims to further extend the clinical evaluation and improve reconstruction accuracy using ventricular shape priors.

Proceedings Article | 3 April 2023 Presentation + Paper

Presentation + Paper

Multi-modality registration of preoperative MR and intraoperative long-length tomosynthesis using GAN synthesis and 3D-2D registration

Y. Huang, X. Zhang, A. Johnston, C. Jones, J. Siewerdsen, W. Zbijewski, P. Helm, B. Judy, A. Sacino, A. Bydon, T. Witham, A. Uneri

Proceedings Volume 12466, 124661G (2023) https://doi.org/10.1117/12.2654329

KEYWORDS: Magnetic resonance imaging, Image registration, 3D image processing, Spine, Computed tomography, Anatomy, Tomosynthesis, X-ray computed tomography, Radiography

Read Abstract +

Proceedings Article | 18 October 2022 Open Access

Open Access

Paper

DL-Recon: combining 3D deep learning image synthesis and model uncertainty with physics-based image reconstruction

Xiaoxuan Zhang, Pengwei Wu, Wojciech Zbijewski, Alejandro Sisniega, Runze Han, Craig Jones, Prasad Vagdargi, Ali Uneri, Patrick Helm, William Anderson, Jeffrey Siewerdsen

Proceedings Volume 12304, 1230408 (2022) https://doi.org/10.1117/12.2646383

KEYWORDS: 3D modeling, Brain, Computed tomography, 3D image reconstruction, 3D image processing, Gallium nitride, Image quality, Neuroimaging, Image restoration, Image analysis

Read Abstract +

High-precision image-guided neurosurgery – especially in the presence of brain shift – would benefit from intraoperative image quality beyond the conventional contrast-resolution limits of cone-beam CT (CBCT) for visualization of the brain parenchyma, ventricles, and intracranial hemorrhage. Deep neural networks for 3D image reconstruction offer a promising basis for noise and artifact reduction, but generalizability can be challenged in scenarios involving features previously unseen in training data. We propose a 3D deep learning reconstruction framework (termed “DL-Recon”) that integrates learning-based image synthesis with physics-based reconstruction to leverage strengths of each. A 3D conditional GAN was developed to generate synthesized CT from CBCT images. Uncertainty in the synthesis image was estimated in a spatially varying, voxel-wise manner via Monte-Carlo dropout and was shown to correlate with abnormalities or pathology not present in training data. The DL-Recon approach improves the fidelity of the resulting image by combining the synthesized image (“DL-Synthesis”) with physics-based reconstruction (filtered back-projection (FBP) or other approaches) in a manner weighted by uncertainty – i.e., drawing more from the physics-based method in regions where model uncertainty is high. The performance of image synthesis, uncertainty estimation, and DL-Recon was investigated for the first time in real CBCT images of the brain. Variable input to the synthesis network was tested – including uncorrected FBP and precorrection with a simple (constant) scatter estimate – hypothesizing the latter to improve synthesis performance. The resulting uncertainty estimation was evaluated for the first time in real anatomical features not included in training (abnormalities and brain shift). The performance of DL-Recon was evaluated in terms of image uniformity, noise, and soft-tissue contrast-to-noise ratio in comparison to DL-Synthesis and FBP with a comprehensive artifact correction framework. DL-Recon was found to leverage the strengths of the learning-based and physics-based reconstruction approaches, providing a high degree of image uniformity similar to DL-Synthesis while accurately preserving soft-tissue contrast as in artifact-corrected FBP.

Proceedings Article | 4 April 2022 Paper

Paper

Deformable registration of MRI to intraoperative cone-beam CT of the brain using a joint synthesis and registration network

R. Han, C. Jones, P. Wu, P. Vagdargi, X. Zhang, A. Uneri, J. Lee, M. Luciano, W. Anderson, P. Helm, J. Siewerdsen

Proceedings Volume 12034, 1203407 (2022) https://doi.org/10.1117/12.2611783

KEYWORDS: Image registration, Magnetic resonance imaging, Brain

Read Abstract +

Proceedings Article | 4 April 2022 Presentation + Paper

Presentation + Paper

Automatic labeling of vertebrae in long-length intraoperative imaging with a multi-view, region-based CNN

Y. Huang, C. Jones, X. Zhang, A. Johnston, N. Aygun, T. Witham, P. Helm, J. Siewerdsen, A. Uneri

Proceedings Volume 12034, 120340U (2022) https://doi.org/10.1117/12.2611912

KEYWORDS: Spine, Computed tomography, Surgery, Medicine, 3D image processing, Diagnostics, Convolutional neural networks

Read Abstract +

Proceedings Article | 4 April 2022 Presentation + Paper

Presentation + Paper

Robot-assisted neuroendoscopy for real-time 3D guidance of transventricular approach to deep-brain targets

P. Vagdargi, A. Uneri, C. Jones, P. Wu, R. Han, M. Luciano, W. Anderson, P. Helm, G. Hager, J. Siewerdsen

Proceedings Volume 12034, 120340E (2022) https://doi.org/10.1117/12.2613231

KEYWORDS: Cameras, 3D acquisition, Endoscopy, Video, Imaging systems, Visualization, 3D modeling, Navigation systems, Image filtering, Augmented reality, Computer vision technology

Read Abstract +

Purpose: Recent neurosurgical techniques require accurate targeting of deep-brain structures even in the presence of deformation due to egress of cerebrospinal fluid (CSF) during surgical access. Prior work reported Structure-from-Motion (SfM) based methods for endoscopic guidance using 3D reconstruction. We are developing feature detection and description methods for a real-time 3D endoscopic navigation system using simultaneous localization and mapping (SLAM) to for accurate and near real-time registration. Methods: Feature detectors and descriptors were evaluated in SLAM reconstruction in anthropomorphic phantom studies emulating neuroendoscopy. The experimental system utilized a mobile UR3e robot (Universal Robots, Denmark) and ventriculoscope (Karl Storz, Tuttlingen, Germany) affixed to the end effector as a repeatable ventriculoscopy platform. Experiments were conducted to quantify optimal feature detection parameters in scale-space. Neuroendoscopic images acquired in traversal of the lateral and third ventricles provided a rich feature space of vessels and other structures on ventricular walls supporting feature detection and 3D point-cloud reconstruction. Performance was evaluated in terms of the mean number of features detected per frame and the algorithm runtime. Results: Parameter search in scale-space for feature detection demonstrated the dependence on the mean number of features per image and the points of diminishing return in parameter selection (e.g., the number of octaves and scale levels) and tradeoffs in runtime. Nominal parameters were identified as 3 octaves and 9 scale levels, with a mean number of features detected as 492 and 806 respectively. Conclusions: The system for neuroendoscopic guidance based on SLAM 3D point-cloud reconstruction provided a promising platform for the development of robot-assisted endoscopic neurosurgery. The studies reported in this work provided an essential basis for rigorous selection of parameters for feature detection. Future work aims to further develop the SLAM framework, assess the geometric accuracy of reconstruction, and translate methods to clinical studies.

SPIE Journal Paper | 13 March 2021 Open Access

Open Access

Sinogram + image domain neural network approach for metal artifact reduction in low-dose cone-beam computed tomography

Michael Ketcha, Michael Marrama, Andre Souza, Ali Uneri, Pengwei Wu, Xiaoxuan Zhang, Patrick Helm, Jeffrey Siewerdsen

JMI, Vol. 8, Issue 05, 052103, (March 2021) https://doi.org/10.1117/12.10.1117/1.JMI.8.5.052103

KEYWORDS: Metals, Image segmentation, Neural networks, Signal attenuation, Data modeling, Head, 3D image processing, Computed tomography, Network architectures, 3D image reconstruction

Read Abstract +

Proceedings Article | 23 February 2021 Presentation + Paper

Presentation + Paper

Data-driven deformable 3D-2D registration for guiding neuroelectrode placement in deep brain stimulation

A. Uneri, P. Wu, C. Jones, M. Ketcha, P. Vagdargi, R. Han, P. Helm, M. Luciano, W. Anderson, J. Siewerdsen

Proceedings Volume 11598, 115981B (2021) https://doi.org/10.1117/12.2582160

KEYWORDS: Brain stimulation, 3D modeling, Image registration, Model-based design, X-rays, Radiography, Performance modeling, Neural networks, Metals, Instrument modeling

Read Abstract +

Proceedings Article | 16 March 2020 Paper

Paper

Multi-slot intraoperative imaging and 3D-2D registration for evaluation of long surgical constructs in spine surgery

A. Uneri, X. Zhang, M. Ketcha, S. Doerr, C. Jones, P. Helm, J. Siewerdsen

Proceedings Volume 11315, 113152M (2020) https://doi.org/10.1117/12.2549876

KEYWORDS: Image registration, Radiography, Sensors, Spine, Prototyping, Collimators, Surgery, Collimation, Imaging systems, Line scan image sensors

Read Abstract +

Proceedings Article | 16 March 2020 Paper

Paper

Multi-slot extended view imaging on the O-Arm: image quality and application to intraoperative assessment of spinal morphology

X. Zhang, A. Uneri, P. Wu, M. Ketcha, S. Doerr, C. Jones, P. Helm, J. Siewerdsen

Proceedings Volume 11315, 113152B (2020) https://doi.org/10.1117/12.2549710

KEYWORDS: Optical spheres, Image quality, Visualization, Imaging systems, Spine, Image resolution, Collimators, Distortion, Sensors, Image visualization

Read Abstract +

Purpose. Surgical treatment of spinal deformity often seeks to achieve a given change in spinal curvature for the desired surgical outcome. However, it can be difficult to reliably evaluate changes in spinal curvature in the operating room based on qualitative evaluation or radiographs covering a limited field of view (FOV). We report a prototype beam filtration hardware configuration constructed on the O-arm imaging system and an image reconstruction algorithm for extended view (EV) imaging to enable such clear, long-length visualization of the spine and long surgical constructs. Methods. EV imaging on the O-arm involves a novel multi-slot collimator and longitudinal translation of the gantry. A weighted-backprojection algorithm was developed for EV image reconstruction. Image quality and geometric accuracy was evaluated in simulation and phantom studies to quantitatively characterize the depth resolution and potential sources of geometric distortion. A cadaver study was conducted to verify the quality of visualization in EV images and the potential for measurement of global spinal alignment (GSA) in the operating room. Results. EV imaging provided images spanning up to 65 cm length. Analogous to tomosynthesis, EV image reconstruction provides a modest degree of depth resolution and out-of-plane clutter rejection. The phantom study presenting highcontrast spheres in foam-core exhibited ~11% reduction in signal magnitude at 60 cm from the specified focal plane. The geometric accuracy of EV image reconstructions was high for objects at the focal plane, and distortion outside the focal plane was accurately described by predictions based on the known system geometry and object location. In addition to extending the FOV length by more than a factor of 3, EV images demonstrated strong improvement in visual image quality compared to a plain radiograph, and provided clear visualization of structures necessary for evaluation of GSA– e.g., vertebral endplates at the cervical-thoracic, thoraco-lumbar, and lumbar-sacral junctions. Conclusions. The multi-slot EV imaging technique offers a promising means for intraoperative visualization and assessment of spinal deformity correction through improved visualization over a long FOV and accurate measurement of distance and angles for GSA analysis. Future work involves integration of EV imaging with automated vertebral labeling, GSA analysis, and registration of surgical instrumentation in long surgical constructs.

Proceedings Article | 16 March 2020 Paper

Paper

Data-driven detection and registration of spine surgery instrumentation in intraoperative images

S. Doerr, A. Uneri, Y. Huang, C. Jones, X. Zhang, M. Ketcha, P. Helm, J. Siewerdsen

Proceedings Volume 11315, 113152P (2020) https://doi.org/10.1117/12.2550052

KEYWORDS: Image registration, 3D modeling, Data modeling, Spine, Model-based design, Process modeling, Surgery, Radiography, Image processing, 3D image processing

Read Abstract +

Purpose. Conventional model-based 3D-2D registration algorithms can be challenged by limited capture range, model validity, and stringent intraoperative runtime requirements. In this work, a deep convolutional neural network was used to provide robust initialization of a registration algorithm (known-component registration, KC-Reg) for 3D localization of spine surgery implants, combining the speed and global support of data-driven approaches with the previously demonstrated accuracy of model-based registration. Methods. The approach uses a Faster R-CNN architecture to detect and localize a broad variety and orientation of spinal pedicle screws in clinical images. Training data were generated using projections from 17 clinical cone-beam CT scans and a library of screw models to simulate implants. Network output was processed to provide screw count and 2D poses. The network was tested on two test datasets of 2,000 images, each depicting real anatomy and realistic spine surgery instrumentation – one dataset involving the same patient data as in the training set (but with different screws, poses, image noise, and affine transformations) and one dataset with five patients unseen in the test data. Assessment of device detection was quantified in terms of accuracy and specificity, and localization accuracy was evaluated in terms of intersection-overunion (IOU) and distance between true and predicted bounding box coordinates. Results. The overall accuracy of pedicle screw detection was ~86.6% (85.3% for the same-patient dataset and 87.8% for the many-patient dataset), suggesting that the screw detection network performed reasonably well irrespective of disparate, complex anatomical backgrounds. The precision of screw detection was ~92.6% (95.0% and 90.2% for the respective same-patient and many-patient datasets). The accuracy of screw localization was within 1.5 mm (median difference of bounding box coordinates), and median IOU exceeded 0.85. For purposes of initializing a 3D-2D registration algorithm, the accuracy was observed to be well within the typical capture range of KC-Reg.¹ Conclusions. Initial evaluation of network performance indicates sufficient accuracy to integrate with algorithms for implant registration, guidance, and verification in spine surgery. Such capability is of potential use in surgical navigation, robotic assistance, and data-intensive analysis of implant placement in large retrospective datasets. Future work includes correspondence of multiple views, 3D localization, screw classification, and expansion of the training dataset to a broader variety of anatomical sites, number of screws, and types of implants.

Proceedings Article | 8 March 2019 Presentation + Paper

Presentation + Paper

3D-2D image registration in virtual long-film imaging: application to spinal deformity correction

A. Uneri, X. Zhang, J. Stayman, P. Helm, G. Osgood, N. Theodore, J. Siewerdsen

Proceedings Volume 10951, 109511H (2019) https://doi.org/10.1117/12.2513679

KEYWORDS: Image registration, Radiography, Collimators, Imaging systems, Sensors, 3D image processing, 3D acquisition, Spine, Collimation, Surgery

Read Abstract +

Proceedings Article | 13 March 2018 Presentation + Paper

Presentation + Paper

Advanced image registration and reconstruction using the O-Arm system: dose reduction, image quality, and guidance using known-component models

A. Uneri, X. Zhang, J. Stayman, P. Helm, G. Osgood, N. Theodore, J. Siewerdsen

Proceedings Volume 10576, 105761G (2018) https://doi.org/10.1117/12.2293874

KEYWORDS: Image registration, Image quality, 3D modeling, 3D image reconstruction, Model-based design, Spatial resolution, Calibration, Reconstruction algorithms, Instrument modeling, Visibility

Read Abstract +

Proceedings Article | 3 March 2012 Paper

Paper

Feasibility study of 3D cardiac imaging using a portable conebeam scanner

Ivailo Petrov, Patrick Helm, Maria Drangova

Proceedings Volume 8313, 83131C (2012) https://doi.org/10.1117/12.912886

KEYWORDS: Imaging systems, 3D image processing, Cardiac imaging, Image quality, Heart, Reconstruction algorithms, 3D scanning, Blood, Laser scanners, In vivo imaging

Read Abstract +

Proceedings Article | 17 March 2011 Paper

Paper

Feasibility study of low-dose intra-operative cone-beam CT for image-guided surgery

Xiao Han, Shuanghe Shi, Junguo Bian, Patrick Helm, Emil Sidky, Xiaochuan Pan

Proceedings Volume 7961, 79615P (2011) https://doi.org/10.1117/12.878915

KEYWORDS: Computed tomography, Reconstruction algorithms, Image-guided intervention, Medical imaging, Physics, Current controlled current source, CT reconstruction, Data acquisition, Computer simulations

Read Abstract +

Proceedings Article | 5 June 1998 Paper

Paper

Sensor technology for image-guided surgery

S. Zinreich, Patrick Helm

Proceedings Volume 3262, (1998) https://doi.org/10.1117/12.309480

KEYWORDS: Surgery, Sensors, Sensor technology, Image-guided intervention, Magnetism, Imaging systems, Electromagnetism, Light emitting diodes, 3D image processing, Visualization

Read Abstract +

Showing 5 of 19 publications

SIGN IN TO:

View contact details

UPDATE YOUR PROFILE

Is this your profile? Update it now.

Sign into your SPIE.org account

Don’t have a profile and want one?

Create an account on SPIE.org

Advertisement

Advertisement