KEYWORDS: Denoising, Optical coherence tomography, Signal to noise ratio, Image classification, Speckle, Medical imaging, Wavelets, Medical diagnostics, Ophthalmology, Classification systems
Noise in speckle-prone optical coherence tomography tends to obfuscate important details necessary for medical diagnosis. In this paper, a denoising approach that preserves disease characteristics on retinal optical coherence tomography images in ophthalmology is presented. We propose semantic denoising autoencoders, which combine a convolutional denoising autoencoder with a priorly trained ResNet image classifier as regularizer during training. This promotes the perceptibility of delicate details in the denoised images that are important for diagnosis and filters out only informationless background noise. With our approach, higher peak signal-to-noise ratios with PSNR = 31.0 dB and higher classification performance of F1 = 0.92 can be achieved for denoised images compared to state-of-the-art denoising. It is shown that semantically regularized autoencoders are capable of denoising retinal OCT images without blurring details of diseases.
Pathologies of protective laryngeal reflexes such as the laryngeal adductor reflex (LAR) can increase the risk of aspiration pneumonia, a potentially life-threatening inflammation of the lungs caused by the intrusion of foreign particles into the lower airways. To estimate this risk, a standardized and non-invasive LAR screening method is highly desirable. In previous work, a commercially available high-speed laryngoscope has been equipped with a pressurized fluid system to shoot droplets onto the laryngeal structures for LAR stimulation and subsequent reflex latency evaluation. This Micro-droplet Impulse Testing of the LAR (MIT-LAR) lacks droplet impact site prediction for an unbiased and stimulation site-dependent reflex latency assessment. In this work, a two- phase algorithm leveraging stereoscopic image data for droplet impact site prediction and visualization of this prediction in the laryngoscopic image is proposed. A high-speed stereolaryngoscope requiring only a single camera was designed from scratch by combining two rod lens optics in one common shaft. This setup was used for stereoscopic high-speed image data acquisition of droplets shot at different muzzle energies and endoscope roll angles. Surface reconstruction of the target region is performed using a phantom of the human larynx. The point of intersection between the reconstructed surface and the droplet trajectory approximation recorded previously at known droplet formation parameters is calculated. The proposed approach allows stimulation site prediction for an enhanced MIT-LAR procedure. Quantitative evaluation of impact site prediction accuracy for different droplet muzzle energies and nozzle inclinations yields an average prediction error of 2.9 mm (n = 20).
In microsurgery, lasers have emerged as precise tools for bone ablation. A challenge is automatic control of laser bone ablation with 4D optical coherence tomography (OCT). OCT as high resolution imaging modality provides volumetric images of tissue and foresees information of bone position and orientation (pose) as well as thickness. However, existing approaches for OCT based laser ablation control rely on external tracking systems or invasively ablated artificial landmarks for tracking the pose of the OCT probe relative to the tissue. This can be superseded by estimating the scene flow caused by relative movement between OCT-based laser ablation system and patient. Therefore, this paper deals with 2.5D scene flow estimation of volumetric OCT images for application in laser ablation. We present a semi-supervised convolutional neural network based tracking scheme for subsequent 3D OCT volumes and apply it to a realistic semi-synthetic data set of ex vivo human temporal bone specimen. The scene flow is estimated in a two-stage approach. In the first stage, 2D lateral scene flow is computed on census-transformed en-face arguments-of-maximum intensity projections. Subsequent to this, the projections are warped by predicted lateral flow and 1D depth flow is estimated. The neural network is trained semi-supervised by combining error to ground truth and the reconstruction error of warped images with assumptions of spatial flow smoothness. Quantitative evaluation reveals a mean endpoint error of (4.7 ± 3.5) voxel or (27.5 ± 20.5) μm for scene flow estimation caused by simulated relative movement between the OCT probe and bone. The scene flow estimation for 4D OCT enables its use for markerless tracking of mastoid bone structures for image guidance in general, and automated laser ablation control.
Projector based augmented reality serves as an alternate visual guidance tool for surgeons when performing complicated open surgeries. In projector based augmented reality, image overlay projection is a technique that allows the surgeon to view the underlying anatomical information such as tissues, tumors etc. directly on the surface of the organ or the patient. This will provide an intuitive view of the surgical navigation data by combining the surgeon’s real world view with the preoperative three dimensional virtual models or instructions. Thus the strain on the surgeon to mentally align and visualize the preoperative data with intraoperative scene is greatly reduced. There are multiple stationary and handheld projectors available in the market today for this purpose. During surgery, stationary projectors mounted on a rack or under the ceiling suffer from a loss of adjustability and further cause shadowing issues when the surgeon occludes the scene. Although hand-held projectors do not have these disadvantages, they have major problems in terms of illuminance and luminous flux. The amount of light at which the hand-held projectors can project virtual additional information on to the patient is very low especially when the surgical lights are switched on. This paper therefore aims to provide an analysis of the requirements for designing such a special hand-held, augmented reality projector system that could be used during surgery, through a user study. Various optical parameters which are a key to design an augmented reality projector such as illuminance, luminance, luminous flux etc. are measured. Apart from that, other parameters such as refresh rate, image size, resolution which are also some important criteria in designing such a special projector, are discussed in this paper with respect to our application.
A common representation of volumetric medical image data is the triplanar view (TV), in which the surgeon manually selects slices showing the anatomical structure of interest. In addition to common medical imaging such as MRI or computed tomography, recent advances in the field of optical coherence tomography (OCT) have enabled live processing and volumetric rendering of four-dimensional images of the human body. Due to the region of interest undergoing motion, it is challenging for the surgeon to simultaneously keep track of an object by continuously adjusting the TV to desired slices. To select these slices in subsequent frames automatically, it is necessary to track movements of the volume of interest (VOI). This has not been addressed with respect to 4DOCT images yet. Therefore, this paper evaluates motion tracking by applying state-of-the-art tracking schemes on maximum intensity projections (MIP) of 4D-OCT images. Estimated VOI location is used to conveniently show corresponding slices and to improve the MIPs by calculating thin-slab MIPs. Tracking performances are evaluated on an in-vivo sequence of human skin, captured at 26 volumes per second. Among investigated tracking schemes, our recently presented tracking scheme for soft tissue motion provides highest accuracy with an error of under 2.2 voxels for the first 80 volumes. Object tracking on 4D-OCT images enables its use for sub-epithelial tracking of microvessels for image-guidance.
Image-to-physical registration based on volumetric data like computed tomography on the one side and intraoperative endoscopic images on the other side is an important method for various surgical applications. In this contribution, we present methods to generate panoramic views from endoscopic recordings for image-to-physical registration of narrow drill holes inside spongy bone. One core application is the registration of drill poses inside the mastoid during minimally invasive cochlear implantations. Besides the development of image processing software for registration, investigations are performed on a miniaturized optical system, achieving 360° radial imaging with one shot by extending a conventional, small, rigid, rod lens endoscope. A reflective cone geometry is used to deflect radially incoming light rays into the endoscope optics. Therefore, a cone mirror is mounted in front of a conventional 0° endoscope. Furthermore, panoramic images of inner drill hole surfaces in artificial bone material are created. Prior to drilling, cone beam computed tomography data is acquired from this artificial bone and simulated endoscopic views are generated from this data. A qualitative and quantitative image comparison of resulting views in terms of image-to-image registration is performed. First results show that downsizing of panoramic optics to a diameter of 3mm is possible. Conventional rigid rod lens endoscopes can be extended to produce suitable panoramic one-shot image data. Using unrolling and stitching methods, images of the inner drill hole surface similar to computed tomography image data of the same surface were created. Registration is performed on ten perturbations of the search space and results in target registration errors of (0:487 ± 0:438)mm at the entry point and (0:957 ± 0:948)mm at the exit as well as an angular error of (1:763 ± 1:536)°. The results show suitability of this image data for image-to-image registration. Analysis of the error components in different directions reveals a strong influence of the pattern structure, meaning higher diversity results into smaller errors.
The introduction of Er:YAG lasers for soft and hard tissue ablation has proven promising results over the last decades due to strong absorption at 2.94 μm wavelength by water molecules. An extension to endoluminal applications demands laser delivery without mirror arms due to dimensional constraints. Therefore, fiber-based solutions are advanced to provide exible access while keeping space requirements to a minimum. Conventional fiber-based treatments aim at laser-tissue interactions in contact mode. However, this procedure is associated with disadvantages such as advancing decrease in power delivery due to particle coverage of the fiber tip, tissue carbonization, and obstructed observation of the ablation progress. The objective of this work is to overcome aforementioned limitations with a customized fiber-based module for non-contact robot-assisted endoluminal surgery and its associated experimental evaluation. Up to the authors knowledge, this approach has not been presented in the context of laser surgery at 2.94 μm wavelength. The preliminary system design is composed of a 3D Er:YAG laser processing unit enabling automatic laser to fiber coupling, a GeO2 solid core fiber, and a customized module combining collimation and focusing unit (focal length of 20 mm, outer diameter of 8 mm). The performance is evaluated with studies on tissue substitutes (agar-agar) as well as porcine samples that are analysed by optical coherence tomography measurements. Cuts (depths up to 3mm) with minimal carbonization have been achieved under adequate moistening and sample movement (1.5mms-1). Furthermore, an early cadaver study is presented. Future work aims at module miniaturization and integration into an endoluminal robot for scanning and focus adaptation.
Laser surgery is an established clinical procedure in dental applications, soft tissue ablation, and ophthalmology. The presented experimental set-up for closed-loop control of laser bone ablation addresses a feedback system and enables safe ablation towards anatomical structures that usually would have high risk of damage. This study is based on combined working volumes of optical coherence tomography (OCT) and Er:YAG cutting laser. High level of automation in fast image data processing and tissue treatment enables reproducible results and shortens the time in the operating room. For registration of the two coordinate systems a cross-like incision is ablated with the Er:YAG laser and segmented with OCT in three distances. The resulting Er:YAG coordinate system is reconstructed. A parameter list defines multiple sets of laser parameters including discrete and specific ablation rates as ablation model. The control algorithm uses this model to plan corrective laser paths for each set of laser parameters and dynamically adapts the distance of the laser focus. With this iterative control cycle consisting of image processing, path planning, ablation, and moistening of tissue the target geometry and desired depth are approximated until no further corrective laser paths can be set. The achieved depth stays within the tolerances of the parameter set with the smallest ablation rate. Specimen trials with fresh porcine bone have been conducted to prove the functionality of the developed concept. Flat bottom surfaces and sharp edges of the outline without visual signs of thermal damage verify the feasibility of automated, OCT controlled laser bone ablation with minimal process time.
Planning and analyzing of surgical interventions are often based on computer models derived from computed tomography images of the patient. In the field of cochlear implant insertion the modeling of several structures of the inner ear is needed. One structure is the overall helical shape of the cochlea itself. In this paper we analyze the cochlea by applying statistical shape models with medial representation. The cochlea is considered as tubular structure. A model representing the skeleton of training data and an atomic composition of the structure is built. We reduce the representation to a linear chain of atoms. As result a compact discrete model is possible. It is demonstrated how to place the atoms and build up their correspondence through a population of training data. The outcome of the applied representation is discussed in terms of impact on automated segmentation algorithms and known advantages of medial models are revisited.
This work proposes new methods for a microstereotactic frame based on bone cement fixation. Microstereotactic frames are under investigation for minimal invasive temporal bone surgery, e.g. cochlear implantation, or for deep brain stimulation, where products are already on the market. The correct pose of the microstereotactic frame is either adjusted outside or inside the operating room and the frame is used for e.g. drill or electrode guidance. We present a patientspecific, disposable frame that allows intraoperative, sterile pose-setting. Key idea of our approach is bone cement between two plates that cures while the plates are positioned with a mechatronics system in the desired pose. This paper includes new designs of microstereotactic frames, a system for alignment and first measurements to analyze accuracy and applicable load.
Recent research has revealed that incision planning in laser surgery deploying stylus and tablet outperforms state-of-the-art micro-manipulator-based laser control. Providing more detailed quantitation regarding that approach, a comparative study of six tablet-based strategies for laser path planning is presented. Reference strategy is defined by monoscopic visualization and continuous path drawing on a graphics tablet. Further concepts deploying stereoscopic or a synthesized laser view, point-based path definition, real-time teleoperation or a pen display are compared with the reference scenario. Volunteers were asked to redraw and ablate stamped lines on a sample. Performance is assessed by measuring planning accuracy, completion time and ease of use. Results demonstrate that significant differences exist between proposed concepts. The reference strategy provides more accurate incision planning than the stereo or laser view scenario. Real-time teleoperation performs best with respect to completion time without indicating any significant deviation in accuracy and usability. Point-based planning as well as the pen display provide most accurate planning and increased ease of use compared to the reference strategy. As a result, combining the pen display approach with point-based planning has potential to become a powerful strategy because of benefiting from improved hand-eye-coordination on the one hand and from a simple but accurate technique for path definition on the other hand. These findings as well as the overall usability scale indicating high acceptance and consistence of proposed strategies motivate further advanced tablet-based planning in laser microsurgery.
This work investigates combination of Optical Coherence Tomography and two cameras, observing a microscopic scene. Stereo vision provides realistic images, but is limited in terms of penetration depth. Optical Coherence Tomography (OCT) enables access to subcutaneous structures, but 3D-OCT volume data do not give the surgeon a familiar view. The extension of the stereo camera setup with OCT imaging combines the benefits of both modalities. In order to provide the surgeon with a convenient integration of OCT into the vision interface, we present an automated image processing analysis of OCT and stereo camera data as well as combined imaging as augmented reality visualization. Therefore, we care about OCT image noise, perform segmentation as well as develop proper registration objects and methods. The registration between stereo camera and OCT results in a Root Mean Square error of 284 μm as average of five measurements. The presented methods are fundamental for fusion of both imaging modalities. Augmented reality is shown as application of the results. Further developments lead to fused visualization of subcutaneous structures, as information of OCT images, into stereo vision.
In recent years, optical coherence tomography (OCT) has gained increasing attention not only as an imaging device, but also as a navigation system for surgical interventions. This approach demands to register intraoperative OCT to pre-operative computed tomography (CT) data. In this study, we evaluate algorithms for multi-modal image registration of OCT and CT data of a human temporal bone specimen. We focus on similarity measures that are common in this field, e.g., normalized mutual information, normalized cross correlation, and iterative closest point. We evaluate and compare their accuracies to the relatively new normal distribution transform (NDT), that is very common in simultaneous localization and mapping applications, but is not widely used in image registration. Matching is realized considering appropriate image pre-processing, the aforementioned similarity measures, and local optimization algorithms, as well as line search optimization. For evaluation purpose, the results of a point-based registration with fiducial landmarks are regarded as ground truth. First results indicate that state of the art similarity functions do not perform with the desired accuracy, when applied to unprocessed image data. In contrast, NDT seems to achieve higher registration accuracy.
In recent years, bone-attached robots and microstereotactic frames have attracted increasing interest due to the promising targeting accuracy they provide. Such devices attach to a patient's skull via bone anchors, which are used as landmarks during intervention planning as well. However, as simulation results reveal, the performance of such mechanisms is limited by errors occurring during the localization of their bone anchors in preoperatively acquired computed tomography images. Therefore, it is desirable to identify the most suitable fiducials as well as the most accurate method for fiducial localization. We present experimental results of a study focusing on the fiducial localization error (FLE) of spheres. Two phantoms equipped with fiducials made from ferromagnetic steel and titanium, respectively, are used to compare two clinically available imaging modalities (multi-slice CT (MSCT) and cone-beam CT (CBCT)), three localization algorithms as well as two methods for approximating the FLE. Furthermore, the impact of cubic interpolation applied to the images is investigated. Results reveal that, generally, the achievable localization accuracy in CBCT image data is significantly higher compared to MSCT imaging. The lowest FLEs (approx. 40 μm) are obtained using spheres made from titanium, CBCT imaging, template matching based on cross correlation for localization, and interpolating the images by a factor of sixteen. Nevertheless, the achievable localization accuracy of spheres made from steel is only slightly inferior. The outcomes of the presented study will be valuable considering the optimization of future microstereotactic frame prototypes as well as the operative workflow.
The choice of a navigation system highly depends on the medical intervention and its accuracy demands. The most commonly used systems for image guided surgery (IGS) are based on optical and magnetic tracking systems. This paper compares two optical systems in terms of accuracy: state of the art triangulation-based optical tracking (OT) and optical coherence tomography (OCT). We use an experimental setup with a combined OCT and cutting laser, and an external OT. We simulate a robotic assisted surgical intervention, including planning, navigation, and processing, and compare the accuracies reached at a specific target with each navigation system.
KEYWORDS: Optical coherence tomography, Calibration, Distortion, Mirrors, Data modeling, Geometrical optics, 3D modeling, Mathematical modeling, 3D metrology, Surgery
This contribution compares two approaches for the geometric calibration of an optical coherence tomography
(OCT) which forms part of a medical navigation system. For this purpose, a one step and a multi step calibration
is performed with a self-produced 3D reference structure and a high-accurate 6 degrees of freedom (DoF)
parallel robot, respectively. These 3D landmark-based geometric calibrations are based on the identification of
a parameterized grey-box OCT model. We show in experimental results that both methods reduce systematic
errors by more than one order of magnitude.
KEYWORDS: Optical coherence tomography, Calibration, Distortion, Data modeling, Mirrors, 3D modeling, 3D metrology, Surgery, Electroluminescent displays, 3D image processing
This article presents a one step geometric calibration for an optical coherence tomography (OCT) which forms part of a medical navigation system. The 3D landmark-based geometric calibration with a self-produced 3D reference structure is based on the identification of a parameterized grey-box OCT model. We show in experimental results by comparing common measurement errors in the field of medical surgery before and after calibration, that the proposed methodology reduces systematic errors by more than one order of magnitude. Due to its simplicity, the calibration can be carried out directly before a surgical intervention enhancing the OCT accuracy.
We introduce a new method of rotational image acquisition for four-dimensional (4D) optical coherence tomography (OCT) of beating embryonic chick hearts. The rotational axis and the central A-scan of the OCT are identical. An out-of-phase image sequence covering multiple heartbeats is acquired at every angle of an incremental rotation of the deflection mirrors of the OCT system. Image acquisition is accomplished after a rotation of 180°. Comparison of a displayed live M-mode of the central A-scan with a reference M-mode allows instant detection of translational movements of the embryo. For calculation of 4D data sets, we apply an image-based retrospective gating algorithm using the phase information of the common central A-scan present in all acquired images. This leads to cylindrical three-dimensional data sets for every time step of the cardiac cycle that can be used for 4D visualization. We demonstrate this approach and provide a video of a beating Hamburger and Hamilton stage 16 embryonic chick heart generated from a 4D OCT data set using rotational image acquisition.
Minimally invasive surgery in combination with ultrasound (US) imaging imposes high demands on the surgeon's hand-eye-coordination capabilities. A possible solution to reduce these requirements is minimally invasive robotic surgery in which the instrument is guided by visual servoing towards the goal defined by the surgeon in the US image. This approach requires robust tracking of the instrument in the US image sequences which is known to be difficult due to poor image quality. This paper presents algorithms and results of first tracking experiments. Adaptive thresholding based on Otsu's method allows to cope with large intensity variations of the instrument echo. Median filtering of the binary image and subsequently applied morphological operations suppress noise and echo artefacts. A fast run length code based labelling algorithm allows for real-time labelling of the regions. A heuristic exploiting region size and region velocity helps to overcome ambiguities. The overall computation time is less than 20 ms per frame on a standard PC. The tracking algorithm requires no information about texture and shape which are known to be very unreliable in US image sequences. Experimental results for two different instrument materials (polyvinyl chloride and polyurethane) are given, showing the performance of the proposed approach. Choosing the appropriate material, trajectories are smooth and only few outliers occur.
Local motion on the beating heart is investigated in the context of minimally invasive robotic surgery. The focus lies on the motion remaining in the mechanically stabilised field of surgery of the heart. Motion is detected by tracking natural landmarks on the heart surface in 2D video images. An appropriate motion model is presented with a discussion of its degrees of freedom and a trajectory analysis of its parameters.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.