We have developed a clinically compatible, real-time ultrasound needle tracking system (UNT) that can be appended to a clinical ultrasound system, superimposing a crosshair onto the ultrasound image at the needle tip position. The UNT was developed under the ISO 13485 Medical Devices quality standard for deployment in the clinic. During handheld ultrasound guidance, the location of the needle tip within the imaging plane is determined from the acoustic signals received by an embedded fibre-topic hydrophone. Assessment of tracking accuracy found that the mean distance between tracked and true positions was 0.7 ± 0.4 mm with a repeatability of 0.3 ± 0.2 mm.

Guidance of epidural needle is important for the safe and efficient epidural anesthesia procedure. In this study, we built an endoscopic system based on polarization-sensitive optical coherence tomography (PS-OCT). We used pig backbones to test the feasibility of our PS-OCT endoscopic system. Different spinal tissue layers that epidural needle punctures through including subcutaneous fat, ligament, ligamentum flavum, epidural space and spinal cord were imaged and analyzed. They showed different imaging features on the PS-OCT imaging results. Furthermore, we applied deep-learning methods to classify those tissue types automatically to improve the recognition efficiency.

We report on the development of a dedicated Raman spectroscopic system employing a thin fiber optic needle for rapid determination of tumor resection margins on fresh intact specimens. This information will enable the surgeon to excise additional tissue if and where needed for complete tumor removal with adequate margins.

Re-excision following breast-conserving surgery (BCS) due to suspected residual cancer left from the primary surgery causes substantial physical, psychological, and financial burdens for patients. This study provides a first-in-human clinical study of in vivo quantitative micro-elastography (QME) for in-cavity identification of residual cancer. A custom-built handheld QME probe is used to directly scan the surgical cavity for imaging the micro-scale tissue stiffness during BCS. In vivo QME of 21 patients, validated by co-registered histopathology of the excised specimens, demonstrates the capability to detect residual cancer based on its elevated micro-scale stiffness, potentially contributing to a more complete cancer removal.

Accurate identification of the interventional medical device during ultrasound-guided minimally-invasive procedures is of critical importance. A real-time 3D needle tracking system has been developed that utilises a fiber-optic, photoacoustic US transmitter integrated into the needle tip and a custom 2D, 4x4 receiver array attached to a clinical US imaging probe. Ultrasound signals received by the array are used to determine the location of source, which is then registered to the imaging probe and visualised. During initial laboratory measurements of tracking accuracy, the mean displacement between tracked and true distances from the array face was 0.8 ± 0.8 mm.

Many liver resection surgeries are performed with fluorescence-guided approaches using near-infrared (NIR) fluorescent dyes, such as indocyanine green (ICG), in an effort to prevent postoperative bile leakage. However, these existing fluorophores have limitations when assessing bile leakages and other bile duct injuries. We have reported a novel NIR fluorescent dye, BL-760, that we believe would be favorable over ICG and other NIR dyes during hepatectomies. We will describe our hepatic resection surgery study conducted with a swine model while comparing the advantages and disadvantages of common fluorescent dyes and the novel BL-760 dye.

In this paper we report the use of a novel multimodal imaging hand-held probe for guiding laser and radiotherapy on nonmelanoma skin cancers (NMSCs) patients. This probe combines the capability of reflectance confocal microscopy (RCM) with that of Optical Coherence Tomography (OCT) to reliably detect cancer markers and measure cancer depth of invasion. These capabilities have shown to be very effective in accurately measuring cancer margins and guiding the therapy.

We present a microscope-integrated Optical Coherence Tomography (OCT) system for aiding ophthalmic surgical manoeuvers. It generates two B-scans dynamically locked on the tip of the surgical tool to provide relevant depth cues. The low-latency architecture of our tool-tracking engine provides a position update rate of 200 Hz in a bid to minimize instantaneous tool position tracking errors. Likewise, the B-scan update rate of 200 Hz helps reduce motion artefacts while allowing for high signal-to-noise ratio averaged OCT images displayed at 50 Hz.

In this work, we investigate the remodeling of uterine tissue that occurs during the presence of cancer. We acquired de-identified normal and tumor tissue from the Columbia University Medical Center Tissue Bank and used an optical coherence tomography (OCT) system with a 6.5-micron axial resolution to optically characterize the samples. Initial results indicate that the collagen fiber orientation in the cancerous tissue is more heterogeneous than that of the normal tissue.

Surgical microscopes provide clear visualization of the tissue and have been increasingly used in operating rooms. Going further from this, we developed optical coherence tomography (OCT), and optical coherence microscopy (OCM) integrated surgical microscope to offer superficial and sub-surface tissue information simultaneously. With the use of an optical switch, OCT and OCM view can be freely converted. After surgical resection of cancer with the guidance of OCT, we performed high-resolution whole cancer imaging with OCM for margin detection. Our proposed system can be a promising tool for intraoperative applications and increase the accuracy of the operation.

Sepsis is a life-threatening medical emergency caused by body's extreme inflammatory response to an infection. The important features of sepsis are the alternations in leukocyte numbers, enhanced adhesiveness of leukocytes on the blood vessel walls, and impaired microvasculature network that can ultimately lead to organ failure and death. To non-invasively quantify these parameters, we have built a miniaturized oblique back-illumination microscope capable of high-resolution and label-free imaging of human microvasculature in vivo. The developed instrument was used to capture high-speed videos of circulating, rolling, and adherent leukocytes. We have also developed a deep learning data analysis algorithm for rapid diagnosis of sepsis without drawing blood.

Hematoxylin and eosin (H&E) staining in histology, the century-old technique, has been the gold standard tool for pathologists to detect anomalies in tissues and detect disorders such as cancer. Even in the modern era, several new label-free imaging techniques have been developed which provide many more precise layers of information, but they have yet to translate to the clinic, mainly due to a lack of direct comparison between these techniques. Here we present the use of laser ablation to produce fiduciary markers to enable the correlation of nonlinear microscopy with the gold-standard tool of histology.

Lung cancer is one of the leading causes of cancer mortality worldwide, with an estimated 2.2 million new cancer cases and 1.8 million deaths in 2020. Adenocarcinoma is the most common type of non-small cell lung cancer (NSCLC), which is usually developed with a mixture of histologic subtypes. Surgery to remove the affected tissue or tumor is the most curative treatment option for the early-stage NSCLC currently. The clinical diagnosis of NSCLC based on pathological analysis of formalin-fixed and paraffin-embedded (FFPE) tissues is laborious and time-consuming, failing to guide surgeons intraoperatively. Although frozen section can serve as a rapid alternative to FFPE histology, it still requires a turnaround time of 20–30 minutes during surgery. Besides, the diagnostic accuracy of the frozen section could be affected due to the tissue freezing artifacts and inadequate sampling of resection margins. Here, we propose a rapid histological imaging method, termed microscopy with ultraviolet single-plane illumination (MUSI), which enables label-free and non-destructive imaging of freshly excised and unprocessed tissues. The MUSI system allows the surgical specimens with large irregular surfaces to be scanned in a label-free manner at a speed of 0.65 mm2/s with a subcellular resolution, showing great potential as an assistive imaging platform that can provide immediate feedback to surgeons and pathologists for intraoperative decision-making. We demonstrate that MUSI can differentiate between different subtypes of human lung adenocarcinomas, revealing diagnostically important features that are comparable to the gold standard FFPE histology, holding great promise to revolutionize the current practice of surgical pathology.

This Conference Presentation, Optical image-guided autonomous robotic surgery, was recorded at Photonic West 2023 held in San Francisco, CA, United States.

Achieving adequate resection margins during breast-conserving surgery is crucial for minimizing the risk of tumor recurrence in patients with breast cancer but remains challenging due to the lack of intraoperative feedback. Here, we evaluated the use of hyperspectral imaging to discriminate healthy tissue from tumor tissue in lumpectomy specimens of 121 patients. A dataset on tissue slices was used to develop and evaluate three convolutional neural networks. Subsequently, these networks were fine-tuned with lumpectomy data to predict the tissue percentages on the lumpectomy resection surface. We achieved a MCC of 0.92 on the tissue slices and an RMSE of 9% on the lumpectomy resection surface.

Specular reflections (SR) commonly found in endoscopy videos can severely disrupt a surgeon’s observation and judgment, but existing methods to inpaint SR regions can result in false clinical interpretations. Therefore, we propose an end-to-end pipeline termed SpecFlow to detect and restore SR regions from endoscopy videos. Our proposed SpecFlow consists of two phases: detection using a reduced U-net model and a novel restoration method using optical flow-guided color propagation. Our detection pipeline achieves a competitive 82.8% Dice score with only 14ms of computational time (near real-time), and our restoration pipeline successfully incorporates temporal information for more accurate restorations.

Within this work we explore texture analysis of optical coherence tomography images and machine learning for automated detect classification of breast biopsies. Under an approved IRB protocol, breast biopsy specimens from 100 patients were imaged with a high resolution OCT system providing 3.7 micron axial resolution. The texture features extracted were first order statistics (histogram distribution) and second order statistics (such as GLCM). Binary classification was carried out for two cases: 1) risk 0 (no risk of cancer) versus everything else and 2) risk 3 (cancer) versus everything else.

In vivo 3D OCT imaging of live animal generally suffers from motion artifacts due to involuntary tissue movement. Here, we propose a real-time 3D OCT imaging approach using a convolution neural network (CNN)/regression-based algorithm to correct tissue motion in vivo. The system first scans four reference images along the slow axis within millisecond-scale acquisition time before acquiring a C-mode image. The algorithm recognizes the tissue surface by CNN, then uses the segmentation result along with reference images to compensate lateral and axial motion. We evaluated the system performance using a fish eye model.

We present a virtual immunohistochemical (IHC) staining method based on label-free autofluorescence imaging and deep learning. Using a trained neural network, we transform multi-band autofluorescence images of unstained tissue sections to their bright-field equivalent HER2 images, matching the microscopic images captured after the standard IHC staining of the same tissue sections. Three pathologists’ blind evaluations of HER2 scores based on virtually stained and IHC-stained whole slide images revealed the statistically equivalent diagnostic values of the two methods. This virtual HER2 staining method provides a rapid, accurate, and low-cost alternative to the standard IHC staining methods and allows tissue preservation.

This work investigates the capability of a novel Raman arthroscopic needle probe to assess cartilage-tissue-specific biomarkers that are predictive of articular cartilage health during the progression of osteoarthritis. Using ex vivo models of cartilage over a range of degenerative states, we demonstrate that Raman probe-derived biomarkers can predict the composition and material properties of cartilage tissue specimens. This work supports the future use of Raman arthroscopy as an intraarticular clinical platform for ‘molecular assessment’ cartilage to achieve diagnostics of osteoarthritis early in the disease process and assess the efficacy of emerging clinical therapies in restoring cartilage health.

Cartilage tissue engineering (TE) is a promising osteoarthritis treatment strategy, whereby cellularized tissue constructs are implanted in cartilage defect sites to promote tissue repair. A significant impediment to developing effective TE strategies is a lack of in vivo diagnostic platforms to monitor engineered cartilage growth. Here, we utilize an ex vivo model system of chondrocyte-seeded tissue constructs, demonstrating the capability of a novel Raman arthroscopic-configured needle probe to monitor the composition and material properties of developing TE cartilage over time. This work supports Raman arthroscopy as a clinical tool for monitoring TE cartilage, providing objective assessments of TE platform efficacies.

Cancers of the upper gastrointestinal (GI) tract remain a major contributor to overall cancer risk. This study uses a sterilisable diffuse reflectance spectroscopic probe for tissue characterisation intraoperatively during stomach and oesophageal cancer surgery. Histopathology correlation is achieved by the surgeon marking the optical probed tissue locations after the acquisition of spectral data. The data is normalised, and significant features are selected to improve tissue discrimination accuracy. Supervised classification algorithms are used for discrimination between tumour and non-tumour tissue and evaluated in terms of accuracy, sensitivity, specificity, and the area under the curve.

Respiratory and pulmonary illnesses such as respiratory distress syndrome and transient tachypnoea of the newborn are leading causes of death among newborns. These morbidities result in lung collapse and reduction in the lung gas volume. While these conditions can be treated using surfactant administration and supplemental oxygen, continuous feedback on the health of the lung during these procedures can be helpful in improving their efficiency and avoiding later complications. Optical techniques like GASMAS (Gas in scattering media absorption spectroscopy) have shown considerable promise in this regard. The technique is non-invasive and non-ionizing and causes no short term or long term discomfort to the infant. It also allows for real time continuous monitoring of the oxygen content which is critical in a clinical setting. In this work, we discuss the results from a pilot clinical study performed at the INFANT Research Centre, Cork. A GASMAS device was used to measure the oxygen concentration of the lung in this healthy cohort of 100 healthy neonatal infants between 1 to 5 days of age. Lung oxygen concentration was measured at multiple locations and across multiple visits for each infant. The huge dataset allows us to understand the influence of different parameters such as the weight of the infant, chest circumference, location etc, on the instrument performance and recovered oxygen concentration. This information and understanding will set the stage for the next phase of the study which is aimed at a similar cohort of term and pre-term infants with respiratory morbidities.

Ghost cytometry is a technique for discrimination of cells by applying structured illumination consisting of multiple micro-spots (0.8um) and machine learning. We have developed a benchtop type device that adds ghost cytometry capabilities to our conventional flow cytometry system and evaluated cell models, showing that our new device enables label-free classification of cell types that cannot be classified by conventional flow cytometry. We will present and discuss the results of cell classification using our hybrid ghost cytometer enabling detailed and high-throughput cell analysis, and the new diagnostic value that ghost cytometry brings.

Blue light cystoscopy (BLC) has been demonstrated to detect bladder tumors with better sensitivity than white light cystoscopy (WLC); however, the use of BLC is limited to the operating room. In this study, we aim to bring BLC to the clinic by transforming WLC frames into digitally-stained BLC-like frames. We collected region-matched WLC and BLC videos from TURBT procedures and generated BLC-like frames, using WLC frames as input and the matched BLC frames as target. We will discuss the staining performances with perfectly registered WLC-BLC datasets, as well as WLC and BLC video clips collected with commercial clinical systems.

Gold nanostars (GNS) have increasingly seen usage in photothermal treatment of diseases such as cancer. Owing to their flexible synthesis, GNS can be easily tuned to absorption wavelengths that bypass the absorption of non-target tissue. The particles can also be easily coupled with other sensing modalities such as molecular imaging using surface-enhanced Raman spectroscopy (SERS) and Immunotherapy in a synergistic manner. We have developed a combination treatment for cancer therapy called Synergistic Immuno Photo Nanotherapy (SYMPHONY) effectively uses photothermal heating as an adjunct treatment with immune check-point inhibitors. Here we develop several methods to simulate and analyze the optical and thermal effects of GNS absorption. In particular, the optimal conditions of GNS heating are investigated for understanding specific targeted therapy with limitations surrounding tissue absorption.