1.

Introduction

Optical coherence tomography (OCT) is an intravascular imaging technique that can play a significant role in the planning and optimization of percutaneous coronary interventions.¹ OCT produces high-contrast, high-resolution cross-sectional images of the coronary arteries using near-infrared rays, which can clearly differentiate the borders between the lumen and vessel wall.² Cardiologists utilizing OCT in the cardiac catheterization laboratory require rapid generation of three-dimensional (3D) lumen models reconstructed from the segmented images for visualization purposes. The manual segmentation of OCT images is a laborious and time-consuming process and requires expertise in OCT imaging segmentation. Therefore, automatic techniques have been developed that can significantly amplify the segmentation speed while maintaining accuracy.^3,4 Prior studies have explored several techniques to enhance the automatic OCT lumen segmentation speed and accuracy. One study employed multiple canny edge detection algorithms to detect lumen edges and concentrated on the contour extraction of stented images.⁵ Another study proposed a preprocessing and segmentation method for OCT lumen images that involves conversion between polar and Cartesian coordinates, noise removal using median and Gaussian blur algorithms, and interpolation to connect the disconnected regions of the lumen.⁶ Although one study suggested a smoothly varying threshold approach,⁷ some studies utilized a global threshold approach or a global Otsu threshold derived from performing a one-level decomposition of the preprocessed images.^8–10 Recent advances in automated segmentation techniques have utilized machine learning to investigate OCT lumen segmentation.^11,12 The success of these automated techniques has been limited by the paucity of appropriate computational and training data resources, however, that can provide results within a suitable time frame and cost efficiency. Furthermore, most of the existing methods have been applicable only to good-quality images without bifurcations⁹ or artifacts,^8,10,11 and their processing times have not been instantaneous (about 1 min per frame).

The existing methods of automated OCT lumen segmentation may be improved in several key areas, including: (i) faster segmentation speed and shorter sample training times (if machine learning is employed); (ii) reliable segmentation in the presence of image artifacts; and (iii) direct 3D vessel reconstruction. To address these limitations, this paper proposes a fully automated approach to OCT lumen segmentation supplemented by manual correction and 3D vessel reconstruction. Specifically, manual correction is necessary for OCT frames that exhibit large bifurcations or significant image artifacts. Our primary objective was to develop a user-friendly, time-efficient, and accurate automated platform for coronary lumen segmentation. Additionally, this tool is intended to facilitate quick and precise 3D reconstruction of the coronary artery lumen.

2.

Materials and Methods

All procedures were conducted in compliance with relevant guidelines and regulations. The OCT data and angiograms used in this study were obtained from a clinical trial named PROPOT (Randomized Trial of the Proximal Optimization Technique in Coronary Bifurcation Lesions). The study was reviewed and approved by the ethics committee of Teikyo University (IRB Approval No. 15-159-2), and informed consent was obtained from all participants.

2.3.

OCT Image Preprocessing

In Fig. 1, we present the preprocessing of OCT images. Initially, the original image was converted into a grayscale image, which discarded the color information and retained only the different shades of gray information. Then median filtering was applied, which is a nonlinear technique utilized for reducing speckle noise. The filter selected a pixel from the input image to become the output pixel location at the center of the coordinates in the kernel frames. The median filter eliminated features smaller than the median filter kernel, whereas it had a trivial effect on large fluctuations in intensity or the edges. Following the median filtering, morphological operations were performed.

Fig. 1

Preprocessing of the OCT images: (a) original image with artifacts, (b) grayscale conversion, (c) median filtered and eroded image, (d) dilated image, (e) binarized image, and (f) Canny edge detection.

Morphological processing is another nonlinear image operation that depends on the shape or properties of the morphology. It is based on the relative ordering of pixel values rather than numerical numbers. The morphological analysis of an image utilizes a small template and form, referred to as a structuring element. The structuring element is positioned at each point in the original image and compared with the matching neighborhood of pixels. Certain operations assess whether a component fits inside a neighborhood, and others determine if it intersects or hits the neighboring components. We used grayscale erosion and dilation as the morphological processing techniques in this study.

Grayscale erosion of an image involves assigning the minimum value obtained from the structuring element to each pixel. In this study, the image was convolved with the kernel of an odd-size 5 matrix. Binary erosion of an image $f$ by a structuring element $p$ $(\text{denoted}f\ominus p)$ resulted in a new binary image $g=f\ominus p$ with ones in all locations $(x,y)$ of a structuring element’s origin at which that structuring element $p$ fits the input image $f$. This means that $g(x,y)=1$ if $p$ fits $f$ and = 0 otherwise, which is repeated for all pixel coordinates $(x,y)$. Similarly, the dilation of an image was convolved with the kernel of an odd-size 5 matrix. When a structuring element $f$ dilated an image $p$ ($\text{denoted}f\oplus p$), the result was a new binary image $g=f\oplus p$ with ones in all of the structuring element’s origin locations $(x,y)$, where the structuring element $p$ fits the input image $f$. Specifically, $g(x,y)=1$ if $p$ fits $f$ and = 0 otherwise, which is repeated for all pixel coordinates $(x,y)$. In this study, the normalized median value of 164 in the whole range was chosen as the threshold, which we found consistently delivered accurate results in distinguishing the lumen from surrounding tissues and artifacts.

Following the morphological processing, binarization was carried out through rough thresholding. Pixels in the input image with brightness below the threshold level were assigned a value of 0 (black) in the output binary image, and those with brightness above the threshold level were assigned a value of 1 (white). As a result, the final image consisted of only two intensity levels, i.e., 0 and 1. The canny edge detection technique was then applied to refine the image further. The Gaussian filter was employed to suppress the remaining noise, and a double threshold was utilized to identify edges.

2.4.

Automated Lumen Segmentation, Manual Correction, and 3D Reconstruction of the Lumen

The proposed segmentation method utilized nonuniform rational B-spline (NURBS) to represent the lumen shape by connecting limited interpolation (or knot) points on the lumen edge. NURBS offers a high degree of flexibility in representing complex shapes, making it particularly suitable for capturing the intricate contours of the lumen based on the segmentation process. The use of knot points in NURBS allows for easy manipulation and refinement of the shape. This feature was instrumental in our approach as it enabled us to correct the segmentation by simply moving the knot points, ensuring a more accurate representation of the lumen. Furthermore, NURBS can represent smooth curves and surfaces, which is essential for accurately modeling the lumen contours. The process is illustrated in Fig. 2. Initially, all points on the edges were obtained through canny edge detection to determine the appropriate knot points. Next, the catheter center was set as the origin to connect all edge points using straight lines, which were then grouped into 30 domains with evenly anticlockwise 12 deg. The shortest line in each domain was chosen, and the attached edge point became the knot point. Unnecessary knot points were removed based on a set criterion of 0.25 mm, determined through trial and error. If the distance between two-knot points fell within this criterion, one of the points was omitted. Finally, the knot points were connected anticlockwise to obtain the lumen segmentation.

Fig. 2

Automatic segmentation method and correction: (a) creation of edge points; (b) all pixel points are connected from the catheter center and are divided into 30 domains based on an axis angle of 12 deg; (c) find the shortest line in each domain; (d) segmented image by connecting the knot points on the image boundary; the green line is the contour estimated by the proposed model; (e) knot point with curvature more than $5{\mathrm{mm}}^{-1}$; and (f) curvature corrected image.

During the segmentation procedure, two additional steps were taken to filter out the interpolation points. First, in domains where the guidewire shadow and branches were present, there were usually few or no edge points ($<6$), and these points were filtered out to remove the influence of the shadow and branches. Second, if the curvature at a single knot point was too high (more than $5{\mathrm{mm}}^{-1}$), this indicated that the knot point might be incorrectly positioned, and it needed to be filtered out.

One of the challenges in automatic segmentation methods is the presence of large branches and abnormal images containing multiple artifacts or image distortions. Previous works have often avoided such frames, but in our method, we chose to manually correct them after the automatic segmentation. We also thoroughly checked all other frames to identify any misplottings, which were also corrected manually. To facilitate the manual correction process, we integrated computer-aided design (CAD) and made NURBS knots easy to control using simple mouse operations. The benefit of this approach is that an expert can correct each frame in just 5 s while ensuring the accuracy of the segmentation.

Following the reconstruction method described in our previous work,¹³ we created a 3D model of the lumen with Rhinoceros using the segmented curves (including corrected ones), which can be further applied for bifurcation reconstruction and analysis. We also reconstructed the 3D lumen models with ground truth segmentation for comparison.

3.

Results

The automated segmentation method was tested on 3300 frames in total, out of which 194 frames had big bifurcations ($>3\mathrm{mm}$), 121 were abnormal frames with artifacts, and 68 frames had misplottings. Large bifurcations approach very close to the image window size, limiting automation around the edges and requiring manual segmentation. The presence of artifacts, such as residual luminal blood due to suboptimal vessel flushing, which causes signal rich blood swirls in the lumen; blood clots causing complete attenuation of the OCT beam; and the sew-up artifact due to rapid artery or imaging wire movement leading to the lumen border misalignment, lead to significant errors in automation. Additionally, the proposed method encounters challenges in accurately identifying the lumen boundary for certain frames with edge dissections and honeycomb-like appearances due to spontaneous recanalization of thrombi, which lead to misplotting the lumen boundary. Therefore, the remaining 2917 frames were used to validate the automated segmentation method, compared with the corresponding ground truth segmentation performed by clinical specialists. Figure 3 displays the effectiveness of the method in segmenting OCT images with side branches and guidewire shadows of varying widths. In the case of big bifurcation frames, the segmentation results were manually modified using Rhino software to ensure a gradual change in the shape of segmented frames that accurately represents the transition from a branch to the main vessel (Fig. 4). Table 1 shows the time required for automatic segmentation, manual correction in Rhinoceros, and expert segmentation for 10 patients. On average, our method takes 0.17 s per frame for automated segmentation.

Fig. 3

Segmentation for frames with small side branches and guidewire shadows of different sizes. (a) Original OCT images; (b) ground truth manual segmentation; and (c) automatic segmented image by connecting the knot points, and the green line is the contour detection by the proposed model.

Fig. 4

Manual correction after automatic segmentation. (a) Automatic segmentation on bifurcation frames. The green line is the contour detection, and the dots are the pixel points on the boundary. (b) Manual correction on bifurcation frames.

Table 1

Processing time with the proposed automatic segmentation algorithm and manual segmentation.

The proposed OCT segmentation approach was found to exhibit good agreement with the ground truth results. Specifically, the bias value for the proposed model was $0.24{\mathrm{mm}}^2$ for the area, 0.07 mm for the average diameter, 0.08 mm for the maximum diameter, and 0.05 mm for the minimum diameter. The 95% limit agreement was calculated for each of these parameters, yielding the following ranges: ($-0.31$ to $0.78{\mathrm{mm}}^2$) for the area, ($-0.13$ to 0.26 mm) for the average diameter, ($-0.17$ to 0.33 mm) for the maximum diameter, and ($-0.17$ to 0.28 mm) for the minimum diameter (Fig. 5).

Fig. 5

Bland–Altman plots for comparison of the proposed algorithm and manual segmentation: (a) area, (b) average diameter, (c) maximum diameter, and (d) minimum diameter.

The scatter graphs demonstrated that the proposed automated segmentation was in closer agreement with the ground truth values, as evidenced by the proximity of the data points to the regression line (Fig. 6). The $R$-squared values for the lumen area, average diameter, maximal diameter, and minimal diameter were 0.98, 0.96, 0.95, and 0.95, respectively. The corresponding equations of the regression lines were as follows: $Y=0.99X+0.27$ for the lumen area, $Y=0.95X+0.18$ for the average lumen diameter, $Y=0.93X+0.24$ for the maximal lumen diameter, and $Y=0.96X-0.12$ for the minimal lumen diameter. The $p$-values for the lumen area, average diameter, maximal diameter, and minimal diameter were all found to be $<0.001$.

Fig. 6

Linear regression analysis of the proposed algorithm and manual segmentation: (a) area, (b) average diameter, (c) maximum diameter, and (d) minimum diameter.

In the last step of our study, we employed Rhino and Grasshopper to directly reconstruct 3D lumen models using the results of the automatic segmentation (with manual corrections) and compared them with the models generated from the ground truth segmentation. The reconstruction process was found to take $\sim 6\mathrm{s}$ per case. The resulting models were found to be fully compatible with the ground truth models, as illustrated in Fig. 7.

Fig. 7

Comparison of the 3D vessel reconstruction between the proposed algorithm (after manual segmentation correction) and the manual segmentation method: (a)–(j) cases 1 to 10.

4.

Discussion

We introduce a rapid methodology and platform for automatic OCT lumen segmentation and 3D lumen reconstruction that represents an advancement over the current state-of-the-art methods. Our technique has been demonstrated to be time-efficient, accurate, robust, and user-friendly when applied to clinical data. This approach has the potential to be widely used in the clinical setting to provide highly accurate information about coronary artery lumen in near real time, with 3D reconstructed models that can aid in clinical planning and decision-making in the cardiac catheterization laboratory.

Lumen segmentation is a crucial beginning step in processing OCT images. The accuracy and speed of this process directly influence the subsequent reconstruction and clinical analysis. OCT images are frequently affected by artifacts, guidewire shadows, and branching, however, which makes efficient processing of these images a significant challenge. This necessitates the preprocessing step to obtain a binary image of the intimal layer border to overcome artifacts and improve the image quality for further assessment. However, guidewire shadow is usually regular and constant in size, which makes correction easier. The gaps caused by bifurcations and other large artifacts can have variable shapes, which can cause the shape of the segmented lumen to be irregular. Interpolation of these regions is needed to draw the final lumen contour as close as possible to the expected values. Although several approaches have been proposed to address these difficulties in a time-efficient manner, most of them have processing times of more than 1 s per frame or require extensive training of the model with large datasets. For instance, Pociask et al.⁶ used 4137 images, and their model required 2 to 5 s per frame. Athanasiou et al.⁸ proposed an approach that took 40 s per frame for 556 images, Moraes et al.¹⁰ required 5.9 s per frame for 290 images, and Macedo et al.¹⁴ suggested a 15-s processing time per frame for 1328 images. Cheimariotis et al.¹⁵ proposed an approach using 1812 images that processed each frame in $<1\mathrm{s}$, and Kim et al.¹⁶ required an average of 0.367 s per image for 30 images. Tsantis et al.² proposed an approach that required $0.937\pm 0.045\mathrm{s}$ per image for 2710 images. A comparative study of the automated segmentation approaches of these studies and the processing times is presented in Table 2.

Table 2

Comparison of the proposed method with the current state-of-the-art methods.

The ideal method for OCT lumen segmentation of coronary vessels should exhibit two essential characteristics: (i) accurate identification of lumen boundaries and (ii) rapid processing speed with minimal human involvement. Our proposed automatic lumen segmentation technique achieves a processing time of 0.17 s per frame, with an additional 5 s required for manual correction for each frame in the bifurcation region. Based on our knowledge, our approach demonstrates one of the fastest image processing speeds while delivering comparable accuracy to other existing methods.

Our methodology’s efficient and precise processing can be attributed to two main factors. First, we have established that a NURBS curve passing through limited knot points can accurately represent the shape of the lumen. This fact has been validated through extensive manual segmentation operations conducted in our previous works.^13,21 The algorithm employed to determine these knot points after image preprocession is straightforward and robust as it involves only line creation, domain division, and length comparison. The filtering methods for knot points apply simple geometrical comparisons, such as point distance and curvature. As a result, our method extracts the essential characteristics of the lumen shape rapidly and accurately without complicated machine learning techniques and coordinate system transformations. By contrast, other works have relied on more complex machine learning methods with varying degrees of success. For example, Balaji et al.¹⁷ proposed a deep learning model for segmenting OCT images that required training the dataset for 3 h and 40 min, which is a significant limitation compared with our proposed model. Similarly, Guo et al.,¹² Kerkeni et al.,²² Abdolmanafi et al.,¹¹ and Macedo et al.²³ used supervised machine learning techniques, such as support vector machines and least squares regression, to segment OCT data. Although these attempts achieved some success, they were often hampered by a lack of training data and the intrinsic limitations of coronary artery OCT. More recently, convolutional neural networks and linear regression algorithms have been used to designate lumens using a points technique, as opposed to successful pixel-wise segmentation.²⁴ However, training machine learning models is challenging, particularly in the medical imaging field due to the exponential increase in the number of trainable parameters as the network depth increases. This can result in extremely lengthy training times and require complex hyperparameter adjustment approaches, particularly when dealing with high-resolution images, often $512\times 512\text{pixels}$ or larger.

Second, our method integrates CAD software (Rhinoceros), which offers a great convenience for manual correction and seamless data transfer from segmentation to reconstruction. This contrasts with other works that often use a third-party code for manual correction and reconstruction. Our method can provide a reconstructed vessel in real time, which is a significant advantage over other methods.^25–27

Accurate segmentation and reconstruction of the bifurcation region of a vessel is a challenging task for any methodology as the merging of a branch into the main vessel leads to a continuous change in the vessel’s shape and lumen contour. Thus accurately segmenting the gradually changing bifurcation frames requires careful manual effort to achieve a reasonably accurate transition from a side branch to the main vessel. Macedo et al.¹⁴ addressed this issue by initially performing automatic segmentation of bifurcations with significant side branches, followed by validation of the automated segmentation with manual segmentation of the same frame by an expert, which served as the ground truth. In the proposed approach, bifurcation frames were automatically segmented and then corrected manually by an expert to achieve the desired vessel shape and ensure a more precise reconstruction.

We performed a comprehensive validation of our methodology by comparing it to ground truth data provided by medical experts. Our validation process involved the use of various dimensional indices, including lumen area and lumen maximum, minimum, and average diameters. We found that our proposed OCT segmentation approach produced results that were in good agreement with the ground truth data. Specifically, we obtained $R^2$ values of 0.98, 0.97, 0.96, and 0.96 for lumen areas, average diameters, maximum diameters, and minimum diameters, respectively. The $p$-values for these measurements were all below 0.0001, indicating strong statistical significance.

Our methodology offers a user-friendly feature due to the visual programming language tool, Grasshopper 3D. In contrast to conventional text-based codes, such as MATLAB, Grasshopper 3D facilitates a modularized workflow, even for operators without programming experience. This tool requires minimal intervention for the parameter settings. The feedback from clinical physicians, who are end users of the methodology, indicates that the learning curve is minimal and that it allows them to become familiar with the tool quickly. This feature improves the usability of the tool in interpreting the results and allows for manual intervention where necessary. Additionally, it enables users to adjust segmentation parameters with ease and view the results in almost real time. Therefore, our methodology is practically feasible and versatile for different types of real patient data and diseased bifurcation anatomies, regardless of the degree of stenosis.

Our proposed method has significant applications in offline clinical studies. The precise segmentation and reconstruction of the lumen can offer valuable insights for researchers in understanding the scope, severity, and precise vessel anatomy of coronary artery stenosis. This can aid in the personalized planning of stenting techniques for modeling and simulation purposes, leading to a deeper understanding of patient outcomes. Furthermore, extending this methodology to other imaging modalities is a possibility. The fundamental concepts of our methodology, particularly the efficient lumen edge extraction, can provide a valuable foundation for future research seeking to adopt similar techniques to diverse imaging modalities, such as intravascular ultrasound imaging or coronary computed tomography angiography, keeping in mind their own distinct challenges and necessitating tailored approaches for successful adaptation.

5.

Limitations

There are several limitations to our work. First, our automatic method is not suitable for analyzing large bifurcations, as there may not be sufficient edge points present. Second, lumen shapes in bifurcations can change gradually, further complicating automatic detection. Therefore, manual correction is necessary for such cases. Third, our method focuses only on lumen edge detection, which means that it is unable to recognize the medial wall and characterize the plaque tissues. Fourth, our method was not tested on stented coronary arteries, limiting the usefulness of this methodology for clinical applications. Future studies will focus on testing this method on the stented vessels. Finally, our study used manual segmentation by clinical experts as the gold standard to validate the performance of the proposed methodology. There is a possibility of unreliable estimation by experts, however, due to the difficult segmentation of certain atypical frames. Thus it requires further validation of the proposed methodology using a more accurate reference. Despite these limitations, we believe that our work can serve as a solid foundation for future studies on lumen walls and tissue segmentation.

6.

Conclusion

We introduced a platform that enables rapid and automated coronary OCT lumen segmentation, with the added convenience of manual correction, followed by a seamless 3D vessel reconstruction process. We evaluated the accuracy of the segmented lumens and reconstructed vessels against ground truth data (covering a wide range of anatomical complexities) and found our method to be accurate and time efficient. Our platform employs NURBS’ distinctive characteristics and is integrated with CAD software and a virtual programming environment, resulting in a time-efficient, accurate, robust, and user-friendly tool for clinical image processing in nearly real time. This platform has the potential to aid clinical planning, education, and decision making for coronary catheterization.