1.

Introduction

Breast cancer is the most frequently diagnosed cancer among women in the Western world.¹ It can be detected by mammography, ultrasound (US), and magnetic resonance imaging (MRI). Positron emission tomography (PET) and X-ray CT scan can be used to evaluate systemic, metastatic disease. These modalities measure tumor size and location,^1–3 but their use is limited by either their availability or cost (MRI and PET). Some involve ionizing radiation (mammography, CT, PET), and the sensitivity and specificity can be suboptimal (mammography, US).

Thermography, which is a nonionizing, noninvasive, portable, and low-cost imaging modality, can be used potentially. Thermography is based on the detection of midinfrared radiation inertly emitted from the surface of a measured object.⁴ Any object with a temperature above the absolute zero degrees emits radiation from its surface. Thermography enables the recording of the temperature distribution of an object using the infrared radiation emitted by the surface of the object at wavelengths between 8 and $12\mu \mathrm{m}$.⁵ The human skin has a high emissivity of 0.98, thus the measured radiation can be converted directly into accurate temperature values. The high sensitivity of thermography to surface changes can be valuable for cancer treatment monitoring. Since malignant tumors are characterized by abnormal metabolic and perfusion rates,⁵ they are expected to show a different temperature distribution compared to the surrounding healthy tissue.^6,7 Metabolic heat production is known to correlate with tumor growth,^8,9 the more aggressive the tumor is, the more heat it produces.^9–12 Therefore, a change in skin temperature map during treatment can be a measure of the tumor’s response to treatment. Thermography has been extensively studied as a breast cancer detection tool^3,4,13–15 and was also investigated as a tool for early detection of mucositis in locally advanced head and neck cancer treated with chemoradiotherapy.¹⁶ The use of thermography to monitor treatment efficacy has yet to be evaluated. González¹⁷ showed that thermal camera with sensitivity of 100 mK can detect tumors with a diameter of 1 cm at a depth of 2 to 3 cm, and if the diameter of the tumor is bigger, it can be detected even deeper. In this study, we present a method for tumor and vasculature detection from a thermal image and propose a quantitative estimation of the changes in tumor and vasculature during radiotherapy. To the best of our knowledge, no other work demonstrates a visualization and quantification of tumor vasculature from a thermal image. The algorithm that we developed (Fig. 1) consists of four main steps:

Fig. 1

The flowchart of the algorithm that we developed and reported in this study.

The first step consists of gray scale conversion, normalization of the image, and determination of the region of interest. In the second step, a filter that highlights the tumor and vasculature is applied. This filter was originally used to show vessels that have high contrast in angiography. However, we use this filter, as will be explained in this section, to identify blood vessels (long, narrow, hot objects) and blobs of heat (malignant tumors) in thermal images. In the third step, the change in regularity (as shown in the filtered images of the tumors) is calculated. In the filtered images of the vasculature, the changes in the vasculature’s shape are calculated. Finally, a quantitative measure of the change in the tumor’s regularity and the change in vasculature shape is produced.

2.

Preprocessing Steps

The preprocessing step includes (Fig. 2): (1) conversion of color image to gray scale and (2) setting a fixed temperature range of 7°C for all images. One can see physiological changes in the human tissue around 7°C. Entropy $H(x)$ calculates the amount of uncertainty of a random variable. It is defined as follows:

Fig. 2

Preprocessing steps explained on one of the thermal images taken in this study.

The probability density $p(x_k)$ is needed for the calculation of the image entropy. This parameter is being estimated using a grayscale histogram. A fixed temperature range enables the comparison of the entropy shown in different images. The tumor area was cropped to enable focusing on the tumor’s change during radiotherapy.

3.

Tumor and Vasculature Filtering and Monitoring

Frangi filter¹⁸ uses the eigenvectors of the Hessian matrix to compute the likelihood of an image region to contain vessels. As explained in Frangi et al. paper: when there is a significant difference between the two eigenvalues ${\lambda }_1$ and ${\lambda }_2$ of the Hessian matrix (one is low and the other is high), the object is tubular, and when there is no significant difference between the two eigenvalues, the object is blob-like. A blood vessel is an example of a tubular object.

Frangi defined the following parameter, which describes to what extent the object is either blob-like or tubular:

The more tubular the object is, the lower the value for this parameter.

An additional parameter used by Frangi is $S$, where

The center of a blood vessel is a deep local minimum point when moving perpendicular to the main axis of the vessel. The second derivative in that direction is therefore high at such points, which yields a large value of $S$. Therefore, $S$ ensures that we are in the center of the blood vessel.

Frangi et al. define a parameter called vesselness that aimed to emphasize the blood vessel in the image:

The constants $\beta$ and $c$ control the sensitivity of the filter.

The reason for setting $V$ to 0 when ${\lambda }_2$ is negative because ${\lambda }_2<0$ when the tubular object is brighter than its surroundings, and this is not the case with blood vessels. The smaller RB gets (we are in a tubular object) and the larger S gets (we are near the center of the object) then the value of the vesselness ($V$) increases. Note that $V$ depends on the width σ of the Gaussian (because the second derivatives at H depend on it).

Hence, the calculation of the parameters that construct $V$ needs to be performed for a series of σ values. We chose values suitable for small blood vessels.

Although the Frangi filter is designed to emphasize tubular structures, in thermal breast images, this filter can highlight both the tumor (which is generally blob-like) and the blood vessel network of the tumor (which is composed of tubular objects).

The tumor is characterized by a high temperature gradient (which means that its center is much warmer than its circumference), and therefore, it appears as a small bright blob showing a high and steep intensity change with the peak at its center. This leads to a high second derivative along the axes and hence to a high value of $S$. Even though the value of RB for a tumor is not small (blob shape), the effect of $S$ yields a high value of vesselness and the tumor appears bright in the filtered image.

Each blob-like tumor contains a network of small tubular blood vessels. Enlarging the tumor (a “zoom” operation) enables us to observe the local temperature changes that characterize the blood vessels. This interpolation adapts the range of vessel widths addressed by the parameter $V$ to the value typical to tumor vasculature. Although $S$ is not as high as the value measured for the tumor before interpolation (now we have smaller and less steep intensity changes), the combined effect of RB and $S$ is sufficient for obtaining high vesselness values in the tubular structures (the blood vessels). In our case, enlarging the image by sevenfold allows the detection of the blood vessels.

4.

Feature Extraction

Entropy characterizes the homogeneity of the image. The vasculature in tumors is disorderly compared to the structure in normal tissue.¹⁹

This may cause high entropy in the thermal image when compared to normal tissue. Tumors affect the homogeneity of a thermal image. Therefore, we investigate the possibility of measuring entropy to evaluate changes in the tumors.

In the feature extraction stage, entropy is calculated in the cropped thermal image of the tumor area as well as in the filtered tumor image.

After all the images of the patient have been filtered, we calculated the change in entropy in the image of the filtered tumor, as described in this equation:

5.

Filtering and Entropy Calculation

During radiation treatment, there was a reduction in entropy in the tumor areas in all patients. Figure 3 shows the entropy of the tumor areas during radiation therapy (as shown in the cropped thermal images).

Fig. 3

Entropy of the thermal images (cropped tumor area) versus total applied dose as measured over night after the last radiation fraction.

Figure 4 shows the thermal imaging of patient #1 before, during (21 Gy), and at the end of treatment (39 Gy). At the top of the thermal image, taken prior to the beginning of the treatment (after vasculature and tumor filtering), the tumor and vasculature are visualized. In the middle thermal image, taken during the radiotherapy (after 21 Gy), the vasculature and tumor are visibly reduced. In the thermal image at the bottom, taken at the end of treatment (after 39 Gy), a sharp decrease in size of the tumor and vasculature is evident.

Fig. 4

Tumor and vasculature filtering from thermal image of patient #1 during radiation therapy.

Figure 5(a) shows a CT-PET scan of patient #1 before radiation therapy. Figure 5(b) shows a CT-PET scan after the treatment. There is a reduction in the tumor avidity to 2-deoxy-2-[fluorine-18] fluoro-d-glucose documenting a treatment response.

Fig. 5

(a) CT-PET scan of patient #1 before radiation therapy and (b) CT-PET scan after treatment. There is a reduction in the affinity to 2-deoxy-2-[fluorine-18] fluoro-d-glucose.

All images had the same gray scale. The last image in Fig. 4 was indeed darker due to heating of the breast during treatment. Radiotherapy involved irradiating the entire breast. This radiation caused inflammation, which in turn caused heating of the breast. The breast at the end of the treatment was therefore warmer than it was during earlier stages of the treatment. This appears as a darker breast in the grayscale image representing the end of the treatment (bottom image in Fig. 4).

As shown in Fig. 6, the entropy of the filtered tumor image is reduced during radiotherapy. There is no data for patient #4, because her tumors were invading the skin with ulceration. In this case, Frangi filtering highlights the edges of the tumor on the skin and not the tumor or the vasculature.

Fig. 6

Entropy of filtered tumor versus total dose.

6.

Quantification of Vascular Shape Changes

Using the filter, we were able to see the vascular network. In order to quantify the changes in the shape of the vascular network, we converted the image into a binary image, as shown in Fig. 7, and counted the number of objects. Figure 8 shows a diagram of the change in the number of objects during radiation therapy.

Fig. 7

Conversion of the filtered vasculature image to binary image (before treatment and at the end treatment for patient #1).

Fig. 8

Change in the number of objects in a binary image before and during treatment for all patients.

In order to find out whether there is a statistically significant difference between the starting values and ending values of the entropy and of the number of objects, a student’s $t$-test was used. The obtained values were $p=0.01208$ for the entropy of the tumor areas, $p=0.0065$ for the entropy of the filtered tumor image, and $p=0.1299$ for the number of objects.

Table 2 shows the changes in the percentage of the entropy, as calculated from the baseline image before treatment and the image after 30 Gy (patient #1 after 39 Gy).

Table 2

Decrease in the entropy of filtered tumor images (%).