1.

Introduction

The rates of melanoma and nonmelanoma skin cancers (NMSC) have been steadily rising,^1,2 and early diagnosis is key for improved outcomes.³ Because there is a shortage of board-certified dermatologists,^4,5 particularly in remote or underserved settings where <10% of dermatologists practice,⁶ most of the burden of diagnosis and treatment falls on primary care physicians (PCPs) who are not extensively trained in dermatological care.^3,7 Dermoscopy is a tool utilized to improve the in vivo diagnostic accuracy of benign versus malignant lesions, a unique skill that requires additional training, even among board-certified dermatologists. In remote settings, dermascopes may capture and document pigmented lesions that can be forwarded to expert colleagues through telemedicine for further analysis.⁸ Unfortunately, dermascopes and their accessories range from hundreds to thousands of dollars,^9,10 which is potentially too expensive for general medical practice. Thus, there is a need for a low-cost, readily available dermoscopy tool to bridge this clinical need.

Lesion evaluation using visual, subjective methods such as the ABCDE criteria and seven-point checklist are useful tools for PCPs.^3,11 The ABCDE criteria predict melanoma by a lesion’s asymmetry, border irregularity, coloration, diameter if $>6\mathrm{mm}$, and evolution, providing a sensitivity of 0.85 and specificity of 0.72.^3,11 The seven-point checklist monitors a lesion’s change in size, shape, color, and looks for diameters $>7\mathrm{mm}$, crusting or bleeding, and sensory change, providing a sensitivity of 0.77 and specificity of 0.80.³ Continuous monitoring has shown to improve outcomes through early detection as evidenced by mole mapping techniques^12,13 and the increase in sensitivity and specificity with the addition of the evolving in the ABCDE criteria.¹¹

Adjunctive tools utilizing objective measures such as polarized multispectral imaging (PMSI) and polarized white-light imaging (PWLI) to map dermal chromophores [hemoglobin, deoxyhemoglobin (Hb), and melanin], quantify erythema, and perform image classification for lesion screening have the potential to increase early detection of melanoma by PCPs and even outside the physician’s office, leading to reduced need for biopsy and improved outcomes.^14–27 We propose a smartphone combined with LED illumination as the ideal platform for an adjunctive medical device, which will provide a portable system with easy-to-operate apps and native image capture, processing, and data transmission. These systems can reduce the costs associated with interference-filter-based^14,15,20 or spectrometer-based^21,23 systems while also providing a more compact, portable geometry for use in any testing environment compared with clinical-grade imaging systems.^17–19

2.

Materials

We have developed two point-of-care dermascope design concepts for skin lesion screening and erythema monitoring, implementing both PMSI and PWLI²⁸ on an LG G5 (LG, Seoul, South Korea) smartphone platform. One system concept utilizes the embedded smartphone camera for imaging while the other uses a USB-connected camera module that connects to the smartphone. Both systems share a common illumination system and software application to enable PWLI and PMSI.

The PMSI and PWLI dermascope using the smartphone’s embedded rear camera is shown in Figs. 1(c)–1(e). The main LG G5 camera consists of a Sony IMX234 Exmor RS sensor with $5312\times 2988$, $1.12\text{-}\mu \mathrm{m}$ pixels and a $5.95\mathrm{mm}\times 3.35\mathrm{mm}$ sensor size. The sensor is paired with a $f/1.8$, 4.42-mm focal length lens.

Fig. 1

Two dermascope implementations. The USB-camera-based PMSI and PWLI dermascope is shown in (a) and (b). (a) Various components of the handheld imaging module (the USB camera is hidden behind the imaging polarizer) and (b) the imaging module paired with the smartphone camera. The smartphone-camera-based PMSI and PWLI dermascope is shown in (c), (d), and (e). (c) The smartphone-based system’s side opposite the smartphone screen with the imaging annulus removed, where the LED PCB and smartphone camera are visible and other components are highlighted; (d) the system with the imaging annulus attached; and (e) the smartphone installed in the dermascope.

To decrease the working distance of the optical system to allow imaging of the epidermis, a 24-mm focal length achromatic doublet (Ross Optical, El Paso, Texas, USA) is placed 4 mm away from the principal plane of the smartphone optical system, providing a magnification of $m=0.187$ and a numerical aperture $\mathrm{NA}=0.04$. After cropping, the field of view (FOV) is $9.96\mathrm{mm}\times 11.67\mathrm{mm}$, as shown in Fig. 2. The imaging achromat is aligned to the smartphone camera using a machined PMMA disk installed in a removable 3D-printed annulus of VeroBlue RGD840 (Stratasys, Eden Prairie, Minnesota, USA) plastic. The annulus serves as an imaging guide; its length equals the optical system working distance (23 mm), so the PCP can contact the patient to stabilize the device and ensure correct focus. An additional 3D-printed structure serves as a mounting platform for the smartphone, imaging annulus, and LED electronics.

Fig. 2

Layout of the LG G5 smartphone camera and the added achromat. The smartphone camera lens system is modeled as a paraxial lens.

The alternative PMSI and PWLI dermascope [Figs. 1(a) and 1(b)] is also based on an LG smartphone platform, but it utilizes an external USB-connected RGB camera (OV5648, Omnivision, Santa Clara, California, USA; 5 MP, $3.67\mathrm{mm}\times 2.74\mathrm{mm}$) with the vendor-supplied $\sim 2.8\text{-}\mathrm{mm}$ focal length lens adjusted to a working distance of 30 mm. After cropping, the FOV is $27.5\mathrm{mm}\times 20\mathrm{mm}$. In addition, the integrated infrared (IR) filter was removed. Again, the mechanical design of the annulus is matched to the working distance of the camera, providing in-focus imaging when the device contacts the patient.

For both systems, multispectral illumination is accomplished using a custom printed circuit board (PCB) with LEDs of various wavelengths (Lumileds, Amsterdam, The Netherlands; Vishay, Malvern, Pennsylvania, USA) installed as shown in Table 1. The color wavelengths were chosen based on commercial availability and the ability to probe both hemoglobin isosbestic points and separate oxygenated from deoxygenated hemoglobin content along the molar attenuation curves (Fig. 3).

Table 1

LED and camera settings for each illumination wavelength and each dermascope. LED wavelength (λ), LED part number, smartphone camera LED-driving current (I), smartphone camera LED flux for a single LED of the given color, smartphone camera International Organization for Standardization setting, smartphone camera exposure time, USB camera LED-driving current (I), USB camera LED flux for a single LED of the given color, USB camera brightness setting, and USB exposure time are provided.

Fig. 3

Molar extinction coefficients, $\epsilon (\lambda )$, for Hb, ${\mathrm{HbO}}_2$, and melanin plotted on a log scale and the LED spectral flux probability density functions, ${\varphi }_{\mathrm{e},\lambda }$, plotted on a linear scale.

For the smartphone-based dermascope, the PMMA disk used for mounting the lens also extends over the illumination LEDs to provide mounting for a linear polarizer (Edmund Optics, Barrington, New Jersey, USA). An orthogonal linear polarizer is installed in front of the imaging channel, enabling both PMSI and PWLI and reducing the effect of specular reflection on the images.²⁸ The LED sources’ spectral fluxes, ${\varphi }_{\mathrm{e},\lambda }$, shown in Fig. 3, were measured with a spectrometer (Ocean Optics).

The USB-camera-based dermascope uses the same LED PCB and wavelengths for illumination along with orthogonal polarizers in the illumination channel (Edmund Optics) and the imaging channel (Moxtek, Orem, Utah, USA). To help normalize white-light image luminance, an 18% gray color reference (Kodak, Rochester, New York, USA) is permanently installed on both sides of the image FOV. Because the 3D-printed mounting foundation does not need to mount the LED board and imaging annulus, a previously designed geometry is used for this system.²⁹

The illumination PCB consists of three LEDs of each color soldered in a symmetrical pattern around the camera aperture to maximize uniformity without additional beam shaping optics. The backside solder mask of the PCB was removed to expose the copper and is attached to a copper heatsink with electrically insulating epoxy (DP240, 3M, St. Paul, Minnesota, USA). Numerous vias were placed on the PCB to ensure a low thermal resistance between the front and backside copper planes. The LEDs are driven with a switching boost power supply (LT3478, Linear Technology, Milpitas, California, USA) powered by two lithium-ion batteries (Orbtronic, Saint Petersburg, Florida, USA). Each LED color string can be turned on individually with a custom power level setting and illumination, and image capture is synchronized by a custom Android application through a Bluetooth-connected microcontroller (MCU, IOIO-OTG, SparkFun Electronics, Niwot, Colorado, USA). The LED-driving currents, fluxes, and dermascopes’ image capture settings are shown in Table 1. In addition, the smartphone camera uses the daylight white balance setting, and the white balance setting of the USB camera is inaccessible. A block diagram of the system electronics is shown in Fig. 4.³⁰ The Android application controls the camera functions, synchronizes the LED illumination, and sets camera exposure time. For the USB camera, the Android app was modified to use the USB camera instead of the on-board smartphone camera. Images are connected to an ID assigned to each patient, removing identifiable information from the smartphone. Screenshots of the app are shown in Fig. 4.

Fig. 4

The system electronics block diagram is provided in (a) and Android application screenshots in (b) and (c).

3.

Methods

3.1.

Data Processing

The algorithms used to process collected dermal images are provided in Algorithms 1 and 2. Descriptions of the steps and related equations are provided in the following sections.

Algorithm 1

Processing of reference images.

1: procedure ProcessReferenceImages (reference images)

2:  for all reference images do

3:   convert sRGB to linear RGB ▹ Eq. (1)

4:   if white-light image then

5:    convert linear RGB to CIEXYZ ▹ Eq. (2)

6:   else if color image then

7:    convert linear RGB to Yequal ▹ Eq. (3)

8:   end if

9:   calculate luminance reference ▹ Eq. (4)

10:   calculate illumination uniformity reference ▹ Eq. (5)

11:  end for

12:  return optical density reference images

13:  return illumination uniformity images

14: end procedure

Algorithm 2

Processing of dermal images.

1: procedure ProcessDermalImages (dermal images)

2:  for all dermal images do

3:   if USB camera then

4:    correct white-light image luminance ▹ Eq. (8)

5:  end if

6:  convert sRGB to linear RGB ▹ Eq. (1)

7:  correct by illumination uniformity ▹ Eq. (6)

8:  if white-light image then

9:   convert linearRGB to CIEXYZ ▹ Eq. (2)

10:   convert CIEXYZ to CIELAB ▹ Eqs. (17) and (18)

11:  else if color image then

12:   convert linear RGB to Yequal ▹ Eq. (3)

13:   calculate optical density ▹ Eq. (7)

14:   calculate melanin content ▹ Eq. (19)

15:   calculate erythema ▹ Eq. (20)

16:   solve chromophore concentration ▹ Eq. (14)

17:   end if

18:  end for

19: end procedure

3.1.1.

Image collection

When the dermascopes were first built, images of an 18% reflective gray card were collected by each system at each wavelength to serve as both the optical density (OD) and illumination uniformity references.

For dermal image collection, a pilot study was performed on human subjects at the University of Arizona College of Medicine, Division of Dermatology to determine feasibility of each multispectral dermascope. This study received institutional review board approval (#1612067061). All patients provided informed written and oral consent.

3.1.2.

Colorspace conversions

The melanin content, erythema, and chromophore concentration measurements rely on conversion to the CIELAB and CIEXYZ colorspaces. The imaging systems natively capture in the sRGB colorspace, and the images are first converted to linear RGB space:³¹

Eq. (1)

$$C_{\text{linear}}=\{\begin{array}{ll}\frac{C_{\mathrm{sRGB}}}{12.92}& C_{\mathrm{sRGB}}\le 0.04045\\ {\left(\frac{C_{\mathrm{sRGB}}+0.055}{1+0.055}\right)}^{2.4}& C_{\mathrm{sRGB}}>0.04045\end{array},$$

where $C_{\mathrm{sRGB}}$ is each channel of the ${\mathbf{I}}_{\mathrm{sRGB}}$ image. Images are then converted from ${\mathrm{RGB}}_{\text{linear}}$ to CIEXYZ using the transformation matrix,³¹

Eq. (2)

$$\left[\begin{array}{c}X\\ Y\\ Z\end{array}\right]=\left[\begin{array}{ccc}0.4124& 0.3576& 0.1805\\ 0.2126& 0.7152& 0.0722\\ 0.0193& 0.1192& 0.9505\end{array}\right]·\left[\begin{array}{c}{R}_{\text{linear}}\\ G_{\text{linear}}\\ B_{\text{linear}}\end{array}\right],$$

where $Y$ is the luminance value and is used to calculate ODs from the dermis images and reference. Luminance is a measure that scales optical radiation by the response of the human visual system.³² Because the images will be processed by a computer, accurate color representation for a human is not required, so an additional luminance measure, $Y_{\text{equal}}$, is created using the equal sum of all three channels:

Eq. (3)

$$Y_{\text{equal}}=[\begin{array}{ccc}1& 1& 1\end{array}]·\left[\begin{array}{c}{R}_{\text{linear}}\\ G_{\text{linear}}\\ B_{\text{linear}}\end{array}\right].$$

3.1.3.

Reference and illumination uniformity correction

Using the reference images that have been converted to CIEXYZ or Yequal, reference luminance images are defined as

Eq. (4)

$${\mathbf{I}}_0=\overline{{\mathbf{Y}}_{\mathrm{ref}}},$$

where ${\mathbf{Y}}_{\mathrm{ref}}$ is the $Y$ (luminance) channel of the CIEXYZ image or Yequal. The reference grayscale image is normalized to serve as the illumination reference for the dermal images.

Eq. (5)

$$\mathbf{U}=\frac{{\mathbf{Y}}_{\mathrm{ref}}}{\max ({\mathbf{Y}}_{\mathrm{ref}})},$$

where $\mathbf{U}$ is now the illumination uniformity correction matrix.

The dermal CIEXYZ and Yequal images are corrected in the same way

Eq. (6)

$${\mathbf{I}}_{\text{dermal},\text{uniformity corrected}}=\frac{{\mathbf{I}}_{\text{dermal}}}{\mathbf{U}}\overline{\mathbf{U}},$$

where ${\mathbf{I}}_{\text{dermal}}$ is the illumination uniformity corrected dermal image with constant mean luminance. Finally, OD dermal images are calculated as

Eq. (7)

$$\mathrm{OD}=-\ln \left(\frac{\mathbf{I}}{{\mathbf{I}}_0}\right).$$

Finally, the USB dermascope has sections of a 18% gray photography card mounted on either side of the FOV [Fig. 1(b)]. Knowing the card image should equal 50% levels of RGB, the luminance of the white-light image is scaled using the following equation:

Eq. (8)

$${\mathbf{I}}_{\text{dermal},\text{luminance corrected}}={\mathbf{I}}_{\text{dermal},\text{uniformity corrected}}\frac{0.5}{\overline{{\mathrm{Y}}_{\text{gray}}}}.$$

3.1.4.

Chromophore concentration

The Beer–Lambert law is utilized to measure the relative concentrations of Hb, oxyhemoglobin (${\mathrm{HbO}}_2$), and melanin:^{17,22,33–35}

Eq. (9)

$$I(\lambda )=I_0(\lambda )\exp [-c_n\epsilon (\lambda )\ell (\lambda )],$$

where $I$ is the resulting intensity, $I_0$ is the incident intensity, $c_n$ is the concentration of the chromophore, $\epsilon (\lambda )$ is the molar attenuation coefficient of the chromophore at a particular wavelength, and $\ell (\lambda )$ is the optical path length of the light in the medium for the incident wavelength. This is restated as OD:

Eq. (10)

$$\begin{gathered} \mathrm{OD}=-\log \left(\frac{I(\lambda )}{I_0(\lambda )}\right)=c_{\mathrm{Hb}}{\epsilon }_{\mathrm{Hb}}(\lambda )\ell (\lambda )+c_{{\mathrm{HbO}}_2}{\epsilon }_{{\mathrm{HbO}}_2}(\lambda )\ell (\lambda ) \\ +c_{\text{melanin}}{\epsilon }_{\text{melanin}}(\lambda )\ell (\lambda )+c_{\text{background}}, \end{gathered}$$

where $c_{\text{background}}$ is due to residual absorption from molecules present in the epidermis and dermis.

The molar extinction coefficients for Hb and ${\mathrm{HbO}}_2$³⁶ and melanin³⁷ are shown in Fig. 3. Jacques’s ${\epsilon }_{\text{melanin}}$³⁷ was fit with an exponential curve to extend the wavelength to 1000 nm, resulting in a fit of

Eq. (11)

$${\epsilon }_{\text{melanin}}=2.2858·{10}^4\exp (-5.5028·{10}^{-3}\lambda ).$$

Optical path lengths, $\ell (\lambda )$, for the chromophores are calculated from a linear fit of Anderson’s data³⁸ in the region of the illumination wavelengths,

Eq. (12)

$$\ell (\lambda )=2.62·{10}^{-4}\lambda -9.87·{10}^{-2},$$

where $\lambda$ is in units of nm and $\ell (\lambda )$ is in units of cm.

Because the LEDs are broad spectrum, we integrate over the wavelength probability density function to calculate a total molar attenuation coefficient^39,40 for each color

Eq. (13)

$${\epsilon }_{\text{total}}=\int {\varphi }_{\mathrm{e},\lambda }(\lambda )\epsilon (\lambda )\mathrm{d}\lambda .$$

The resulting molar attenuation coefficients for all of the chromophores are shown in Table 2.

Table 2

Molar extinction coefficients calculated using Eq. (13) for each illumination wavelength compared with the molar extinction coefficients for the peak wavelength.

Wavelength (nm)	Coefficients from Eq. (13)			Coefficients at peak LED wavelength
	Hb (cm−1 M−1)	HbO2 (cm−1 M−1)	Melanin (cm−1 M−1)	Hb (cm−1 M−1)	HbO2 (cm−1 M−1)	Melanin (cm−1 M−1)
450	199,864	82,747	1922	103,292	62,816	1921
470	35,937	36,662	1706	16,156	33,209	1721
500	26,659	25,521	1392	20,862	20,932	1459
530	37,824	34,851	1241	39,036	39,957	1237
580	22,606	13,258	869	37,010	50,104	940
660	3380	352	611	3227	320	605
810	845	812	281	717	864	265
940	656	1185	142	693	1214	130

A system of equations is built from the multispectral datacube and the molar attenuation coefficients shown in Table 2

Eq. (14)

$$\left[\begin{array}{cccc}{\epsilon }_{\mathrm{Hb}}({\lambda }_1)\ell ({\lambda }_1)& {\epsilon }_{{\mathrm{HbO}}_2}({\lambda }_1)\ell ({\lambda }_1)& {\epsilon }_{\text{melanin}}({\lambda }_1)\ell ({\lambda }_1)& 1\\ {\epsilon }_{\mathrm{Hb}}({\lambda }_2)\ell ({\lambda }_2)& {\epsilon }_{{\mathrm{HbO}}_2}({\lambda }_2)\ell ({\lambda }_2)& {\epsilon }_{\text{melanin}}({\lambda }_2)\ell ({\lambda }_2)& 1\\ {\epsilon }_{\mathrm{Hb}}({\lambda }_3)\ell ({\lambda }_3)& {\epsilon }_{{\mathrm{HbO}}_2}({\lambda }_3)\ell ({\lambda }_3)& {\epsilon }_{\text{melanin}}({\lambda }_3)\ell ({\lambda }_3)& 1\\ ⋮& ⋮& ⋮& ⋮\\ {\epsilon }_{\mathrm{Hb}}({\lambda }_{\mathrm{n}})\ell ({\lambda }_{\mathrm{n}})& {\epsilon }_{{\mathrm{HbO}}_2}({\lambda }_{\mathrm{n}})\ell ({\lambda }_{\mathrm{n}})& {\epsilon }_{\text{melanin}}({\lambda }_{\mathrm{n}})\ell ({\lambda }_{\mathrm{n}})& 1\end{array}\right]\left[\begin{array}{c}{c}_{\mathrm{Hb}}\\ c_{{\mathrm{HbO}}_2}\\ c_{\text{melanin}}\\ c_{\text{background}}\end{array}\right]=\left[\begin{array}{c}\mathrm{OD}({\lambda }_1)\\ \mathrm{OD}({\lambda }_2)\\ \mathrm{OD}({\lambda }_3)\\ ⋮\\ \mathrm{OD}({\lambda }_{\mathrm{n}})\end{array}\right].$$

and the system is solved by linear algebra least-squares techniques³³ where $\mathrm{OD}({\lambda }_{\mathrm{n}})$ are calculated OD matrices for each illumination wavelength.

The ability of the dermascopes to properly measure relative chromophore concentrations was validated using a finger occlusion test. Images were taken with both dermascopes and the chromophores mapped preocclusion, after 2 min of occlusion, postocclusion, and 5 min after ending the occlusion.⁴¹

3.1.5.

Melanin and erythema

To measure melanin content and erythema, the white-light image is converted to the CIELAB⁴² colorspace using lightness ($L^{*}$) as a measure of relative melanin content and the direction of red color stimuli ($a^{*}$) as a measure of redness, with more positive values indicating higher levels of erythema.⁴³ Before converting to CIELAB, normalization constants must be calculated from the white-LED spectral content. Using the color matching functions,⁴⁴ $\overline{x}(\lambda )$, $\bar{y}(\lambda )$, $\overline{z}(\lambda )$ (Fig. 5), $X$, $Y$, and $Z$ are calculated as⁴²

Eq. (15)

$$X={\int }_{380\mathrm{nm}}^{780\mathrm{nm}}\overline{x}(\lambda ){\varphi }_{\mathrm{e},\lambda }\mathrm{d}\lambda ;Y={\int }_{380\mathrm{nm}}^{780\mathrm{nm}}\overline{y}(\lambda ){\varphi }_{\mathrm{e},\lambda }\mathrm{d}\lambda ;Z={\int }_{380\mathrm{nm}}^{780\mathrm{nm}}\overline{z}(\lambda ){\varphi }_{\mathrm{e},\lambda }\mathrm{d}\lambda ,$$

where ${\varphi }_{\mathrm{e},\lambda }$ is the relative spectral flux of the white-LED source as shown in Fig 3. The normalization constants $X_{\mathrm{n}}$, $Y_{\mathrm{n}}$, and $Z_{\mathrm{n}}$ are calculated by

Eq. (16)

$$X_{\mathrm{n}}=\frac{X}{Y};Y_{\mathrm{n}}=\frac{Y}{Y};Z_{\mathrm{n}}=\frac{Z}{Y}.$$

Fig. 5

Color matching curves used to determine normalization constants to convert to CIELAB along with the 4000 K white-LED spectrum.

The image is then converted to CIELAB by⁴²

Eq. (17)

$$\begin{gathered} L^{*}=116f\left(\frac{Y}{Y_{\mathrm{n}}}\right)-16, \\ {a}^{*}=500\left[f\right(\frac{X}{X_{\mathrm{n}}})-f(\frac{Y}{Y_{\mathrm{n}}}\left)\right], \\ {b}^{*}=200\left[f\right(\frac{Y}{Y_{\mathrm{n}}})-f(\frac{Z}{Z_{\mathrm{n}}}\left)\right], \end{gathered}$$

where

Eq. (18)

$$f(x)=\{\begin{array}{ll}{x}^{1/3}& x>{(24/116)}^3\\ (841/108)x+16/116& x\le {(24/116)}^3\end{array}.$$

In addition to the white-light image measures, melanin and erythema measures are constructed from the color-OD images. Melanin content^16,45 is calculated as

Eq. (19)

$$\text{Melanin}={\mathrm{OD}}_{660}-{\mathrm{OD}}_{940}.$$

As shown in Table 2, these two wavelengths maximize the difference in melanin absorption and minimize the effect of Hb and ${\mathrm{HbO}}_2$ absorption.

Erythema, due to increased blood content, results in increased blue light absorption but little change in red light absorption⁴⁶ as shown in Table 2. Therefore, an erythema index is constructed as

Eq. (20)

$$\text{Erythema}={\mathrm{OD}}_{470}-{\mathrm{OD}}_{660}.$$

4.

Results

4.1.

Clinical Results

Following are the RGB, chromophore, melanin, and erythema measures for cases of junctional nevus (JN) (Fig. 6) and squamous cell carcinoma (SCC) (Fig. 7); each case was captured with both the USB camera dermascope and the smartphone camera dermascope.

Fig. 6

The same JN imaged by both the smartphone and USB dermascopes. For the smartphone dermascope, (a) the RGB images after illumination uniformity correction, (b) the relative chromophore concentrations, and (d) lightness as measured by $L^{*}$, redness as measured by $a^{*}$, melanin calculated from Eq. (19), and erythema calculated from Eq. (20). The same measures are shown for the USB dermascope in (f), (c), and (e), respectively. A 5-mm scale bar is provided for both the smartphone-camera images and the USB camera images above the RGB image grids.

Fig. 7

The same SCC imaged by both the smartphone and USB dermascopes. For the smartphone dermascope, (a) the RGB images after illumination uniformity correction, (b) the relative chromophore concentrations, and (d) lightness as measured by $L^{*}$, redness as measured by $a^{*}$, melanin calculated from Eq. (19), and erythema calculated from Eq. (20). The same measures are shown for the USB dermascope in (f), (c), and (e), respectively. A 5-mm scale bar is provided for both the smartphone-camera images and the USB camera images above the RGB image grids.

The chromophore maps for both dermascopes at the chosen time points for the occlusion test are shown in Fig. 8.

Fig. 8

Finger occlusion test results for the smartphone camera and USB camera at preocclusion, after occlusion for 2 min, and postocclusion. The bottom plot provides the mean relative concentration for Hb and ${\mathrm{HbO}}_2$ inside the rectangle showing a dip in ${\mathrm{HbO}}_2$ and increase in Hb after occlusion.

4.2.

Optical System Performance

Figure 9 shows the changes in the mean of the sum of the red, green, and blue image channels over varying exposure times for the smartphone-based camera and over brightness settings for the USB camera.

Fig. 9

Mean of the sum of the red, green, and blue channels over changing exposure times for the smartphone-based camera and changing brightness settings for the USB camera.

Figure 10 shows full-field and zoomed 1951 USAF resolution test chart images after cropping along with measured MTF data using the slanted-edge test for both dermascopes.

Fig. 10

(a)–(c) Smartphone camera and (d)–(f) USB camera results of 1951 USAF resolution test chart imaging along with measured MTFs from a slanted-edge test. The smartphone camera’s measured MTF matches the USAF cutoff frequency of group 5 to 6 ($57\mathrm{lp}/\mathrm{mm}$). The USB camera’s measured MTF matches the USAF cutoff frequency of group 3 to 6 ($14.25\mathrm{lp}/\mathrm{mm}$).

Maps of the illumination uniformities of both systems are shown in Fig. 11, and the coefficient of variations are given in Table 3.

Fig. 11

Normalized luminance maps showing illumination uniformity of each device and each illumination wavelength corresponding to $\mathbf{U}$ in Eq. (5)

Table 3

LED illumination uniformity according to Eq. (21).

Wavelength	Smartphone camera	USB camera
White	0.954	0.880
450	0.974	0.980
470	0.982	0.977
500	0.935	0.969
530	0.977	0.955
580	0.980	0.949
660	0.916	0.972
810	0.852	0.900
940	0.907	0.870

5.

Discussion

The distribution of polarized multispectral dermascopes based on smartphone platforms and low-cost color LEDs to PCPs (and eventually to consumers) has the potential to democratize dermal chromophore and melanoma mapping along with erythema monitoring, improving quantitative monitoring of lesions and increasing early detection of skin cancers.

This platform demonstrates a number of advantages compared with previous systems targeting chromophore mapping and skin cancer screening.^{14,15,17,18,20,22–24} The smartphone platform is a compact, low-cost, portable, easy-to-use system with native image capture and processing capabilities, which removes the need for expensive, clinical-grade imaging systems.^17–19 The platform is flexible enough to use either the embedded camera for imaging or a separate USB-connected camera, depending on the desired ergonomics of the user. Both system implementations can still use the built-in smartphone camera for wide-field, white-light, and dermal imaging [the annulus in Fig. 1(c) can be removed]. Additionally, the smartphone camera can be used for large area image capture either using the smartphone-camera-based dermascope with the imaging annulus removed or using the USB camera’s host smartphone.

The use of low-cost, compact, high-power, high-efficacy, surface mount LEDs improves on the costs and complexities associated with laser-based,^22,24 interference-filter-based,^14,15,20 and spectrometer-based^21,23 systems. While these systems likely allow for better discrimination due to their narrow-bandwidth sources or detection schemes, the costs involved (with the possible exception of the laser-based systems) are prohibitive. High-reliability LEDs are available in myriad wavelengths to probe various points along the chromophore molar attenuation curves (Fig. 3) and can be powered with simple driving circuits. Surface-mount packages remove the bulk of transistor outline can packages (or larger packages) necessary for edge-emitting lasers, and the broad wavelength selection is wider than that of surface mount laser packages such as vertical-cavity surface-emitting lasers. The cost of LED sources compared with laser sources or interference filters allows for the use of multiple wavelengths in a single system while keeping bill of materials (BOM) costs low.

6.

Conclusion

Two geometries of smartphone-based dermascopes for dermal lesion screening and erythema monitoring using PMSI and PWLI are described. These devices augment the capabilities of PCPs, with the potential for earlier detection of melanoma and NMSC along with quantitative monitoring of erythema. The combination of LED sources, 3D-printing, and smartphone-based imaging enables the creation of low-cost (a high-volume BOM cost of $<\$40$ excluding the smartphone should be easily achievable), feature-rich, easy-to-use medical imaging devices using either the smartphone camera or a USB camera. While initial results are promising, a longitudinal clinical trial along with histopathology gold-standards will be necessary to validate the diagnostic performance of the devices across multiple lesion types and skin types.

7.

Appendix A: Processing Algorithms Visualized

Figures 12Fig. 13–14 provide flowcharts to help visualize the processing algorithms for the reference, white-light images, and color images provided in Algorithms 1 and 2.

Fig. 12

Visual process flow for the reference images as provided in Algorithm 1

Fig. 13

Visual process flow for the white-light images as provided in Algorithm 2

Fig. 14

Visual process flow for the multispectral images as provided in Algorithm 2.

8.

Appendix B: Finger Occlusion

While the Hb and ${\mathrm{HbO}}_2$ chromophore levels should change during the finger occlusion test as shown in Fig. 8, the melanin and background measures should remain constant. Figure 15 shows the additional measures during the occlusion test. Here, the melanin measure has been divided by 100 and the background measure divided by 10,000 for easier comparison of changes between measures. Here, the USB camera’s melanin and background measurements are more stable over the time points compared with the smartphone camera.

Fig. 15

Finger occlusion test results for the smartphone camera and USB camera at preocclusion, after occlusion for 2 min, and postocclusion for Hb, ${\mathrm{HbO}}_2$, melanin, and background measures resulting from Eq. (14). Here, the melanin measure has been divided by 100 and the background measure divided by 10,000 for easier comparison with the changes in Hb and ${\mathrm{HbO}}_2$.