1.

Introduction

Wide-field microscopy provides images with short acquisition time and a wide field of view. However, its inability to resolve depth information limits it to only surface measurements or thin samples. In order to provide both lateral and axial information, many techniques in microscopy have been explored. In particular, confocal microscopy¹ has gained popularity for its ability to section individual planes by passing light through a pinhole and eliminating light. In particular, confocal reflectance microscopy (CRM) has been used for imaging skin.² However, CRM requires the ability to scan individual points on an object leading to expensive and elaborate point scanning equipment. There has been some success recently by using a dual-wedge scanner³ or a two-dimensional (2-D) microelectromechanical scanner⁴ to efficiently traverse a full 2-D plane. With respect to live imaging, confocal reflectance theta-line scanning has been successful in producing in vivo images without the use of fluorescence.⁵ More recently, a technique called structured illumination has been studied. This technique uses a known pattern, typically at a constant spatial frequency, which is projected onto a sample. Reflected light from areas conjugate to the pattern, which is modulated at that spatial frequency and can be separated from the out-of-focus regions.⁶ This technique requires that the spatial frequency of the pattern is known a priori, such that an exact $1/3$ phase shift can be applied to resolve the entire image. An extension of this idea, dynamic speckle illumination (DSI)^7,8 uses a randomly distributed speckle pattern that is decorrelated from image to image by either translation or randomization. Similar techniques have been proposed leveraging pseudo-random patterns,⁹ but these are confocal techniques that depend on the pattern being in the illumination path and the detection path. DSI has been successful, but the number of images needed to produce sectioning increases with depth, reducing the quality of sectioning when imaging deeper into the specimen. Also, areas where speckles are correlated can result in streaking within the image, resulting in undesired artifacts within the final image. This technique has been applied using a coherent light source in conjunction with fluorescence to provide depth information about various specimens.

We present here a complementary technique that is similar to that of DSI; by using a random intensity pattern translated in a pseudo-random manner, giving the ability to section an image using an light emitting diode (LED) in reflectance only. We call this technique random intensity illumination (RII). Furthermore, to extend the depth capabilities of RII a new technique was developed to leverage multiple diffusers to create an additional spatial spectrum and light intensities to provided three-dimensional sectioning. We named this new technique cascaded random intensity illumination (CRII), which creates variations by cascading multiple diffusers¹⁰ and then imaging them onto the specimen. Similar to Wilson et al.’s application of an incoherent light source to confocal microscopy, the use of an LED over a laser is desirable for the reduced cost and increased line width.¹¹

2.

Methods and Setup

The projection pattern in the illumination path modulates the in-focus plane of the specimen and allows for sectioning using reflectance only and thus providing depth discrimination with endogenous index-of-refraction contrast, similar to CRM. Furthermore, we have used an LED of wavelength 635 nm as an incoherent light source, with a $40\times /0.66$ NA Leica Achro objective, and a Thorlabs 1544M charge coupled device (CCD). Light from the LED is focused onto a piece of ground glass (Fig. 1), which is projected onto the specimen.

Fig. 1

Optical layout.

There are several advantages to this approach; due to the fact that this system is using reflectance only, it provides strong contrast and depth discrimination without the use of potentially toxic reagents required for fluorescence. That fact also makes this a good candidate for imaging human skin in vivo as an alternative to biopsy. The ability to use an LED light source further reduces the cost, complexity, and speckles associated with lasers. Also, since the processing does not require the modulation pattern to be at a discrete frequency, the exact layout of the pattern is completely arbitrary, leading to a lower cost for developing the instrument. Many aspects of the system explored here, reflect similar developments in research of CRM which has proven to be an extremely useful tool in dermatology.

The core of the setup uses a wide-field microscope at a total magnification of $30\times$, with the addition of a modulation pattern; here we’ve used ground glass, between the light source and the objective, in the image plane conjugate to the object. In order to provide sectioning capabilities, there is a need to translate the modulation pattern such that all areas of the specimen are uniformly illuminated. Attached directly to the ground glass is a motor controller which can be used to rotate the orientation of the pattern. Ideally, we would be able to randomize the pattern from image to image to decorrelate each consecutive frame. While rotating the projection pattern is not actually random, it provides a simple method to sampling the specimen while exposed to various light intensities. Care has been taken to ensure that the fundamentals of this method are not dependent on rotation, thus this could be extended to any type of scheme for translating the pattern.

Two types of projection patterns have been explored in an attempt to maximize sectioning efficiency and removing potential artifacts. RII uses a diffuse piece of ground glass to create random intensities in the light that is projected onto the specimen [Fig. 2(a)]. CRII attempts to increase the energy with respect to the low frequency components by cascading multiple diffusers together. The tradeoff here is related to the depth of sectioning and the resolution of sectioning. Using a high spatial frequency projection pattern allows for high resolution sectioning but poor sectioning depth, while low spatial frequency gives good depth, but poor resolution. Because we are imaging using an LED in reflectance, the ability to section deep into the skin is considerably more difficult than using a laser with fluorescence. CRII was developed in an attempt to rectify some of the lost depth due to the high frequencies of RII. By placing a second diffuser in specific locations directly in front of the ground glass, an additional low frequency intensity component has been added [Fig. 2(b)]. The two patterns are perfectly coupled and can be considered as a new single projection pattern with respect to any translation and location within the system. In this experiment, we have formed a line pattern, which is not random, but the processing does not rely directly on this being known a priori and any arbitrary location for the second diffuser could be used.

Fig. 2

Illumination pattern imaged onto a mirror of (a) RII—a single diffuse pattern (ground glass) and (b) CRII—a cascaded diffuse pattern (layered ground glass). The white lines represent areas sampled in the $X$ and $Y$ direction for quantitative analysis used in Tables 1 and 2.

A model describing the system and the projection of the pattern onto the image is described by:

In order to minimize the number of frames needed, the pattern must be perfectly decorrelated from frame to frame. As a first attempt to pseudo-randomly translate the illumination pattern, the diffuser is simply rotated slightly off axis, simultaneously moving all of the parts of the pattern on the specimen; this ensures that no point is in the same location from frame to frame. Using this type of translation, the illumination pattern is still radially correlated which may result in streaking artifacts in the processed image. These artifacts are a pitfall of RII which we attempt to address with CRII later in this paper. $N$ images are taken (on average, $N\approx 40$) and RMS difference is computed using:

3.

Instrument Properties

An optimization was performed and it was found that selecting a value of $t>5$ samples returned much better results than $t=1$ samples. A 5-sample interval corresponds to a rotation of the pattern by about 14-deg. rather then using the 2.8-deg. from the use of consecutive frames. When computing the RMS difference between two frames, a value of $t=5$ results in the pattern being offset by about 50 pixels.

As depth is increased, it is found that contrast is lost in the case of a single diffuse pattern (RII). Thus, it is required to take additional images to compensate for the loss in signal strength. In an attempt to maintain sectioning at larger depths without increasing the number of images, we will now consider CRII, with its additional energy in the low frequency bands.

Figure 3(a) and 3(b) compares the two dimensional Fourier transform of the single and cascaded illumination patterns, computed from Fig. 2(a) and 2(b), respectively. In Fig. 3(a) it is apparent that the distribution is nominally uniform across a wide range of frequencies and in Fig. 3(b) the pattern with multiple diffusers has additional power along the axis of modulation.

Fig. 3

Graphical representation of the 2D-FFT of the (a) single diffuse pattern and (b) the cascaded diffuse pattern.

When the illumination pattern is projected onto a specimen, it is found that the highest frequencies have the strongest contrast at the surface of the object and decay as a function of depth. This loss of contrast at depth is the reason why additional images are needed in order to section into a thick specimen. CRII attempts to overcome this issue by leveraging a lower spatial frequency which does not attenuate as quickly at depth. There is however, the trade-off of using a larger spatial frequency, which manifests itself as a loss in axial resolution. In order to quantify the resolutions of the two techniques, the patterns were projected onto a mirror, 1D slices were taken from each pattern at various depths, and then the slices were compared against each other.

Comparing two images that are in and out of focus axial resolution of the system can be calculated. Figure 4(a) shows the pattern when the mirror is perfectly conjugate to the CCD (0 μm depth), as well as at displacements of 0.5 μm [(Fig. 4(b)] and 1.0 μm [Fig. 4(c)]. It is shown that there is a significant loss in modulated signal strength as the pattern goes out of focus. First considering Fig. 4(a), it can be seen that both patterns contain a strong high frequency component throughout the entire signal. Looking at Fig. 4(b), it is apparent that the pattern is far enough out of focus that the high frequency pattern is completely lost. Here both signals have some low frequency components associated with them, but the contrast is much stronger in the case of the cascaded diffuser. This strong contrast accounts for the efficiency of CRII at depth. Finally, in Fig. 4(c), both patterns have been lost and we are left with only slight contrast changes due to the distribution of light at the detector.

Fig. 4

1D slices of RII and CRII projected onto a mirror of depths (a) at focus, (b) 0.5 μm from the plane of focus, and (c) 1.0 μm from the plane of focus.

To further quantify the signal loss in the system, the power spectral density can be taken at various depths as shown in Fig. 5. The highest energy signal is when the mirror is perfectly conjugate to the CCD, i.e., 0 μm where there is energy across the entire spectrum. As depth is increased to 0.5 μm, there is a loss in the high frequency information (about 0.2 cycles/pixel) of nearly 10 dB, where the lower frequencies decrease with a 5 dB loss (about 0.1 cycles/pixel). At 1.0 μm, the entire spectrum decays by about 15 dB and the signal is lost.

Fig. 5

PSD of the CRII pattern projected against a mirror at depths of 0 μm, 0.5 μm, and 1.0 μm.

This implies that the sectioning process reduces signals outside of 1.0 μm of focus by about 15 dB or that the axial resolution of this system is about 1.0 μm for any plane in focus. The sectioning ability of this system is comparable to confocal systems attempting to achieve in vivo images using reflectance only.⁵

In order to verify an increase in the contrast using this cascaded technique, the standard deviation of $I_d$ has been characterized. This was done by projecting each illumination pattern onto first, a mirrored target and second, onto a thick sample, and for each integrating about 40 frames while the pattern was rotated. The assumption is that, because the pattern from the CRII will produce higher contrast than RII, the standard deviation of $I_d$ will be greater. First, a single pixel is selected from Fig. 2 at the center of each of the images (RII and CRII, respectively). While the illumination pattern is translated, the standard deviation is taken of that pixel over the 40 frames. Next, from a single image we select 500 pixels in either the horizontal ($500\times 1$) or 500 pixels in the vertical directions ($1\times 500$), indicated by the white lines in Fig. 2. Then the average standard deviation is taken over all 500 pixels to produce the results in Tables 1 and 2.

Table 1

STD of pixel values against a mirror.

Table 2

STD of pixel values against a leaf at a depth of 6 μm.

These measurements were integrated over 40 frames using the same pixels locations, gain settings, and background image to ensure consistency across each method. The ground glass was divided into two parts such that RII and CRII measurements could be taken consecutively.

These results show that the CRII technique is in fact producing a higher contrast both at the surface and at depth compared to a RII as expected. Because there is a 2 to $3\times$ greater contrast at depth, the system is able to section the image with out requiring a large amount of additional images.

It is also found that the cascaded diffuser technique has the added benefit of reducing the amount of streaking that is created in the image. Since the spots are rotated on a fixed axis, areas where the pattern has a slightly greater correlation create local maxima in $I_r$ and areas with very little correlation create a local minima in $I_r$. The result of this effect is a pattern of parallel lines, or streaks, as seen in Fig. 6(a). The addition of a new low frequency pattern, “fills in” any local minimas found in the high frequency pattern, resulting in the smoothing of many of these artifacts [Fig. 6(b)]. Figure 6 shows a comparison between the two techniques and the change in streaking. These processed images were the results of projecting the illumination pattern directly onto a mirror and then collecting 40 images each at the same intensity and exposure.

Fig. 6

Mirror target showing (a) streaking from single frequency illumination pattern (RII) compared against the (b) reduction in streaking from the cascaded illumination pattern (CRII).

4.

Results

The use of an LED also provides images that do not contain the speckle associated with the use of a laser. Using an Ocean Optics USB2000 spectrometer, we measured the spectrum of the LED. It peaked at 635 nm with a full width at half max of about 17 nm. The use of an incoherent light source with a broader line width kept the images speckle free. The reduction in artifacts such as laser speckling greatly smooths the image and returns a higher quality image. Figure 7 compares (a) CRM to (b) CRII against a leaf at a depth of 0.6 μm, where Fig. 7(b) clearly shows the reduced speckle.

Fig. 7

Image of a leaf at depth of 6 um using (a) CRM and (b) CRII over 40 images.

Furthermore, to demonstrate sectioning, the processed images can be compared to the original wide-field images. By taking the mean of all the CRII raw images with different realizations of the diffuse pattern, a synthetic wide-field image can be computed [Fig. 8(a)].⁷ Tissue paper was used as a target at a depth of about 10 μm. Because the fibers of paper are layered at various depths, the rejection of out-of-focus light in both the foreground and the background of the image can clearly be seen [Fig. 8(b)].

Fig. 8

Image of a tissue paper at 10 μm (a) wide-field and (b) CRII.

Finally, the system was configured for in vivo imaging and the forearm of a human was imaged at two different depths. Figure 9 shows a synthetic wide-field image at the surface of the skin, reconstructed from the the images used in CRII. The image processed with CRII is shown in Fig. 10. It is clear that many areas have been rejected due to being out of focus, thus the contrast of the object at focus is greatly increased.

Fig. 9

Wide-field in vivo image at the surface.

Fig. 10

CRII in vivo image showing the stratum corneum.

Figure 11 shows the synthetic wide-field image constructed where minimal structure is resolvable. After processing the images with CRII, a considerable amount of detail is restored as seen in Fig. 12. This detail strongly resembles the stratum granulosum just below the surface of the skin.² This provides a good basis for the system’s ability to resolve subsurface detail in a living organism without the need for fluorescence.

Fig. 11

Wide-field in vivo image at depth.

Fig. 12

CRII in vivo image showing the stratum granulosum.

5.

Summary

CRII has the advantage of providing resolution in depth and also greatly enhances the contrast of the image by removing the “clutter” of out-of-focus light that otherwise degrades contrast. There are a number of limitations that will still need to be resolved for future experiments. Our current camera only has an 8-bit depth and a maximum frame rate of 25 Hz, which made live-imaging challenging. More appropriate equipment and better processing techniques could lead to a real-time system and greater depth resolution. Extending the structured illumination method to an incoherent light source and reflectance has also been a challenge. We’ve provided a good benchmark for axial resolution and sectioning depth, but hope that further research can optimize the process. Overall, early work with CRII has shown great promise and would provide a safe, compact system that, in some applications, would be competitive CRM and could improve research in the biomedical industries.