Dither removing Fourier ptychographic microscope based on a two-axis rotation stage

Kaicheng Huang; Wangwei Hui; Qing Ye; Hongyang Zhao; Qiushuai Shi; Jianguo Tian; Wenyuan Zhou

doi:10.1117/1.JBO.26.3.036501

4 March 2021 Dither removing Fourier ptychographic microscope based on a two-axis rotation stage

Kaicheng Huang, Wangwei Hui, Qing Ye, Hongyang Zhao, Qiushuai Shi, Jianguo Tian, Wenyuan Zhou

Author Affiliations +

Journal of Biomedical Optics, Vol. 26, Issue 3, 036501 (March 2021). https://doi.org/10.1117/1.JBO.26.3.036501

Abstract

Significance: Large space-bandwidth product is highly desirable in many biomedical imaging. Fourier ptychographic microscopy (FPM) is a computational imaging technique that can significantly increase the space-bandwidth product of a standard microscope. The illuminator of a Fourier ptychographic microscope is not flexible at present, and it is inconvenient to meet different imaging needs.

Aim: An illuminator based on a two-axis motorized rotation stage was presented to provide a more flexible illuminating way with the goal of meeting different imaging needs.

Approach: The illuminator adopts a concentric illuminating method to provide coherent illumination in any direction on the sample plane. The sampling pattern can be freely designed and changed according to the parameters of the imaging system. A dither removing algorithm was proposed to remove the potential dither influence introduced in the image acquisition process.

Results: The illuminator could be conveniently integrated into different imaging systems. The feasibility and flexibility were demonstrated by applying it to imaging systems with numerical aperture of 0.045 and 0.01. The resolution gain is about 4- and 13-fold, respectively. The effectiveness of the dither removing algorithm was validated in both simulation and experiment.

Conclusions: A more flexible illuminator for FPM was presented to meet different imaging needs. A dither removing algorithm was proposed to remove dither influence.

1. Introduction

Fourier ptychographic microscopy (FPM) is a recent computational imaging technique that is capable of increasing the resolution of a standard microscope without sacrificing the field of view (FOV).¹^–⁴ By applying a phase retrieval algorithm, FPM stitches a series of low-resolution images to produce a high-resolution image.⁵^,⁶ In addition, the phase of the reconstructed high-resolution image can be recovered without knowing the phases of the low-resolution images.⁴ FPM has made great progress in many aspects, such as apparatus structure,⁷^–¹¹ experimental method,¹²^–²¹ parameter correction,³^,²²^–²⁴ and reconstruction algorithm.²⁵^–²⁸

So far, most FPM platforms are using a light-emitting diode (LED) array as an illuminator. An LED array can provide stable and fast illumination without mechanical movement. However, an LED array has its own drawbacks as an illuminator in FPM. First, the intensity of an LED is relatively weak, especially for large-angle illumination. Second, the distribution density of LEDs is fixed, which is inflexible for adjusting the sampling density in spectrum. Third, the largest illumination angle is limited, which is inflexible for synthesizing a very large NA. Fourth, the positions of all LEDs are fixed; therefore, the sampling patterns with an LED array are limited. These drawbacks of an LED array lead to the emergence of some different illuminators in FPM. Chung et al.⁹ have developed an illuminator using laser source guided by a galvo-scanner and a mirror array and achieved fast acquisition, but it is inflexible for the mirror array to change the sampling pattern. Guo et al.²⁹ have developed a flexible illuminator using liquid crystal display, the illuminator is convenient to change the sampling pattern but is inconvenient for large incident angles since the maximum incident angle is limited by the condenser. There are other different illuminators, such as quasi-dome LED array,³⁰ hemispherical condenser,¹⁰^,³¹ and illuminator with digital micromirror device.⁸^,³² These illuminators are still inflexible in some aspects; therefore, they could not be conveniently utilized to investigate the FPM technique under different conditions.

To provide a more convenient way to investigate FPM technique in depth, we present a more flexible illuminator for FPM. The illuminator uses a laser as its illumination source mounted on a two-axis motorized rotation stage. We adopt a concentric illuminating method, that is, the laser always illuminates the intersection of the two axes. The illuminator is capable of providing coherent illumination in any direction with an angular resolution of 0.0003125 deg and a maximum incident angle of 90 deg. Therefore, it is easy to design and change the sampling pattern in the FPM with a very high degree of freedom. The illuminator can be easily integrated into different FPM platforms with different parameters. It is quite convenient to use this illuminator to investigate various deep-seated problems related to FPM.

To reconstruct high-resolution image, FPM needs to capture a series of low-resolution images. Keeping the acquisition process stable is essential for reconstructing high-quality image. Our illuminator is driven in mechanical way, which inevitably introduces little dither during the FPM acquisition process. One feasible approach is to wait until the platform is stable before each image acquisition. However, this approach would greatly increase the acquisition time. To obtain the next image quickly without being disturbed by dither, we propose a dither removing algorithm for FPM. The dither removing algorithm is able to reconstruct a high-resolution image successfully from a series of low-resolution images polluted by dither. With the dither removing algorithm, the stability requirement can be relaxed and the acquisition time can be shortened.

The remainder of this paper is structured as follows. In Sec. 2, we describe the schematic of the setup that we used in the experiments and the principle of the dither removing algorithm. In Sec. 3, we first verify the effectiveness of our proposed dither removing algorithm by simulation. Then we perform three experiments: The first experiment demonstrates the feasibility of our illuminator and the effectiveness of our proposed dither removing algorithm. The second experiment shows the flexibility and wide applicability of the illuminator. The last experiment shows that the illuminator is applicable for real biological specimens. In Sec. 4, we present the conclusions of our works.

2. Method

2.1.

Experimental Setup

We integrate our illuminator into an FPM platform, which is described as follows. The illumination source is a laser whose wavelength is 635 nm. The laser is mounted on a two-axis motorized rotation stage. We adopt a concentric illuminating method, that is, the laser always illuminates the intersection of the two axes. We denote the angle associated with the vertical axis as $θ$ and the angle associated with the horizontal axis as $φ$ . $θ$ and $φ$ can cover the whole spatial angle with an accuracy of 0.0003125 deg. We adjust the laser beam to coincide with the vertical axis at the beginning. Benefitting from the flexibility of the rotation stage, the sampling pattern can be freely designed and changed according to the system parameters, making the illuminator widely applicable. The specimen is placed at the intersection of the two axes. The imaging lens is a $1 \times$ telecentric lens (WWH10-110CT-G, Coolens), whose numerical aperture (NA) is adjustable in the range of 0.005 to 0.045. A complementary metal-oxide-semiconductor camera is used with a pixel size of $3.45 μ m$ (GS3-U3-123S6M-C, FLIR). The structure of the imaging platform is shown in Fig. 1.

Fig. 1

Schematic of the imaging platform.

2.2.

Dither Removing Algorithm

In a general microscope imaging platform, the pixel size on the object plane can be calculated by $d / m a g$ , which is about a few hundred nanometers to a few microns, where $d$ is the pixel size of the camera and $m a g$ is the magnification of the system. It is inevitable to introduce little dither especially when starting and stopping mechanical movement. In FPM, the presence of dither will degrade the quality of the reconstructed image. It becomes especially essential to find the unknown individual shifted value of each low-resolution image correctly. Bian et al.²³ have tackled the sample motion problem using an annealing method. However, the dither in our application is relatively small, so here we use a more lightweight method to solve the dither problem. The main idea of our algorithm is straightforward. For each illumination direction, after updating the spectrum in conventional FPM recovery procedure,¹ we use this spectrum to correct the dither value of next image. We simulate the imaging process of the corresponding illumination direction. We assume that the captured image has the identical distribution as the simulated image except for a lateral shift. We apply the phase correlation algorithm to find the shifted value of the captured image³³^,³⁴ and use the found shifted value to shift the captured image back. Then the square root of the dither removed image is used to substitute the amplitude of the simulated image. The algorithm is described in detail below.

As suggested by the FPM recovery algorithm,¹ we initialize an estimation of the high-resolution spatial spectrum $\tilde{O} (f)$ first, where $f$ represents the spatial frequency coordinate. Since the estimated spectrum could be different from the true spectrum, if necessary, we could perform a few iterations of the conventional FPM recovery procedure (experiments show that two iterations are sufficient). Then we calculate the simulated low-resolution image $I_{i}^{s} (r)$ under illumination of the $i$ ’th direction,²²

Eq. (1)

I_{i}^{s} (r) = {| F^{- 1} {\tilde{O} (f - f_{i}) \cdot \tilde{P} (f)} |}^{2},

where

r

represents the spatial coordinate,

F^{- 1}

represents the inverse Fourier transform,

f_{i}

is the illumination wavenumber of the

i

’th image, and

\tilde{P} (f)

represents the pupil function. Let

I_{i} (r)

denote the recorded image under illumination of the

i

’th direction. Under our assumption, we have

Eq. (2)

I_{i}^{s} (r) = α_{i} I_{i} (r + r_{i}),

where

α_{i}

is a constant factor and

r_{i}

is the unknown

i

’th shifted value introduced by dither. If we can find the exact value of

r_{i}

, then we can remove the influence of dither by simply replacing

I_{i} (r)

with

I_{i} (r + r_{i})

. When we calculate the Fourier transform on both sides of Eq. (2), we have

Eq. (3)

{\tilde{I}}_{i}^{s} (f) = α_{i} {\tilde{I}}_{i} (f) e^{j 2 π f \cdot r_{i}},

where

{\tilde{I}}_{i}^{s} (f)

represents the Fourier transform of

I_{i}^{s} (r)

and

{\tilde{I}}_{i} (f)

represents the Fourier transform of

I_{i} (r)

. We multiply both sides of Eq. (3) by

{\tilde{I}}_{i}^{*} (f) / {| {\tilde{I}}_{i} (f) |}^{2}

and calculate the inverse Fourier transform, and we mark it as

D_{i} (r)

,

Eq. (4)

D_{i} (r) = F^{- 1} {\frac{{\tilde{I}}_{i}^{s} (f) {\tilde{I}}_{i}^{*} (f)}{{| {\tilde{I}}_{i} (f) |}^{2}}} = α_{i} F^{- 1} {e^{j 2 π f \cdot r_{i}}} = α_{i} δ (r + r_{i}),

where

δ (r)

is the Dirac delta function. We determine

r_{i}

by

Eq. (5)

r_{i} = - \underset{r}{\arg \max} | D_{i} (r) | .

In practice, Eq. (2) usually does not hold, nor is $D_{i} (r)$ a shifted Dirac function. However, as long as $I_{i}^{s} (r)$ is similar with $I_{i} (r)$ , $D_{i} (r)$ is a unimodal function, and Eq. (5) still works. We will not prove it analytically, as it will be demonstrated in simulation and experiments later. Then $I_{i} (r)$ is replaced with $I_{i} (r + r_{i})$ to participate in the FPM recovery procedure. The flowchart of the dither removing algorithm is summarized in Fig. 2.

Fig. 2

Flowchart of the dither removing algorithm.

2.3.

Dither Calibration

The dither removing algorithm we proposed does not consider the motion blur problem. To theoretically analyze the feasibility of the dither removing algorithm without considering the motion blur, we calibrated the dither of our experimental setup. Then we gave a sufficient condition that does not need to consider the motion blur.

The method that we calibrate the dither is described as follows. First, we start to capture images of the sample in a subregion of $352 \times 360 pixels$ in static state at a framerate of 200 fps. Immediately we rotate the stage by an angle. We keep capturing images for about 30 s. When the image acquisition process is finished, we use the first image as the standard and calculate the shifted values for the remaining images using phase correlation method.³³^,³⁴ We calibrated the dither for three types of movements: (1) $(θ, φ)$ from (1 deg, 0) to (0, 0); (2) $(θ, φ)$ from (0, 1 deg) to (0, 0); and (3). $(θ, φ)$ from (90 deg, 1 deg) to (90 deg, 0). The angular speed of the rotation stage is set to be $8 \deg / s$ . The calibration results are plotted in Fig. 3.

Fig. 3

Dither calibration results. (a1) The dither of the first type of movements in $x$ direction over time, (a2) the frequency spectrum of (a1), (a3) and (a4) have similar meanings to (a1) and (a2) but in $y$ direction. (b1)–(b4) and (c1)–(c4) are similar to (a1)–(a4) but belong to the second and third types of movements.

Figure 3(a1) shows the dither of the first type of movements in $x$ direction over time. Figure 3(a2) shows the frequency spectrum of Fig. 3(a1). Figures 3(a3) and 3(a4) have similar meanings to Figs. 3(a1) and 3(a2) but in $y$ direction. Figures 3(b1)–3(b4) and 3(c1)–3(c4) are similar to Figs. 3(a1)–3(a4) but belong to the second and third types of movements. Comparing the results of the three types of movements, we can conclude that the dither is more severe when rotating the stage along the vertical axis than along the horizontal axis. Comparing the results in $x$ and $y$ directions, we can conclude that the dither is more severe in $y$ direction than in $x$ direction. This conclusion can be also obtained by comparing the results of the last two types of movements since the dither is a little more severe when $θ$ is 90 deg than 0. Results in Fig. 3 show that the dominant frequency when rotating the stage along the vertical axis is about 20 Hz and the dominant frequency when rotating the stage along the horizontal axis is about 10 Hz.

Fig. 4

Simulation results. (a1) The true image intensity of the simulated object, (a2) the raw image under normal illumination, and (a3) the reconstruction intensity of the traditional FPM algorithm applied to the dither-free dataset. (b1)–(b3) The intensities recovered by the traditional FPM algorithm whose maximum shifted pixels are 1, 3, and 5. (c1)–(c3) Similar with (b1)–(b3) except that they are reconstructed by the dither removing algorithm. The true and recovered shifted values are plotted in (d). (e1) and (e2) The reconstructed intensities without and with the dither removing algorithm of non-integer case, respectively.

It is reasonable to use simple harmonic vibration model to represent the dither in a short time duration ( $T$ to $T + Δ T$ ). We represent the dither $D$ as

Eq. (6)

D (t) = N_{T} \sin (2 π f t), T \leq t \leq T + Δ T,

where

N_{T}

is the amplitude in pixel around time

T

and

f

is the dominant frequency. The maximum velocity can be expressed as

2 π f N_{T}

. When we capture an image, we think the motion blur can be neglected if the image travels less than 1 pixel. If the exposure time is

Δ t

, we give a sufficient condition for not considering the motion blur by Eq. (7),

Eq. (7)

2 π f N_{T} Δ t < 1 .

For rotating the stage along the horizontal axis, most $N_{T}$ is no more than 1 pixel, $f$ is about 10 Hz, we can deduce that if $Δ t \leq 16 ms$ , we can neglect the motion blur. For rotating the stage along the vertical axis, the case will be worse. $f$ is about 20 Hz. If $N_{T}$ is 20, $Δ t$ should be less than 0.4 ms, and if $N_{T}$ is 5, $Δ t$ should be less than 1.6 ms.

In practice, we rotate the stage along the vertical axis to a specified $θ$ , and then rotate the stage along the horizontal axis to all $φ s$ we needed. So the number of rotations along vertical axis is much less than that along horizontal axis. When changing $θ$ , we wait for 3 s before capturing images to reduce the $N_{T}$ to about 5. The exposure time of bright-field images is set to be 0.6 ms. We increase the exposure time as $φ$ goes up until the exposure time reaches 19.2 ms. Then we increase the gain instead if $φ$ continues to go up. (But we cannot increase the gain too much since it will bring in noise.) In this way, we can ensure that most of the images are free of motion blur, and images that could be possibly polluted with motion blur correspond to the largest angles. This strategy works well in our experiments.

3. Results

3.1.

Simulation

We first demonstrate the effectiveness of the dither removing algorithm in simulation. We generate a dither-polluted simulation dataset by artificially adding a random shifted value to each simulated image. Concretely, in the process of simulating image acquisition, we artificially add the influence of dither for the $i$ ’th image by shifting the image with $(Δ x_{i}, Δ y_{i})$ pixels, where $Δ x_{i}$ and $Δ y_{i}$ are independent discrete random variables and have the identical probability distribution:

Eq. (8)

P (Δ x_{i} = Δ) = P (Δ y_{i} = Δ) = {\begin{cases} \frac{1}{2 p + 1}, & if Δ = - p, - p + 1, \dots, p - 1, p \\ 0, & otherwise \end{cases},

where

p > 0

is a preset positive integer that indicates the maximum shifted pixels due to dither. We simulate the situation where

p

is 1, 3, and 5. We compare the reconstruction results of the dither removing algorithm and the traditional FPM algorithm. The reconstruction results of simulation are summarized in Fig. 4.

Figure 4(a1) shows the true image intensity of the simulated object, Fig. 4(a2) shows the raw image under normal illumination, and Fig. 4(a3) shows the reconstruction intensity of the traditional FPM algorithm applied to the dither-free dataset. Group (b) and (c) in Fig. 4 are the results reconstructed from the dataset polluted by dither. Figures 4(b1)–4(b3) show the intensities recovered by the traditional FPM algorithm whose maximum shifted pixels are 1, 3, and 5 while Figs. 4(c1)–4(c3) show the intensities recovered by the dither removing algorithm whose maximum shifted pixels are 1, 3, and 5. Results of group (b) in Fig. 4 show that dither can severely degrade the reconstruction quality. Results of group (c) in Fig. 4 show that the dither removing algorithm can effectively remove the influence of dither and successfully recover the image.

It is worth noting that the reconstructed image is allowed to have an uncertain global shifted value. Suppose $H (r)$ is the reconstructed image through the dither removing algorithm and ${r_{i} | i = 1,2, \dots, n}$ is the shifted value set of the image set, $n$ is the total number of raw images. It is easy to understand that $H (r + r_{0})$ could also be a reconstructed image with the shifted value set ${r_{i} + r_{0} | i = 1,2, \dots, n}$ , where the Euclidean norm of $r_{0}$ is much smaller than the range of the image. In practice, when the high-resolution image $H (r)$ is reconstructed, we shift the image by $\bar{r}$ , i.e., $H (r - \bar{r})$ , where $\bar{r} = \sum_{i = 1}^{n} r_{i} / n$ . And the corresponding shifted value set would be ${r_{i} - \bar{r} | i = 1, 2, \dots, n}$ , which is zero mean. So $H (r - \bar{r})$ would not shift too far from the true image since the dither value is also zero mean.

We also examine the recovered shifted values with the case $p$ is 5. The result is plotted in Fig. 4(d). The blue circles represent the true shifted values while the red asterisks represent the recovered shifted values. The black crosses represent the recovered errors. The image with index 1 is corresponding to the spectrum center. The position in spectrum goes outward in a spiral way with an increasing image index. Figure 4(d) shows that in this case, all images retrieved the correct shifted values. Figure 4(d) demonstrates that the recovered shifted value by dither removing algorithm is quite reliable.

The shifted values artificially added to the image sequence by Eq. (8) are all integers. In practice, the shifted values cannot be exactly integers. To examine the recovery quality in non-integer cases, we simulate the non-integer dither by interpolation. First, we upsample all the captured images by 10 times. Then we add the influence of dither as before, where $p$ is 50. Finally, we downsample all images to their original size. In thus constructed images, the precision of shifted values is 0.1 pixel and the maximum shifted value is 5. Figures 4(e1) and 4(e2) show the reconstructed intensities without and with the dither removing algorithm. Results demonstrate that the dither removing algorithm is also effective in non-integer cases.

3.2.

Experiments

To verify the feasibility of the illuminator and the effectiveness of the dither removing algorithm in realistic imaging, we conduct an experiment on the imaging platform using the dither removing algorithm. In this experiment, we adjust the NA of the lens to the maximum value of 0.045. We use a non-uniform sampling pattern for better sampling efficiency and image quality.¹³ We set $Δ φ = Δ φ_{1}$ when $φ \leq φ_{threshold}$ and $Δ φ = Δ φ_{2}$ when $φ_{threshold} \leq φ \leq φ_{\max}$ (the value of $Δ φ$ should meet the overlapping condition).³⁵ We adopt a simple strategy to design the sampling pattern: $φ_{0} = 0$ , for $φ_{n} = φ_{n - 1} + Δ φ \leq φ_{\max}$ , we set $θ$ to be evenly distributed on the circumference. When $n$ is 0, we only sample one angle where $θ$ is 0. When $n$ is 1, we sample four angles where $θ$ is 0 deg, 90 deg, 180 deg, and 270 deg, respectively. Otherwise, we temporarily set the number of $θ$ to be the same as in the case of $n - 1$ . If the adjacent subareas of the spectrum do not overlap with each other, we double the number of $θ$ . Here, we set $Δ φ_{1} = 1 \deg$ , $Δ φ_{2} = 2 \deg$ , $φ_{threshold} = 4 \deg$ , and $φ_{\max} = 8 \deg$ . The total number of sampling angles is 45. The maximum exposure time is 5.4 ms with gain of 0.

The sample in this experiment is a USAF resolution target. We compare the reconstruction results of the dither removing algorithm and the traditional FPM algorithm. The experimental results are summarized in Fig. 5.

Fig. 5

Experimental comparison results. (a) The raw image; (b1) and (c1) the intensities reconstructed by FPM without and with dither removing, respectively; (b2) and (c2) zoomed-in subarea of (b1) and (c1), respectively. (b3) and (c3) The spectra reconstructed by FPM without and with dither removing, respectively. (d) The designed sampling pattern.

Figures 5(a) shows the raw image. Figures 5(b1) and 5(c1) show the intensities reconstructed by FPM without and with dither removing, respectively. Figures 5(b2) and 5(c2) show a zoomed-in subarea of Figs. 5(b1) and 5(c1), respectively. Figures 5(b3) and 5(c3) show the spectra of the complex amplitudes of Figs. 5(b1) and 5(c1), respectively. Figures 5(b1) and 5(b2) show that the quality degration of the reconstructed image is more severe in $y$ direction than in $x$ direction. It is consistant with the calibration results in Sec. 2.3. Figures 5(c1) and 5(c2) indicate that the quality of reconstructed image can be improved by dither removing algorithm. The sampling pattern in this experiment is portrayed in Fig. 5(d). The system NA is 0.045 (the red circle) and the theoretical synthetic NA is 0.1842 (the big dashed black circle), and the effective resolution gain is about 4-fold. This experiment verifies the feasibility of our illuminator and the effectiveness of the dither removing algorithm. The application of non-uniform sampling pattern also shows the incident angle selection flexibility of our illuminator.

Then we apply the imaging platform to an experiment with very low system NA to show the flexibility and wide applicability of the illuminator. In this experiment, we adopt the same strategy with the first experiment to design the sampling pattern. We set NA to be 0.01, $Δ φ_{1} = 0.3 \deg$ , $Δ φ_{2} = 0.6 \deg$ , $φ_{threshold} = 3 \deg$ , and $φ_{\max} = 7.2 \deg$ . The total number of sampling angles is 457. The maximum exposure time is 19.2 ms with gain of 12.11. The experimental result is shown in Fig. 6.

Fig. 6

Experimental results with very low NA. (a) The raw image, (b) the reconstructed image of (a), (c) the zoomed-in subarea of the red dashed square in (b), (d) the reconstructed spectrum, and (e) the designed sampling pattern.

Figure 6(a) shows the raw image, Fig. 6(b) shows the reconstructed intensity, and Fig. 6(c) shows the zoomed-in subarea of the red dashed square in Fig. 6(b). Figure 6(d) shows the reconstructed spectrum. The sampling pattern is portrayed in Fig. 6(e). The system NA is 0.01 (the red circle) and the theoretical synthetic NA is 0.1353 (the big dashed black circle), and the effective resolution gain is about 13-fold. Figure 6 shows that our imaging platform is also applicable to a very low-NA objective lens. It demonstrated that our illuminator is quite flexible and widely applicable.

To demonstrate that the illuminator is also feasible for a real biological specimen, we apply the imaging platform to well-differentiated squamous cell carcinoma. We use the same parameters and sampling pattern as the first experiment. Figure 7 shows the experimental results. Figure 7(a) shows the FOV image. Figures 7(b1)–7(d1) show the zoomed-in subareas of different squares in Fig. 7(a). Figures 7(b2)–7(d2) and 7(b3)–7(d3) show the reconstructed intensities and phases of (b1)-(d1), respectively. Results demonstrated that our platform is also applicable for a real biological specimen.

Fig. 7

Experimental results of well-differentiated squamous cell carcinoma. (a) The FOV raw image, (b1)–(d1) three zoomed-in subareas of the squares in (a), (b2)–(d2) the reconstructed intensities, and (b3)–(d3) the reconstructed phases.

4. Conclusion and Discussion

In conclusion, we have presented a quite flexible illuminator for FPM based on a two-axis motorized rotation stage. It is capable of providing coherent illumination in any direction; therefore, the sampling pattern in FPM can be freely designed and changed. This illuminator provides a quite convenient way for investigating FPM in a deeper level and a wider range. To prevent tiny dither introduced by mechanical movements from degrading the reconstruction quality, we proposed a dither removing algorithm for FPM. The dither removing algorithm is able to retrieve a high-resolution image successfully from a sequence of low-resolution images polluted by dither. Simulation and experiment have verified the feasibility of our illuminator as well as the effectiveness of our dither removing algorithm. We demonstrated the high flexibility of our illuminator by applying it in FPM experiments with a 0.045 NA lens as well as a 0.01 NA lens and achieved a resolution gain of 4- and 13-fold, respectively.

A large FOV is very appealing in pathological section. However, a large FOV usually means a low NA, which requires illumination with a much denser angular distribution. Our illuminator can be integrated into a low-NA FPM platform conveniently. Our illuminator is quite suitable for investigating extremely high-resolution gain FPM, such as from 0.04 to 0.8 or more.

We use a laser as the illumination source, which is sensitive to any optical imperfection. It will bring fluctuating background signals and speckles in the captured images, which could be seen in Fig. 7(a). These fluctuations could then contribute negatively to the reconstructed phase. For a thin sample (with small phase), one can use the differential phase contrast deconvolution in the FP algorithm to further improve the quality of reconstructed phase.⁹ We will use a more appropriate illumination source to further improve the reconstructed complex image in the future.

In many other cases, the motion blur problem cannot be neglected. We will investigate the motion blur problem and improve the dither removing algorithm in the future.

Due to the mechanical scanning, the slow acquisition speed is a disadvantage of our illuminator. So it is not suitable for the cases that have high requirement for time.

We believe that our illuminator and the dither removing algorithm will help the application and theory investigation for FPM.

Disclosures

The authors have no relevant financial interests in this paper and no potential conflicts of interest.

Acknowledgments

This research was supported by the National Natural Science Foundation of China (No. 31527801).

References

1.

G. Zheng, R. Horstmeyer and C. Yang, “Wide-field, high-resolution Fourier ptychographic microscopy,” Nat. Photonics, 7 (9), 739 –745 (2013). https://doi.org/10.1038/nphoton.2013.187 NPAHBY 1749-4885 Google Scholar

2.

X. Ou et al., “Quantitative phase imaging via Fourier ptychographic microscopy,” Opt. Lett., 38 (22), 4845 –4848 (2013). https://doi.org/10.1364/OL.38.004845 OPLEDP 0146-9592 Google Scholar

3.

Z. Bian, S. Dong and G. Zheng, “Adaptive system correction for robust Fourier ptychographic imaging,” Opt. Express, 21 (26), 32400 –32410 (2013). https://doi.org/10.1364/OE.21.032400 OPEXFF 1094-4087 Google Scholar

4.

R. Horstmeyer and C. Yang, “A phase space model of Fourier ptychographic microscopy,” Opt. Express, 22 (1), 338 –358 (2014). https://doi.org/10.1364/OE.22.000338 OPEXFF 1094-4087 Google Scholar

5.

R. W. Gerchberg and W. O. Saxton, “A practical algorithm for the determination of phase from image and diffraction plane pictures,” Optik, 35 227 –246 (1972). OTIKAJ 0030-4026 Google Scholar

6.

J. R. Fienup, “Phase retrieval algorithms: a comparison,” Appl. Opt., 21 (15), 2758 –2769 (1982). https://doi.org/10.1364/AO.21.002758 Google Scholar

7.

S. Dong et al., “Aperture-scanning Fourier ptychography for 3D refocusing and super-resolution macroscopic imaging,” Opt. Express, 22 (11), 13586 –13599 (2014). https://doi.org/10.1364/OE.22.013586 OPEXFF 1094-4087 Google Scholar

8.

C. Kuang et al., “Digital micromirror device-based laser-illumination Fourier ptychographic microscopy,” Opt. Express, 23 (21), 26999 –27010 (2015). https://doi.org/10.1364/OE.23.026999 OPEXFF 1094-4087 Google Scholar

9.

J. Chung et al., “Wide-field Fourier ptychographic microscopy using laser illumination source,” Biomed. Opt. Express, 7 (11), 4787 –4802 (2016). https://doi.org/10.1364/BOE.7.004787 Google Scholar

10.

M. Alotaibi et al., “Illumination-direction multiplexing Fourier ptychographic microscopy using hemispherical digital condensers,” Appl. Opt., 56 (14), 4052 –4057 (2017). https://doi.org/10.1364/AO.56.004052 APOPAI 0003-6935 Google Scholar

11.

H. Lee, B. H. Chon and H. K. Ahn, “Reflective Fourier ptychographic microscopy using a parabolic mirror,” Opt. Express, 27 (23), 34382 –34391 (2019). https://doi.org/10.1364/OE.27.034382 OPEXFF 1094-4087 Google Scholar

12.

L. Tian et al., “Multiplexed coded illumination for Fourier ptychography with an LED array microscope,” Biomed. Opt. Express, 5 (7), 2376 –2389 (2014). https://doi.org/10.1364/BOE.5.002376 BOEICL 2156-7085 Google Scholar

13.

K. Guo et al., “Optimization of sampling pattern and the design of Fourier ptychographic illuminator,” Opt. Express, 23 (5), 6171 –6180 (2015). https://doi.org/10.1364/OE.23.006171 OPEXFF 1094-4087 Google Scholar

14.

Y. Zhang et al., “Self-learning based Fourier ptychographic microscopy,” Opt. Express, 23 (14), 18471 –18486 (2015). https://doi.org/10.1364/OE.23.018471 OPEXFF 1094-4087 Google Scholar

15.

R. Cao et al., “Pattern-illuminated Fourier ptychography microscopy with a pattern-estimation algorithm,” Appl. Opt., 56 (24), 6930 –6935 (2017). https://doi.org/10.1364/AO.56.006930 APOPAI 0003-6935 Google Scholar

16.

Y. F. Cheng et al., “Illumination pattern design with deep learning for single-shot Fourier ptychographic microscopy,” Opt. Express, 27 (2), 644 –656 (2019). https://doi.org/10.1364/OE.27.000644 OPEXFF 1094-4087 Google Scholar

17.

K. Huang et al., “Compressed-sampling-based Fourier ptychographic microscopy,” Opt. Commun., 452 18 –24 (2019). https://doi.org/10.1016/j.optcom.2019.07.009 OPCOB8 0030-4018 Google Scholar

18.

S. Li et al., “Predictive searching algorithm for Fourier ptychography,” J. Opt., 19 125605 (2017). https://doi.org/10.1088/2040-8986/aa95d5 Google Scholar

19.

Y. Li et al., “Adaptive and efficient Fourier ptychographic microscopy based on information entropy,” J. Opt., 22 (4), 045702 (2020). https://doi.org/10.1088/2040-8986/ab78a8 Google Scholar

20.

M. Zhang, D. Yang and Y. Liang, “Color Fourier ptychographic microscopy based on symmetrical illumination and wavelength multiplexing,” J. Opt., 22 (6), 065604 (2020). https://doi.org/10.1088/2040-8986/ab8c12 Google Scholar

21.

A. Zhou et al., “Analysis of Fourier ptychographic microscopy with half of the captured images,” J. Opt., 20 (9), 095701 (2018). https://doi.org/10.1088/2040-8986/aad453 Google Scholar

22.

X. Ou, G. Zheng and C. Yang, “Embedded pupil function recovery for Fourier ptychographic microscopy,” Opt. Express, 22 (5), 4960 –4972 (2014). https://doi.org/10.1364/OE.22.004960 OPEXFF 1094-4087 Google Scholar

23.

L. Bian et al., “Motion-corrected Fourier ptychography,” Biomed. Opt. Express, 7 (11), 4543 –4553 (2016). https://doi.org/10.1364/BOE.7.004543 Google Scholar

24.

S. Chen et al., “Random positional deviations correction for each LED via ePIE in Fourier ptychographic microscopy,” IEEE Access, 6 33399 –33409 (2018). https://doi.org/10.1109/ACCESS.2018.2849010 Google Scholar

25.

L. Bian et al., “Fourier ptychographic reconstruction using Wirtinger flow optimization,” Opt. Express, 23 (4), 4856 –4866 (2015). https://doi.org/10.1364/OE.23.004856 OPEXFF 1094-4087 Google Scholar

26.

Y. Zhang, W. Jiang and Q. Dai, “Nonlinear optimization approach for Fourier ptychographic microscopy,” Opt. Express, 23 (26), 33822 –33835 (2015). https://doi.org/10.1364/OE.23.033822 OPEXFF 1094-4087 Google Scholar

27.

C. Zuo, J. Sun and Q. Chen, “Adaptive step-size strategy for noise-robust Fourier ptychographic microscopy,” Opt. Express, 24 (18), 20724 –20744 (2016). https://doi.org/10.1364/OE.24.020724 OPEXFF 1094-4087 Google Scholar

28.

Y. Fan et al., “Adaptive denoising method for Fourier ptychographic microscopy,” Opt. Commun., 404 23 –31 (2017). https://doi.org/10.1016/j.optcom.2017.05.026 OPCOB8 0030-4018 Google Scholar

29.

K. Guo et al., “Microscopy illumination engineering using a low-cost liquid crystal display,” Biomed. Opt. Express, 6 (2), 574 –579 (2015). https://doi.org/10.1364/BOE.6.000574 BOEICL 2156-7085 Google Scholar

30.

R. Eckert, Z. F. Phillips and L. Waller, “Efficient illumination angle self-calibration in Fourier ptychography,” Appl. Opt., 57 (19), 5434 –5442 (2018). https://doi.org/10.1364/AO.57.005434 APOPAI 0003-6935 Google Scholar

31.

A. Pan et al., “Subwavelength resolution Fourier ptychography with hemispherical digital condensers,” Opt. Express, 26 (18), 23119 –23131 (2018). https://doi.org/10.1364/OE.26.023119 OPEXFF 1094-4087 Google Scholar

32.

X. Tao et al., “Tunable-illumination for laser Fourier ptychographic microscopy based on a background noise-reducing system,” Opt. Commun., 468 125764 (2020). https://doi.org/10.1016/j.optcom.2020.125764 OPCOB8 0030-4018 Google Scholar

33.

B. S. Reddy and B. N. Chatterji, “An FFT-based technique for translation, rotation, and scale-invariant image registration,” IEEE Trans. Image Process., 5 (8), 1266 –1271 (1996). https://doi.org/10.1109/83.506761 IIPRE4 1057-7149 Google Scholar

34.

H. Foroosh, J. B. Zerubia and M. Berthod, “Extension of phase correlation to subpixel registration,” IEEE Trans. Image Process., 11 (3), 188 –200 (2002). https://doi.org/10.1109/83.988953 IIPRE4 1057-7149 Google Scholar

35.

J. Sun et al., “Sampling criteria for Fourier ptychographic microscopy in object space and frequency space,” Opt. Express, 24 (14), 15765 –15781 (2016). https://doi.org/10.1364/OE.24.015765 OPEXFF 1094-4087 Google Scholar

Biography

Kaicheng Huang received his BS degree in physics from the School of Physics, Nankai University in 2016. Currently, he is a PhD student at Nankai University. His current research interests include Fourier optics imaging and photonics.

Wangwei Hui received his BS degree in electronic information engineering from the School of Electronic Information Engineering, Shandong University in 2007. He received his MS degree in photonics and technology from the School of Physics, Nankai University in 2010. Currently, he is a teacher in the School of Physics, Nankai University. His current research interests include photo-electronic imaging and compression sensing optical imaging.

Qing Ye received his PhD in optics from Nankai University, China, in 2008, and then joined the School of Physics, Nankai University. Since 2010, he has been an associate professor of optics at Nankai University. His specialization includes optical coherence tomography, optical clearing, and optical properties measurement of biotissue.

Hongyang Zhao received his BS degrees in physics from the School of Physics, Nankai University in 2016. Currently, he is a PhD student at Nankai University. His current research interests include Fourier optics imaging and photonics.

Qiushuai Shi received his BS degree in physics from the School of Physics, Nankai University in 2016. Currently, he is a PhD student at Nankai University. His current research interests include Fourier optics imaging and compression sensing optical imaging.

Jianguo Tian received his PhD from Nankai University in 1991 and then started to teach in the same university as an assistant professor. In 1995, he became a professor. Since 2001, he has become a specially appointed professor at Nankai University. He is mainly engaged in condensed matter physics and photonics, involving optical nonlinear mechanism and application of photoelectric material properties.

Wenyuan Zhou received his master’s and doctoral degrees in 1996 and 2002, respectively. From 2003 to 2012, he worked as an associate professor at Nankai University. Since 2012, he has been a professor of optics at Nankai University. His main research interests are high-sensitive photon detection, photon imaging technology, and application of biological sensors.

CC BY: © The Authors. Published by SPIE under a Creative Commons Attribution 4.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.

Citation Download Citation

Kaicheng Huang, Wangwei Hui, Qing Ye, Hongyang Zhao, Qiushuai Shi, Jianguo Tian, and Wenyuan Zhou "Dither removing Fourier ptychographic microscope based on a two-axis rotation stage," Journal of Biomedical Optics 26(3), 036501 (4 March 2021). https://doi.org/10.1117/1.JBO.26.3.036501

Received: 9 December 2020; Accepted: 12 February 2021; Published: 4 March 2021

Access the abstract

JOURNAL ARTICLE
12 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

KEYWORDS

Fiber optic illuminators

Reconstruction algorithms

Microscopes

Computer simulations

Image processing

Imaging systems

Light emitting diodes

1.

Introduction

2.

Method

2.1.

Experimental Setup

Fig. 1

2.2.

Dither Removing Algorithm

Eq. (1)

Eq. (2)

Eq. (3)

Eq. (4)

Eq. (5)

Fig. 2

2.3.

Dither Calibration

Fig. 3

Fig. 4

Eq. (6)

Eq. (7)

3.

Results

3.1.

Simulation

Eq. (8)

3.2.

Experiments

Fig. 5

Fig. 6

Fig. 7

4.

Conclusion and Discussion

Disclosures

Acknowledgments

References

Biography

Show All Keywords

Keywords/Phrases

Search In:

Publication Years