2.1.

Overall DMD-Based TFMPM System Setup

The DMD-based TFMPM system is shown in Fig. 1. To have enough pulse energy to excite widefield or line-shaped MPEF signals, the Ti:Sapphire regenerative amplifier (Spitfire Pro XP, Spectra-Physics) seeded by a Ti:Sapphire ultrafast oscillator (Tsunami, Spectra-Physics) was adopted as the laser source and provides a maximum pulse energy of $400\mu \mathrm{J}/\text{pulse}$ with a pulse width of 120 fs. The center wavelength is 800 nm and the repetition rate is 10 kHz. The fast mechanical shutter (VS14S-2-ZM-0-R3, Uniblitz) is open during the imaging process and can block the laser to avoid unnecessary exposure. After passing through the half-wave plate and the polarization beam splitter, it governs the laser output power and remains horizontal polarization. The DMD (DLP6500FYE, Texas Instruments) included in the evaluation module (DLP^® LightCrafter™ 6500, Texas Instruments) has a resolution of $1920\times 1080\text{pixels}$ (1080 p) within a 0.65-in. chip. The pixel pitch is $7.56\mu \mathrm{m}$ and the mechanical tilt angle of every micromirror is $\pm 12°$. The binary pattern rate can be as high as 9.5 kHz with the DLPC900 digital controller. The image patterns were designed offline by MATLAB and uploaded to the internal memory of the controller to achieve the maximum pattern rate. Because the micromirror array structure has an orthogonal arrangement, and the hinge axis of the micromirrors runs along the diagonal of the chip, the DMD is rotated 45° to make the hinge axis parallel to $y$ axis in Fig. 1. The DMD behaves like a brazed grating and the effective grating pitch is $10.69\mu \mathrm{m}$ with a blazed angle of 12°. The chosen sixth-order diffraction beam ensures that the overall efficiency is maximized according to the diffraction equation. Lenses $L_1$ ($f=-75\mathrm{mm}$) and $L_2$ ($f=150\mathrm{mm}$) expand the beam to make full use of the micromirrors by filling the DMD; meanwhile, $L_3$ ($f=500\mathrm{mm}$) and $L_4$ ($f=200\mathrm{mm}$) form a relay pair that extends the beam into the upright microscope (Axio Imager.A2m, Carl Zeiss, Germany). Lastly, $L_5$ ($f=250\mathrm{mm}$) and the water immersion objective (UPlanSApo 60XW/NA 1.2, Olympus, Japan) create the temporal focusing excitation. The optics were chosen to ensure that the beam shape on the back-focal plane of the objective nearly matches the diameter of the objective rear aperture for optimal AEC performance. The induced group velocity dispersion (GVD) from the system shifts the temporal focusing plane away from the conjugate plane of the generated pattern on the DMD. The resultant temporal focusing excitation and spatial focal plane mismatch would greatly lower the excitation efficiency and pattern contrast.^56,63–65 In response, the built-in pulse compressor in the regenerative amplifier can be adjusted to compensate the overall system GVD and shift the temporal focusing excitation plane to match the spatial focal plane. Then, the generated optical pattern at the focal plane of the objective has superior excitation efficiency and could be imaged with high contrast. In this manner, the optics performs the optimal setup for temporal focusing excitation and patterned illumination. The MPEF and SHG pass through a dichroic mirror and optical filter and are detected by the electron-multiplying charge-coupled device (EMCCD) (iXon Ultra 897, Andor), which has $512\times 512\text{pixels}$ and a pixel size of $16\mu \mathrm{m}$. The $2.5\times$ camera adaptor is used to magnify the excited fluorescence image to fill the EMCCD for optimal digital resolution. The single-pixel size of the EMCCD corresponds to $0.118\mu \mathrm{m}$ on the temporal focusing plane by calibration with the stage micrometer (#36-121, Edmund Optics Inc.) and is almost half of the theoretical diffraction limit of the system so that the digital resolution is sufficient according to the Nyquist criterion. By imaging the patterns on the DMD, the size of a single pixel on the DMD is calibrated and corresponds to $0.035\mu \mathrm{m}$ on the temporal focusing plane. The specimen is placed on the motorized stage (H101A ProScan, Prior Scientific) with a three-axis encoder to have precise position accuracy. The 3D images can then be captured by axially scanning with the fast piezo focusing stage (NanoScanZ 200, Prior Scientific) with a maximum traveling range of $200\mu \mathrm{m}$. The PC includes a data acquisition card (PCIe-7842R, National Instruments) with a Virtex-5 LX50 field-programmable gate array that controls and communicates with all peripheral instruments and components.

Fig. 1

(a) Optical setup of the DMD-based TFMPM with parallel multiline scanning. (b) An illustration of DMD shows the line profile with the micromirrors tilted at $-12\mathrm{deg}$ in an inactive (OFF) state, and the profile with micromirrors tilted at 12° in an active (ON) state.

2.2.

Theoretical Simulation

A conventional temporal focusing setup with a blazed grating, collimating lens, and an objective lens is shown in Fig. 2(a). The incident ultrashort laser pulse consists of different wavelengths that are interfered constructively in phase. The electric field $U_G$ at the grating plane is represented as a function of the superposition of different wavelengths with its initial phase according to the diffraction equation:

Fig. 2

(a) Temporal focusing system setup. (b) Simulation results of axial intensity distribution with different laser spectral bandwidths. The red, green, and blue circles stand for bandwidth of 2, 4, and 8 nm. The solid lines are the fitted curves and the estimated FWHM are 22.8, 6.5, and $2.3\mu \mathrm{m}$, respectively.

It is noted that $U_F$ also represents the angular spectrum of $U_G$. On the other hand, it is assumed that the back-aperture acts like a spatial filter $L_F$ that passes only a limited bandwidth of angular spectrum of $U_F$. The final electrical field $U_{TF}$ on the temporal focusing plane could be represented as

As can be seen, $U_{TF}$ is obtained by summing all the frequency components. The two-photon excited fluorescence signal $I_{2p}$ is proportional to the square of the intensity profile $I_{TF}$ after Fourier transforming the frequency domain into the time domain on the temporal focusing plane:

To obtain the overall temporal focusing intensity distribution along the $z$ direction, we start with the angular spectrum propagation. Based on Helmholtz equation, the effect of the axial propagation of the angular spectrum is the product of the relative phase term and its corresponding component.⁶⁶ Furthermore, the corresponding angular spectrum of the electric field on the temporal focusing plane is the electric field on the Fourier plane, so we can simply apply the propagation phase term before we perform the Fourier transform for $U_{TF}$ ($x_{TF}$, $y_{TF}$, $\mathrm{\Delta }z$; $\omega$). Finally, the $I_{TF}$ ($x_{TF}$, $y_{TF}$, $\mathrm{\Delta }z$; $t$) can be calculated:

2.3.

System Performance

The AEC of the DMD-based TFMPM shown in Fig. 1 is estimated by experimentally measuring the axial intensity profile of a thin film doped with Rhodamine 6G (R6G) dye ($<200\mathrm{nm}$ thickness). Figure 3(a) shows the data points of the average fluorescence intensity at every $z$ step of $0.5\mu \mathrm{m}$ (blue circles). According to the fitted curve (blue line) in Fig. 3(a), the AEC of the system is estimated to be $3.5\mu \mathrm{m}$. To find the effective laser spectral bandwidth, the system in Fig. 1 is converted into the conventional temporal focusing system setup as shown in Fig. 2 and performs the simulation based on the theoretical model described in Sec. 2.2. The sixth-order diffraction beam of the DMD used in TFMPM effectively equals to the first-order diffraction beam of a $561\text{grooves}/\mathrm{mm}$ blazed grating. The relay pair $L_3$ and $L_4$ together with $L_5$ match the collimating lens of 625 mm in Fig. 2; meanwhile, the $60\times$ Olympus objective lens has an effective focal length of 3 mm. With Eq. (6), we can simulate the axial intensity change for every $\mathrm{\Delta }z$ of $0.2\mu \mathrm{m}$ in one direction from the temporal focusing plane ($z=0$) with the assumption that the curve is symmetric in order to reduce the overall computation time for simulation in MATLAB. With different laser spectral bandwidth value, the AEC is calculated from the curve that is fitted to the simulated axial intensity data points in each case. The simulation result of the AEC of the TFMPM varies with respect to the laser spectral bandwidth is shown in Fig. 3(b), and the effective full spectral bandwidth of laser is estimated to be 5.7 nm according to the experimental measurement of the $3.5\mu \mathrm{m}$ AEC. With this estimated effective spectral bandwidth value, we can discuss how the system AEC would be influenced by the generated pattern on the DMD and the limited size of the back-aperture of the objective, $L_F$. Furthermore, the DMD pattern is able to be well-designed to improve the AEC and verified by simulation.

Fig. 3

(a) AEC of the DMD-based TFMPM. Experimental measurement (blue circles) and fitted curve (blue line). The estimated FWHM is $3.5\mu \mathrm{m}$. (b) Simulation result of the AECs of the TFMPM with different laser spectral bandwidths.

3.1.

Line Scanning Performance with Line Pattern of Different Line Width

The line-scanning TFMPM has a superior AEC theoretically because of the full utilization of the NA of the objective. With the DMD-based TFMPM, we can generate line patterns on the DMD to achieve a line-scanning mechanism. The line was chosen parallel to the blazed direction. Although synchronizing the DMD with the PC via the video interface is more convenient and intuitive, serious background noise due to the micromirrors switching noise⁶⁷ will arise, which in turn limits the pattern switching rate. Instead of using the video mode, the patterns are first uploaded into the internal memory of the DMD controller and displayed on the DMD in sequence when triggered so that the background micromirror switching noise could be eliminated. Patterns with different line widths are applied and the AEC curves are experimentally acquired by axially scanning the R6G thin film. The fitted AEC values with their corresponding line widths are plotted in Fig. 4 as blue circles. When increasing the line width, the AEC will converge to the value of $3.5\mu \mathrm{m}$, which is reasonable since the line pattern is so thick that it acts as the widefield TFMPM. On the other hand, reducing the line width can push the AEC to nearly $1.5\mu \mathrm{m}$, which is equivalent to line-scanning TFMPM. The trend for how the AEC changes according to the influence of the pattern $P_{\mathrm{DMD}}$ is validated by simulation described in detail in Sec. 2.2:

Fig. 4

Curve of the AEC changes with different line width patterns on the DMD. The blue and red circles are experimental measurements and simulation data, respectively.

The laser parameter of the effective full spectral bandwidth is estimated by the measured AEC value according to the simulated relationship between the AEC and spectral bandwidth in Fig. 3. The ${U^{\prime }}_F$ on the Fourier plane is the electric field that is the Fourier transform of the product of $U_G$ and the applied pattern $P_{\mathrm{DMD}}$ on the DMD. The ${U^{\prime }}_{TF}$ on the temporal focusing plane is the Fourier transform of the ${U^{\prime }}_F$ after the spatial low-pass filter due to the limited back-aperture size of the objective. The theoretical AEC profile can be performed according to Eq. (6) with ${U^{\prime }}_{TF}$. The simulated AECs with respect to different $P_{\mathrm{DMD}}$ of different line widths are shown in Fig. 4 and plotted as red circles. The simulation matches the experimental data and confirms the correctness of the simulation method. According to Fig. 4, the line pattern with fewer than 20 pixels does not have much AEC improvement and has low reflection efficiency for excitation due to fewer active micromirrors on the DMD. For the single-line scanning, we take 20 pixels to be the minimum width of a single line before the AEC starts to grow. In addition, the main central peak of the diffraction limit focus for the system shown in Fig. 1 at an excitation wavelength of 800 nm based on the Rayleigh criterion is $0.81\mu \mathrm{m}$ which corresponds to around 23 DMD pixels, so a line pattern has a line width $<20\text{pixels}$ on DMD would be meaningless.

3.2.

Parallel Multiline Scanning

Conventional line-scanning TFMPM requires scanning a line to form a widefield image. If the image has $512\times 512\text{pixels}$, it takes 512 scans to form the full image. The DMD-based TFMPM has the ability to generate multiline patterns and is capable of scanning the patterns to render an image. We have verified that a 20-pixel-width line can have an AEC of $1.7\mu \mathrm{m}$, which approaches the theoretical limit according to the simulation. Next, we experimentally measure the AEC with the multiline pattern which has the single-line width of 20 pixels and different spatial periods. In Fig. 5(a), the blue circles show the experimental data while red circles indicate the simulation results. When the spatial period is shorter, the pattern becomes similar to the pattern of all pixels are in the “ON” state and its AEC approaches the performance of conventional TFMPM. On the other side, together with the reduction of the pattern period, the subsidiary maxima of AEC induced by the Talbot effect move toward to the main peak and affect the resultant AEC. In the simulation, the relative intensities of the subsidiary maxima of the Talbot effect decrease together with the increased pattern period as shown in Fig. 5(b). The resultant AEC is improved when the Talbot effect is ceased. In the experiment, Figure 5(c) shows the measured axial intensity curve with different spatial periods of the pattern. The red, green, magenta, and blue circles stand for 24-, 44-, 60-, and 80-pixel periods, respectively. The simulated data of the same pattern periods as Fig. 5(c) are shown in Fig. 5(d). The overall AEC changes, together with the pattern period and the subsidiary peaks status trend, can all be verified with the theoretical simulation. The Talbot effect on the AEC can be minimized when the subsidiary peaks are moved away from the main peak and the relative intensity of subsidiary peaks to the main central peak reduces to below 10% in Fig. 5(b). In other words, the widened AEC due to the subsidiary peaks can be relieved when the pattern period is larger than three times the width of a single line. In practice, four times the width of a single line would result in the best AEC, while longer period patterns seem not to provide much AEC improvement and more patterns are required to form a complete image. Ideally, since the pattern has 25% duty cycle, four patterns with different spatial phases (0, $\pi /2$, $\pi$, and $3\pi /2$) should be sufficient to form a widefield image with the improved AEC. However, the limited spatial bandwidth of the optical system would result in a periodic background on the final image when these four multiline patterns are applied. In practice, the eight multiline patterns with $\pi /4$ phase increment, $P_n$ and $n=0,1,\dots ,7$, are designed for scanning to ensure a uniform excitation image:

Fig. 5

(a) The AEC curve changes with the pattern of the parallel 20-pixel lines with different periods on the DMD. The blue circles are the experimental measurement, while the red circles are the simulation data. (b) The Talbot effect-induced side lobe intensity relative to the main lobe intensity with respect to different pattern periods. The blue circles denote the simulation data and the black dashed line indicates 0.1; axial intensity curves with different period patterns: (c) experimental measurement and (d) simulation. The red, green, magenta, and blue circles stand for 24-, 44-, 60-, and 80-pixel period patterns, respectively. The side-lobes induced by the Talbot effect are clearly observed both in the experiment and simulation. The side-lobes move outwards, but the effect on the AEC is minimized when the pattern period is increased.

Fig. 6

Widefield two-photon excited fluorescence image of an R6G thin film with the DMD-based TFMPM with (a) all DMD pixels active and (b) parallel multiline scanning. The bright signal on edges are due to the fixed margin of DMD which would keep reflecting the laser for excitation during scanning. (c) AEC of (b), with the experimental measurement (blue circles) and fitted curve (blue line). The FWHM is estimated as $1.7\mu \mathrm{m}$.

Figure 7(a) shows the back-aperture image of the conventional TFMPM. The line-shaped beam captured with the digital camera (COOLPIX P7000, Nikon, Japan) is horizontally expanded due to the angular dispersion induced by the periodic structure of the DMD. The simulation described in Sec. 2.2 yields a similar beam shape as shown in Fig. 7(d). The white-dashed circle indicates the boundary of the objective back aperture. It was noted that the low filling ratio of the incident beam on the back-aperture plane deteriorates the effective objective NA and lowers the system AEC. On the other hand, the parallel multiline-scanning mechanism means that every multiline pattern on DMD induces its spatial-modulated shape in addition to the dispersion line-shape, both of which result in a diffraction pattern that has a higher filling ratio and a more uniform beam shape on the back-focal aperture of the objective. In this manner, both a higher utilization of effective objective NA and superior AEC are realized. Since all the multiline patterns described in Eq. (9) have the same line width and period except the phase term, the resulting beam shapes remain the same on the back-aperture plane and the multiline patterns are continuously switched for parallel multiline scanning. An experimental back-aperture image captured during scanning and a simulated image are shown in Figs. 7(b) and 7(e), respectively. On the other side, if the scanning multiline pattern sequences include any unwanted pattern, e.g., uniform pattern with all DMD pixels active when the video mode of the DMD controller is adopted, the beam shape on the back-focal aperture of the objective is shown in Fig. 7(c). The beam shape is less uniform than Fig. 7(b) and more intense in central line. The non-uniform beam shape will result wider AEC and deteriorate the system performance.

Fig. 7

The beam shape on the back aperture of the objective with: (a) all DMD pixels active and captured by digital camera; (b) in-process parallel multiline pattern scanning; (c) multiline pattern sequences include a uniform pattern with all DMD pixels active (ON state); (d) all DMD pixels active in the simulation; and (e) parallel multiline pattern scanning in simulation. The white dotted circle indicates the boundary of the objective aperture.

3.3.

Biotissue Imaging with Parallel Multiline Scanning-Based TFMPM

The developed parallel multiline scanning-based TFMPM system was demonstrated to deliver an improved AEC of $1.7\mu \mathrm{m}$. With the enhanced AEC, the axial excitation region of the specimen is better confined, and less background noise will be induced from the out-of-focus region. With the improved axial-resolving ability, the image contrast is obviously superior to the original TFMPM image. The complex biotissue of an eosin-stained mouse skin specimen is used for demonstration. Figure 8(a) shows the mouse hair in dermis, and the medulla structural pattern can be clearly observed. The sectional image on the $yz$ plane of the white-dashed line is shown in Fig. 8(b). The image blur is resolved with the parallel multiline scanning technique that improved the AEC. The images are shown in Figs. 8(c) and 8(d). The details can be clearly observed in the adjacent dermis and in the sectional images of Fig. 8(d) as well. The adipose tissue in the subcutaneous layer imaged with a conventional TFMPM on the $xy$ plane and the $yz$ plane indicated by the white-dashed line are shown in Figs. 9(a) and 9(b), respectively. In comparison, a fluorescence image of the $xy$ plane and sectional $yz$ plane produced with the AEC-improved and parallel multiline scanning-based TFMPM is shown in Figs. 9(c) and 9(d), respectively. The structural shadow that appears above the image in Fig. 9(a) is attributed to the original broad AEC and scattering. With the parallel multiline scanning, Fig. 9(c) shows an image with improved contrast and effectively eliminated background disturbance. Moreover, the reconstructed images on the sectional $yz$ plane display improved contrast and more observable details. The power efficiency of the DMD-based TFMPM is around 20% when all DMD pixels are active. To avoid the damage to DMD by the regenerative amplifier with long exposure time, the laser power to DMD is limited to 70 mW, which corresponds to 14 mW on specimen. With the multiline pattern with 25% duty cycle for the parallel multiline scanning, the actual power on the specimen is expected to be around 3.5 mW. The limited power excitation only allows the system to excite and receive the fluorescence signal at the depth of $60\mu \mathrm{m}$ in this complex eosin-stained mouse skin specimen. However, the contrast improvement of the reconstructed image is clearly observed in the limited depth.

Fig. 8

Widefield two-photon excited fluorescence images of mouse hair in eosin-stained skin with a conventional TFMPM on (a) the $xy$ plane and (b) the $yz$ plane, and with the parallel multiline pattern scanning-based TFMPM in (c) the $xy$ plane and (d) the $yz$ plane.

Fig. 9

Widefield two-photon excited fluorescence images of adipose tissue in the subcutaneous layer of eosin-stained mouse skin with a conventional TFMPM on (a) the $xy$ plane and (b) the $yz$ plane, and with the parallel multiline pattern scanning-based TFMPM in (c) the $xy$ plane and (d) the $yz$ plane. A comparison between these two different imaging methods at different $z$-stacks is given in Video 1 (mp4, 1.35 MB) https://doi.org/10.1117/1.JBO.26.1.016501.1, with both different sectional images shown in Video 2 (mp4, 1.0 MB) https://doi.org/10.1117/1.JBO.26.1.016501.2.