1.

Introduction

Imaging in the shortwave infrared (SWIR) band (i.e., at wavelengths in the range $\sim 1.1$ to $1.7\mu \mathrm{m}$) has distinct advantages over other reflective bands, including higher atmospheric transmission, reduced optical turbulence, superior haze penetration, enhanced target/background contrast, and improved spectral discrimination.^1,2 Additionally, SWIR permits seeing laser designators and leveraging airglow in otherwise low-light environments.^3,4 SWIR capabilities have also reached technological maturity due to minimal losses in fiber-optic telecommunications and uncooled operation of indium gallium arsenide (InGaAs) detectors.^5,6 For all its benefits, however, passive SWIR imaging can suffer from low signal-to-noise ratios (SNRs) under reduced illumination conditions (e.g., at night).⁷ Providing an active illumination source can remedy this problem in exchange for added size, weight, power, and cost (SWaP-C).

Another known issue with imaging in the SWIR band is that commercial off-the-shelf (COTS) cameras are often severely read-noise limited (in the hundreds of electrons),⁸ potentially masking any performance boost over imagers in other bands with lower read noise (also known as temporal dark noise⁹). Improved readout integrated circuit design is one possible path forward. Another is to exploit the photoelectric effect by one of several mechanisms that provide amplification in the charge domain (i.e., electron multiplication), prior to both voltage conversion and digitization (after which analog and digital gain may be applied, respectively). Electron multiplication generally allows for higher dynamic range than analog gain by employing multiplication registers with their own full-well capacities between the serial registers and output nodes, so as not to affect the imaging area’s well depth.

One such gain architecture is the patented electron bombarded active pixel sensor (EBAPS) from EOTECH (formerly Intevac Photonics).^10–12 The principle of EBAPS operation is that a GaAs photocathode absorbs incoming photons, which then accelerate through high voltage to a CMOS anode, generating a photocurrent for amplification prior to readout. The drawback of this photoemissive scheme is that excess noise—i.e., multiplicative noise arising from uncertainty in the electron multiplying process—becomes significant due to electron scattering within the multiplication layer,¹³ effectively acting as a shot-noise multiplier. Consequently, the high dark current associated with EBAPS is poorly suited to long exposures with sources operating in continuous-wave (CW) mode. A solution to this problem is to implement laser range gating (LRG) with a pulsed laser source, thereby minimizing integration time (on the order of nanoseconds) and preventing significant accumulation of dark shot noise. LRG has many benefits, including contrast improvements by decreasing the influence of laser backscatter and solar path radiance, as well as reduction in scene clutter by effectively isolating the target from the background.^14,15

A mercury cadmium telluride avalanche photodiode (MCT-APD) sensor can offer comparable gain that is virtually free of excess noise (albeit at a substantially higher market price),^16,17 enabling CW-mode imaging as needed with minimal dark current. APDs operate in reverse bias just below the breakdown voltage, causing incident photons to excite electrons that generate secondary carriers in an avalanche process. The favorable sensitivity of this device (as compared to, say, an InGaAs-APD) follows from avalanche breakdown being initiated by only a single carrier (i.e., electron or hole injection) as a result of MCT’s unique band structure and scattering behavior.^18,19 An example COTS product incorporating MCT-APD technology is First Light Imaging’s C-RED One, which claims subelectron total dark noise (i.e., shot noise and read noise combined).²⁰

Figure 1 uses pristine drone imagery with real noise parameters to illustrate the advantages of both gain cameras: In CW mode (top row), higher dark current counteracts the low read noise that EBAPS affords, making MCT-APD the preferred option; in LRG mode (bottom row), the two gain solutions would appear more or less indistinguishable from one another if their quantum efficiencies (QEs) were equal. Note that all noise throughout this paper is treated as temporal only; i.e., we assume no spatial noise in our modeling.

Fig. 1

Illustrative example of DJI Phantom 4 Pro V2.0 (CAD model) with simulated noise corresponding to (a) InGaAs, (b) EBAPS, and (c) MCT-APD detector specifications (c.f., Tables 1Table 2–3) in CW mode; same for (d)–(f) but in LRG mode. Target is taken to be at a range of 10 km through a clear atmosphere (without regard for resolution in this instance) and a peak integrated photon energy of 1225 (at $1.6\mu \mathrm{m}$).²¹

A third class of charge-domain amplification that includes photomultiplier tubes and image intensifier tubes is outside the scope of our present discussion, as these devices are generally restricted to ultraviolet, visible (VIS), and near-infrared radiation only. Given these considerations, we set out to compare performance of three SWIR imagers: a standard InGaAs (so-called “nongain”), an EBAPS, and an MCT-APD camera. In doing so, we outline a multifaceted tradespace with field-testing applicability in mind: 2 imaging geometries (ground to air [G2A], air to air [A2A]) × 3 target objects (white quadcopter drone, steel cylinder, and aluminum sheet) × 2 meteorological conditions (23-km visibility and 5-km visibility) × 2 illumination modes (CW and LRG) × 3 ambient lighting scenarios (day w/sun behind sensor [SBS], day w/sun behind target [SBT], night) = 72 excursions with each camera. Our goal in this performance comparison is to set expectations for a series of upcoming field tests designed to gauge image quality at $1.6\mu \mathrm{m}$ with an emphasis on active tracking. The coming sections tell this story by first discussing theory and inputs to our model, then detailing hardware parameters of a notional active testbed, and finally presenting results of our trade studies from which we can draw conclusions going forward.

2.

Theory and Modeling

2.1.

Sensitivity and Resolution

The metric of interest for sensitivity is generally an SNR or contrast-to-noise ratio (CNR), depending on the imaging task at hand. We have observed in preliminary tracking studies that aimpoint maintenance is more of a target detection problem than recognition or identification,²² and so CNR is more pertinent to our interests. A general definition for this quantity is²³

Eq. (1)

$$\mathrm{CNR}=\frac{\sqrt{{({\mu }_{\mathrm{tg}}-{\mu }_{\mathrm{bg}})}^2+{\sigma }_{\mathrm{tg}}^2}}{\sqrt{{(F{\sigma }_{\mathrm{ps}})}^2+{(F{\sigma }_{\mathrm{ds}})}^2+{({\sigma }_{\mathrm{rd}}/G)}^2}},$$

where ${\mu }_{\mathrm{tg}}$, ${\mu }_{\mathrm{bg}}$, ${\sigma }_{\mathrm{tg}}$, ${\sigma }_{\mathrm{ps}}$, ${\sigma }_{\mathrm{ds}}$, and ${\sigma }_{\mathrm{rd}}$ are, respectively, the target signal mean, background signal mean, target signal variation, photon shot noise, dark shot noise, and read noise. The notation of Eq. (1) assumes all terms expressed in the charge domain (${\mathrm{e}}^{-}$), where the latter three refer specifically to background noise. Because shot noise is modeled as a Poisson process of equal mean and variance, ${\sigma }_{\mathrm{ps}}=\sqrt{{\mu }_{\mathrm{bg}}}$ and ${\sigma }_{\mathrm{ds}}=\sqrt{i_{\mathrm{d}}t}$ with $i_{\mathrm{d}}$ and $t$ being the per-pixel dark current (${\mathrm{e}}^{-}/\mathrm{px}/\mathrm{s}$) and integration time (s), respectively. $F$ and $G$ in Eq. (1) are, respectively, the excess noise factor and electron-multiplying gain of such devices as EBAPS and MCT-APD; as a reference, $F\approx \sqrt{2}$ for electron-multiplying charge-coupled devices.²³ We rely primarily on the Night Vision Integrated Performance Model (NV-IPM) to evaluate each of these terms,^24,25 though we have also developed our own radiometric model in Python for further verification with generally strong agreement.²⁶ Note that the denominator of Eq. (1) takes the form of a quadrature sum due to the uncorrelated nature of individual noise contributions. Also note that we are neglecting sources of coherent noise (e.g., speckle and scintillation) in this work as they affect only noise across the target and not the background.

We assess limiting image resolution simply in terms of nominal pixels on target (POT), estimated as²⁷

Eq. (2)

$$\mathrm{POT}={\left(\frac{W/R}{d/f}\right)}^2$$

in the case of sampling-limited imagery.²⁸ Here $W$ is the target’s characteristic dimension (i.e., the square root of its surface area), $R$ is the range from target to entrance pupil, $d$ is the pixel pitch/width, and $f$ is the effective focal length. Under the paraxial approximation, the numerator and denominator of Eq. (2) are the target’s angular extent and the camera’s instantaneous field of view (IFOV), respectively.

Empirical evidence from tracking simulations has repeatedly suggested that a minimum CNR of 10 and POT of 50 are sufficient to consistently track without risking a breaklock event.²² As such, we define a pass/fail figure of merit (FOM) as

Eq. (3)

$$\mathrm{FOM}(\mathrm{CNR},\mathrm{POT})=\text{step}(\mathrm{CNR}-10)\times \text{step}(\mathrm{POT}-50),$$

where²⁹

Eq. (4)

$$\text{step}(w)=\{\begin{array}{cc}1,& w>0\\ 1/2,& w=0\\ 0,& w<0\end{array}$$

is the Heaviside step function and $\mathrm{FOM}>0$ to maintain a successful track. Figure 2 provides a visual for this FOM with darker shading indicative of acceptable CNR and POT.

Fig. 2

Heatmap of pass/fail FOM as a function of both CNR and POT.

2.2.

NV-IPM Modifications

NV-IPM has a rich history of use in the passive electro-optical/infrared targeting community, and it is rigorously parameterized as such to predict targeting task performance.³⁰ We refer the interested reader to Teaney and Reynolds²⁴ for a broad overview of native NV-IPM interfacing, noting that all noise sources are modeled in NV-IPM as zero-mean Gaussian processes. Some retrofitting via custom components³¹ is required to apply the model in cases of active imaging, however. For instance, NV-IPM does not natively account for laser backscatter through an aerosol volume. To implement this missing functionality, we assume a layered atmospheric model by dividing the space between target and sensor into $M$ layers of equal thickness $\mathrm{\Delta }r$ (km). With a known aerosol phase function $f_{\mathrm{p}}({\theta }_{\mathrm{i}},{\theta }_{\mathrm{s}})$ that factors in respective incident and backscattered angles ${\theta }_{\mathrm{i}}$ and ${\theta }_{\mathrm{s}}$, the backscattered path radiance from layer $m$ is then

Eq. (5)

$$L_m=E_m·N\sigma f_{\mathrm{p}}(0,0)\mathrm{\Delta }r.$$

In the above, $E_m$ is the irradiance at layer $m$, $N$ is the aerosol number density, and $\sigma$ is the scattering cross section, and the 0-deg arguments denote normal incidence and on-axis observation. In turn, transmittance ($0\le \tau \le 1$) through the $m$’th layer is

Eq. (6)

$${\tau }_m={\tau }_{1\mathrm{k}}^{m\mathrm{\Delta }r},$$

where ${\tau }_{1\mathrm{k}}$ is a reference transmittance derived from the MODTRAN radiative-transfer code³² for a 1-km path that accurately portrays the imaging geometry.

We estimate the unknowns in Eq. (5) from a literature review of Mie scattering theory, which describes particles on the order of an optical wavelength.^33,34 Elterman found in his own literature review that the Mie scattering coefficient (${\beta }_{\mathrm{M}}=N\sigma$) decreases monotonically with increasing wavelength, as does the aerosol number density with increasing altitude $h$ (at least within the first 10 km above sea level).³⁵ Based on these findings, together with³⁶

Eq. (7)

$${\beta }_{\mathrm{M}}(h)={\beta }_{\mathrm{M}}(0)·\frac{N(h)}{N(0)},$$

we plot the Mie scattering coefficient as a function of altitude in Fig. 3(a). At sea level and wavelength $\lambda =1.6\mu \mathrm{m}$, this gives us ${\beta }_{\mathrm{M}}(0)\approx 9.89\times {10}^{-2}{\mathrm{km}}^{-1}$. We then use the built-in Mie model of FRED Optical Engineering Software to derive the normally incident phase function of Fig. 3(b) from a uniform particle-size distribution between 0.001 and $10\mu \mathrm{m}$³⁷ with a water refractive index of $n=1.32$ at $\lambda =1.6\mu \mathrm{m}$.³⁸ This leads to an on-axis value of $f_{\mathrm{p}}(0,0)\approx 3.64\times {10}^{-4}{\mathrm{sr}}^{-1}$, along with an unnormalized phase function of ${\beta }_{\mathrm{M}}(0)f_{\mathrm{p}}(0,0)\approx 3.60\times {10}^{-8}{(\mathrm{m}·\mathrm{sr})}^{-1}$ that is consistent with the peer-reviewed literature.³⁹

Fig. 3

(a) Mie scattering coefficient as a function of altitude. (b) Aerosol phase function at normal incidence as a function of scattering angle.

NV-IPM also lacks native capability to include solar path radiance between the target and sensor, which we must include in daytime engagements where active and passive signals combine at the receiver. We accomplish this by applying the expression

Eq. (8)

$$L(r)=L_{1\mathrm{k}}·\frac{1-\tau (r)}{1-{\tau }_{1\mathrm{k}}},$$

where $L_{1\mathrm{k}}$ is a 1-km reference path radiance output from MODTRAN, $\tau (r)$ is a range-dependent transmittance output from MODTRAN, and ${\tau }_{1\mathrm{k}}$ is the reference transmittance we defined previously in Eq. (6).

3.

Hardware Parameters

With a modeling approach in place, we now tailor our parameter space to hardware specifications for our notional testbed. Table 1 lists all parameters that are common to different excursions. For our pulsed system, we choose a somewhat arbitrary but realistic pulse energy of 200 mJ with a duration of 4 ns at a 10-Hz repetition rate. This translates to 2 W of optical power in a CW system producing an equivalent number of photons per frame interval. 2.2-mrad laser divergence corresponds to a Thorlabs F230SMA-1550 fiber collimator, and we assume the 50-nm bandwidth of an Edmund Optics #87-872 bandpass filter to suppress unwanted ambient light. We select an $f/4$ optical system based on COTS telescope availability with an acceptable balance between SWaP-C, FOV, and throughput.

Table 1

Source and sensor parameters that remain constant across excursions.

We base our notional detector model on the specifications of an Allied Vision Goldeye CL-033 TEC1,⁴⁰ prominently featured in a number of active and passive SWIR systems. For our purposes, this means holding pixel size/pitch constant while varying noise parameters between camera models, in order to make a fair comparison between core technologies rather than discuss design differences. In particular, we assume the Goldeye’s $5\text{-}\mu \mathrm{m}$ pixel size/pitch. Naturally, smaller pixels would have a beneficial effect on dark current, but our interest is in studying true noise specifications of commercially available devices. On the other hand, modeling actual differences in pixel size would expose somewhat trivial performance trades attributed to system resolution. Thus our hybrid approach offers a reasonable compromise to isolate noise performance through modeling. Table 2 summarizes all relevant noise parameters, with notable differences including a sharp decrease in read noise for both gain cameras but a drastic increase in dark current with EBAPS. The EBAPS system also has the lowest QE of the three, as shown in Fig. 4. The QE from each of these curves at $\lambda =1.6\mu \mathrm{m}$ makes up the third column of Table 2.

Table 2

Dark current, read noise, and QE values (at 1.6  μm) of all three camera systems under test.

Fig. 4

QE of various cameras in the SWIR band per manufacturer datasheets.

Table 3 shows the defining traits for our three targets of interest in order of increasing size (by area), and Fig. 5(a) provides a visualization of these target objects acquired for future testing. Using these dimensions in conjunction with the parameters in Table 1, we plot POT as a function of range for all three targets in Fig. 5(b). $F\lambda /d$ (where $F=f/D$ is the $f$-stop or focal ratio) according to Table 1 is 1.28, placing our system in the broad transition region between being sampling and diffraction limited.^41,42 It is, therefore, possible that our targets will become unresolved by the optics well before sampling, but we consider nominal pixel projections for the $\mathrm{POT}\ge 50$ criterion since object sizes on this order of magnitude are well resolved in our system by both definitions. To provide a reference value for extreme ranges at which CNR is of no consequence, we take the simplified approach of calling a target “unresolved” when it reaches $\mathrm{POT}=1$.

Table 3

Characteristic dimensions and reflectivity values (at 1.6  μm) of all three target objects under test; obtained from material characterization of real objects.

Fig. 5

(a) Photograph of target objects planned for use in upcoming field tests. (b) POT with respect to sampling as a function of range to various targets; solid and dotted orange lines represent $\mathrm{POT}=50$ and “unresolved” range (at which $\mathrm{POT}=1$), respectively.

4.

NV-IPM Results and Discussion

For reference throughout this section, Fig. 6 shows the relative placement of Sun, sensor, and target in both SBS and SBT scenarios. Note that, for modeling purposes, the only passive signal is ambient sunlight diffusely scattered through clouds (modeled as a sum of transmitted and path radiances). Under the assumption that background pixels see ground projections of the IFOV, the irradiance on such pixels is therefore independent of range.

Fig. 6

Diagram illustrating toy model of active imaging at range $R$ in SBS and SBT scenarios.

Looking closely at one subset of excursions, Fig. 7 displays all CNR versus range results in the case of an aluminum sheet in a G2A imaging geometry. Specifically, its first, second, and third rows comprise a daytime SBS scenario, a daytime SBT scenario, and a nighttime scenario, respectively; its left and right columns, respectively, represent 23-km and 5-km visibility conditions.

Fig. 7

CNR as a function of range for G2A imaging of an aluminum sheet; orange lines represent $\mathrm{CNR}=10$ (solid horizontal), $\mathrm{POT}=50$ (solid vertical) and “unresolved” range (dotted vertical). (a) Daytime SBS 23-km visibility, (b) daytime SBS 5-km visibility, (c) daytime SBT 23-km visibility, (d) daytime SBT 5-km visibility, (e) nighttime 23-km visibility, and (f) nighttime 5-km visibility.

The general shape comparison between LRG and CW curves is quite intuitive, as the pulsed laser provides a stronger signal at close range that falls off with distance as an inverse-square law. Passive illumination, on the other hand, maintains virtually constant irradiance with target distance. For this reason, long-range applications may benefit from CW mode in the presence of ambient lighting with little benefit from a divergent laser source. InGaAs operating in the read-noise limit experiences none of the usual LRG advantages, with performance either suffering compared to CW during daytime or matching CW at nighttime.

Performance in CW mode generally degrades with successive rows, as path radiance is most severe with SBT and the only available signal is subject to strong laser backscatter at nighttime. Columnwise, CNR suffers more in 5-km than 23-km visibility across the board. The degradation from SBS to SBT is less dramatic for LRG, however, and nighttime LRG imaging is superior due to a strong signal with minimal backscatter and zero contribution from solar path radiance. Beyond our binary $\mathrm{FOM}>0$ test, it is noteworthy that the order of best to worst performance always goes from MCT-APD to EBAPS to InGaAs, with the difference between gain imagers owing to a combination of increased dark current and decreased QE in EBAPS. Also worth noting is that the CW-illuminated target and sky background undergo contrast reversals in Fig. 7(d) that give rise to local minima at longer ranges. Such reversals simply result from viewing targets of range-dependent brightness against a background of constant brightness: Its brightness exceeds that of the background at close ranges but eventually becomes dimmer than the background with increasing range.

In the interest of visualizing data over a wider array of cases, Fig. 8 compares maximum ranges at which $\mathrm{CNR}\ge 10$ for both G2A [(a) and (b)] and A2A [(c) and (d)] geometries with the target again being an aluminum sheet. The columns once again represent 23-km and 5-km visibility from left to right. In a clear atmosphere with 23-km visibility, daytime CNR remains above 10 well past the “unresolved” range for any system operating in CW mode. When paired with a gain camera, CW illumination is more than sufficient for $\mathrm{CNR}\ge 10$ to surpass the $\mathrm{POT}\ge 50$ cutoff in all daytime scenarios. The reason for CW outperforming LRG in daylight is that CNR benefits more from longer integration of ambient sunlight than it suffers from increased path radiance and dark current. When nighttime imaging is also a priority, however, one of the gain systems operating in LRG mode is generally necessary to reach the POT limit. Again we can see here that CW and LRG perform identically in all nighttime cases with an InGaAs camera as limited by read noise. The MCT-APD either outperforms or is tied with other systems in all cases, as it has no inherent drawback other than price. EBAPS would perform similar to MCT-APD in all LRG cases (where dark current is negligible) if not for its comparatively low QE. Overall, LRG performance of a given camera stays relatively constant in different lighting scenarios because gating reduces the impact of both ambient signal and path radiance.

Fig. 8

Comparison between maximum ranges for an aluminum sheet; orange vertical lines represent $\mathrm{POT}=50$ (solid) and “unresolved” range (dotted). (a) G2A 23-km visibility (b) G2A 5-km visibility, (c) A2A 23-km visibility, and (d) A2A 5-km visibility.

For further abstraction of results, we plot maximum ranges, in which $\mathrm{CNR}\ge 10$ for each gain camera versus InGaAs across all 72 excursions in Fig. 9(a). We immediately notice that gain cameras outperform nongain as expected, but distances from the diagonal provide a sense of relative performance distribution (keeping in mind that points overlap in some instances). Rather than consider maximum range, Fig. 9(b) compares CNR values for each gain camera versus InGaAs at the ranges where $\mathrm{POT}=50$. Here we see a clear divide between high and low visibility, as well as division between gain cameras within each of these regimes. Together these two plots illustrate that MCT-APD will deliver better results than EBAPS or InGaAs when considering either $\mathrm{CNR}\ge 10$ or $\mathrm{POT}\ge 50$ alone, but EBAPS performance is likely acceptable when considering both criteria together (especially with LRG rather than CW illumination).

Fig. 9

(a), (b) Summary of gain versus nongain relative performance for all excursions within tradespace; unfilled markers in (a) indicate $\mathrm{CNR}\ge 10$ beyond “unresolved” range. Diagonal reference line marks equivalent performance and thus no improvement.

The nighttime-only results of Fig. 10 show the same trends, with fewer cases out at longer ranges but a similar distribution of cases otherwise. The main takeaway from Figs. 9 and 10 is that EBAPS can be a lower-cost alternative to MCT-APD and still a marked improvement over InGaAs for an LRG system to achieve $\mathrm{FOM}>0$, but MCT-APD is the best all-purpose camera that may justify its cost if nighttime CW imaging at long range is a must. It is also important to note, however, that COTS detector arrays are currently limited to larger pixel sizes/pitches with MCT-APD ($\sim 25\mu \mathrm{m}$) than with EBAPS ($\sim 15\mu \mathrm{m}$). Since we have studied only G2A and A2A imaging with nominally zero background reflectivity up to this point, we can expect that LRG would compare even more favorably to CW in G2G or A2G geometries with a substantial background to isolate from the target. The same is true of harsher environments, such as haze, rain, or smoke, where one could expect backscatter to limit performance even more severely than in cases we have explored here.

Fig. 10

(a), (b) Summary of gain versus nongain relative performance for “nighttime-only” excursions within tradespace; unfilled shapes in (a) indicate $\mathrm{CNR}\ge 10$ beyond “unresolved” range. Diagonal reference line marks equivalent performance and thus no improvement.

5.

Conclusion

The sensitivity of active targeting systems in the SWIR band is currently limited by high read noise associated with conventional readout integrated circuitry. This limit imposes a barrier to leveraging other performance trades, such as source power, illumination wavelength, and temporal coherence. Introducing gain in the charge domain prior to signal readout can reduce the impact of read noise, to the point that it no longer limits performance. In preparation for a series of planned active-imaging field tests, we demonstrated improved system performance on a modeling basis with two different charge-domain gain cameras: the EBAPS and the MCT-APD sensor. We found that both solutions mitigate read noise to make either one suitable for LRG, but the high dark current associated with EBAPS may make it unsuitable for CW imaging in some scenarios. These results aid in our understanding of expected performance in field testing of charge-domain gain systems.