2.1.

Datasets

Three different image modalities containing 87 stacks of images were used to evaluate the performance of individual and hybrid FMFs. Three diverse data types covering ZN (CM), FM, and multispectral (MS) images were used to evaluate HFMFs. Detailed description for each imaging modality is given below.

2.1.1.

Ziehl–Neelsen sputum smear conventional microscopy

A total of 31 autofocusing stacks were extracted from ZN Sputum Smear Microscopy Image Database.²⁰ These stacks were prepared from 10 different ZN-stained sputum smear slides of tuberculosis patient using three different microscopes. Each stack contains 20 images captured at different focus points over the same view field [Fig. 2(a)]. Acquired images were diverse as image contents ranged from medium to high noisy backgrounds. Image contents also varied due to improper use of staining dye (over- and under-staining).

Fig. 2

Image modalities used to evaluate HFMFs. (a) Image acquired from ZN sputum smear CM, (b) image acquired from sputum smear fluorescent microscopy (FM), (c) image acquired in VS, (d) image acquired in near-IS, (e) image acquired in TS, (f) depiction of the 50% region sampled area, and (g) depiction of the 25% region sampled area used to evaluate FMFs.

2.2.

Focus Measure Functions

The eight most common FMFs were included in this study as their performances in ZN (CM) and FM images were good.^1,2,21 Other FMFs such as Laplacian-based operator and wavelet-based operator drastically failed on CM images; therefore, they were not included in the current study. These eight FMFs were hybridized, and their performances were evaluated in this study to identify the best-focused images from ZN, FM, and MS images (Table 2). An FMF has the highest value at the best focus position, and the values reduce sequentially in both directions as focusing decreased. Three major categories of FMFs and their HFMFs were implemented in MATLAB (Table 2).

Table 2

Category of FMFs used to form the hybrid FMF for identifying the best-focused images.

2.3.

Hybridization

Eight FMFs were hybridized in pairs of two, and a total of 36 combinations were obtained. Hybridization of FMFs means two FMFs are implemented simultaneously on a single stack to calculate the hybrid focus measure using the following equation:

2.4.

Region Sampling

Region sampling was performed to implement FMFs on 25%, 50%, and 75% central parts of whole image. For 25% region sampling, a total of 25% of pixels from the central part of the original image were retained. For $100\times 100$ size (10,000 pixels) image, 25 pixels from each end of the rows and columns were removed to get an image of $50\times 50$ dimensions (2500 pixels). The resultant image was sampled to 25% as the total number of pixels was reduced to one fourth of the original image [Fig. 2(g)]. Similarly, region sampling of 50% [Fig. 1(f)] and 75% was performed. Region sampling was performed to achieve better accuracy as well as to reduce computation time.^9,27

2.5.

Image Preprocessing

Poisson noise was added to check the robustness of HFMFs to noise. A MATLAB function “imnoise” is used to add Poisson noise generated from the image itself. A scaling factor $1\times 10{}^{-10}$ is used for the significant effect of noise on the image. In general, FMFs are more sensitive to the higher level of noise.²

The saturation level of an image also alters the performance of FMFs, which was previously tested on normal images.²⁸ To check the efficacy of HFMFs with respect to an increase in the saturation level, ZN and FM images were converted to hue, saturation, and value (HSV) color space. Furthermore, saturation of HSV images was increased by 25% using MATLAB.

Reduced contrast level leads to smoothening of edges in images, which reduces differentiation of the best focus image from the defocused one. Contrast was incorporated in the preprocessing step to verify the effectiveness of HFMF at the low contrast level. Generally better focused methods are not perturbed by low contrast, which was reduced for every stack by mapping the image pixel values to a narrow range.²⁸

Uneven illumination was incorporated into images using a luminance gradient to test the effectiveness of HFMFs in low signal-to-noise ratio conditions due to poor illumination. Grayscale image is used to represent luminance gradient using quadratic polynomial function, and it is multiplied by the original images to get resultant images.

2.6.

Evaluation of Focus Measures

The following three criteria were used to evaluate the performance of FMFs and HFMs.²⁹

Accuracy criterion: The accuracy value was assigned a score of 1, 0.5, or 0 if a stack was correctly classified; if the second best focus was classified as the best focus when the difference between the best and second best image differs marginally; or if the stack was misclassified, respectively. Finally, the accuracy rate in percent was calculated using the following equation:¹

A higher score represents a more accurate FMF.

Focus error: It determines the difference between the manually obtained and predicted best-focused image.²

Number of false maxima: This criterion was used to calculate the number of false maxima produced by an HFMF or FMF. A number of maxima present in a sharpness curve of the FMF or HFMF excluding global maximum was determined.¹

2.7.

Convergence Rate of Focus Measure Functions

Finally, “sharpness of focus curve” was used to identify the FMF and HFMFs with better convergence rate. It is used to calculate the narrowness of the peak. Narrower peak of FMF represents rapid convergence to the best focus point; hence, FMF would be implementable in the real system.¹⁶

3.1.

Region Sampling and Hybridization of Focus Measure Function

Different parameters and configuration were checked prior to evaluating the performance of HFMFs. Images regions were sampled to 25%, 50%, and 75% to evaluate the accuracy of FMFs on different region sampling rates in comparison with original images [Fig. 2(a)]. Overall accuracy of most of the FMFs was reduced by 1% to 11% at 25% region sampling, while it was increased by 1% to 4% at 50% and 75% region sampled images (Fig. 3). HFMFs analyses were performed only on 50% region sampled images as the result was optimal and the mean computation time was minimal at this level. Improved performance of FMFs on 50% and 75% region sampling might be due to better focusing on the central part of the image than boundaries. Hybridization of two and three FMFs implemented on separate locations of the same view field was evaluated, but performance of most of these FMFs was inconsistent and poor due to the different imaging contents. Therefore, the FMFs were superimposed on the same location of view field image to calculate the unbiased focus measure. A combination of three FMFs yielded poor accuracy in most of the HFMFs, while combinations of two FMFs have provided a better accuracy rate. Therefore, only two FMFs were superimposed and used as the final configuration. Performances of HFMFs were evaluated on overall datasets as well as separately on three individual types of datasets (ZN, FM, and MS) to determine the effect of different imaging modalities on HFMFs. ZN datasets also contain microscopic images captured from a Smartphone camera to evaluate the performance of HFMFs. Mean computational time taken by each FMF or HFMF was determined on Intel^® Core™ i3-3220 CPU at 3.30 GHz with eight GB RAM (Table 4 of Appendix). The comparative performances of HFMFs without preprocessing and post preprocessing are provided in the following sections.

Fig. 3

Accuracy of FMFs in percent with different region sampling data, i.e., 25%, 50%, 75%, and original image.

3.2.

Without Image Preprocessing

Average performances of 36 (eight individual and twenty-eight HFMFs) FMFs were computed separately on each dataset at different region sampling rates (Fig. 3). More than 90% overall accuracy at 50% region sampling was obtained using 19 HFMFs, which indicated that HFMFs were consistent w.r.t different imaging modalities [Fig. 4(a) and Table 3]. Focus error and false maxima rate of these 27 FMFs (eight individual and nineteen HFMFs) were also computed to validate the analysis. Most of these HFMFs performed accurately and had less focus errors [Fig. 4(b)] and false maxima [Fig. 4(c)], whereas most of the individual FMFs provided an accuracy $<90\%$, with higher focus error and false maxima (except GDR and TGR). HELMnTGR, SFMnTGR, VGRnGDR, VGRnHELM, and VGRnSFB HFMFs obtained $>95\%$ accuracy and outperformed most of the individual FMFs. Measures such as accuracy, focus error, and false maxima along with standard deviation, mean, and combined results for each dataset are also provided (Table 3). Mean accuracy, its standard deviation, and combined accuracies showed that VGRnHELM and VGRnSFB were the most accurate and consistent HFMFs with minimal standard deviation of 1.78 and 1.62, respectively (Table 3).

Fig. 4

Performance of FMFs and HFMFs without preprocessing at 50% region sampling data. (a) Accuracy in percent, (b) focus error, and (c) false maxima.

Table 3

Accuracy in percent, focus error and false maxima of FMF and HFMF without preprocessing at 50% subsampling.

3.3.

Image Preprocessing

Effectiveness of HFMFs to different imaging conditions is very important because the occurrence of noise, poor contrast, illumination, etc. may affect its performance. Poisson noise addition, saturation-level increment, contrast reduction, and uneven illumination were incorporated to find out the effect of image distortion on FMFs and HFMFs performance.

In the first step, Poisson noise was added to all the images. Generally, a higher level of noise in an image significantly affects FMF performance.^2,28 Most of the HFMFs were more robust than individual FMFs after noise addition (Fig. 5). Though GDR and TGR FMFs produce higher accuracy in focused image identification, they failed drastically after noise addition [Fig. 5(a)]. GDR FMF accuracies dropped to 25.8%, 67.1%, and 65% for ZN, FM, and MS datasets, respectively. Similarly, the TGR accuracies dropped to 90%, 30%, and 80% for the above datasets after noise addition. Focus error [Fig. 5(b)] and false maxima [Fig. 5(c)] rate were also increased in the above two individual FMFs. VGRnSFB HFMF was least affected by noise addition and outperformed all the individual FMFs in terms of accuracy, focus error, and false maxima, whereas VGRnGNV HFMF ranked second after noise addition [Fig. 5(a)].

Fig. 5

Performance of FMFs and HFMFs after noise addition at 50% region sampling data. (a) Accuracy in percent, (b) focus error, and (c) false maxima.

In the second step, the saturation was increased by 25% in all images. Generally, performance of all FMFs decreases as the saturation level increases.²⁸ On MS datasets, GDR and TGR have shown a poor accuracy rate of 65% and 75%, respectively. The performances of most of the HFMFs were better than individual FMFs after increased saturation levels. The performance of VGRnSFB was altered slightly and showed highest accuracy rate with less focus error and false maxima (Fig. 6).

Fig. 6

Performance of FMFs and HFMFs after 25% saturation increment at 50% region sampling data. (a) Accuracy in percent, (b) focus error, and (c) false maxima.

In the third step, the contrasts of all images were reduced using the imadjust function of MATLAB. Generally, marginal reduction of contrast has no or minimum effects on FMFs performance.^22,28 Performance of all the HFMFs was affected marginally by contrast reduction, and accuracies slightly dropped (Fig. 7). VGRnSFB remained consistent to reduced contrast level and obtained an overall accuracy of 91.3%.

Fig. 7

Accuracy of FMFs and HFMFs in percent after contrast reduction at 50% region sampling data.

Uneven illumination has a minimal effect on performance of HFMFs in FM images. In some cases, performance has been improved.² Uneven illumination was incorporated in all the images. Most of the HFMFs have shown relatively consistent performance and have marginal changes in accuracy (Fig. 8).

Fig. 8

Accuracy of FMFs and HFMFs in percent after uneven illumination at 50% region sampling data.

Evaluation of FMFs and HFMFS in various imaging conditions (such as without preprocessing, noise addition, saturation increment, etc.) shows that VGRnSFB was the most robust and accurate HFMF with an overall accuracy $>90\%$, less focus error, and false maxima.

3.4.

Discussion

The main objective of this study is to propose the most accurate robust HFMF applicable to all the imaging modalities. Eight FMFs, evaluated in this study, were earlier implemented in different applications, such as CM, FM, and shape from focus.^1,2,28 VCR, BGR, and ELP were reported as the best FMFs in CM.¹ Mateos-Pérez et al.² established that midfrequency discrete cosine transform (96.67%), VCR (89%), and TGR (89%) FMFs performed better in FM images. Pertuz et al.²⁸ found that Laplacian-based operators were outperformed when preprocessing was not applied. Zukal et al.¹⁶ proposed interest point detection-based FMFs (the Harris–Laplace detector, fast Hessian detector, and the features from accelerated segment test detector) for MS datasets. Performance of these methods is poor on VS and IS images, and only fast Hessian detector performed better on TS datasets. None of the previously reported FMFs were consistent to diverse imaging modalities, such as ZN, FM, and MS images. These inconsistencies of results have emphasized the importance of robust HFMFs that may capture focused images automatically irrespective of imaging system.

The performances of 36 (twenty-eight hybrid and eight individual) HFMFs were evaluated on the datasets covering diverse image contents with high, medium, and low density backgrounds and lack of sharp edges in images. Initially, 19 HFMFs provided an overall accuracy rate $\ge 90\%$. VGRnSFB HFMF has been identified as the most robust and consistent after evaluating performance in different imaging conditions, such as noise addition, contrast reduction, saturation increment, and uneven illumination. VGRnSFB HFMF has also shown consistent performance in all three modalities of MS dataset (100%, 100%, and 92.8% accuracies without preprocessing for TS, IS, and VS, respectively). An efficient HFMF has lots of application potential as it is easier to implement when intervention of preprocessing and other requirements is minimal. Better performance of VGRnSFB is significant as there was no HFMF reported earlier that was robust to ZN, FM, and MS images simultaneously.

Finally, the sharpness of the focus curve was evaluated for eight individual FMFs (GDR, GNV, HELM, SFB, SFM, TGR, VCR, and VGR) and an HFMF (VGRnSFB) (Fig. 9). VGRnSFB, VGR, and SFB are rapidly converged to the best focus position. Though the sharpness curve of VGR is better, the VGRnSFB HFMF curve is comparable to it and found to be suitable for implementation in real systems. This HFMF has also produced a comparable sharpness curve in MS images (Fig. 10 of Appendix).

Fig. 9

Sharpness curve of nine FMFs including HFMF (VGRnSFB). Narrow peak represents rapid convergence of FMF. (a) VGRnSFB (HFMF), VGR, and SFB were rapidly converged to the best focus position in ZN (CM) images and (b) VGRnSFB and VGR were rapidly converged to the best focus position in FM images.

Fig. 10

Sharpness curve of nine FMFs including VGRnSFB HFMF on MS datasets. Narrow peak represent rapid convergence of FMF. (a) Sharpness curve on headphones images of VS, (b) sharpness curve on headphones building images of near-IS, and (c) sharpness curve on breaker images of TS.

Based on content, the images for autofocusing could be categorized into three broad categories, namely low, medium, and high density background. A current study contains a stack of images from diverse modalities of MS, FM, and ZN with low, medium, and high densities. It was evident from the earlier studies that the performances of FMFs were dependent on imaging contents and varied when the focus area changed.^1,2,16,21 However, the current study (VGRnSFB FMF) achieved better and consistent results in all five diverse imaging modalities, such as ZN, FM, and MS (TS, IS, and VS) (Figs. 4Fig. 5Fig. 6Fig. 7–8). Furthermore, various parameters such as accuracy and focus error have indicated that VGRnSFB is robust in different experimental setup and imaging conditions. However, HFMFs performance could be validated on other image modalities for their universal applications. In the future, the effectiveness of these HFMFs on live imaging techniques may be evaluated for their applications in detecting objects or microorganisms, which are not static.³⁰ The region sampling incorporated in this study has increased the accuracy of FMFs as well as HFMFs. As computation time is not a major factor for FMF evaluated in this study, image compression by means of subsampling has not been performed and may be evaluated in the future to optimize the performance of HFMFs. Nonetheless, the HFMFs have outperformed individual FMFs in all the tested imaging modalities, which are diverse and have low, medium, and high density backgrounds with different levels of noise.