Digital image processing is at the base of everyday applications aiding humans in several fields, such as underwater monitoring, analysis of cultural heritage drawings, and medical imaging for computer-aided diagnosis. The starting point of all such application regards the image enhancement step. A desirable image enhancement step should simultaneously standardize the illumination in the image set, possibly removing bad or not-uniform illumination effects, and reveal all hidden details. In 2002, a successful perceptual image enhancement model, the automatic color equalization (ACE) algorithm, was proposed, which mimics the color and contrast adjustment of the human visual system (HVS). Given its widespread usage, its correlation with the HVS, and since it is easily implementable, we propose a scoping review to identify and classify the available evidence on ACE, starting from the papers citing the two funding papers on the algorithm. The aim of this work is the identification of what extent and in which ways ACE may have influenced the research in the color imaging field. Thanks to an accurate process of papers tagging, classification, and validation, we provide an overview of the main application domains in which ACE was successfully used and of the different ways in which this algorithm was implemented, modified, used, or compared.

This guest editorial summarizes the Special Section on Perceptually Optimized Imaging.

We propose a video stabilization algorithm based on the rotation of a virtual sphere. Unlike traditional video stabilization algorithms relying on two-dimensional motion models or reconstruction of (3D) camera motions, the proposed virtual sphere model stabilizes video by projecting each frame onto the sphere and performing corrective rotations. Specifically, matching feature points between two adjacent frames are first projected onto two virtual spheres to obtain pairs of spherical points. Then, the rotation matrix between the previous and current frame is calculated. The resulting 3D rotation matrix sequence is used to represent the camera motion, and it is smoothed using the geodesic distance on a Riemannian manifold. Finally, the difference between the smoothed and original path allows obtaining the rotation matrix that causes camera jitter, and the virtual spheres are rotated reversely to suppress jitter. Experimental results show that the proposed algorithm can effectively reduce random jitter, outperforming state-of-the-art methods.

Single-image super-resolution (SISR) refers to reconstructing a high-resolution image from given low-resolution observation. Recently, convolutional neural network (CNN)-based SISR methods have achieved remarkable results in terms of peak-signal-to-noise ratio and structural similarity measures. These models use pixel-wise loss functions to optimize their models, which results in blurry images. However, the generative adversarial network (GAN) has the ability to generate visually plausible solutions. The different GAN-based SISR methods obtain perceptually better SR results when compared to that with the existing CNN-based methods. However, the existing GAN-based SISR methods need a large number of training parameters in the architecture to obtain better SR performance, which makes them unsuitable for many real-world applications. We propose a computationally efficient enhanced progressive approach for SISR task using GAN, which we referred as E-ProSRGAN. In the proposed method, we introduce a novel design of residual block called enhanced parallel densely connected residual network, which helps to obtain better SR performance with less number of training parameters. The quantitative performance of the proposed E-ProSRNet (i.e., generator network of E-ProSRGAN) model is better for higher upscaling factors ×3 and ×4 for most of datasets when compared to the same obtained using different CNN-based methods whose trainable parameters are less than 7 M. In the case of upscaling factor ×2, E-ProSRNet obtains second highest structural similarity index measure values for Set5 and BSD100 datasets. The proposed E-ProSRGAN model generates SR samples with better high-frequency details and perception measures than that of the other existing GAN-based SISR methods with significant reduction in the number of training parameters for larger upscaling factor.

Single-pixel camera is developed to mitigate the constraints faced by the conventional cameras especially in invisible wavelengths and low light conditions. Nyquist–Shannon theorem requires as many measurements as the image pixels to reconstruct images flawlessly. In practice, obtaining more measurements increases the cost and acquisition time, which are the major drawbacks of single-pixel imaging (SPI). Therefore, compressive sensing was proposed to enable image reconstruction with fewer measurements. We present a design of sensing patterns to obtain image information by utilizing spatially variant resolution (SVR) technique in SPI. The proposed method reduces the measurements by prioritizing the resolution in the region of interest (ROI). It successfully achieves the programmable imaging concept where multiresolution adaptively optimizes the balance between the image quality and the measurements number. Results show that SVR images can be reconstructed from significantly fewer measurements yet able to achieve better image quality than uniform resolution images. In addition, the SVR images can be further enhanced by integrating the dynamic supersampling technique. Consequently, the concerns of image quality, long acquisition, and processing time can be addressed. The proposed method potentially benefits imaging applications where the target ROI is prioritized over the background and most importantly it requires fewer measurements.

Haze scatters light transmitted in the air and reduces image quality, which greatly decreases the interpretability and intelligibility of an image. To solve these problems, we propose an improved real-time image dehazing algorithm based on dark channel prior and fast weighted guided filtering. First, the image is divided into dark areas and bright areas by the K-means clustering algorithm, and the atmospheric light value is calculated according to the proportion of the bright area in the whole image. Second, the fast weighted guided filtering algorithm is employed to generate a refined transmission map, which removes the halo artifact from around the sharp edges. Finally, the gamma correction and automatic contrast enhancement algorithms are used to adjust the brightness and contrast of the dehazing image. Experimental results demonstrate that the proposed method can effectively remove the halo artifacts, improve the color deviation, and retain more details in the images.

We propose accelerated implementations of bilateral filter (BF) and nonlocal means (NLM) called color-compressive bilateral filter (CCBF) and color-compressive nonlocal means (CCNLM). CCBF and CCNLM are random filters, whose Monte-Carlo averaged output images are identical to the output images of conventional BF and NLM, respectively. However, CCBF and CCNLM are considerably faster because the spatial processing of multiple color channels are combined into a single random filtering process. This implies that the complexity of CCBF and CCNLM is less sensitive to color dimension (e.g., hyperspectral images) relatively to other BF and NLM methods. We experimentally verified that the execution time of CCBF and CCNLM are faster than the existing “fast” implementations of BF and NLM, respectively.

The new video coding standard, known as versatile video coding (VVC) is projected to be concluded by the end of 2020. This standard is conducted mainly to address 8k videos and emerging applications such as 360 deg and high dynamic range. Intraprediction is a part of the prediction step in the video coding that exploits spatial redundancy. This module has been improved, compared to the high-efficiency video coding (HEVC), by increasing the set of angular intraprediction modes (IPM) from 33 to 65 to model directional textures more accurately. Moreover, a quadtree plus binary tree (QTBT) structure replaced the QT of the HEVC. These improvements targeting at enhancing the coding efficiency resulted in significant coding complexity, especially in terms of encoding time. This paper fits into this context. It evokes the optimizations of the intramode and coding unit size decisions using statistical methods of fast decision and deep learning. A fast intramode decision algorithm is proposed for the different binary depths of the QTBT structure. Thus, an optimization by deep learning for square blocks is also included. Results show that the combinations of these two approaches can significantly reduce the complexity of the VVC encoder. Under the all intra (AI) configuration, a reduction of about 61.04% of the intraencoding time is achieved while maintaining an acceptable rate-distortion performance.

Face aging and rejuvenating work effectively in public security criminal investigation, cross-age recognition, and entertainment. However, three main problems still exist: the lack of accurate and sufficient dataset, low aging effect, and poor preservation of personal information. We propose a semi-supervised face aging and rejuvenating method for face aging and rejuvenating. In particular, a conditional encoder is utilized to map an input face into a latent vector, which is used by the generator network with age conditions to produce a new face. The latent vector preserves identity information, whereas the age label controls face aging or rejuvenating. To make generated features closer to prior features, the discriminator network is designed to assist the generator network. In addition, a cycle optimized method is utilized to preserve the personal information of the generated face. Experimental results demonstrate that our network can generate more realistic faces, both in personal identity and age consistency.

A multi-secret image sharing (MSIS) scheme is a procedure to break down secret images into numerous shares and distribute each share to each authorized participant. Most existing   (  n  ,  n  )   schemes face the problem of all-or-nothing that is all the n shares are required to reconstruct a secret image. If a single share is lost, then the secret image will not be recovered. Moreover, the existing   (  k  ,  n  )   MSIS schemes either require k consecutive shares or general access structure to recover the secret images. The proposed scheme addresses the previous schemes’ issues. It is based on Arnold cat map, Chinese remainder theorem (CRT), and Boolean operation. It is a   (  k  ,  n  )   threshold scheme where k is the threshold, and n is the number of participants. Arnold cat map is adopted for the randomization of images, and Boolean operation is used for producing public share images. CRT is utilized for the recovery of images. It has high fault tolerance capability as any of the k participants can submit their shares to reconstruct images and consume less computational time due to employing the modular method. Moreover, it can withstand differential as well as statistical attacks.

Video saliency detection, which suffers from the interference of complicated motion and complex background, aims at discovering the motion-related and the most noticeable object in a video sequence while maintaining the spatiotemporal consistency of saliency maps. We propose a video saliency detection approach by constructing the moving object properties, meaning that motion is in concurrence with a general object for detected regions. With the assistance of different key frame strategies, the consistency propagation, which is respectively conducted to refine the obtained results of motion saliency and object proposals to improve low-quality detections and avoid the excessive accumulation of errors, is orderly implemented through sparse reconstruction based on the constructed adjacent relationships. Meanwhile, by integrating the refined object proposals with the refined salient motion detection, a pixel-based fusion strategy is applied to locate the most trustworthy motion object regions and suppress detection noises, such as dynamic background and stationary objects. Moreover, a Bayesian fusion framework that incorporates the global features obtained using low-rank constraints is employed to further enhance the accuracy and global temporal consistency for the obtained initial motion object regions. Then based on the obtained priors of motion objects and background regions, the spatial saliency map is estimated using the geodesic distances transform to discover more complete and spatially salient motion objects. Finally, an energy optimization function is proposed to intuitively integrate multiple saliency clues (i.e., spatial saliency, global features, and motion objects) and generate the global spatiotemporal consistency saliency map. Experimental results on three different benchmark datasets demonstrate that the proposed method successfully infers video moving objects and extracts the most salient regions, outperforming the state-of-the-art methods.

Robust steganography enables secret information to be transmitted stealthily and accurately in lossy channels such as social channels and wireless channels. With the development of deep learning, robust steganography can be constructed using the generative model of deep neural networks. Two new robust steganographic frameworks are proposed on the basis of generative models, and two algorithms are proposed on these two frameworks to verify the effectiveness of the proposed framework. Experiments show that the two frameworks proposed are more flexible than existing robust steganographic frameworks. To further verify the validity of the framework, when compared with existing robust steganography based on deep learning, the generative robust steganography algorithm is shown to have a higher secret information embedding capacity and higher steganography image quality.

Sparse prior to image denoising is a classical research field with a long history in computer vision. We propose an end-to-end supervised neural network, named DnMLCSC-net, which is inspired via multi-layer convolutional sparse coding model embedded with symbiotic analysis–synthesis priors for natural image denoising. Unfolding a multi-layer, learned iterative soft thresholding algorithm (ML-LISTA) and developing into a convolutional recurrent neural network, all parameters in the model are updated adaptively to minimize mixed loss via gradient descent using backpropagation. In addition, a combined ReLU function is taken as the activation function. Inconsistent dilated convolution and batch normalization were empirically introduced into the encoding layers corresponding to the first iteration of ML-LISTA. Experimental results show that our network achieves a competitive denoising effect in comparison with several state-of-the-art denoising methods.

3D object recognition from point clouds is a fast-growing field of research in computer vision. 3D object recognition methods can be classified into two categories: global feature-based and local feature-based methods. The local feature-based methods are more popular than global ones. Because the global feature-based methods need a prior segmentation of the scene, they are not suitable for real-world scenes. Many previous local descriptor methods limit their performance by introducing a local reference frame or axis (LRF/A). Estimating the LRF/A for each keypoint leads to extra computational time and error. We use the fundamental theorem of surface theory to introduce a simple and efficient local feature descriptor based on the coefficients of discrete first and second fundamental forms. The proposed method overrides the necessity of an LRF/A, and it required a small feature dimension of seven, which means it is a low-complexity and fast procedure. To assess the proposed method, we have compared it with eight state-of-the-art descriptors and applied it to the three popular datasets to extract features and recognize the correspondences. Experimental results demonstrate the superiority of the proposed approach to the compared methods in terms of pairwise registration measurements, recall versus 1-precision curve, and the computational time.

Rapid and accurate detection of crowd abnormal events, such as stampedes and violent attacks in public places, has great research significance and application value. Due to the diversity and uncertainty of abnormal events, almost all existing methods tackle the problem by minimizing the reconstruction errors of training data, which cannot guarantee a larger reconstruction error for all abnormal events. According to the idea that “normal events can be predicted, abnormal events cannot be predicted,” we proposed a future frame prediction-based anomaly detection algorithm. First, the generative adversarial network (GAN) is trained by the normal videos to predict normal future frames. Then, it can determine the existence of abnormal events by identifying the difference between the ground truth and predicted video frame. In the design of the GAN, the attention module is introduced to improve the prediction level of the network. At the same time, the optical flow information is added for motion constraint to improve the constraint ability on the appearance characteristics. In the testing stage, the appearance gap and optical flow gap between the ground truth and the predicted video frame are fused to determine whether the frame is abnormal. The experimental results on the datasets of CUHK Avenue, UCSD, and ShanghaiTech show that the proposed algorithm is superior to that of the current mainstream anomaly detection algorithms.

Underwater images suffer from haze, color distortion, and low contrast due to the absorption and scattering of transmitted light. A systematic approach including accurate background light (BL) estimation and transmission (TM) estimation is proposed based on color-line model for underwater images restoration. The estimation of BL candidate region is improved by combining quadtree subdivision and minimum unary gray entropy evaluation, to realize a high-quality restoration and improve accuracy of TM estimation with color-line model. Given incomplete intersection of color lines with BL, a convex optimization function is proposed to improve accuracy of TM estimation. On the basis of restorations with BL and TM estimations, pixel distribution stretching and Unsharp Masking are employed to correct color and enhance edge information for better performance of vision. Comprehensive evaluations illustrate that the proposed method could efficiently restore underwater images with visibility improvement, natural color correction, and contrast enhancement.

Image retrieval based on deep learning of hash has made great progress. The hash method increases retrieval speed greatly while saving storage space. However, some problems exist, such as the distinctiveness of image feature still needs to be improved and some image features are still redundant. We propose a new deep learning to hash method, namely, deep multiscale divergence hashing, which provides a high diversity and compact image feature for image retrieval. The discriminative features from deep neural networks are identified by the importance criterion in network pruning techniques and the feature redundancy is reduced. Then, the selected features across different layers are fused in a certain proportion to increase the diversity of features for image retrieval. We also present a hybrid loss function in hash space, which consists of the weighted pairwise cross-entropy loss function and the KL-divergence. It tends to minimize the hamming distance between similar images and maximize the hamming distance between different images, which helps improve the accuracy. Massive experimental results show that our method achieves better feature distinguishability and more advanced image retrieval accuracy, and surpasses HashNet by 11.46%, 7.58%, and 13.86% on MS COCO, NUS-WIDE, and CIFAR-10 datasets, respectively.

Currently, there is an increasing demand for wooden furniture and products, especially in the decorating business where wood textures are widely used. These textures have different artificial designs, and because different consumers have different needs for wood texture, the trend of using computer algorithms to design wood textures emerged. We propose a method for synthesizing wood’s heterogeneous texture. It can analyze the characteristics of different wood textures, select the most appropriate input sample block size, and then generate a new image with a sample texture. Compared with the deep learning method, our method reduces pressure on system resources and production costs. The proposed method also generates higher quality reconstructed images than traditional algorithms.

The denoising of interferometric phase images attracts many researchers. Natural image denoising algorithms based on neural networks are often proposed in the development of deep learning methods. A neural network, In-CNN, is derived from an advanced natural image denoising network and proposed for interferometric phase image denoising. Preactivation and residual learning methods are combined and applied to the function of the neural network nodes. Considering the particularity of the interferometric phase image, we propose a neural network based on the rational application of the preactivation mode and feedforward mapping, which is different from previous natural image denoising networks. We also construct a training set for an interferometric phase image denoising neural network. We experimentally verify that our model performs better than state-of-the-art interferometric phase image denoising methods based on sparse representation and advanced natural image denoising networks. We discuss the complexity of traditional interferometric phase image denoising algorithms to demonstrate the efficiency of the proposed method.

Few-shot learning aims to learn classification with only a few training examples per class. The metric-based approaches aim to learn a set of embedding functions, so that when represented in this embedding, images are easy to recognize. For metric-based few-shot learning, how to get the class feature representation under a few support samples and what metric to use are important. We propose a multiclass triplet metric-learning network combined with a simple foreground–background feature mixing block. With the foreground–background feature mixing block, we “hallucinate” the information from few support examples to get conceptual representation of classes, which is effective to promote few-shot learning. Furthermore, using the multiclass triplet loss, it learns a feature embedding function that could bring similar samples close to each other and keep samples of different classes far apart. Our proposed network is trained in an end-to-end manner from scratch, so as to learn a good embedding function, conceptual representation of classes, and a nonlinear metric simultaneously. Experimental results on the challenging datasets show that our method with Conv-64F feature extracting block is competitive and effective compared to the metric-based baselines with Conv-64F.

Recently, anchor-free methods have brought new ideas to the field of object detection that eliminate the need for anchor boxes in object detection and provide a simpler detection structure. CenterNet is the representative anchor-free method. However, this method still has the problem of obtaining high-resolution representation from low-resolution representation using upsampling, and the predicted heatmap is not accurate enough in space and does not make full use of the shallow low-level features of the network. We introduce CenterNet-HRA to solve this problem. An attention module is proposed to calibrate the high-level semantic features of the network output using the shallow low-level features from different receptive fields; HRNet is used as the backbone to maintain high-resolution feature representation through the whole process rather than using upsampling to generate high-resolution feature representation as HourglassNet. Considering that the feature representations with different resolutions have different contributions to the network but HRNet fuses them without distinction, a novel weighted feature fusion HRNet is designed to achieve higher detection precision. Our method achieves an average precision (AP) of 42.3% at 13.5 frames-per-second (FPS) (40.3% AP at 13.3 FPS for CenterNet-HG) on the MS-COCO benchmark.

We present an image inpainting technique using frequency-domain information. Prior works on image inpainting predict the missing pixels by training neural networks using only the spatial-domain information. However, these methods still struggle to reconstruct high-frequency details for real complex scenes, leading to a discrepancy in color, boundary artifacts, distorted patterns, and blurry textures. To alleviate these problems, we investigate if it is possible to obtain better performance by training the networks using frequency-domain information (discrete Fourier transform) along with the spatial-domain information. To this end, we propose a frequency-based deconvolution module that enables the network to learn the global context while selectively reconstructing the high-frequency components. We evaluate our proposed method on the publicly available datasets: celebFaces attribute (CelebA) dataset, Paris streetview, and describable textures dataset and show that our method outperforms current state-of-the-art image inpainting techniques both qualitatively and quantitatively.

Self-supervised learning of depth from monocular videos has recently drawn attention as it has notable advantages over supervised ones in a training framework. We propose a self-supervised monocular depth estimation method with a set of improvements, which together result in both quantitatively and qualitatively improved depth maps compared to competing self-supervised methods. Our architecture amends current deep convolutional neural network backbone combined with attention mechanism to boost depth estimation performance. Additionally, for addressing moving objects and occlusion, we propose a learnable outlier-masking technique to exclude invalid pixels in photometric error map. Extensive experiments show the effectiveness of the proposed improvements. Our proposed model achieves state-of-the-art performance on KITTI dataset compared with other competing methods.

Near-infrared (NIR) face recognition (FR) has demonstrated robustness against changes in ambient illumination, which makes it suitable for surveillance even under weak illumination conditions. However, the existing database for NIR FR only contains frontal face images, and the impact of pose variation on the robustness of NIR FR remains unascertained. We developed an NIR face database with 57 pose variations in a dark environment, which can be used in pose-invariant FR research. Convolutional neural networks (CNNs) were designed and tested in comparison to the traditional method in the database. The experimental results showed that a difference of even 10 deg between the gallery and testing sets can dramatically reduce the recognition performance. Additionally, an average accuracy of 90.58% was obtained for pose-invariant recognition by employing more pose variations in the gallery set using the CNN-based method.

A deep learning framework is proposed to recognize large-scale plant species by integrating attention-based deep feature extraction network and plant taxonomy-guided path-based tree classifier. First, a plant taxonomy is constructed for organizing large-scale fine-grained plant species hierarchically in a coarse-to-fine fashion. Second, a deep learning framework is proposed, where attention mechanism is used to remove useless feature components and a plant taxonomy-guided path-based two-layer tree classifier is used to replace the flat softmax classifier in traditional deep convolutional neural network structure. Furthermore, a specific path-based loss function and back-propagation method are proposed to optimize the weight parameters in both deep network and tree classifier. Experimental results on the Orchid2608 plant dataset can also prove that proposed deep attention network with path-based tree classifier can achieve improvements on large-scale plant species identification task.

Underwater fish recognition is an important task in fish stock assessment and marine ecosystem studies. Machine learning techniques have been applied to train high-performance fish recognition models from underwater images. However, underwater images often contain extremely noisy backgrounds, hindering the training of accurate recognition models. Traditional methods exploit handcrafted features to train traditional classifiers. These methods often suffer from low recognition accuracy and limited scalability to large-scale datasets. While deep learning approaches have been proposed, the challenge of learning with noisy underwater images has not yet been fully addressed. We propose a discriminative feature learning (DFL) framework to train accurate fish recognition models on noisy underwater images. By leveraging the idea of contrastive learning, DFL encourages the model to learn more discriminative features for images in different classes and similar features for images in the same class. To better address the noisy background problem, DFL also utilizes a regularization technique called attention suppression to prevent the model from paying too much attention to the noisy background. Experimental results on three benchmark datasets validate the superior performance of DFL over the current state-of-the-art deep learning approaches.

Deep convolutional neural networks (CNNs) have achieved considerable success in terms of image denoising. However, previous CNN denoisers have been restricted by rigid kernel convolution that applies equal spatial treatment across images. To fully utilize the local differences, we propose a kernel prediction network that examines each pixel region and predicts unique pixel-wise kernels. Several optimizations have been further designed to gather sufficient information for single-image denoising task. We adopt dilated residual blocks to view the local pixel region at varying receptive fields. Then, kernel fusion assembles the information from different scopes and generates accurate kernels for each pixel. Instead of applying the predicted kernels to the original image, we construct a compressed feature map as a substitution such that more relevant local features are collected. Experiments are used to demonstrate that our network achieves favorable results compared with state-of-the-art methods and is adequate for practical applications.

Over the past few decades, we have witnessed a large family of algorithms that has been designed to provide different solutions to the problem of dimensionality reduction (DR). DR is an essential tool to excavate important information from the high-dimensional data by mapping the data to a low-dimensional subspace. Even though most DR algorithms can achieve satisfactory performance for many practical applications, they fail to fully consider the information from multiple views. Therefore, how to learn the subspace for high-dimensional features by utilizing the consistency and complementary properties of multi-view features is important and challenging in the present. To tackle this problem, we propose an effective multi-view DR algorithm named multi-view smooth representation preserve projection (MvSMRP²). MvSMRP² seeks to find a set of linear transformations to project each view features into one common low-dimensional subspace where the multi-view smooth reconstructive weights are preserved as much as possible. A pair-wise-consistent scheme is designed by fully exploiting the Hilbert–Schmidt independence criterion to maximize the dependence among different views, which can obtain one common subspace and force multiple views to learn from each other jointly. Moreover, MvSMRP² can allocate ideal weight for each view automatically according to view importance without explicit weight definition. Finally, an iterative alternating strategy is proposed to obtain the optimal solution of MvSMRP². Plenty of experiments on various datasets show the excellent performances of the proposed method.

Neural networks are vulnerable to various adversarial perturbations added to the input. Highly sparse adversarial perturbations are difficult to identify, which is especially dangerous to network security. Previous research has shown that ℓ₀-norm has good sparsity but is challenging to solve. We use ℓ_q-norm to approach ℓ₀-norm and propose a new white-box algorithm to generate adversarial examples aiming at minimizing ℓ_q distance of the original image. Meanwhile, we extend the adversarial attack to facial anti-spoofing task in the field of face recognition security. This extension enables us to generate sparse and unobservable facial attack perturbation. To increase the diversity of the data set, we make a new data set of real and fake facial images containing images produced by various latest spoofing methods. Extensive experiments show that our proposed method can effectively generate a sparse perturbation and successfully mislead the classifier in multi-classification tasks and facial anti-spoofing tasks.

Recently, the attention mechanism and part-based architectures have greatly boosted the research of person re-identification (Re-ID). However, most attention-based works extract first-order information and lack diversity. Meanwhile, the classic part-based works are not able to make use of cross-part information because of the unified partitions. These two kinds of methods ignore visual relationships in the global scope. Accordingly, we propose multi-subspace non-local attention (MSNA) and reinforced loss (R-Loss) to alleviate the issues above. MSNA is an improved attention module. It can be integrated into existing networks to utilize rich low-level information and extract the global relationships from different subspaces. R-Loss module is motivated to reinforce the capability of extracting fine-grained features by making full use of intra-part and cross-part information. We combine them and provide a global-locally reinforced feature extraction strategy. In addition, we design a feature fusion module to combine features from different branches. Equipped with the modules above, our model can extract important local and fine-grained features by identifying diverse visual relationships in the global scope. The models with our proposed modules achieve significant improvements over the baselines on four public datasets and establish new state-of-the-art results.

Intensity inhomogeneity, which is also called bias field, is ubiquitous in digital images. The causes of intensity inhomogeneity are complex and include uneven illumination and defects of imaging equipment. For images with local intensity inhomogeneity, an array of existing segmentation algorithms has poor performance on efficiency, accuracy, or initial robustness. To tackle this problem, an active contour model based on local prefitting bias estimation is proposed. The bias field is approximated through a new function based on a mean filtering algorithm, which can credibly represent the distribution of bias field of an input image. Then, the bias field is incorporated into the optimized energy functional based on the level set method to implement the segmentation process. Specifically, the bias field is computed before iterations and the mean filtering algorithm is much faster than traditional clustering algorithm, so the efficiency is greatly raised. Moreover, a new regularization function is formulated to improve the robustness of the initial contour and noise. Comparing with some traditional models, the proposed model achieves better results on some challenging images.

Recently, data hiding and encryption methods have been very important for transmitting information to the relevant people correctly and reliably. Although each newly developed computer-aided data hiding and encryption method is stronger than previous methods, there may be security gaps or speed factors in future use. Therefore, faster and more secure encryption methods are needed. Chaotic systems can be used for high-performance random number generators because their frequency spectrum is wide. Moreover, they show extreme sensitivity to the initial conditions and eliminate the need for large sized keying. This means that a faster and more secure encryption can be made with random numbers generated from a chaotic system. All of these features make chaotic systems stand out in data hiding and encryption applications. Chaos-based image encryption and decryption applications were performed differently than in traditional encryption algorithms in the literature. For the application, a three-dimensional chaotic system was discretized by the Runge–Kutta-4 method, and random numbers were generated. These random numbers were tested by the statistical NIST-800-22 tests, the international top random standard, and their suitability was determined. With the random numbers that passed the tests successfully, the image was successfully encrypted and sent via e-mail; then the encrypted image was decrypted back to the original image by the receiver. Finally, an electronic circuit realization of the chaotic encryption approach was depicted using LabVIEW to confirm the operability of the application.

Noise reduction is an essential preprocess in applications of complex-valued images. A method of complex-valued image denoising based on complex-valued dictionary learning and an adaptive complex-valued dictionary filter (OCDL-ACDF) is proposed. Our dictionary is first trained by the online dictionary learning method. Then, to further reduce the noise contained in the dictionary atoms, we design a complex-valued dictionary filter based on the feature similarity between the atoms of redundant dictionaries. By combining the advantages of online dictionary learning and denoising methods of real-valued images, an effective complex-valued dictionary is obtained. The orthogonal matching tracking method, which is a greedy algorithm, is used in the process of sparse coding. The simulation experiments show that the denoising effect of the proposed method is not only better than the current advanced algorithms but also effective at avoiding overfitting. The detail fidelity was also relatively high.

Image inpainting attempts to fill the missing areas of an image with plausible content that is visually coherent with the image context. Semantic image inpainting has remained a challenging task even with the emergence of deep learning-based approaches. We propose a deep semantic inpainting model built upon a generative adversarial network and a dense U-Net network. Such a design helps achieve feature reuse while avoiding feature explosion along the upsampling path of the U-Net. The model also uses a composite loss function for the generator network to enforce a joint global and local content consistency constraint. More specifically, our new loss function combines the global reconstruction loss characterizing the semantic similarity between the missing and known image regions with the local total variation loss characterizing the natural transitions among adjacent regions. Experimental results on CelebA-HQ and Paris StreetView datasets have demonstrated encouraging performance when compared with other state-of-the-art methods in terms of both quantitative and qualitative metrics. For the CelebA-HQ dataset, the proposed method can more faithfully infer the semantics of human faces; for the StreetView dataset, our method achieves improved inpainting results in terms of more natural texture transitions, better structural consistency, and enriched textural details.

We propose a dual-band fusion method using multiscale visual saliency extraction based on spatial weight matrix. There are two main contributions in this paper. The first major contribution is to use the local window based on the gray distance of the spatial weight for saliency extraction. Second, to emphasize those potential targets and details with different sizes from source images, we based on the saliency extraction method of multi-window and fusion of different scales to achieve the preservation of more information. The proposed method is mainly divided into four steps. First, we use a spatial weight matrix of different window sizes to enhance targets of different sizes in the image. Then, through the different processing of each enhanced image to get detailed images, we use a special method to fuse the obtained details of each scale. Then, reconstruct the results obtained at each scale. Finally, we have the exact weight index selection to get better fusion results. This method solves the problem of improper weight selection and the result deteriorates. Through comparison and verification, our results retain more detailed information from the original images.

The most common method for video-level classification is to operate models over a whole series of fixed-temporal-length clips in a long video. However, a video usually consists of several meaningless sections, and similar clips that are fed into models could lead to a suboptimal recognition accuracy and waste of computing resources. To address this issue, we introduced an adaptive discriminative clip identification network to evaluate every video clip with respect to its relevance. We adaptively choose top-ranked clips as inputs for prediction and filter out irrelevant clips. Specifically, for a given trained cumbersome convolutional neural network action recognition model, we use a lightweight hallucination network (H-Net) to study its generalization ability based on distillation. Then, the evaluation of relevant clips is performed by considering the imitated features of H-Net. Thus, heavy calculations of overlapping content or meaningless clips can be avoided during inferences. We validate our approach by examining its performance on two datasets: UCF101 and HMDB51. Our strategy can be applied to any clip-based action recognition model. Experimental results demonstrate that on UCF101, we reduced the computational cost by 70% and increased the accuracy.

We present an effective blind image deblurring method based on the reweighted L₁ norm prior. The prior is motivated by that the traditional L₁ norm highly depends on the pixel itself, that is, the larger the pixel value, the greater the penalty. However, the blur reduces the high-frequency components of the clear image, and minimizing the high-frequency part will result in a blur solution or delta function kernel. To overcome this limitation, we employ the reweighted L₁ norm and it eliminates this dependence within wisely weighting. The image prior compensates for the degeneration of high intensities and greatly stabilizes the intermediate image estimation process. However, the prior proposed introduces a challenging optimization problem. We develop an efficient optimization scheme to obtain a reliable intermediate image for estimating the blur kernel. Extensive experiments on different kinds of challenging blurry images demonstrate the superiority of our proposed method over the state-of-the-art blind deblurring methods. Moreover, our blind deblurring algorithm is effective in various scenarios, such as natural, text, and low-light images.

We present a gradient-domain image reconstruction framework with the chroma-preserving pixel-wise intensity-range constraint and the base-structure constraint. The existing methods for manipulating base structures and detailed textures are classifiable into two major streams: gradient-domain and layer-decomposition. To generate detail-preserving and artifact-free output images, we combine the two approaches’ benefits into the proposed framework by introducing the chroma-preserving intensity-range constraint and the base-structure constraint. To preserve details of the input image, the proposed method takes advantage of reconstructing the output image in the gradient domain, whereas the output intensity is guaranteed to lie within the specified intensity range by the intensity-range constraint. The reconstructed image lies close to the base structure by the base-structure constraint, which is effective for restraining artifacts. Using this chroma-preserving pixel-wise luminance constraint, the proposed algorithm does not require post-processing such as intensity clipping or rescaling. The proposed framework directly generates the output luminance, which guarantees that the output RGB intensities are within the target intensity range, preserving the chromatic component. Experiments demonstrated that (1) the proposed framework is effective for various applications such as tone mapping, seamless image cloning, detail enhancement, and image restoration, and (2) the proposed framework can preserve chroma components compared with the existing methods.

Unexpected situations, such as object movement and camera shaking, cause interference and produce blurry, degraded images. These degraded images can adversely affect visual processing tasks, such as target detection and facial recognition. In addition, the super-resolution (SR) of a single image is a basic task to improve the image quality in visual applications. However, due to the existence of artifacts, it is difficult to generate clear SR images from rough and degraded images. In this work, we propose a double-branch projection feedback network that can generate clear high-resolution (HR) images from severely blurred low-resolution (LR) images. One is a deep feature projection feedback branch that can obtain texture-rich SR feature maps and reduce the accumulative errors with depth, and a channel adjustment and multi-scale feature fusion mechanism are proposed to better obtain the mapping relation between LR and HR images. The other is an autoencoder deblurring module, which is used to get deblurred features of an input image. Moreover, a mapping associated feature block is applied to match non-local feature information in the feature fusion module. Extensive experiments show that the network is effective and performs better than the state-of-the-art methods in single image deblurring and SR.