Image inpainting techniques based on deep learning have shown significant improvements by introducing structure priors, but still generate structure distortion or textures fuzzy for large missing areas. This is mainly because series networks have inherent disadvantages: employing unreasonable structural priors will inevitably lead to severe mistakes in the second stage of cascade inpainting framework. To address this issue, an appearance flow-based structure prior (AFSP) guided image inpainting is proposed. In the first stage, a structure generator regards edge-preserved smooth images as global structures of images and then appearance flow warps small-scale features in input and flows to corrupted regions. In the second stage, a texture generator using contextual attention is designed to yield image high-frequency details after obtaining reasonable structure priors. Compared with state-of-the-art approaches, the proposed AFSP achieved visually more realistic results. Compared on the Places2 dataset, the most challenging with 1.8 million high-resolution images of 365 complex scenes, shows that AFSP was 1.1731 dB higher than the average peak signal-to-noise ratio for EdgeConnect.
At present, two-stage networks are widely used in image restoration methods, but existing two-stage network often generates inpainting results with distorted structures and blurry textures, especially when reconstructed object is more complex. The main reason is insufficient structure prior and inaccurate, which leads to generating wrong results in texture generation stage. In order to solve this problem, a novel Image Inpainting based on Edge and Smooth Structures Prediction is proposed. The edge structure and smooth structure are completed in structure reconstruction stage, and reconstructed edge structure and smooth structure are simultaneously used as a prior to guide texture generation stage fills in damaged area. The proposed method is evaluated on publicly available datasets Paris StreetView, CelebA-HQ and Places2, and many experiments show that proposed method obtains excellent results under subjective and objective indexes compared with mainstream approaches.
Tensor ring (TR) decomposition is an effective method to achieve deep neural network (DNN) compression. However, there are two problems with TR decomposition: setting TR rank to equal in TR decomposition and selecting rank through an iterative process is time-consuming. To address the two problems, A TR network compression method by Bayesian optimization (TR-BO) is proposed. TR-BO involves selecting rank via Bayesian optimization, compressing the neural network layer via TR decomposition using rank obtained in the previous step, and, finally, further fine-tuning the compressed model to overcome some of the performance loss due to compression. Experimental results show that TR-BO achieves the best results in terms of Top-1 accuracy, parameter, and training time. For example, on the CIFAR-10 dataset Resnet20 network, TR-BO-1 achieves 87.67% accuracy with a compression ratio of 13.66 and a running time of only 2.4 hours. Furthermore, TR-BO has achieved state-of-the-art performance on the CIFAR-10/100 benchmark tests.
Tensor decomposition has been extensively studied for convolutional neural networks (CNN) model compression. However, the direct decomposition of an uncompressed model into low-rank form causes unavoidable approximation error due to the lack of low-rank property of a pre-trained model. In this manuscript, a CNN model compression method using alternating constraint optimization framework (ACOF) is proposed. Firstly, ACOF formulates tensor decomposition-based model compression as a constraint optimization problem with low tensor rank constraints. This optimization problem is then solved systematically in an iterative manner using alternating direction method of multipliers (ADMM). During the alternating process, the uncompressed model gradually exhibits low-rank tensor property, and then the approximation error in low-rank tensor decomposition can be negligible. Finally, a high-performance CNN compression network can be effectively obtained by SGD-based fine-tuning. Extensive experimental results on image classification show that ACOF produces the optimal compressed model with high performance and low computational complexity. Notably, ACOF compresses Resnet56 to 28% without accuracy drop, and the compressed model have 1.14% higher accuracy than learning-compression (LC) method.
Recently, discriminative object trackers based on deep learning have demonstrated excellent performance. However, the tracking accuracy is facing a challenge due to contaminated training samples and different complex scenarios. For this reason, we propose a tracker based on sparse robust samples and convolutional residual learning with multi-feature fusion (SR_MFCRL). First, a sparse robust sample set (SRSS) is introduced to improve robustness of the network. In this process, we first employ sparse representation to estimate the best candidate and then utilize joint detection with response peak value and occlusion detection to determine the contamination degree of the sample. Second, a multifeature fusion residual network (MRN) is proposed and its two base branches to capture response output of different features in order to achieve higher positioning accuracy. Extensive experimental results conducted on OTB-2013 illustrate that the proposed tracker achieves outstanding performance in terms of tracking accuracy and robustness.
Image super-resolution methods based on forward-feed convolutional neural networks (CNN) reconstruct the image with more details and sharper texture. However, most of these methods do not consider the influence of high level semantic feature to improve image perceptual effect. In this paper, we propose a deep CNN architecture jointing low-high level feature for image super-resolution. Our method uses 17 weight layers to predict residual between the high resolution and low resolution image. And we joint the low level and high level image features to constraint the network parameters updating. Experimental results validate that our method reconstruct the high resolution images with clear edge and less warp.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.