Perceptually motivated deep neural network for video compression artifact removal

Darren Ramsook; Anil Kokaram; Neil Birkbeck; Yeping Su; Balu Adsumilli

doi:10.1117/12.2633552

3 October 2022 Perceptually motivated deep neural network for video compression artifact removal

Darren Ramsook, Anil Kokaram, Neil Birkbeck, Yeping Su, Balu Adsumilli

Proceedings Volume 12226, Applications of Digital Image Processing XLV; 122260H (2022) https://doi.org/10.1117/12.2633552
Event: SPIE Optical Engineering + Applications, 2022, San Diego, California, United States

Abstract

Recent advances have shown that latent representations of pre-trained Deep Convolutional Neural Networks (DCNNs) for classification can be modelled to generate scores that are well correlated with human perceptual judgement. In this paper we seek to extend the use of perceptually relevant losses in training a DCNN for video compression artefact removal. We will use internal representations of a pre-trained classification network as the basis of the loss functions. Specifically, the LPIPS metric and a perceptual discriminator will be responsible for low-level and high-level features respectively. The perceptual discriminator uses differing internal feature representations of the VGG network as its first stage of feature extraction. Initial results shows an increase in performance in perceptually based metrics such VMAF, LPIPS and BRISQUE, while showing a decrease in performance in PSNR.

Conference Presentation

Citation Download Citation

Darren Ramsook, Anil Kokaram, Neil Birkbeck, Yeping Su, and Balu Adsumilli "Perceptually motivated deep neural network for video compression artifact removal", Proc. SPIE 12226, Applications of Digital Image Processing XLV, 122260H (3 October 2022); https://doi.org/10.1117/12.2633552

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available