AV1 In-loop super-resolution framework

Urvang Joshi; Debargha Mukherjee; Yue Chen; Sarah Parker; Adrian Grange; Hui Su

doi:10.1117/12.2534538

6 September 2019 AV1 In-loop super-resolution framework

Urvang Joshi, Debargha Mukherjee, Yue Chen, Sarah Parker, Adrian Grange, Hui Su

Proceedings Volume 11137, Applications of Digital Image Processing XLII; 1113718 (2019) https://doi.org/10.1117/12.2534538
Event: SPIE Optical Engineering + Applications, 2019, San Diego, California, United States

Abstract

The AV1 codec added a new in-loop super-resolution mode that allows a frame to be encoded at a lower resolution, and then super-resolved normatively to the full resolution, before updating the reference buffers. While encoding at lower resolution and super-resolving to a higher resolution is not a new concept, this is the first time such a mode has been normatively incorporated in a standardized video codec. To this end, AV1 has not only added support for across-scale motion prediction to allow predicting a lower resolution version of a frame from higher resolution reference buffers, but also made various simplifications to the super-resolving process itself after reconstruction to make it both software and hardware-friendly in implementation. Specifically, the super-resolving process in AV1 comprises of normative linear upscaling, followed by a restoration operation to recover the high frequencies using another AV1 tool called loop-restoration that includes a Wiener or Self-guided filter selected in a block switchable manner. Further, in order to enable a cost-effective hardware solution with limited line-buffers, this mode only allows the upscaling/downscaling operation to be horizontal.

In this paper, we provide the details of the super-resolution mode in AV1 and some results showcasing the benefits of the same.

Conference Presentation

Citation Download Citation

Urvang Joshi, Debargha Mukherjee, Yue Chen, Sarah Parker, Adrian Grange, and Hui Su "AV1 In-loop super-resolution framework", Proc. SPIE 11137, Applications of Digital Image Processing XLII, 1113718 (6 September 2019); https://doi.org/10.1117/12.2534538

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available