Research on image restoration combining lightweight transformer and dense feature inference

Shen Zhang; Xuan Ma; Zijia Wen

doi:10.1117/12.3029158

6 May 2024 Research on image restoration combining lightweight transformer and dense feature inference

Shen Zhang, Xuan Ma, Zijia Wen

Proceedings Volume 13107, Fourth International Conference on Sensors and Information Technology (ICSI 2024); 131070Q (2024) https://doi.org/10.1117/12.3029158
Event: Fourth International Conference on Sensors and Information Technology (ICSI 2024), 2024, Xiamen, China

Abstract

Limited by the nature of traditional convolutional neural networks such as local prior and parameter sharing, the ability of image restoration to cope with complex semantic environments and large-scale hole repair needs to be improved. In recent years, transformer network architectures based on self-attentive mechanisms have performed well in NLP and high-resolution visual tasks. Compared with traditional CNNs, it is more effective in long-range feature migration applications, but there is high computational complexity in directly using the self-attention mechanism in high-dimensional image tasks. To address this problem, this work improves on the self-attention mechanism by using a linear attention mechanism with gating, a dense feature reasoning module (DFR) embedded in the middle of the U-Net style network, and feature fusion of different codec layers through jump connections. Comparative experiments on publicly available datasets (Paris Street View, CelebA-HQ, Places2), qualitative analyses and quantitative evaluations demonstrate the ability of the approach to reduce computational complexity and to perform well for the repair of broken images with complex semantics of largescale holes.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Shen Zhang, Xuan Ma, and Zijia Wen "Research on image restoration combining lightweight transformer and dense feature inference", Proc. SPIE 13107, Fourth International Conference on Sensors and Information Technology (ICSI 2024), 131070Q (6 May 2024); https://doi.org/10.1117/12.3029158

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
12 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Image restoration

Transformers

Image processing

Convolution

Semantics

Data modeling

Education and training

Show All Keywords

Keywords/Phrases

Search In:

Publication Years