Text to multi-object images synthesis based on non-local self-attention

Pengxiong Wang; Wu Yang

doi:10.1117/12.2681140

23 May 2023 Text to multi-object images synthesis based on non-local self-attention

Pengxiong Wang, Wu Yang

Proceedings Volume 12645, International Conference on Computer, Artificial Intelligence, and Control Engineering (CAICE 2023); 126451D (2023) https://doi.org/10.1117/12.2681140
Event: International Conference on Computer, Artificial Intelligence, and Control Engineering (CAICE 2023), 2023, Hangzhou, China

Abstract

Deep Convolutional Network (CNN) can make the pictures generated by GAN more reasonable, but limited by the local receptive field of CNN, there are still many unreasonable places in the authenticity and semantics of the multi-object images generated from text. Therefore, a GAN-based method that incorporates a non-local self-attention mechanism is proposed. By embedding a non-local self-attention structure in the network, the network obtains global semantic information and detailed features, and uses the obtained information to perform level-by-level encoding to generate the final relatively reasonable image. The amount of parameters and calculation of the entire model is also reduced a lot. The proposed method is verified on the public COCO-stuff dataset and uses multiple indicators such as Inception Score, FID and classification accuracy score to evaluate the authenticity and diversity of the generated images. Experimental results show that the quality of the generated images is superior to that of previously proposed methods.

Citation Download Citation

Pengxiong Wang and Wu Yang "Text to multi-object images synthesis based on non-local self-attention", Proc. SPIE 12645, International Conference on Computer, Artificial Intelligence, and Control Engineering (CAICE 2023), 126451D (23 May 2023); https://doi.org/10.1117/12.2681140

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available