Prof. Gong Lei Profile

Prof. Gong Lei

SPIE Involvement:

Author

Publications (1)

This will count as one of your downloads.

You will have access to both the presentation and article (if available).

DOWNLOAD NOW

This content is available for download via your institution's subscription. To access this item, please sign in to your personal account.

Email or Username Forgot your username?

Password Forgot your password?

Show

Keep me signed in

No SPIE account? Create an account

Proceedings Article | 8 May 2023 Paper

RAIN: robust and adaptive image manipulation based on GAN inversion with a text condition

Haoyu Cai , Longshan Shang, Lei Gong , Chao Wang

Proceedings Volume 12635, 126351D (2023) https://doi.org/10.1117/12.2678914

KEYWORDS: Gallium nitride, Rain, Semantics, Image quality, Image segmentation, Education and training, Visual process modeling, Image compression, Prior knowledge, Image processing

Read Abstract +

Recent work has shown various interesting semantic image manipulation methods based on GAN guided by text descriptions. A method based on GAN inversion can achieve versatile image manipulation functions without a time-consuming preprocessing stage. However, the method suffers from a lack of self-adaptation due to the intrinsic conflict between multi-objective losses. Meanwhile, the method applied in image manipulation guided by text conditions is not robust due to the vast and ambiguous search space. To solve the above problems, we propose a novel framework RAIN based on GAN inversion, which can achieve robust and adaptive text-driven image manipulation. As shown in Fig. 1(c), RAIN contains two main parts: CEV Initialization and RAGAN inversion. CEV Initialization can adaptively provide a Candidate Editing Vector (CEV) in a short time. RGAN inversion is a multi-stage optimization scheme utilizing the CEV as prior knowledge to prune search space. In RAGAN inversion, we explore how to improve the vision-language model's perception capability to restrict search space further. The objective of the paper is guaranteeing semantic correctness and image quality in a time-constrained scenario compared to the SOTA image manipulation methods guided by text descriptions. Extensive experiments show that RAIN can manipulate images guided by text description while meeting robustness and self-adaptation.

My Library

You currently do not have any folders to save your paper to! Create a new folder below.

Folder Name

Folder Description

View contact details

UPDATE YOUR PROFILE

Is this your profile? Update it now.

Sign into your SPIE.org account

Don’t have a profile and want one?

Create an account on SPIE.org

Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks. You are receiving this notice because your organization may not have SPIE eBooks access.*

*Shibboleth/Open Athens users─please sign in to access your institution's subscriptions.

To obtain this item, you may purchase the complete book in print or electronic format on SPIE.org.

ORGANIZATIONAL
Sign in with credentials provided by your organization.

Organizational Username

Organizational Password

Show/Hide Password

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members:

Non-members: ADD TO CART

Keywords/Phrases

Search In:

Publication Years