MyStyle++: A Controllable Personalized Generative Prior

Libing Zeng*

Lele Chen

Yi Xu*

Nima Kalantari*

* External authors

SIGGRAPH Asia 2023

2023

Abstract

In this paper, we propose an approach to obtain a personalized generative prior with explicit control over a set of attributes. We build upon MyStyle, a recently introduced method, that tunes the weights of a pre-trained StyleGAN face generator on a few images of an individual. This system allows synthesizing, editing, and enhancing images of the target individual with high fidelity to their facial features. However, it relies on the latent space of StyleGAN, and thus inherits its limitations in controllability. We propose to address this problem through a novel optimization system that organizes the latent space in addition to tuning the generator. Our key contribution is to formulate a loss that arrange the latent codes, corresponding to the input images, along a set of specific directions according to their attribute. We demonstrate that our approach, dubbed MyStyle++, is able to synthesize, edit, and enhance images of an individual with great control over the attributes, while preserving the unique facial characteristics of that individual.