StyleRetoucher: Generalized Portrait Image Retouching with GAN Priors

Wanchao Su¹

Can Wang²

Chen Liu³

Fangzhou Han²

Hongbo Fu⁴

Jing Liao^2,*

1 Department of Human Centred Computing, Faculty of Information Technology, Monash University

2 Department of Computer Science, City University of Hong Kong

3 School of Computer Science and Engineering, Nanyang Technological University

4 Division of Emerging Interdisciplinary Areas, Hong Kong University of Science and Technology

* Corresponding Author

[Paper]

Our StyleRetoucher automatically removes skin blemishes in portrait photos and preserves the characteristics (i.e., fine details) of the original inputs (Left), producing high-quality photo retouching results (Right). Four pairs of close-ups in the bottom row show the regions before and after retouching. Please zoom in to examine the results in detail.

Abstract

Creating fine-retouched portrait images is tedious and time-consuming even for professional artists.There exist automatic retouching methods, but they either suffer from over-smoothing artifacts or lack generalization ability. To address such issues, we present StyleRetoucher, a novel automatic portrait image retouching framework, leveraging StyleGAN’s generation and generalization ability to improve an input portrait image's skin condition while preserving its facial details. Harnessing the priors of pretrained StyleGAN, our method shows superior robustness: a). performing stably with fewer training samples and b). generalizing well on the out-domain data. Moreover, by blending the spatial features of the input image and intermediate features of the StyleGAN layers, our method preserves the input characteristics to the largest extent. We further propose a novel blemish-aware feature selection mechanism to effectively identify and remove the skin blemishes, improving the image skin condition. Qualitative and quantitative evaluations validate the great generalization capability of our method. Further experiments show StyleRetoucher's superior performance to the alternative solutions in the image retouching task.We also conduct a user perceptive study to confirm the superior retouching performance of our method over the existing state-of-the-art alternatives.

Bibtex

If you find our work useful for your research, please cite our work as:

@article{su2024styleretoucher,
    author = {Su, Wanchao and Wang, Can and Liu, Chen and Han, Fangzhou and Fu, Hongbo and Liao, Jing},
    title = {StyleRetoucher: Generalized Portrait Image Retouching with GAN Priors},
    journal = {IEEE Transactions on Visualization and Computer Graphics},
    year = {2024},
    volume={},
    number={},
    pages={1-12},
    doi={10.1109/TVCG.2024.3432910},
    publisher={IEEE}
}