Unsupervised Photo-to-Caricature Generation with Adaptive Select Layer-Instance Normalization and Semi-Cycle Consistency

Abstract

Unpaired photo to caricature generation is a challenging but meaningful task. Generating high quality caricatures with rich texture/color and plausible exaggeration is important. Previous methods often respectively deal with the shape transformation and texture/color style. We argue that shape transformation can be treated as same as texture/color. Thereby, shape transformation and texture/color can be transferred at the same time. In this paper, we proposed a new method namely AdsSe-GAN for photo-to-caricature generation, which consists of a new normalization function called AdaSLIN and a new semi-cycle consistency loss. The AdaSLIN adaptively selects Layer Normalization or Instance Normalization to simultaneously transfer texture/color and shape transformation. Besides we present semi-cycle consistency loss which only imposes L1 norm on caricature-to-photo process, which is different from existing methods that apply cycle consistency loss to preserve the original domain information. In fact, while generating caricature, taking no account of the cycle restriction makes our model generate caricature with more distinct exaggeration and higher quality. Experimental results on a public caricature dataset, WebCaricature, show the effectiveness of our proposed method compared with the state-of-the-art models.