Random Projection-Induced Gaussian Latent Features for Arbitrary Style Transfer

Abstract

The feature transfer technique centered on mean and variance statistics, widely known as AdaIN, lies at the core of current style transfer research. This technique relies on the assumption that latent features for style transfer follow Gaussian distributions. In practice, however, this assumption is often hard to meet, as the features typically exhibit sparse distributions due to the significant spatial correlation inherent in natural images. To tackle this issue, we propose first performing a random projection on the sparse features, and then conducting style transfer on these projections. Statistically, the projections will satisfy or approximate Gaussian distributions, thereby better aligning with AdaIN's requirements and enhancing transfer performance. With the stylized projections, we can further reconstruct them back to the original feature space by leveraging compressed sensing theory, thereby obtaining the stylized features. The entire process constitutes a projection-stylization-reconstruction module, which can be seamlessly integrated into AdaIN without necessitating network retraining. Additionally, our proposed module can also be incorporated into another promising style transfer technique based on cumulative distribution functions, dubbed EFDM. This technique faces limitations when there are substantial differences in sparsity levels between content and style features. By projecting both types of features into dense Gaussian distributions, random projection can reduce their sparsity disparity, thereby improving performance. Experiments demonstrate that the aforementioned performance improvements can be achieved on existing state-of-the-art approaches.