Separating Style and Content

Abstract

We seek to analyze and manipulate two factors, which we call style and content, underlying a set of observations. We fit training data with bilinear models which explicitly represent the two-factor struc(cid:173) ture. These models can adapt easily during testing to new styles or content, allowing us to solve three general tasks: extrapolation of a new style to unobserved content; classification of content observed in a new style; and translation of new content observed in a new style. For classification, we embed bilinear models in a probabilistic framework, Separable Mixture Models (SMMsj, which generalizes earlier work on factorial mixture models [7, 3]. Significant per(cid:173) formance improvement on a benchmark speech dataset shows the benefits of our approach.