Supervised Dimensionality Reduction Using Mixture Models

Abstract

Given a classification problem, our goal is to find a\nlow-dimensional linear transformation of the feature vectors which retains\ninformation needed to predict the class labels. We present a method based on\nmaximum conditional likelihood estimation of mixture models. Use of mixture\nmodels allows us to approximate the distributions to any desired accuracy while\nuse of conditional likelihood as the contrast function ensures that the\nselected subspace retains maximum possible mutual information between feature\nvectors and class labels. Classification experiments using Gaussian mixture\ncomponents show that this method compares favorably to related dimension\nreduction techniques. Other distributions belonging to the exponential family\ncan be used to reduce dimensions when data is of a special type, for example\nbinary or integer valued data. We provide an EM-like algorithm for model\nestimation and present visualization experiments using both the Gaussian and\nthe Bernoulli mixture models. Pre-2018 CSE ID: CS2004-0810