Group Feature Selection Using Non-Class Data

Abstract

Existing embedded feature selection methods barely let non-class data contribute to feature selection. However, in some learning tasks, when non-class data have contribution to classification, they should also have an influence to the selection of useful features. For instance, $F_\infty$ F ∞ -norm support vector machine is an effective embedded group feature selection method that performs classification simultaneously. In this paper, we find out that it implicitly uses a kind of non-class data formulated as coordinate Universum when implementing group feature selection, and the information contained in this non-class data could be a meaningful group-wise $F_{\infty }$ F ∞ -norm penalization. As far as we know, this is the first time that $F_{\infty }$ F ∞ -norm penalization is understood from this angle. We prove that useful features can be identified through this non-class data that contribute to classifier construction. In addition, to fully explore the classification information provided by this non-class data, we improve $F_\infty$ F ∞ -norm support vector machine by deeming the non-class data as a middle class to better classify positive and negative classes. Experiments show that the non-class data in the proposed method help reduce the labelled data in some sense. Furthermore, it improves $F_\infty$ F ∞ -norm support vector machine in terms of both classification and group feature selection.