Comprehensible Interpretation of Relief's Estimates

Abstract

Attribute estimation is an important machine learning problem. It is contained in many tasks, e.g., feature subset selection, constructive induction, decision and regression tree building. Relief algorithms are one of the most successful heuristic measures for solving this problem. The quality estimates of the Relief algorithms have commonly been interpreted as the difference of two probabilities, which make them rather difficult for human comprehension. We present a new insight on how these algorithms work by analyzing their behavior with abundance of training data. We show that Relief’s weight of an attribute converges to the ratio between the number of explained changes in the concept and the number of examined instances. We show how this new interpretation of quality estimates can be used to explain some behaviors of Relief algorithms and give examples of better human comprehension in two real world problems.