Boosting a Strong Learner: Evidence Against the Minimum Margin

Abstract

Boosting is a method by which an ensemble of classiers can be assembled with, generally speaking, much improved results over individual classiers and other ensemble methods. It works by re-weighting the training set after each classier is induced, so that misclassied training instances are given increased weight in the subsequently induced classier. The focus on misclassied instances has meant that boosting cannot continue if the learning system creates a classi- er that ts the data. This article presents a simple boosting-like method for tted decision trees with empirical results very similar to boosting. This provides an alternative view of the reasons for performance improvements due to boosting, and provides a further counter example for minimum margin as an explanation for boosting performance. 1 INTRODUCTION Ensemble methods for improving classication accuracy have recently become a focus for the machine learning community. These methods combine lear...