Feature Selection and Multiple Model Approach in Discriminant Analysis

Abstract

Significant improvement of model stability and prediction accuracy in classification and regression can be obtained by using the multiple model approach. In classification multiple models are built on the basis of training subsets (selected from the training set) and combined into an ensemble or a committee. Then the component models (classification trees) determine the predicted class by voting. In this paper some problems of feature selection for ensembles will be discussed. We propose a new correlation-based feature selection method combined with the wrapper approach.