Ensemble methods like Bagging and Boosting which combine the decisions of multiple hypotheses are some of the strongest existing machine learning methods. The diversity of the members of an ensemble is known to be an important factor in determining its generalization error. We present a new method for generating ensembles, DECORATE (Diverse Ensemble Creation by Oppositional Relabeling of Artificial Training Examples), that directly constructs diverse hypotheses using additional artificially-constructed training examples. The technique is a simple, general meta-learner that can use any strong learner as a base classifier to build diverse committees. Experimental results using decision-tree induction as a base learner demonstrate that this approach consistently achieves higher predictive accuracy than both the base classifier and Bagging. DECORATE also obtains higher accuracy than Boosting early in the learning curve when training data is limited.
We propose to show that DECORATE can also be effectively used for (1) active learning, to reduce the number of training examples required to achieve high accuracy; (2) exploiting unlabeled data to improve accuracy in a semi-supervised learning setting; (3) combining active learning with semi-supervision for improved results; (4) obtaining better class membership probability estimates; (5) reducing the error of regressors; and (6) improving the accuracy of relational learners.

PDF, PS

Technical Report UT-AI-TR-03-306, Department of Computer Sciences, University of Texas at Austin. Ph.D. proposal.

@techreport{melville:techproposal03, title={Creating Diverse Ensemble Classifiers}, author={Prem Melville}, number={UT-AI-TR-03-306}, month={December}, institution={Department of Computer Sciences, University of Texas at Austin}, pages={54 pages}, note={Ph.D. proposal}, url="http://www.cs.utexas.edu/users/ai-lab?melville:techproposal03", year={2003} }