Gradient Boosting and bagging applied to regressors can reduce the error due to bias and variance respectively. Alternatively, Stochastic Gradient Boosting (SGB) and Iterated Bagging (IB) attempt to simultaneously reduce the contribution of both bias and variance to error. We provide an extensive empirical analysis of these methods, along with two alternate bias-variance reduction approaches --- bagging Gradient Boosting (BagGB) and bagging Stochastic Gradient Boosting (BagSGB). Experimental results demonstrate that SGB does not perform as well as IB or the alternate approaches. Furthermore, results show that, while BagGB and BagSGB perform competitively for low-bias learners, in general, Iterated Bagging is the most effective of these methods.

PDF, PS

Technical Report UT-AI-TR-05-321, University of Texas at Austin. www.cs.utexas.edu/~ml/publication.

@TechReport{suen:bv-tr05, title={Combining Bias and Variance Reduction Techniques for Regression}, author={Yuk Lai Suen and Prem Melville and Raymond J. Mooney}, number={UT-AI-TR-05-321}, month={July}, institution={University of Texas at Austin}, note={www.cs.utexas.edu/~ml/publication}, url="http://www.cs.utexas.edu/users/ai-lab?suen:bv-tr05", year={2005} }