A novel approach for regularization of ensemble learning in classification and regression analysis Jena Prakash Chandra1, Kuhoo2, Mishra Debahuti3,*, Pani Subhendu Kumar4 1Research Scholar, Department of Computer Science and Engineering, Siksha ‘O'Anusandhan Deemed to be University, Bhubaneswar, Odisha, India, 2Student, Department of Mechanical Engineering, College of Engineering and Technology, Bhubaneswar, Odisha, India, 3Professor, Department of Computer Science and Engineering, Siksha ‘O'Anusandhan Deemed to be University, Bhubaneswar, Odisha, India 4Associate Professor Department of Computer Science and Engineering, Orissa Engineering College, Odisha, India *Corresponding Author: Debahuti Mishra Professor, Department of Computer Science and Engineering, Siksha ‘O’ Anusandhan Deemed to be University, Bhubaneswar, Odisha, India. debahutimishra@soa.ac.in
Online published on 16 October, 2018. Abstract Ensemble method requires in classification or regression model, to obtain better accuracy performance. It uses multiple learning algorithms, for which it can easily approaches to a better classifier or predictor compared to a single model. Gradient Boosting is a machine learning technique that can be used for both classification and regression problems. Basically this method is applied to enhance the performance of decision tree models. It builds a model in a step wise manner by optimizing a differential loss function or cost function. The main objective of this paper is to adjust the parameters of Gradient Boosting to avoid the problem of over fitting. This technique is generally called as regularization and the problem of over fitting can be solved by two regularization methods such as shrinkage and sub-sampling. In this work, those two regularization methods have been explored with the Gradient Boosting for both the classification and regression problems. The results have been found to be very interesting while compared with non regularized methods. Top Keywords Ensemble learning, Gradient Boosting, Regularization, Shrinkage, Sub-sampling. Top |