Tune Learning Rate for Gradient Boosting with XGBoost

A problem with gradient boosted decision trees is that they are quick to learn and overfit training data.

One effective way to slow down learning in the gradient boosting model is to use a learning rate, also called shrinkage (or eta in XGBoost documentation).

Here you will discover the effect of the learning rate in gradient boosting and how to tune it on your machine learning problem using the XGBoost library in Python.

By the end you will know:

• The effect learning rate has on the gradient boosting model.
• How to tune learning rate on your machine learning on your problem?
• How to tune the trade-off between the number of boosted trees and learning rate on your problem?

Source Code

Report

Next: Stochastic Gradient Boosting with XGBoost