GradientBoostingRegressor

A high-performance ensemble learner that builds additive regression models by optimizing differentiable loss functions through sequential decision trees.

Scikit-learn's GradientBoostingRegressor implements the Friedman algorithm to minimize mean squared error or absolute deviation. It constructs an ensemble by fitting each new tree to the negative gradient of the loss function (the residual) of the previous iteration. This sequential process allows the model to correct errors in real time. Key parameters like learning_rate (shrinkage) and n_estimators control the trade-off between speed and accuracy. It is a workhorse for structured data, frequently outperforming single trees and linear models on complex datasets like the California Housing or Ames datasets.

https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingRegressor.html