Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
For example, if I run a single round (nrounds=1), how does XGBoost go about making predictions? I thought it would simply return a linear regression model, but I quickly shot that theory down by working out an example.
Please note that I've already seen this and this and most of Google's top search results.
gblinear uses linear functions, in contrast to dart which use tree based functions.
One primary difference between linear functions and tree-based functions is the decision boundary. Tree-based models decision boundaries are only piece-wise, perpendicular rules to each feature. Linear functions are monotonic lines through the feature space.
Tree-based functions tend to be more flexible. If the data is distributed closer to a line, then a linear function will be more parsimonious.
