i've been studying gradient boosting so realize the pure algorithm requires a function F/model to get boosted.What is the explicit F on gradient boosting trees?
$\begingroup$
$\endgroup$
3
-
1$\begingroup$ it is not (theoreticaly) necessary for $F$ to have a closed-form formula. Gradient can be numericaly computed $\endgroup$– Nikos M.Jan 15, 2021 at 16:09
-
$\begingroup$ gnosisda.gr/wp-content/uploads/2020/07/… $\endgroup$– Nikos M.Jan 15, 2021 at 16:12
-
$\begingroup$ Nikos M. even though it isn't necessary for F to have a closed formula, Can I express the functional form as sum of indicator terms and work to that? $\endgroup$– Nyut NyukaFeb 4, 2021 at 0:33
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
It can be expressed as a linear combination of indicators, where the weights are the training averages on each partition of the space, and the indicator is the indicator for that partition.
Having said that, I haven't come across any software that automatically does it for you. I also think the analytic expressions might become very large when dealing with moderately wide trees.
answered Feb 4, 2021 at 7:27