Skip to main content
added 315 characters in body
Source Link
Ben Reiniger
  • 12.3k
  • 3
  • 19
  • 58

There are two versions of Adaboost: one does not require the weak learner to accept sample weights, and instead just resamples the data according to the weights, using the resampled set for the weak learner.

As for how trees make use of sample weights: the impurity calculation is done making use of the weights, and also the leaf scores become weighted averages. See e.g. this SO post and this DS.SE post.

Finally, many other algorithms operate by minimizing a loss function over some hypothesis space; those can incorporate weights into their loss functions directly. (Examples include GLMs, SVMs, NNs, though none of those are particularly weak learners, so probably not desirable for an adaptive boosting ensemble.)

There are two versions of Adaboost: one does not require the weak learner to accept sample weights, and instead just resamples the data according to the weights, using the resampled set for the weak learner.

As for how trees make use of sample weights: the impurity calculation is done making use of the weights, and also the leaf scores become weighted averages. See e.g. this SO post and this DS.SE post.

There are two versions of Adaboost: one does not require the weak learner to accept sample weights, and instead just resamples the data according to the weights, using the resampled set for the weak learner.

As for how trees make use of sample weights: the impurity calculation is done making use of the weights, and also the leaf scores become weighted averages. See e.g. this SO post and this DS.SE post.

Finally, many other algorithms operate by minimizing a loss function over some hypothesis space; those can incorporate weights into their loss functions directly. (Examples include GLMs, SVMs, NNs, though none of those are particularly weak learners, so probably not desirable for an adaptive boosting ensemble.)

Source Link
Ben Reiniger
  • 12.3k
  • 3
  • 19
  • 58

There are two versions of Adaboost: one does not require the weak learner to accept sample weights, and instead just resamples the data according to the weights, using the resampled set for the weak learner.

As for how trees make use of sample weights: the impurity calculation is done making use of the weights, and also the leaf scores become weighted averages. See e.g. this SO post and this DS.SE post.