1
$\begingroup$

I went through adaboost tutorial and below are my simplified understanding:

  1. Sample weight of equal value is given to all sample in dataset.
  2. Stumps are created which uses only one feature from data set.
  3. Using total error and sample weight stump importance is calculated.
  4. Samples weights are changed i.e. samples which were predicted wrongly by stumps with high importance get sample weight increased and rest of the sample weight are decreased in the same order.
  5. The process is repeated till the number of iterations mentioned where sample weight provides path for training.

Does adaboost contain multiple stumps with different splitting value for a single feature? As mentioned, are the stumps created as first step or is it a continuous process?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

Firstly, i want to notice that adaboost is a ensemble of stumps, and each stump is added to the ensemble sequentially, trying to compensate the errors of the existing ensemble.

Saying that:

  • In every step you have a new weighted dataset, and the Adaboost algorithm tries to fit a stump that splits the new dataset best. So yes, it is possible, and it does happen, that a single feature is used more than once (but the splitting value calculated by the stump should be different in order to reduce the ensemble's error.)

  • Stumps' creation is a continuous process, they are calculated in every step for the new weighted dataset.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks for answer. My follow up question. by added sequential does that mean bunch of stumps are created and best of the stumps is taken as first stump and using it the sample weight is tuned. Next the process is repeated and second stump is acquired ? or one stump is created as model and sample weights are tuned as per the stump result and then second stump is added to reduce the first stump error ? $\endgroup$
    – user123987
    Commented Jan 9, 2020 at 11:34
  • $\begingroup$ Only one stump is created and it is the best based on a criterion, for decision trees a standard criterion is gini criterion.In this case, the feature that minimizes gini index should be selected as the root. Note, that as the weights of the samples are changing, the feature that minimizes this criterion will change too. $\endgroup$
    – Giannis Krilis
    Commented Jan 9, 2020 at 13:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.