3
$\begingroup$

I'm currently studying Boosting techniques in Machine Learning and I happened to understand that in Algorithms like Adaboost, each of the training samples is given a weight depending on whether it was misclassified or not by the previous model in sequential boosting.

Although I intuitively understand that by weighting examples, we are letting the model pay more attention to examples that were previously misclassified, I do not understand "how" the weights are taken into account by a machine learning algorithm. Are the weights taken as a separate feature? I want to know "how" a simple machine learning model, lets say a decision tree, understands that a particular example carries more weight and therefore needs to be given more attention/ classified correctly.

Can someone please explain to me in simple words/equations how weighted training samples are handled by machine learning algorithms? How do they force themselves to classify samples with higher weights correctly? Should there be modifications made to existing ML algos to account for weights?

$\endgroup$
0
$\begingroup$

Adaboost is an Ensemble Classifier, which means that it consists of several weak classifiers that are able to consider weights in the training set. These weights can be used to inform the training of each weak classifier. For example, decision trees can be grown in a way that favor splitting sets of samples with higher weights.

$\endgroup$
0
$\begingroup$

I think below link has better explanation and it may help you to understand logic behind Boosting algorithms.

For boosting algorithms

https://youtu.be/LsK-xG1cLYA

To understand random sampling with replacement based on weight of each sample.

https://stackoverflow.com/questions/3679694/a-weighted-version-of-random-choice

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.