Skip to main content
added 55 characters in body
Source Link
Aditya
  • 2.5k
  • 2
  • 17
  • 35

(answer isn't strictly acc to your Question ordering)

(answer isn't strictly acc to your Question ordering)

added 273 characters in body
Source Link
Aditya
  • 2.5k
  • 2
  • 17
  • 35

Intuitive Understanding(The Docs is quite clear if not read below..)

Hope this helps..

Also your n_estimators is Quite High which will lead you to Overffiting

Also RF isn't a good option when we have severe imbalanceness As Random forests are built on decision trees, and decision trees are sensitive to class imbalance. Therefore each tree will be biased in the same direction and magnitude (on average) by class imbalance.

Just a side note,

Bagging and Boosting might sound similar but aren't similar.. Xgboost can take care of Imbalanceness whereas RF's don't..

Hope it helps..

Intuitive Understanding

Hope this helps..

Also your n_estimators is Quite High which will lead you to Overffiting

Intuitive Understanding(The Docs is quite clear if not read below..)

Also your n_estimators is Quite High which will lead you to Overffiting

Also RF isn't a good option when we have severe imbalanceness As Random forests are built on decision trees, and decision trees are sensitive to class imbalance. Therefore each tree will be biased in the same direction and magnitude (on average) by class imbalance.

Just a side note,

Bagging and Boosting might sound similar but aren't similar.. Xgboost can take care of Imbalanceness whereas RF's don't..

Hope it helps..

Source Link
Aditya
  • 2.5k
  • 2
  • 17
  • 35

From The Docs

The RandomForestClassifier is trained using bootstrap aggregation, where each new tree is fit from a bootstrap sample of the training observations z_i = (x_i, y_i). The out-of-bag (OOB) error is the average error for each z_i calculated using predictions from the trees that do not contain z_i in their respective bootstrap sample. This allows the RandomForestClassifier to be fit and validated whilst being trained.. So we don't actually need to a Validation dataset or k-fold in this case

Intuitive Understanding

Suppose our training data set is represented by T and suppose data set has M features (or attributes or variables).

T = {(X1,y1), (X2,y2), ... (Xn, yn)} and Xi is input vector {xi1, xi2, ... xiM} and yi is the label (or output or class).

Summary of RF:

Random Forests algorithm is a classifier based on primarily two methods - bagging and random subspace method.

Suppose we decide to have S number of trees in our forest then we first create S datasets of "same size as original" created from random resampling of data in T with-replacement (n times for each dataset). This will result in {T1, T2, ... TS} datasets. Each of these is called a bootstrap dataset. Due to "with-replacement" every dataset Ti can have duplicate data records and Ti can be missing several data records from original datasets. This is called Bagging.

Now, RF creates S trees and uses m (=sqrt(M) or =floor(lnM+1)) random subfeatures out of M possible features to create any tree. This is called random subspace method. 

So for each Ti bootstrap dataset you create a tree Ki. If you want to classify some input data D = {x1, x2, ..., xM} you let it pass through each tree and produce S outputs (one for each tree) which can be denoted by Y = {y1, y2, ..., ys}. Final prediction is a majority vote on this set.

Out-of-bag error:

After creating the classifiers (S trees), for each(Xi,yi)in the original training set i.e. T, select all Tk which does not include (Xi,yi). This subset, pay attention, is a set of boostrap datasets which does not contain a particular record from the original dataset. This set is called out-of-bag examples. There are n such subsets (one for each data record in original dataset T). OOB classifier is the aggregation of votes ONLY over Tk such that it does not contain (xi,yi). 

Out-of-bag estimate for the generalization error is the error rate of the out-of-bag classifier on the training set (compare it with known yi's).

Hope this helps..

Also your n_estimators is Quite High which will lead you to Overffiting