New answers tagged random-forest
4
The (maximum) number of features is a hyperparameter, i.e. it depends on what you set it to or what the default of the implementation you are using is. In "Introduction to Machine Learning with Python" by Mueller and Guido the authors recommend the following:
As described earlier, max_features determines how random each tree is,
and a smaller ...
4
_feature_importance of a random forest calculates the average feature importance across all trees in the forest. While tree.feature_importances_ is the feature importance for a single tree.
Since feature importance is calculated as the contribution of a feature to maximize the split criterion (or equivalently: minimize impurity of child nodes) higher is ...
4
A random forest model is an agglomeration of Decision Trees. tree.feature_importance_ defines the feature importance for each individual tree, but model.feature_importance_ is the feature importance for the forest as a whole. The docs give the explanation for calculation as:
The relative rank (i.e. depth) of a feature used as a decision node in a tree can ...
2
Problem is the way you're onehot encoding.
Best practice for any type of encoding :
You should train an estimator for Onehot encoding on the training data only, and when encoding test data, you should use the same estimator used on training data.
Eg : sklearn.preprocessing.OneHotEncoder does this, and it has a parameter called : handle_unknown.
...
0
Decision trees usually base feature importance on the impurity reduction achieved by splitting on the features. In classification a usual choice is gini impurity, while regression trees typically use the mean squared error or node sample variance. This is also the case in scikit learn.
For a given (binary) node $m$ with left and right child nodes the ...
0
My 2 cents: I strongly recommend displaying ALL PARAMETERS such as criterion, max_depth, max_leaf_nodes... in order to learn tree algorithms. In my experience, you'll learn a ton such as overfitting.
2
To plot feature importance using gridsearch use:
x= X_train_v1.columns,y= rf_grid_search_v1.best_estimator_.feature_importances_
1
If you want to see what is the best parameters choosen for your model you can use
rf_grid_search_v1.best_estimator_
0
Using all default values for the RandomForestClassifier class leads to overfitting. As stated in scikit-learn documentation,
The default values for the parameters controlling the size of the trees (e.g. max_depth, min_samples_leaf, etc.) lead to fully grown and unpruned trees which can potentially be very large on some data sets. To reduce memory ...
0
Yes, you can do that. However, how accurate your new labels are depends on the ability of your model to generalize to new data and the similarity of the new data to your training data. Therefore, it is something you need to test for in the first place by using a separate test dataset to assess model performance. In contrast to that, a validation dataset is ...
1
A 1/3 - 2/3 repartition is not that unbalanced. Your problem shouldn't require balancing.
The train/test set partition seems to be done correctly, as it seems implied by checking data histograms. Doing that randomly is usually ok, and when it's not it will inflate your test performance with data leakage, which doesn't seems to be the case here.
Imo the ...
0
You are correct, such a difference between training and test implies that the model is overfitting.
Here are some best practices to improve the process:
1. Accuracy is not a great metric for imbalanced classes and I would recommend moving to f1-score.
2. Balance the training set by over-sampling the minority class or under-sampling the majority class.
3. ...
0
Random Forest is based on bagging and voting.
It means all the individual model has to persist to be used in prediction. Hence increasing the count of estimators will need more RAM.
The good news is that the same property makes it eligible to be trained in parallel if Cores/CPU is available. Please try that.@Brady Gilg has suggested the same.
I have ...
0
Features maybe good enough but obviously you have covariate shift, or some similiar distrubancd. In other words distribution of your train and test features is different and that confuses your model, in other words it doesnt learn to differentiate on train dataset.
3
For the Random Forests algorithm, the time complexity for building a complete un-pruned tree is $O(m.n\log(n))$, where $n$ is the number of records/instances and $m$ is the number of variables. The algorithm is embarrassingly parallel so in many cases companies with available resources will simply use sufficient compute nodes to enable the model to run in a ...
Top 50 recent answers are included
