# Tag Info

4

You need to set bootstrap=False in the random forest to disable the subsampling. (I originally commented because I expected there to be more impediments [in addition to your already-coded random_states and max_features=None], but I guess there aren't any!) You probably don't want to do this in general; by stripping out all the randomness so that the first ...

1

What helps the model more, keeping all features or removing correlated ones? There is some theory about it but in the end Machine Learning is try and error. You should give it a try with all features and then doing a feature selection to see if you are able to improve your model. What works for some models doesn´t necessarily have to work for the rest of ...

1

My experience is with sklearn and python. A Random Forest is a bagging of decision trees, the sklearn package uses a Decision Tree Classifier which is a CART. You can see in this post that I made a bit ago of how to build a Random Forest tree the exact same as a decision tree.

1

Experimentally: using cross-validation on a subset of your training data, compute the performance of every option that you want to consider. Then select the best option and train the final model using this option. // different settings for hyper-parameters, // for instance different pruning criteria: hpSet = { hp1, hp2, ...} trainSet, testSet = split(...

1

By performing GridSearch I understand you want to say searching for the best hyperparameters. For sake of simplicity, let say that you want to fit a linear regression with a penalty (lasso/ridge) with 1 feature and with 100 features. The hyperparameter that you are looking for is the $\lambda$ penalty. It is easy to see that with 1 feature your model ...

1

Since CARTs (Classification And Regression Tree) are a non-parametric algorithm, they should be able to find interactions between variables and non-linear behaviors. Nevertheless, building polynomials can help them have a better performance.

1

You can use categorical features with decision trees in scikit-learn, but you'll need to encode them as numbers. If your categorical features are ordinal (such as ranking ‘bad’, ‘fair’, ‘good’), they are easy to encode in numbers that respect the underlying ordering (e.g. 0, 1, 2). For nominal features, given the high cardinality you mention, you can try ...

1

Now with the new version 0.22.1, you can! It does pruning based on minimal cost-complexity pruning: the subtree with the largest cost complexity that is smaller than ccp_alpha will be chosen. https://scikit-learn.org/stable/auto_examples/tree/plot_cost_complexity_pruning.html

Only top voted, non community-wiki answers of a minimum length are eligible