Your grid search dictionary contains the argument names with the pipeline step name in front of it, i.e. 'randomforestclassifier__max_depth'. Instead, the RandomForestClassifier has argument names without the pipeline step name, i.e. max_depth. You therefore need to remove the first part of the string which denotes the name of the step in your original ...
If the sum of the two feature makes sense on the domain semantically, it might be a good idea.
But while trees can handle redundant features pretty well, increasing the number of features without adding any extra "value" or "information" can lead to lower performance in certain situations. For example, if there is no added value and you ...
To me those are separate things since both models have a different cost function to be optimized.
On the other hand you could combine those models by constructing embeddings based on random forest splits and then using those embeddings as inputs for a neural network.
Toy example shows that there is a non-trivial configuration of a neural net that can get as ...
You are on the right path. It appears you might have analysis paralysis. You should start building, then see what works and what does not work.
Here is code to get you started:
from sklearn.ensemble import RandomForestRegressor
from sklearn.feature_selection import VarianceThreshold
from sklearn.model_selection import GridSearchCV
I had a similar issue, but I realized that I had included my target variable while predicting the test outcomes.
predict(object = model_nb, test[,])
void of error:
predict(object = model_nb, test[,-16])
where the 16th column was for the dependent variable.