3
There are many ways to try to estimate feature importance. Personally I think the random forest measures get overused simply due to the fact that they have “importance” in their name and many people have heard of them. However, what people don’t realize is that those features that the random forest deems important are important for the random forest. They ...
2
Global Explanation:
The overall importance of a feature in a decision tree(and also applied to random forest and GBDT) can be computed in the following way:
‘weight’: the number of times a feature is used to split the data across all trees.
‘gain’: the average gain across all splits the feature is used in.
‘cover’: the average coverage across all splits ...
1
Normalization/Standardization is suggested because it makes the convergence easy and faster.
Andrew Ng has explained the process and the reason with a bowl shape 2-dimensional loss space in his course.
If you are not doing that, you should be very slow with your LR and spend a lot more epochs.
I found this combination working
lr = LogisticRegression(...
1
There are some different methods you can use to measure your model's accuracy. As @BeamsAdept mentioned in his own answer you can use the ROC AUC metric. Alternatively you mind find the Odds Ratio useful and if your dataset is big enough I would highly encourage you look into something like K-fold cross validation in order to get more representative results. ...
1
The effect will be increasing the intercept. I don't recommend doing oversampling unless any other solution doesn't work. Besides, 10% is not such a big imbalance.
I've been in kaggle competitions with way more imbalance where no imbalance solutions were adopted, logistic regression and random forest work quite well without the need of these.
Edit
After @...
1
To have a probability of 1 in a RF, it means that your algorithm can construct a leaf containing only positive sample. Since it doesn't, this means that your features are not explaining the variance of the output or that your algorithm is under-fitted.
I suggest that you try optimize the hyper-parameters of your RF by using cross-validation and use some ...
1
I've often had LogisticRegression "not converge" yet be quite stable (meaning the coefficients don't change much between iterations).
Maybe there's some multicolinearity that's leading to coefficients that change substantially without actually affecting many predictions/scores.
Another possibility (that seems to be the case, thanks for testing ...
Only top voted, non community-wiki answers of a minimum length are eligible
