New answers tagged random-forest
1
The hyper parameters that you could tune in any boosting technique are:
Depth of each tree: As you rightly pointed out this is very important because each tree in boosting technique learns from the errors of the previous trees. Hence underfitting the initial trees ensure that the later trees learn actual patterns and not noise.
Number of trees: this is kind ...
0
While the bagging of random forests is meant to reduce overfitting, they generally will overfit more than e.g. a parametric model like logistic regression.
Having a larger gap between training and testing scores is not necessarily a problem; you may still prefer the model if its testing score is higher than other models'. In a purely statistical perfect ...
1
If your values are larger than float32, try to run some scaler first. It'd be rather unusual to have deviation spanning more than float32.
1
In most cases getting rid of infinite and null values solve this problem.
get rid of infinite values.
df.replace([np.inf, -np.inf], np.nan, inplace=True)
get rid of null values the way you like, specific value such as 999, mean, or create your own function to impute missing values
df.fillna(999, inplace=True)
or
df.fillna(df.mean(), inplace=True)
0
Before training and testing any Machine Learning model, we should perform some prerequisite steps to tune the accuracy of model and to avoid Overfitting or Underfitting.
First you should understand the data.
Analyse statistics of data like range, min, max, std dev etc.
Visualize the data as it will give you better insight of data.
If each features in data ...
3
Overfitting is a common problem in random forest, you can use cross Validation methods (Cross validation is a technique to build models that are not prone to overfitting) example - K-fold cross validation, stratified k fold..
Try this using train package-
control <- trainControl(method = "cv", number = 5)
and add this as a control parameter to the model....
2
You are right. Its looks like typical overfitting.
How to handle it?
try different test:train ratio
K-Fold Cross-Validation
use feature selection techniques
Early stopping
Regularization
1
I dont think its necessary combining 4 models into one by averaging probabilities (how do you know that they have same weights? let the learner handle those weights) since you are using same features and learner which is an ensemble (it combines weak learner into one natively already). Therefore it's better to think about labeling and class balances of those ...
1
This is more like a comment than an answer but my newbie status doesn't allow me to comment yet.
You might be interested in reading this article:
Random forest
4 samples appears to me being low a number of samples, I would recommend that you try using more samples and compare your results.
Besides, there are some good implementation of the algorithm out ...
2
RandomForest advantage compared to newer GBM models is that it is easy to tune and robust to parameter changes. It is robust for most use cases although the peak performance might not be as good as a properly-tuned GBM. Another advantage is that you do not need to care a lot about parameter. You can compare the number of parameter for randomforest model and ...
2
Forest:
Forest paper
"We show that random forest variable importance measures are a sensible means for variable selection in many applications, but are not reliable in situations where potential predictor variables vary in their scale of measurement or their number of categories."
This is saying that if a feature varies on its ability to detect based on ...
1
It is a common problem that - with unbalanced classes - some model tends to predict mostly the majority class. You could try to oversample the minority classes. In addition, RF tends to perform weak here. Boosting or NN are often able to recover more details, which can be important to predict the minority classes.
Edit: Okay, now that you clarified that you ...
0
I think you correctly identified the issue: if your model tries to classify whether an employee stays or leaves, by definition every employee "stays" as long as they are an employee in the company.
A possible direction would be to design the response variable as "does the employee leave within a year?" (or any specific period of time). This way you can use ...
0
Your df row count is 14,999
Your test data is 33% ~ 4950
So, your y_predict should be (4950,1) i.e. a binary prediction for all test rows.
This is what I am getting when running your code which you have posted here.
import pandas as pd
df = pd.read_csv('https://raw.githubusercontent.com/bhaskoro-muthohar/DataScienceLearning/master/HR_comma_sep.csv')
...
-1
This looks like a classic case of class imbalance,
You try oversampling the data.
Since you are targeting an event that is less likely to happen, It is good to optimize on high precision and you can compromise on recall.
Looks like you got lucky there, try using k-fold cross-validation while splitting the dataset into test and train, this looks a little ...
Top 50 recent answers are included
