New answers tagged random-forest
0
If you use tree-based algorithms like random forests the data distribution should not be an issue. Linear algorithms are more dependent on the distribution of your variables. To check if you overfit can try to predict your training data and compare the result with test data. The score depends on your evaluation metric. If you use scikit-learn you get R^2 as ...
1
The decision trees used in gradient boosting are typically shallow decision trees (with only a few nodes). Limiting the depth or number of nodes in the decision tree makes them simple. This is different from fully developed decision trees used as standalone models.
1
Can I conclude that the error at my node is 60 +-13 i.e my values in this particular sample split ranges from 60-13 to 60+13.
No you cannot, because the actual error values depend on the data. For example you might have 1 instance with error 41.11 and 9 instances with error 0:
$$MSE=\frac{41.11^2+0^2+...+0^2}{10}=169$$
This example shows that the only ...
0
Catboost and LightGBM can handle categorical features.
They're based on Decision trees (Random Forest is based on decision trees too), so you can use them (they're usually better than Random Forest), but they use more computational power comparing to Random Forest, yet you still can fine tune them (it's very easy with Catboost, LightGBM needs a little bit ...
0
Because sklearn uses the CART algorithm it cannot accept categorical data as-is (as you have pointed out). There is an existing ticket out to change this.
The issue really should be rephrased as supporting categorical splitting on Decision Trees - not Random Forest, as Random Forrest is simply the ensemble method using these decision trees as fitters.
I don'...
1
If you are talking about testing accuracy in this case (ie you are comparing results on data you didn't train with) - it's possible that adding more estimators is overfitting on your training set and is therefore performing poorly on your holdout set. If this is the case I would recommend approaching the problem with a more basic method such as ...
1
If you're just doing multiclass classification, you should specify the weights as a single dictionary, e.g. {0: 1.0, 1: 1.5, 2: 3.2} for a three-class problem. (Or use the convenience modes "balanced" or "balanced_subsample").
The list of dictionaries is used for multilabel classification (where each row can have multiple true labels). ...
2
From your confusion matrix it actually looks like your class imbalance is over-favoring the positive class (90.04% to 9.96%). Because you have such an extreme class imbalance in your data your random forest is likely just classifying almost everything as positive class and calling it a day.
To solve this issue you need to either down-sample your larger class ...
1
Edit: I misread your post: the answer is no it shouldn't matter which interval is used. So long as the order is not changed the splits your tree finds on this data will effectively be the same.
See my original answer if you want more context:
In scikit-learn specifically (I cannot speak for other implementations of tree-based models) it does not accept ...
1
First, note that Random Forests can handle categorical variables (moreover, if you have too much categories, reducing this number is a good practice).
If you want to apply a filter to your data, I'd suggest you using sklearn transformers (like OneHot Encoder, Label Encoding, ... pick the one you need according to what you want to do).
In this case, you have ...
0
You can get all the Trees using
clf.estimators_
Then you can traverse the Tree using custom code.[Check here]
Then you can take it to whatever format you want i.e. Array/DataFrame
A sample code to view one of the Tree
from sklearn import datasets
iris = datasets.load_iris()
X = iris.data
y = iris.target
clf = RandomForestClassifier(max_depth=3, ...
0
Note that there are works to optimally combine feature selection and getting SVM weights.
1
It is impossible to retrieve column names from a trained Random forest classifier from my experience, there is also a previous answer for an identical question.
0
After giving much consideration, let's review the mechanism of Random Forest Regression (RFR):
So this idea or process of averaging models is a technique called Ensembling. Additionally, Random forest is a Supervised Learning algorithm which uses an ensemble learning method for classification and regression. Random forest is a bagging technique and not a ...
0
In the case of Random Forest, a new tree is built without any input from the previously built trees. If the number of trees built is high, dropping any one tree when making a decision won't affect the final output of the random forest model unless the dropped tree holds information about an extreme outlier that impacts the ensemble model.
In the case of ...
3
In Random Forest, each trea of the forest is trained independant from the others. There's no relation between trees.
To summarise very quickly, if you have a dataset with 10 attributes, each tree will select n (a parameter you have to fix) attributes among the 10, and create a basic decision tree (like C4.5 style) only with those n attributes knowledge. Then,...
10
The two slightly-smaller models will perform exactly the same, on average. There is no difference baked in to the different trees: "the last tree will be the best trained" is not true. The only difference among the trees is the random subsample they work with and random effects while building the tree (feature subsetting, e.g.).
Gradient boosted ...
1
The way to understand Max features is "Number of features allowed to make the best split while building the tree". The reason to use this hyperparameter is, if you allow all the features for each split you are going to end up exactly the same trees in the entire random forest which might not be useful. To overcome this we let the model select a ...
0
I can't add comments yet, so that's why I'm making a post.
It is natural that for a time series data (I assume that's the kind of data you have if you don't want to shuffle) without shuffling you get worse results. Imagine this toy example. You want to predict sales of a grocery store where your data is sales of the store for every day. If you shuffle you ...
2
With XGBoost you can come up with your own loss and metric. It is relatively simple to just add a custom loss. However, I have no experiance with problems described by you, so you would need to see if what you have in mind will fit into the standard XGB.
Find an implementation of custom loss (R) here: https://github.com/Bixi81/R-ml/blob/master/...
2
First some prior and known aclarations (that you probably already know).
Metric is what we want to optimize.
Optimization Loss is what the model optimizes.
Obviously, we would like the Metric and the optimization loss to be the same, but this is always not possible. How to deal with this?
Run the right model. Some models can optimize different loss ...
Top 50 recent answers are included
