4
No, extremely-random trees does still optimize splits. It does only pick one random splitting point for each feature (out of those randomly chosen max_features) but then which feature is actually used for the split depends on the criterion chosen.
https://scikit-learn.org/stable/modules/ensemble.html#extremely-randomized-trees
3
Your chart seems to show that light GBM models are very inconsistent in terms of F1 score. The other two types of model tend to have lower validation accuracy than training accuracy, suggesting overfitting is occurring to some extent (but this is ubiquitous in machine learning so it’s not a deal breaker by any means). The best median validation performance ...
2
A Random Forest model can definitely be used to help you determine feature importances. Actually, it is used as a very common strategy for feature selection. If your data is too small, my recommendation would be to treat it as if you were to make predictions with this model, meaning that you should watch out for overfitting and do a proper hyperparameter ...
2
This comes from the Binomial distribution, where you have $n=1000$ independent trials (models), $p=0.51$ of each model being right and since you care about the majority vote you want to have at least $k=500$ successful trials. That leads to:
$$\text{Pr}(k\geq500 \text{ models are right}) = \sum^{1000}_{k=500}\binom{1000}{k}0.51^{k}(1-0.51)^{1000-k}=0.74675\...
1
A random subset of features than using the best split logic as done in normal Tree
The Random Forest algorithm introduces extra randomness when growing trees; instead of searching for the very best feature when splitting a node, it searches for the best feature among a random subset of features. The algorithm results in greater tree diversity, which (again) ...
1
Most probably, sum across one of the rows is coming out as zero in this code
matrix = matrix.astype('float') / matrix.sum(axis=1)[:, np.newaxis]
It is throwing a runtime warning and only that particular cell will be np.inf. Rest all division will be fine. That's why the plot is showing some data.
You may see the same in this sample code
import numpy as np
...
1
Where are you evaluating the performance of your algorithm?
Are you making a train test split and evaluating in the test split? It might be that you overfitted your train and you are just measuring the accuracy there.
If you have made correctly the train/test split and the evaluation it could be that the images that you are predicting do not have the same ...
1
Gini decrease is calculated based on the mean decrease in Gini i.e. $p_i(i-p_i)$ each time when the Tree is splitted on that Feature. Value is so high because the r package weight the impurities by the raw counts, not the proportions.
Accuracy decrease is calculated on OOB dataset by randomly shuffling the data for that particular feature in the OOB. Then ...
1
If you are looking for a statistical trick, I don't know, but
Recently Andrew NG team recently published about NGBoost.
NGBoost is a new boosting algorithm, which uses Natural Gradient Boosting, a modular boosting algorithm for probabilistic predictions.
In this Towards Data Science toy example you can see how to use the Python API:
Quoting the TDS author:
...
1
The criterion parameter is used to measure the quality of the split when selected, it is not involved in the initial splitting algorithm (the features used for the split are chosen randomly)
ExtraTreesRegressor:
mse and mae are the only options available for use, and mse is the default. mae was added after version 0.18. Check your version if it is available....
1
You have to do
RF <- randomForest(sale ~ v1 + v2 + v3, data = TrainSet, importance = TRUE)
this is the formula notation for R. It doesn't make a lot of sense for random forest models, but it is how it works.
1
The performance of the machine learning models significantly depends on the type of data that you are using. Also, it is required to use a statistical test to compare the performance of two models given the data used for training and testing. I want to say that it might be misleading to say that one machine learning model performs better than the other.
In ...
1
Since size seems to be a categorical variable you can just go ahead and treat all blank values as an additional variable level. This is regardless of the specific algorithm you're using.
1
In theory decision trees (and random forests) are able to deal with missing values in the data. But whether a particular implementation of the algorithm allows this (and how to use it with this implementation) depends on the specific package.
1
I would use temporal difference learning from reinforcement learning. Temporal difference learning employs TD propagation rather than backpropagation. The difference being that TD takes into account the time delay aspect. In fact, it is likely in this scenario that combining the two propagation methods would be optimal.
1
I have less than enough karma to leave a comment, but I'd like to support Derek O's assessment but also add 1 more point: if your 50K observations are repeated measures (multiple rows coming from the same individual or unit) - then you'll want to make sure that in your cross-fold setup that you are ensuring that 100% of each individual's observations are ...
1
If you can find a column that has a value to select on, you can use stratify in the train_test_split function. Stratify will try to select an equal number of cases of each value, similar to what you are using. You won't capture all of them, just an equal sample of value vs non-value, but this would be a better approach than forcing a non-random sampling on ...
Only top voted, non community-wiki answers of a minimum length are eligible
