New answers tagged decision-trees
0
Decision tree are deterministic so will always make the same split if given the same data.
A single decision tree will be make splits conditional on previous splits (greedily taking the best split for either the previous split feature or other features). Separate trees per feature will only split conditional on the feature the tree has access to.
In general, ...
0
The splitting criteria for float-type features is covered in paper in section "3.3 Weighted Quantile Sketch" and the Appendix.
Quantile sketch divides the data into weighted percentages and does the split based on that approximation.
The split value does not have to unique value in the feature because the algorithm uses approximations.
1
how can I get from scikit learn BOTH the result and the probability?
You can simply run both:
The results will always be consistent because there is no randomness involved at the prediction stage, only at training stage.
The computations required for predicting are not intensive, so I don't think there can be any major efficiency issue running it twice.
...
0
If you plot with sklearn.tree.plot_tree, there is a parameter for feature_names:
feature_names: list of strings, default=None
Names of each of the features. If None, generic names will be used (“X[0]”, “X[1]”, …).
0
It totally makes sense. You can also use count encodings. So, rare values tend to have similar counts (with values like 1 or 2), so you can classify rare values together at prediction time. Common values with large counts are unlikely to have the same exact count as other values. So, the common values get their own grouping with these way.
For overfitting, I ...
0
You can frame this issue as feature importance. Which features have the greatest influence on the target value of churn rate?
There are many ways to approach feature importance. In decision trees, permutation importance can be used.
0
It sounds like the problem can solved with business rules - have an expert write down what choices should be made by whom under what circumstances.
Not all problems can be solved with machine learning.
0
Lets take an example. The league is Premier League and the teams playing are Chelsea, X, Y and Z (sorry I don't follow football!). So now you have data for all 4 teams for Premier League. Now comes Champions League and the teams playing are Chelsea, Y and Z (X did not get selected for some reason).
Now ask yourself if you should consider data only for ...
0
You can create hierarchical models. The first model in the hierarchy would only get the single feature. The next model in the hierarchy would get the other features.
Scikit-learn does not natively support hierarchical modeling. You would have to write custom code.
1
You need to include all competitions for a simple reason: you'll not have enough data if you do not. (Keep in mind that ML models generelly need large datasets while you only have a couple of matches for a given team in a given year in a given competition if it is not the national league)
In their paper Learning to predict soccer results from relational data ...
1
Your main problem (it turns out, thanks for following up in the comments) is that you used the raw coefficients from the logistic regression as a measure of importance, but the scale of the features makes such comparisons invalid. You should either scale the features before training, or process the coefficients after.
I find it helpful to emphasize that ...
0
Resampling is generally a good idea when dealing with heavily imbalanced datasets. I would recommend using smote which can either resample or undersample depending on your task (if your dataset is small I would say to use resampling). Another thing you could try is using class weights when training. Models like lightgbm and neural nets can use weighted ...
0
As you well mentioned, tree-based models are not sensitive to feature scaling, but on the contrary it might help with the convergency of finding the minimum in the optimization on boosted models
I replicated your code and I found pretty much the same metrics in both scaling and no scaling versions of the model.
from sklearn.datasets import load_boston
from ...
0
Elo rating system is a very useful way to model sport matches by calculating the relative skill levels of different competitors. The difference in Elo ratings between two competitors serves as a predictor of the outcome of a match.
One formula for soccer Elo is:
$$Rn = Ro + K × (W - We)$$
Rn is the new rating
Ro is the old (pre-match) rating;
K is the ...
0
The important point here is the distinction between rule-based and data-driven:
A rule-based predicting system is an algorithm which calculates the target variable based on rules which have been predetermined and implemented by a human expert.
A data-driven predicting system is an algorithm which is first trained on some labelled training data in order to ...
-1
Keep the data as is and then predict since the data outside the competitions does not make any difference to the performance of the player.
Try using Random forest since multiple variables like home team, away team, league, home score, and away score are involved, and since it uses ensemble techniques thus provides a more accurate result as compared to other ...
1
The core principle in supervised machine learning is that the training data is a representative sample of the true distribution (i.e. the possibly infinite full set of instances that could happen).
Under this assumption, the intuition and the numerical information gain (or other statistical measure) are expected to be more or less in agreement, because if ...
0
Every machine learning algorithm suffers from curse of dimensional to different degrees. The primary issue is the sparsity of observations as the number of dimensions increase.
Since decision trees are greedy, they are one of the most robust machine learning algorithms to sparsity. Decision trees will automatically find the single feature to that best ...
Top 50 recent answers are included
