New answers tagged decision-trees
1
##Entropy
def entropy(Y):
"""
Also known as Shanon Entropy
Reference: https://en.wikipedia.org/wiki/Entropy_(information_theory)
"""
unique, count = np.unique(Y, return_counts=True, axis=0)
prob = count/len(Y)
en = np.sum((-1)*prob*np.log2(prob))
return en
#Joint Entropy
def jEntropy(Y,X):
"""
H(Y;X)
Reference: ...
1
You should impute missing values, try with:
from sklearn.impute import SimpleImputer
imp = SimpleImputer(missing_values=np.nan, strategy='mean')
imp.fit(x_train)
x_train = imp.transform(x_train)
x_test = imp.transform(x_test)
Notice that I am fiting just in the train data, so you are not leaking information to the test.
2
My experience is with sklearn and python.
A Random Forest is a bagging of decision trees, the sklearn package uses a Decision Tree Classifier which is a CART.
You can see in this post that I made a bit ago of how to build a Random Forest tree the exact same as a decision tree.
2
What helps the model more, keeping all features or removing correlated ones?
There is some theory about it but in the end Machine Learning is try and error. You should give it a try with all features and then doing a feature selection to see if you are able to improve your model. What works for some models doesn´t necessarily have to work for the rest of ...
1
Experimentally: using cross-validation on a subset of your training data, compute the performance of every option that you want to consider. Then select the best option and train the final model using this option.
// different settings for hyper-parameters,
// for instance different pruning criteria:
hpSet = { hp1, hp2, ...}
trainSet, testSet = split(...
0
Your question is not very clear: do you mean that your test data never contains this feature?
If yes, you should remove this column from the training data. The train and test data must have the same features.
If no, i.e. only some instances might not have a value for this column, then it's about having missing values in your data. In this case you could ...
1
Now with the new version 0.22.1, you can! It does pruning based on minimal cost-complexity pruning: the subtree with the largest cost complexity that is smaller than ccp_alpha will be chosen.
https://scikit-learn.org/stable/auto_examples/tree/plot_cost_complexity_pruning.html
1
You can use categorical features with decision trees in scikit-learn, but you'll need to encode them as numbers. If your categorical features are ordinal (such as ranking ‘bad’, ‘fair’, ‘good’), they are easy to encode in numbers that respect the underlying ordering (e.g. 0, 1, 2). For nominal features, given the high cardinality you mention, you can try ...
0
Decision trees are supervised methods, so they need to be trained on some annotated data. Thus the general idea is the same as for any text classification: given a set of documents (for instance represented as TFIDF vectors) together with their labels, the algorithm will calculate which how much each word correlates with a particular label.
For instance it ...
0
Which function are you using at Loss? Using the right one is important when dealing with imbalanced datasets. 7% is imbalanced, but not that bad.
Have you tried any eXplainable Artificial Intelligence (XAI) method? Normally I use Shap.
It is really good to see which feature contributes in which direction. You can see an example here.
You can check the ...
0
LightGBM and XGBoost Libraries can handle missing values
LightGBM: will ignore missing values during a split, then allocate
them to whichever side reduces the loss the most
XGBoost: the instance is classified into a default direction (the optimal default directions are learnt from the data somehow)
Finally, it is NOT a general property of ...
0
Your problem can be considered as multiclass classification problem. So, you have a dataset of features X and the predictor y. Where X contain Income, age, sex,etc. and y is an item that one customer will buy with higher probability.
To achieve your goal and predict the probability of a customer you can use any classifier from scikit-learn Library (if you ...
1
By performing GridSearch I understand you want to say searching for the best hyperparameters.
For sake of simplicity, let say that you want to fit a linear regression with a penalty (lasso/ridge) with 1 feature and with 100 features. The hyperparameter that you are looking for is the $\lambda$ penalty.
It is easy to see that with 1 feature your model ...
0
I don't think there is a C4.5 implementation in a popular python library. Your options are :
Try github implementations such as : https://github.com/geerk/C45algorithm
Try getting R implementations with rpy package : see over there for an exemple of how that would work (C5.0) https://stackoverflow.com/questions/41070087/calling-c5-0-in-python
4
You need to set bootstrap=False in the random forest to disable the subsampling. (I originally commented because I expected there to be more impediments [in addition to your already-coded random_states and max_features=None], but I guess there aren't any!)
You probably don't want to do this in general; by stripping out all the randomness so that the first ...
Top 50 recent answers are included
