New answers tagged

0

The default strategy for calibration_curve is 'uniform', i.e. each of the bins has equal width. If, after calibration, your model makes no predictions inside a bin, there will be no point plotted for that range. You could change to strategy='quantile', which would guarantee 10 points plotted for each curve; you'll get many more of the red/yellow dots ...


3

You're also scaling $y$, then of course you are getting lower error. That question was regarding scaling $X$. The same model will have very different error metrics when units on $y$ are changed: if I multiply all $y$ values by 100, the error will be 100 times larger, if I divide all $y$ values by 100 the error will be divided by 100.


2

These are several things you can try: Use quartic error, $(y - \hat{y})^4$, instead of quadratic error. This is going to penalize a lot big errors, way more than MSE. The issue is that this is not implemented in xgboost, and you would need to develop a custom loss. If your target is always positive, you can use the target as training weights. This will give ...


1

When you are working with time-series data, the most recent data captures the most relevant information possible, so it is more prudent to include them in training data. So a more prudent decision would be to opt for Roll-Forward Partitioning. Roll-Forward Partitioning: We start with a short training period and we gradually increase it, at each iteration of ...


1

What you were told is a worst case scenario. With 5 labels, 20.01% is the lowest possible value that a model would need to choose one class over the other. If the probability for each of the 5 classes are almost equal then the probabilities for each would be approximately 20%. In this case, the model would be having trouble deciding which class is correct. ...


2

The amount of data you need depends on the problem (see this great article on learning curves), but in general xgboost is very data efficient like random forests and has found a lot of use where data is expensive to produce as in medicine. Try it out on your data and plot a learning curve - if it is under-fitting, you need more data.


1

No. Xgboost is more like traditional ML algorithm. It doesn't need too much data and it also perform better than almost all ML algorithm


1

If you mean "how many times the same feature can appear in an [individual] tree", then you can use max_depth to indirectly limit the number of features included in a single tree, even down to one feature. Since XGBoost is designed to use weak learners, having a lower depth value is ok. model = XGBClassifier(max_depth=n) However, I think the ...


0

You can use this chunk of code to plot the feature importance of your data. It is also possible that your data is columned with decreasing importance. from xgboost import XGBClassifier from xgboost import plot_importance from matplotlib import pyplot # load data X = y = model = XGBClassifier() model.fit(X, y) plot_importance(model) pyplot.show()


2

GradientBoostClassifier is slower but more precise. In your case, it could be finding a better model without suffering from overfitting. Here are some of the main differences you are looking for. XGBClassifier was designed to be faster. However, XGBClassifier takes a few shortcuts in order to run faster. For example, to save time, XGBClassifier will use an ...


1

With Success already being the larger class, you probably shouldn't be using a scale_pos_weight larger than one: you want to scale the positive class's contribution to the loss function down rather than up. I suspect that's what's happening in the first case. With scale_pos_weight=75, the model ends up basically only caring about the positive class, ...


1

Performance of a model can be judged along several dimensions: Accuracy - Is the train-test validation performing well? Overfitting - Is the difference between training score and the validation score minimal? Efficiency - Is the model light-weight, does it compute and calculate fast? Complexity - Is the model easy to explain, does it use minimal ...


0

Yeah, permutation tests are very slow when having relatively big data. For this reason, the question you've cited answers that boostrapping is an alternative to the permutation test. If you want to do a permutation test, there's code for it in that question. However, some practitioners think that doing cross-validation is a better way to compare models. This ...


1

You can find a good explanation and example on creating a custom objective function here: https://xgboost.readthedocs.io/en/latest/tutorials/custom_metric_obj.html https://github.com/dmlc/xgboost/blob/master/demo/guide-python/custom_objective.py https://github.com/dmlc/xgboost/blob/master/demo/guide-python/custom_rmsle.py Original XGB code repo is here, you ...


1

Writing a custom loss function could be handy, but it may be simpler to just try to treat this as a class balance problem for your regression model. For starters, just try undersampling all of the higher and medium grades until they're close to balanced with your failing students. Given your number of data points and features you can probably still just ...


1

Try writing a custom loss function for a regression model! Keras' neural networks support this, for example. See https://stackoverflow.com/q/43818584/745868 (But many other libraries give support for this as well) The only special thing about your custom loss function is that it doesn't add up the error of a datapoint if min(pred_y, actual_y) >= THRESHOLD


1

I would try two different approaches: interpolate the missing values on a user level. work with the sunset of rows for which we actually have the glucose level. Then, I would compare the test accuracy of the model built with both methods. Remember that your test set has to be composed of rows for which you have the glucose level - you cannot build it with ...


1

The first question about missing data is always why is it missing? Have you checked or know why the data is missing and whether it is MAR, MCAR or not missing at random? If your data is MCAR imputation is generally fine and your lower test metric might simply indicate a suboptimal imputation strategy. In this case you could try MICE or similar more advanced ...


1

First you should define a metric that suits the problem $R^2$ in your case. Do a correct cross-validation and train test splits. And then choose in the cross validation which option has the best results for your model (imputing missing or xgboost no imputing). This way you are doing an empirical experiment and selecting the best result. Probably you want to ...


0

Your modelfit prints the training score, but GridSearchCV bases its decisions on the out-of-fold average (and in particular best_score_ is an out-of-fold average score). This is an unfair comparison, and in particular your 0.577 is probably quite optimistically biased.


1

Flier values/skewed predictors will have a high influence on the regression model. If you want to counteract that, you have a few choices. 1) If your target is always non-zero, and if you expect the regression to be close to linear, you can try to use a log(), sqrt() or even boxcox() conversion transform on the target variable. This will help keep the large ...


0

Since this might be a question for other people, here are the results of my findings: I have tried two options using XGB Regression with different objective functions including: A linear regression objective function ("reg:linear" or "reg:squarederror") and transformed the target to the log space A gamma objective function ("reg:...


2

I am working almost on the same problem these days: I have tried two options using XGB Regression with different objective functions including: Using a linear regression objectiive function ("reg:linear" or "reg:squarederror") and transformed the target to the log space Using the gamma objective function ("reg:gamma"), which is useful for a skewed target ...


Top 50 recent answers are included