2
GradientBoostClassifier is slower but more precise. In your case, it could be finding a better model without suffering from overfitting.
Here are some of the main differences you are looking for.
XGBClassifier was designed to be faster. However, XGBClassifier takes a few shortcuts in order to run faster. For example, to save time, XGBClassifier will use an ...
2
I am working almost on the same problem these days:
I have tried two options using XGB Regression with different objective functions including:
Using a linear regression objectiive function ("reg:linear" or "reg:squarederror") and transformed the target to the log space
Using the gamma objective function ("reg:gamma"), which is useful for a skewed target ...
2
The amount of data you need depends on the problem (see this great article on learning curves), but in general xgboost is very data efficient like random forests and has found a lot of use where data is expensive to produce as in medicine. Try it out on your data and plot a learning curve - if it is under-fitting, you need more data.
2
These are several things you can try:
Use quartic error, $(y - \hat{y})^4$, instead of quadratic error. This is going to penalize a lot big errors, way more than MSE. The issue is that this is not implemented in xgboost, and you would need to develop a custom loss.
If your target is always positive, you can use the target as training weights. This will give ...
1
What you were told is a worst case scenario. With 5 labels, 20.01% is the lowest possible value that a model would need to choose one class over the other. If the probability for each of the 5 classes are almost equal then the probabilities for each would be approximately 20%. In this case, the model would be having trouble deciding which class is correct.
...
1
No. Xgboost is more like traditional ML algorithm. It doesn't need too much data and it also perform better than almost all ML algorithm
1
If you mean "how many times the same feature can appear in an [individual] tree", then
you can use max_depth to indirectly limit the number of features included in a single tree, even down to one feature. Since XGBoost is designed to use weak learners, having a lower depth value is ok.
model = XGBClassifier(max_depth=n)
However, I think the ...
1
With Success already being the larger class, you probably shouldn't be using a scale_pos_weight larger than one: you want to scale the positive class's contribution to the loss function down rather than up.
I suspect that's what's happening in the first case. With scale_pos_weight=75, the model ends up basically only caring about the positive class, ...
1
Performance of a model can be judged along several dimensions:
Accuracy - Is the train-test validation performing well?
Overfitting - Is the difference between training score and the validation score minimal?
Efficiency - Is the model light-weight, does it compute and calculate fast?
Complexity - Is the model easy to explain, does it use minimal ...
1
You can find a good explanation and example on creating a custom objective function here:
https://xgboost.readthedocs.io/en/latest/tutorials/custom_metric_obj.html
https://github.com/dmlc/xgboost/blob/master/demo/guide-python/custom_objective.py
https://github.com/dmlc/xgboost/blob/master/demo/guide-python/custom_rmsle.py
Original XGB code repo is here, you ...
1
I would try two different approaches:
interpolate the missing values on a user level.
work with the sunset of rows for which we actually have the glucose level.
Then, I would compare the test accuracy of the model built with both methods. Remember that your test set has to be composed of rows for which you have the glucose level - you cannot build it with ...
1
Writing a custom loss function could be handy, but it may be simpler to just try to treat this as a class balance problem for your regression model. For starters, just try undersampling all of the higher and medium grades until they're close to balanced with your failing students. Given your number of data points and features you can probably still just ...
1
Try writing a custom loss function for a regression model!
Keras' neural networks support this, for example. See https://stackoverflow.com/q/43818584/745868
(But many other libraries give support for this as well)
The only special thing about your custom loss function is that it doesn't add up the error of a datapoint if min(pred_y, actual_y) >= THRESHOLD
1
The first question about missing data is always why is it missing?
Have you checked or know why the data is missing and whether it is MAR, MCAR or not missing at random?
If your data is MCAR imputation is generally fine and your lower test metric might simply indicate a suboptimal imputation strategy. In this case you could try MICE or similar more advanced ...
1
First you should define a metric that suits the problem $R^2$ in your case.
Do a correct cross-validation and train test splits.
And then choose in the cross validation which option has the best results for your model (imputing missing or xgboost no imputing). This way you are doing an empirical experiment and selecting the best result.
Probably you want to ...
1
Flier values/skewed predictors will have a high influence on the regression model. If you want to counteract that, you have a few choices.
1) If your target is always non-zero, and if you expect the regression to be close to linear, you can try to use a log(), sqrt() or even boxcox() conversion transform on the target variable. This will help keep the large ...
Only top voted, non community-wiki answers of a minimum length are eligible
