Think carefully before you do this. You have no idea what the underlying height distribution is. Here are four possibilities.
If you were building a regression model, each of these sets of height data would be interpreted differently. However, if replaced by your ordinal variable, they all look numerically equivalent. If you use this variable as a ...
You should not use Label Encoding for Categorical data unless there is a known ranking and that also in the specified ratio between the level values.
In this case, the model will assume 10 as 2 times of 5.
One-hot will add a lot of dimensions as I can see in your data.
You must try other Categorical encoding techniques esp. Sum Coding Or Helmert.
Taking the log doesn't result in a normally-distributed target; it would tend to if the target was log-normally distributed, and you have something normalish there, not quite. But, this distribution isn't actually what matters.
What taking the log does is change your model of how errors arise when fitting a regressor. You're now saying that the target ...
I am working almost on the same problem these days:
I have tried two options using XGB Regression with different objective functions including:
Using a linear regression objectiive function ("reg:linear" or "reg:squarederror") and transformed the target to the log space
Using the gamma objective function ("reg:gamma"), which is useful for a skewed target ...
It's impossible in general, simply because a particular value or range for feature A might correspond to class 'good' if feature B has a certain value/range but correspond to class 'bad' otherwise. In other words, the features are inter-dependent so there's no way to be sure that a certain range for a particular feature is always associated with a particular ...
Writing a custom loss function could be handy, but it may be simpler to just try to treat this as a class balance problem for your regression model. For starters, just try undersampling all of the higher and medium grades until they're close to balanced with your failing students. Given your number of data points and features you can probably still just ...
Try writing a custom loss function for a regression model!
Keras' neural networks support this, for example. See https://stackoverflow.com/q/43818584/745868
(But many other libraries give support for this as well)
The only special thing about your custom loss function is that it doesn't add up the error of a datapoint if min(pred_y, actual_y) >= THRESHOLD
The first question about missing data is always why is it missing?
Have you checked or know why the data is missing and whether it is MAR, MCAR or not missing at random?
If your data is MCAR imputation is generally fine and your lower test metric might simply indicate a suboptimal imputation strategy. In this case you could try MICE or similar more advanced ...
First you should define a metric that suits the problem $R^2$ in your case.
Do a correct cross-validation and train test splits.
And then choose in the cross validation which option has the best results for your model (imputing missing or xgboost no imputing). This way you are doing an empirical experiment and selecting the best result.
Probably you want to ...
Flier values/skewed predictors will have a high influence on the regression model. If you want to counteract that, you have a few choices.
1) If your target is always non-zero, and if you expect the regression to be close to linear, you can try to use a log(), sqrt() or even boxcox() conversion transform on the target variable. This will help keep the large ...
@Ben Norris found out that the relaimpo packages has a hard minimum number of observations, so if I wanted to pursue this path I have to up my sample size.
As I only have the data that I have, I pursued a "hacky" solution which I am going to describe for completionists sake. The steps were as follows:
Assign each individual to one of k groups randomly, so ...
Of course your error rate is going to decrease. Remember that your changes in MAE values may come from the fact that the scales of your original variable and that variable transformed by a logarithm ARE NOT THE SAME, and mean is scale-dependant.
About your second question, is exactly that!
If you would like to compare the use (or not) of logarithms. You ...
I think it's complete alright. In fact, the second model mathematical expression is given by
y=x3f(x1, x2, x3),
which is just like the first model but with some specific feature engineering. I don't see any possibility for data leakage.