I have a GradientBoostingRegressor from
scikit-learn which I trained. Afterwards, I obviously would like to know how good the model is. So, on a non-scaled dataset I would just use the
mean_squared_error function from scikit and it would output a certain value that made sense (regarding the dataset).
Now, I scaled (/ transform) the targets in my dataset using the scikit
QuantileTransformer(output_distribution='uniform'). The target is now scaled between 0 - 1. This is fine during training etc.
After training the model, I ran the following code to get a few metrics:
test_pred = gb.predict(X_test) mse_test = mean_squared_error(y_test, test_pred) print("RMSE on Test:", np.sqrt(mse_test)) print("MSE on Test:", mse_test) mae_test = mean_absolute_error(y_test, test_pred) print("MAE on Test:", mae_test)
Because the target values are scaled, the output is something similar to this:
RMSE on Test: 0.23563730007705744 MSE on Test: 0.05552493718760521 MAE on Test: 0.19235478752773819
I assumed that I could get the 'actual' non-scaled metrics back by applying the
QuantileTransformer.inverse_transform function to the output.
So then I got:
RMSE on Test: 2231.21330222 MSE on Test: 807.28588575 MAE on Test: 1888.23406628
Which doesn't seem very correct to me. Normally, the RMSE would be smaller than the MSE, but that isn't the case. If you get that the sqrt of a (MSE) value under 1. Also, the MAE should probably be smaller than the MSE.
My question is, how do you interpret those scaled metric values? Is the inverse_transform output correct? How do I get correct, non-scaled values for the metrics?
I'd appreciate some help on this.
QuantileTransformer is only an example. The question also applies to the
MinMaxScaler and other scalers in general.