I am using XGBoost for a classification problem. The model has multiple inputs and a probability as output.
For simplicity's sake, let's say the model only has one continous input and the function to be learned is f(x) = x.
I am creating training samples for it using the f(x) = x relationship. Of course, in reality, I cannot create an endless amount of training samples and am restricted by the given data set.
The model output would look something like this:
As can be seen, the model predictions are noisy. As the x variable goes from 0 to 1, the y variable jitters around the true relationship. I would want to reduce / penalize this jitter. For every delta x, I want to to keep the delta y as small as possible but as high as necessary. Are there any model parameters to do this? Of course, I can just smooth the predictions ex post but I'd like to tell it to the model directly to make the smoothness vs. optimal prediction trade-off.
