I have practical machine learning problem. I have trained a LightGBM model to predict house prices. Compared with other models I have tried, the loss (RMSE) is quite low and overall I'm quite happy with the predictions.
When investigating the predictions by tweaking features, I have noticed one problem. The trend is correct thus that when I increase size of house, the price generally goes up. However, in many cases a small change in a feature can result in prediction change in the wrong direction. Let's say all features remain the same but I increase house size a little and the predicted price goes down for example. If I increase it further it goes up again.
Are there any some techniques to handle this problem, and get "smoother", more continuous predictions?