In a regression problem that I'm currently working on, it seems that my model is doing well on higher values but significantly worse on lower values (e.g. values from 100,000,000 to 105,000,000 are being accurately predicted/ having lower error scores while values from 1,000,000 to 5,000,000 don't).
One approach that I am planning to test out is using multiple regression models, with one trained on the lower values and one on the higher values. I've seen scikit-learn's VotingRegressor, but if I understand correctly it seems that in predicting the value it'll only average the result from the estimators.
Other than using average values from the estimators, are there any other approaches to do the voting from multiple regression models? Since classification problems might use soft/hard voting, wondering if there are alternative approaches in regression problems as well.