1
$\begingroup$

I trained a model which does well on unseen data,but after deployed on production the data I got is very different,like the highest values in train & test data is ~23,but the data I got from production have values which is ~80,and there is just 1 feature and it is regression problem. Can anyone please recommend what can I do to tackle this problems.

$\endgroup$
1
  • $\begingroup$ By reading some articles I got to know for tackling these types of problem we use 'convariate shift' but I cannot use that because for 'covariate shift' there should be more than 1 feature. $\endgroup$ Sep 10, 2019 at 6:28

1 Answer 1

1
$\begingroup$

To me it seems like you are training on the wrong data. The results of your algorithms will only be as good as the data you train it on. First question you should ask yourself is, if it is possible to transform or update your dataset to make it more realistic. Without deeper insight into the nature of the problem, it is, unfortunately, very hard to help in this regard. Still, in my honest opinion, this will do for the cleanest solution.

If you can not update your training data to represent the real-world distribution, your problem is quite similar to trying to transfer simulation data to real-world applications, like often seen in robotics. Usually, the authors try to learn the transformation between the simulation and real-world data explicitly like shown in this nice paper about optimization. Briefly summarized, the authors model the data by two Gaussian Processes (GPs), one resembling simulation data, whereas the other is the transformation I mentioned above. Thus, your real world data would result as the sum of the two kernels from the GPs. I am sure a similar approach is applicable for your problem, even without using GPs as you (most likely) are not interested in Bayesian Optimization.

Still, if you want go for this method, you will also need to gather some more data from your real distribution, just not a whole new dataset.

$\endgroup$
1
  • $\begingroup$ Thanks for suggestions. :) $\endgroup$ Sep 10, 2019 at 11:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.