0
$\begingroup$

I have a time series data that handled using GDBT to predict the next value. I always use previous 30 days data to train daily, but overtime the data to predict and train is increased because the number of combination things increased.

My question is, how often do we need to hypertune our model? and what eval number combination considered to be enough for hyperparameter tuning? 50? 100? or just 10 is enough?

right now I do it daily, but it getting costly and costly, previously it just aroung 10 minutes, but now about more than hour. The system need to do other things hourly so this hypertuning model will be a deadlock for the system.

Improve this question
$\endgroup$
1
  • $\begingroup$ Are you actually asking just about hyperparameter tuning, or retraining scheduling in general? $\endgroup$
    – Ben Reiniger ♦
    Commented Aug 11, 2019 at 15:35

1 Answer 1

Reset to default
1
$\begingroup$

Some ideas:

  • Number of previous observations to use: depends on the process you are modelling. If the target is related in some way to many of the previous values, you may need to use more data in your tuning process. On the other hand, if the target is largely unrelated to previous observations you can use fewer. You will need to check the relationship - lag plots and auto-correlation plots may be useful.
  • Frequency of parameter tuning: again, depends on your model and process. It lay be a good idea to monitor the prediction accuracy of your model, and to only perform more parameter tuning if the model accuracy drops below a certain threshold.
Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.