I am mentioning Time-GPT here as a placeholder example. It can be any pretrained model.

Suppose I have a dataset that requires some time series prediction. How can I leverage a well-trained model and transfer that learning to make a better prediction for my model?

Asking this because, in almost all cases the I/P and O/P dimensions of Time-GPT and my dataset would be different. Then how can I use it?

Could you share some resourceful git repo with similar examples?

  • 1
    $\begingroup$ Hi @MohammadMosiur, welcome to the site. For transfer learning, usually you need the pretraining task to be somewhat related to the downstream task. In this case, language modeling is not related at all to time series prediction. At least you should take a model that is also pretrained in time series, like TimeGPT. $\endgroup$
    – noe
    Feb 10 at 15:13
  • $\begingroup$ Welcome to the forum. I'm not sure I understand your question - in general, time series are solved by using techniques like ARIMA, recurrent neural networks, and such. Llama-2 is a language model, useful for language tasks. Getting started with time series, you can follow e.g. the kaggle time series tutorial: kaggle.com/learn/time-series $\endgroup$ Feb 10 at 15:15
  • $\begingroup$ Thanks for your comments. I have edited my question. Actually I want to know how can I leverage a pretrained model for training my model. Almost all cases I/P and O/P dimensions do not match. $\endgroup$ Feb 11 at 3:07
  • $\begingroup$ Are I/P and O/P abbreviations for "input" and "output"? It might be worth describing the type of input you want to use, and showing an example, and the same for your desired model output, so people can give you a more concrete example. $\endgroup$ Feb 12 at 17:35
  • $\begingroup$ @noe Re "language modeling is not related at all to time series prediction". Both can be phrased as given a sequence of previous tokens, predict the best next token, so they are not that dissimilar. It does depend on the length of the signal though. 12 months of daily bars (finance data) is not too far off a sentence. But a 5 minute song recorded at 48Khz is (probably) not going to work. $\endgroup$ Feb 12 at 17:42


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.