I am confused that both are same or not, and then how can I differentiate with the online training model.

  • $\begingroup$ Pretrained models are models which are trained on larger datasets along with high computation power. They tend to have high accuracies and have a higher level of generalisation. Offline models could be models which are trained on a local machine and not on a server or a cloud. $\endgroup$ Mar 28, 2019 at 8:10

2 Answers 2


Every pre-trained model is an offline-trained model, but not the reverse.

Offline training is any training that leaves the model unchanged when new observations arrive, i.e. it has an end. Online training constantly updates the model with the help of new, incoming observations without using the previous training points, although, having a limited memory of previous samples compared to all seen samples is OK. Therefore, training offline periodically on all the training points, no matter how frequent, is not the same as online training.

  • We can use offline training for a model that supports online training, however, to train online, the model must allow such training. For example, common implementation of SVM cannot be trained on N new data points without re-using the previous training points. In contrast, Bayesian models are natural candidates for online learning, as they are trained by updating the belief (model) upon new observations, i.e. posterior update.

Pre-training is an offline training followed by a main, task-specific training, hence the prefix "pre".

For example, a $512 \times 512 \times 3 \rightarrow 128$ dimensionality reduction model is pre-trained on a large dataset of RGB images (either supervised, or self-supervised). Then, the pre-trained model is used to reduce the dimension of our [new] images to $128$, which is then being fed to the main, task-specific model.

Note that the word "self-supervised" (supervised by input itself) is currently used for models that try to reconstruct, or predict the whole, or part of the input as close as possible; e.g., auto-encoders, or some language models such as Word2Vec.


enter image description here

Online Model :

Model that continuously learns in production. If 10 new training samples are available, we do not need to retrain with all previous samples.

Pre-trained model

Model has already been trained on large data sets. This is a useful technique via transfer learning.





Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.