I am confused that both are same or not, and then how can I differentiate with the online training model.
$\begingroup$
$\endgroup$
1
-
$\begingroup$ Pretrained models are models which are trained on larger datasets along with high computation power. They tend to have high accuracies and have a higher level of generalisation. Offline models could be models which are trained on a local machine and not on a server or a cloud. $\endgroup$– Shubham PanchalMar 28, 2019 at 8:10
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
Every pre-trained model is an offline-trained model, but not the reverse.
Offline training is any training that leaves the model unchanged when new observations arrive, i.e. it has an end. Online training constantly updates the model with the help of new, incoming observations without using the previous training points, although, having a limited memory of previous samples compared to all seen samples is OK. Therefore, training offline periodically on all the training points, no matter how frequent, is not the same as online training.
- We can use offline training for a model that supports online training, however, to train online, the model must allow such training. For example, common implementation of SVM cannot be trained on N new data points without re-using the previous training points. In contrast, Bayesian models are natural candidates for online learning, as they are trained by updating the belief (model) upon new observations, i.e. posterior update.
Pre-training is an offline training followed by a main, task-specific training, hence the prefix "pre".
For example, a $512 \times 512 \times 3 \rightarrow 128$ dimensionality reduction model is pre-trained on a large dataset of RGB images (either supervised, or self-supervised). Then, the pre-trained model is used to reduce the dimension of our [new] images to $128$, which is then being fed to the main, task-specific model.
Note that the word "self-supervised" (supervised by input itself) is currently used for models that try to reconstruct, or predict the whole, or part of the input as close as possible; e.g., auto-encoders, or some language models such as Word2Vec.
answered Mar 28, 2019 at 13:37
$\begingroup$
$\endgroup$
Online Model :
Model that continuously learns in production. If 10 new training samples are available, we do not need to retrain with all previous samples.
Pre-trained model
Model has already been trained on large data sets. This is a useful technique via transfer learning.
https://www.slideshare.net/queirozfcom/online-machine-learning-introduction-and-examples
https://www.analyticsvidhya.com/blog/2015/01/introduction-online-machine-learning-simplified-2/
answered Mar 28, 2019 at 9:52