# Train new data to pre-trained model

Let's say I've trained my model and made my predictions.

My question is... How can I append some new data to my pre-trained model without retrain the model from the beginning.

• Fine tune or use pre-trained weights Mar 2, 2018 at 10:18
• Thanks @Aditya but i don't want to predict on different dataset .... i want to enrich my train dataset and then predict my results without retrain my train dataset from the beginning Mar 2, 2018 at 10:29
• Which algorithm? Are you using Scikit learn? Mar 2, 2018 at 13:57
• My question was not for a specific algorithm ......it was a general question.....why? Mar 2, 2018 at 14:04
• Possible duplicate of Using a pre trained CNN classifier and apply it on a different image dataset Mar 10, 2018 at 6:54

I cannot comment yet. If you just load the model and use a fit method it will update the weights, not reinstance all the weights. It will just perform a number of weights update that you can chose, using the new data.

• So @Louis you are telling me that if i load my pre-trained model and use a fit method with new data e.g ( loaded_model.fit(x_new_data, y_new_data) ) will update my model??? Mar 2, 2018 at 13:11
• Yes, fit is not going to reset the weights, so you can do : load, compile, fit, save and you'll have trained your model a little bit more. You can also use train_on_batch to perform a single gradient update. Mar 2, 2018 at 13:38
• This is false, at least for sklearn. As per the official documentation, calling fit() more than once will overwrite what was learned by any previous fit().
Sep 24, 2018 at 7:15
• hey all... which is right ?... what fit() method do ? update or overwrite from scratch ? Jan 3, 2019 at 5:40
• Calling fit more than once will overwrite the previous fit. See the SKlearn documentation. Jan 31, 2020 at 15:02

It all depends on the specific algorithm you're using. Some of them support incremental learning, while others don't.

For example, in the case of scikit-learn, using fit() more than once on the same model will simply overwrite the model's weights each time (see here for more details).

What you can do however, is look for algorithms that implement the partial_fit api, and use this to retrain your existing models - see the documentation for a list of algorithms that support incremental learning and thus implement this api.

An alternative solution is to look for algorithms that support the warm_start parameter, e.g. LogisticRegression. Note that warm_start might also be influenced by other parameters, so you need to pay attention to their values, too - e.g. in the case of LogisticRegression, warm_start won't work if you use the liblinear solver (which is the default).

I would say it depends upon the ML framework you are using. I have worked on Scikit and Tensorflow.

Both works in a different way.

Scikit:

1. partial_fit() is one way. If we call partial_fit() multiple times, framework will update the existing weights instead of re-initializing it again.
2. warm_state is another way which is provided by many algo. For example RandomForestRegressor(), it will add new estimators(new tress) which gets trained with new data we pass to it. Refer Scikit link for more explanation

Tensorflow: Consider a basic TF code as mentioned below:

import numpy as np
import tensorflow as tf
from sklearn.datasets import fetch_california_housing
from sklearn.preprocessing import StandardScaler
housing = fetch_california_housing()
m, n = housing.data.shape
target = housing.target.reshape(-1,1)
scaler = StandardScaler()
scaled_housing_data = scaler.fit_transform(housing.data)
housing_data_plus_bias = np.c_[np.ones((m, 1)), scaled_housing_data]
n_epochs = 1000
learning_rate = 0.01
X = tf.placeholder(shape = (None, n+1), dtype=tf.float32, name="X")
y = tf.placeholder(shape=(None,1), dtype=tf.float32, name="y")
theta = tf.Variable(tf.random_uniform([n + 1, 1], -1.0, 1.0, seed=42), name="theta")
y_pred = tf.matmul(X, theta, name="predictions")
error = y_pred - y
mse = tf.reduce_mean(tf.square(error), name="mse")
gradients = 2/m * tf.matmul(tf.transpose(X), error)
training_op = tf.assign(theta, theta - learning_rate * gradients)
init = tf.global_variables_initializer()
with tf.Session() as sess:
sess.run(init)
for epoch in range(n_epochs):
fn,wights, loss, pred, op = sess.run((error, theta, mse, y_pred,training_op), feed_dict={X:housing_data_plus_bias, y:target})
print(loss)
best_theta = theta.eval()
print(best_theta)


Consider I have trained and deployed this model(as a tf.saver() object). Now if I want to predict new values I have to feed my model with new X values and identify "y_pred".. This process will not update my theta(weights) values(tensor graph, evaluate its dependent tensors only) and for prediction model will use existing theta. In case I want to update theta, using the new samples, I have to evaluate training_op.

In this way we can improve TF model later with the new data.

Note: training_op, has a dependency on theta and gradient.