I have trained a deep nn model based on some existing data. In the meantime, I have collected more data and label them so that I can feed it to the model to improve its performance. The questions is, should I feed:
Option 1- New data to the already trained model?
Option 2- New data to a new model with initialized weights?
Option 3- The entire data (old+new) to a model with initialized weights?
Which method should I choose? Do I have to use only the new data or I have to combine the new data to the old dataset.
I asked this question so that I can choose the option which can improve the overall accuracy and to consider the data drift in the new data.