2
$\begingroup$

I am relatively new to Data Science and I've recently embarked on a project. Long story short, I've trained a CNN model to distinguish between Male and Female genders. However, I wish to tune my model. I've saved the weights and bias of the existing model.

  1. Can I use the current model with weights and bias to retrain on augmented data (from the same initial dataset)? I am guessing there are 2 possible outcomes, it tunes the model or it "resets" the weights and bias.

  2. Can I perform hyperparameter tuning with RandomSearchCV or GridSearchCV using on my existing model? I believe this will tune the model.

My aim is to reduce overfitting.

$\endgroup$
1
$\begingroup$

Before answering your questions, let's understand a few points -

1. Model training is nothing but finding the right weights going through the guiding path based on the loss function o/p using y_true and y_pred.
Training starts with random initialization of weights(definitely following some rules)

2. Neural Network training is an incremental training approach i.e. it doesn't see all the data in one go like the DecisionTree

Can I use the current model with weights and bias to retrain on augmented data (from the same initial dataset)? I am guessing there are 2 possible outcomes, it tunes the model or it "resets" the weights and bias

Yes, learning will continue without any big disruption as augmented images will produce a similar gradient like the real images.
It will not reset, as I said the training is an incremental process.
Any new image which has completely different features can definitely cause a large gradient to flow and might disrupt the weights but with augmented images, I don't think any such thing should happen.

Can I perform hyperparameter tuning with RandomSearchCV or GridSearchCV using on my existing model? I believe this will tune the model.

You can do this but you don't have too many hyperparameters left as Model is fixed. You may tune Learning rate.

$\endgroup$
2
  • $\begingroup$ I understand the points, thank you. However, I am still a bit confused. Does that mean my model will be more generalised after training it with augmented data? $\endgroup$ – peanutsee Oct 7 '20 at 17:41
  • $\begingroup$ Yes. Should be. Augmentation is an accepted approach to regularized $\endgroup$ – 10xAI Oct 8 '20 at 15:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.