0
$\begingroup$

I need to train 5 models each for 10 times using tensorflow and keras for a homework. 2 of the models are multilayer perceptron models and remaining 3 of them are CNN models. I am training using my GPU, GTX1660Ti. I have no issue training MLP models, but when I try to train the CNN models I get the error below:

enter image description here

I have added these two lines of code to solve this issue:

physical_devices = tf.config.list_physical_devices('GPU')
tf.config.experimental.set_memory_growth(physical_devices[0], True)

However, when I try to train all of the models in a single script, my process gets killed after succesfully training 3 models:

enter image description here

Training dataset contains 27000 images, test dataset contains 5000 images, and validation dataset contains 3000 images each 28*28. I suspect gpu memory is the issue but I do not know the details. Any help is appreciated.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.