I want to use ResNet50 model to perform binary classification on a dataset spectrogram dataset. In order to do that I had to make a couple of modifications to the model's architecture:

  1. Modified the first convolutional layer, since spectrogramas are grayscale images and ResNet50 was initally built to clasify RGB images.
  2. Changed last dense layer to a simple MLP of three layers, output features are 1024, 512 and 1, where the last is a single neuron along with sigmoid activation. The two former layers have PReLu activation.

After 1000 epochs of training I obtained this: Loss


I'm using SGD for learning with 0.95 momentum, and static learning rates: 0.08 for convolutional layers and 0.1 for fully connected layers. Also for the non-modified layers I used pre-trained weights on the ImageNet dataset.

My question is, since results are not looking good (those plots are a clear example of overfitting), how can I improve results on the validation set? I have tried many different learning rates, and the ones I comment here give the best results. I'm thinking on lowering learning rate as epochs progress, but will this allow accuracy to improve? Any advice would be really appreciated. Sorry if explanation took to long and thanks in advance :)

PS: Everything is implemented in PyTorch


1 Answer 1


Static learning rate is usually a bad thing, and 0.1/0.08 is usually a very high learning rate after a few epoch (for some networks even 0.001 is too high). Easiest thing to do is going with Adam instead of SGD, as it will automatically adapt the learning rate. You can choose a starting learning rate such as 0.01.

Add shuffling in your dataloader if not already done.

Look at the examples that are wrongly classified, it may give you an hint about what does not work or what is the issue for the model.

If you have imbalanced dataset, you may want to take it into account (e.g using weights or change the loss for imbalanced dataset).

Some sort of data augmentation could help (e.g maybe you could add some random noise to the spectrogram during the training).

Try a smaller resnet (18 or 34).

  • $\begingroup$ Thanks for your answer, I really appreciate it. My dataset is balanced and I'm already using data augmentation, namely, sliding window. The thing is that I generate spectrograms with 'librosa' library, and there was a time were I tried to change parameters on spectrogram generation to see if it affected on model performance, but it was kind of arbitrary since my knowledge about spectrograms is limited. I will try with Adam, even though I think I've already tried when I started with this problem. Do you have any good reference on this topic that I can check? Thanks again :) $\endgroup$ Commented Oct 23, 2023 at 6:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.