I've created spectrogramms of different classes of low-hertz signals. They all have a plain blue foreground with hardly other coloured pixels, even for me its not easy to distinguish the classes by human sight. Now I'd like to train a CNN to do binary classification on these spectrogramm images. No matter how I build the network with no matter what configurations and parameters - it doesn't learn, loss doesn't decrease.

I figured out the reason: the convolutional filters of CNNs are good at distinguishing forms and shapes, but not really plain colours.

How can CNNs learn colours? Am I missing something? Perhaps there are more suitable models than CNNs?

Here is an example of the two classes that hardly differ, only in the colour hue:

class1 class2

2 more images:

enter image description here enter image description here

EDIT1: The two classes have images that not all look the same with the same prevailing colour; some have yellow/greenish stripes in them, some have lighter or darker colour etc. So the classes do not only consist of the feature colour. My goal is to classify these images using CNNs, but somehow CNNs fail at learning from images, that almost only consist of colour, whereas CNNs learn well if there are sharp edges, boundaries, object-like elements in the image.

  • $\begingroup$ I have a few questions : Are your classes only defined by the colour of the image ? Is an image of a precise colour always classified in the same classes ? If that's the case, machine learning seems not necessary, since you have defined rules to classify your image (Colour between . and . >> Class = C1 ; Colour between . and . >> Class = C2, etc) If you're not only analysing the colour but it's distribution, you can create variables yourself and use way easier models than CNN : example, create for each R, V and B values, a variable computing the mean value in the image, the difference between $\endgroup$
    – Adept
    Oct 6, 2020 at 13:45
  • $\begingroup$ How many Images and Classes does the dataset has got? $\endgroup$
    – 10xAI
    Oct 6, 2020 at 15:50
  • $\begingroup$ 2 classes and perhaps 1500 images per class $\endgroup$
    – MJimitater
    Oct 6, 2020 at 19:34
  • $\begingroup$ What models are trying? There might be better architectures than the ones you choose. Are you using transfer learning? $\endgroup$ Oct 7, 2020 at 14:52
  • $\begingroup$ Currently, I've tried vanilla CNNs (conv filters, max pooling etc.), playing with all architecture sizes. Im haven't used transfer learning yet $\endgroup$
    – MJimitater
    Oct 7, 2020 at 18:50

1 Answer 1


CNNs can learn colors, there are CNNs developed just to check colors on cars. But there are some points worth analyzing when dealing with classification when color is more relevant feature than shape.

1 - Color Space

Digital Color Images are usually represented as a pixel 2D Grid, where each pixel is a vector with 3 or 4 elements (the later case to define opacity). Usually, we deal with images using the RGB Color Space, as this is a representation used by our retinas to color cells.

But there are multiple Color Spaces that can be used to represent images, mostly 3 dimensional as RGB but each one has a different approaches and the transformation between them is not always linear. Check using HSV and HSL to check which is best for your problem. (The previously mentioned paper, has other examples).

2 - Architecture:

There are a bunch CNNs architectures, developed with different intuitions and objectives. For example U-Net is developed with image segmentation in mind, reducing information loss by keeping shallow paths along with the deep paths.

Most architectures are created with shape in mind, trying to catch local nuances in the image, and usually use small kernels such as 3x3, 5x5 etc.

Dealing with colors as the main feature, these local differences might not be the most relevant feature you want your kernels to learn to extract, more global patchs (i.e larger kernels) might be better.

Specifically for your problem:

These plots can be seem as surfaces and have some known relations between vertical and horizontal axes, maybe non-symmetric kernels might be more functional, and also avoid the speed loss in using too large kernels.

I would go with layers using alternating sizing. For example 30x3 then 3x30 kernels, with no padding.

  • $\begingroup$ Thanks @Pedro Henrique Monforte for your answer! Do you think a dual branch CNN (as in the paper) is the clue to detecting colours? Thanks, I'll try alternating kernels! Do you know how to have non-symmetric kernels in keras? $\endgroup$
    – MJimitater
    Oct 7, 2020 at 18:59
  • 1
    $\begingroup$ If I am not mistaken, you should pass a tuple instead of an integer to set non symmetric kernel size. $\endgroup$ Oct 8, 2020 at 0:10
  • 1
    $\begingroup$ I don't think the dual branch has anyrhing to do with accuracy in this case. They probably used it to manage their low GPU Memory issues, just like it was done by AlexNet. $\endgroup$ Oct 8, 2020 at 0:11
  • $\begingroup$ Okay, I tried with non-symmetric (small and large) kernels, but no improvement. Training always seems to get stuck at a local minimum quite quickly after 3 epochs or so, loss won't decrease then, keeping accuracy for this binary classification at 50% steady - thats like guessing, so no learning has happened $\endgroup$
    – MJimitater
    Oct 8, 2020 at 11:46
  • $\begingroup$ Could you share code? This seems so weird. $\endgroup$ Oct 8, 2020 at 22:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.