Should we keep all channels when doing image classification?

Question

I am discovering the world of image recognition and now trying to build an image classifier. The set of images I have have the shape (101,101,3) which means that it has 3 channels.

If I'm not mistaken the channels corresponds to the Red, Green and Blue channels (Please correct me if I am wrong).

In which classification tasks should we keep the three channels ? I've seen people converting the Image to a grayscale so that the images shape will be (101,101,1). What difference does it make when you do so ?

I am aware that when you keep the 3 channels the computation needs grow with it, here I am more looking for an answer about the intuition behind keeping the 3 channels, move to grayscale or add new custom channels.

Answer 1

Well, it depends on your images and how you want to classify them.

If we take a really basic example, if you want to classify images by their color, you obviously should keep all 3 channels.

In the other hand, if you want to classify images by a shape. For example squares and circle, you don't necessarily need to use all 3 channels.

But changing the number of channel isn't the only preprocessing you can do on your images before classifying them.

You can also change the color space for example (the colorspace LAB can be usefull in some case) , or even use Morphological Image Processing methods to improve your image by, for exemple, bringing out the contrasts of your images or cleaning impurities.

All of this depends of your images, and what you want to do with them.

Answer 2

Images are rich data sets. Like dealing with any data, consider it as feature selection. I would go deeper in this (more then keep or leave). Have all available scenarios in mind and breakdown what you have.

Pixel domain, which is about pixel itself. This can be represented in one value (gray channel) or, as you posted, three (RGB- red green blue). More handy is other representations like HSV (Hue, saturation and value) those three channels are telling different story about your image. See more at (https://en.m.wikipedia.org/wiki/HSV)

Spatial domain, which is about areas (re-representing pixels by its neighbours). This domain have way much to discover (matricesand filters). For example, texture analysis (try have spatial variance/STDEV filter and see). See more at (https://www.google.com.sa/url?sa=t&source=web&rct=j&url=http://www.mv.helsinki.fi/home/khoramsh/4-Image%2520Enhancement%2520in%2520Spatial%2520Domain.pdf&ved=2ahUKEwi11cLC673cAhVJUhoKHdh_B8oQFjAZegQIAxAB&usg=AOvVaw2x_FaO0ohMopZkEcLXTjta)

If you find yourself in need fore more check this thesis as well (https://zone.biblio.laurentian.ca/bitstream/10219/2616/1/Abdulkareem%20Althwaini%20Thesis%20Final.pdf).

After listening all possible features of your image, you can start.