1
$\begingroup$

I am wondering how to convert caffe reference model trained with ImageNet (color pics) for grayscale image to save memory and to speed up.

The filters in the caffe convolution layers are different for RGB but this can be, for example, averaged to form one channel filter.

Since my data is grayscale and speed at test (prediction) is important for me, this may help.

I can of course convert my grayscale images in RGB representation to reuse pretrained convolution layers as they are. But I want to avoid it if possible.

My questions are:

Would this be possible?

How exactly one should combine filters? (Average?)

After filters converted, weights between conv layers need to be re-trained?

$\endgroup$
4
  • $\begingroup$ I think it is an interesting question, but the general case is likely possible but the performance will degrade. Also converting greyscale to RGB would also be worse in a similar way. In both cases this is because the colour channel information is being used by the filters separately. In specific cases, the impact of this could be small enough you could get away with it . . . as usual the only way to be sure is to try. I'd suggest trying with grayscale images converted to RGB first to get a sense of whether the accuracy loss is acceptable. $\endgroup$ Jul 11, 2016 at 14:42
  • $\begingroup$ Would you then recommend to use pre-train model for MNIST digit (grayscale) instead? But digit data would not represent more complex shapes... And can you elaborate the last part about color information? $\endgroup$
    – otterb
    Jul 11, 2016 at 14:49
  • $\begingroup$ oh, i did not expand your comment. Ok, so maybe I will just try with averaging and see what I get. Thanks! $\endgroup$
    – otterb
    Jul 11, 2016 at 14:52
  • $\begingroup$ sir i am interesting about your question . does it possible to convert filters from 3 channels int one instead convert our data images into RGB thanks $\endgroup$
    – Sam Ab
    Nov 19, 2017 at 6:16

2 Answers 2

1
$\begingroup$

I doubt either of your proposed approaches will work, these convolutional layers learn all kind of relationships between the colors which will be non-existent in grayscale, upsampling it to RGB makes your input space very different from the input space that ImageNet was trained on and averaging these filters of the trained ImageNet model makes a whole lot of assumptions that are unlikely to hold. The second approach does seem more promising than the first, so if you want to test an approach I would go for the second one. It's possible that it is at least a better initialization than randomly. I think you will end up needing a lot of data and computation anyway, so it might be better to start off with your own network shape and start from scratch.

$\endgroup$
1
  • $\begingroup$ Thanks for your intuition. I feel that at least it's not a totally bad idea to try. And comments from you and Neil really help me guess how much time I should invest. I will go with 2nd option and see. $\endgroup$
    – otterb
    Jul 13, 2016 at 10:51
0
$\begingroup$

What about pre training your own greyscale Imagenet model? Create a single channel resnet architecture and train it on images that have already been converted to greyscale. Training a model from scratch on Imagenet may seem like a daunting task but it can be done quite quickly and cheaply these days.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.