I am wondering how to convert caffe reference model trained with ImageNet (color pics) for grayscale image to save memory and to speed up.
The filters in the caffe convolution layers are different for RGB but this can be, for example, averaged to form one channel filter.
Since my data is grayscale and speed at test (prediction) is important for me, this may help.
I can of course convert my grayscale images in RGB representation to reuse pretrained convolution layers as they are. But I want to avoid it if possible.
My questions are:
Would this be possible?
How exactly one should combine filters? (Average?)
After filters converted, weights between conv layers need to be re-trained?