I have a classification task using grayscale images and I want to leverage from pretrained networks.
There are a lot of resources out there presenting how to fine tune large neural nets like resnet, alexnet, etc for our custom task (usually with less data). However, I stumble on the problem that I want to use Resnet learned features on data which is not RGB (3 channels). In fact, I'm using grayscale images. I thought about using an "embedder" as a first layer before resnet which can transform my
height x width x 1 image to a
224 x 224 x 3 resnet like input.
I wanted to know if there are works presenting this as an working approach (I couldn't get the right terminology to search for it). If so, is there a generic code to do so ?
class NeuralNet(nn.Module): def __init__(self,n_output): super().__init__() repr_size = 1024 self.embedder = # The magic convolution code here that transform a 1 channel input to a 224 * 224 * 3 output self.backbone = resnet50 self.classifier = nn.Sequential( nn.Flatten(), nn.Linear(repr_size,128), nn.ReLU(), nn.Linear(128,n_output), # nn.Softmax(), )