If I have to create a model in pytorch for images having only single channel. How can I transform my model to adopt to this new architecture without having the need to compromise the pre-trained weights on which it has been trained upon.
$\begingroup$
$\endgroup$
$\begingroup$
$\endgroup$
I came across a code where the user had this very innovative method to tackle this problem. Here is the small trick to convert any pre-trained network to accept 1 channel images without loosing pertained weights.
arch = models.resnet50(num_classes=1000, pretrained=True)
arch = list(arch.children())
w = arch[0].weight
arch[0] = nn.Conv2d(1, 64, kernel_size=7, stride=2, padding=2, bias=False)
arch[0].weight = nn.Parameter(torch.mean(w, dim=1, keepdim=True))
arch = nn.Sequential(*arch)
Basically here, weights of the first Conv2d is taken and stored in w. Afterwards, we are creating new Conv2d which has 1 channel and replacing its parameter with w for which we take mean. I am pretty sure this can be achieved in keras or tenserflow too.
Let me know if you happen to try it.
$\begingroup$
$\endgroup$
You could convert 1-channel image to 3-channel with same values, so it's still black and white. This way you won't loose filters on other channels.
answered Jan 3 at 10:20
-
$\begingroup$ Well certainly we can but this would add extra parameters in the model. Here the stated technique is also going to save on parameters too discarding 2 channels and affecting the fine-tuning time for the model. $\endgroup$ – thanatoz Jan 3 at 10:22
