Why is there an extra convolution layer in VGG19 for some neural style transfer implementations?

Question

I've been exploring neural style transfer and noticed that some implementations, such as the official PyTorch versions of Universal Style Transfer via Feature Transforms and unofficial Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization's include an additional 1x1 convolution layer in the VGG19 model architecture. This layer seems to perform a channel swap from RGB to BGR and scales the values, rather than acting as a feature extraction layer.

This extra layer is not mentioned in the papers for these methods, which only state that VGG19 was used as the encoder. I’m curious about the rationale behind this modification. Why is this preprocessing layer used instead of the standard VGG19 architecture? How does it affect the model’s performance or results, and why might the weights of these modified layers differ from those in the common VGG19 model found in PyTorch?

Any insights or explanations would be greatly appreciated. Thank you!

Update: I unexpectedly found the answer. I mostly use pytorch for deep learning. The pretrained vgg19 model in pytorch was trained with normalization values mean=[0.485, 0.456, 0.406] and std=[0.229, 0.224, 0.225]. Where in the paper they used the original model that was trained in vgg paper. In the vgg paper they only performed mean substraction from pixel values ranging from 0to255. So the extra conv layer only converts rgb channel to bgr channel and subtracts mean. It is actually a preprocessing layer depended on the the pretrained model used

Answer 1

From the VGG paper:

The incorporation of 1 × 1 conv. layers (configuration C, Table 1) is a way to increase the nonlinearity of the decision function without affecting the receptive fields of the conv. layers. Even though in our case the 1 × 1 convolution is essentially a linear projection onto the space of the same dimensionality (the number of input and output channels is the same), an additional non-linearity is introduced by the rectification function.

The VGG paper looks at multiple variations of the model. The model with 1x1 convolutions was one such variation. It looks like the 1x1 conv didn't add much to performance:

Second, we observe that the classification error decreases with the increased ConvNet depth: from 11 layers in A to 19 layers in E. Notably, in spite of the same depth, the configuration C (which contains three 1 × 1 conv. layers), performs worse than the configuration D, which uses 3 × 3 conv. layers throughout the network. This indicates that while the additional non-linearity does help (C is better than B), it is also important to capture spatial context by using conv. filters with non-trivial receptive fields (D is better than C). The error rate of our architecture saturates when the depth reaches 19 layers, but even deeper models might be beneficial for larger datasets. We also compared the net B with a shallow net with five 5 × 5 conv. layers, which was derived from B by replacing each pair of 3 × 3 conv. layers with a single 5 × 5 conv. layer (which has the same receptive field as explained in Sect. 2.3). The top-1 error of the shallow net was measured to be 7% higher than that of B (on a center crop), which confirms that a deep net with small filters outperforms a shallow net with larger filters.