I'm training a convolutional neural network to classify images on fog conditions (3 classes). However, for each of about 150.000 images I also have four meteorological variables available that might help in predicting the classes of the images. I was wondering how I could add the meteorological variables (e.g. temperature, wind speed) to the existing CNN structure so that it can help in the classification.
One way I can already think of is creating another (small) feedforward neural net alongside the CNN and then concatenating the outputs of the CNN layers and the hidden layers of the non-image neural net to each other at the dense layer.
The second way I could think of is just contacting these features to the dense layer. However, in this case, the non-image variables will (I think) only be able to make linear predictions.
Are there any other (better) ways that the non-image features could be included in the model? And what would be the advisable method considering the amount of data I have?
Another question I have is whether or not I should unfreeze the convolutional layers while training with these non-image features? These layers of a Resnet-18 (which were initialized as pre-trained on ImageNet) have already been fine-tuned using the images. My guess is that I should keep them frozen and only unfreeze the dense layer since it is only here that the non-image features come into 'contact' with the image features (not earlier in the CNN). If I'm wrong on this, please say so!