I have the following task Computer Vision/prediction task which I’m interested in hearing whether you guys think is feasible.
I have a dataset of 1000 respondents coming from a survey, where respondents had to give a liking score (on a 5-point likert scale) to a picture of a campaign for a transportation company. The pictures could, for instance, be of some people in a train doing something, people driving a car or whatever. They are basically Ad’s for a transportation company.
Now, I would like to see whether one could predict this liking score based on the pictures alone. Therefore, I would like to build a prediction model. I have done this several times before, with various variables from tabular data. However, I have never dabbed in Computer vision and used an image as the input to my model.
I must clarify, that I have access to around 50 of these datasets. This should leave me with around 50*3 images and 50x1000 output scores (the liking). The same image will therefore have different scores (maybe one person gave it a score of 5 and another a score of 2 and so on).
Researching Computer Vision tasks had led me to the following thoughts:
It is a common task to do image classification, where you want to classify the image (a picture of a dog has the label “dog”). I want to do the same, except instead of my label being “dog” it is 1, 2, 3, 4 or 5 (on the liking scale).
My strategy is the following:
Apply basic image preprocessing to the images (resizing, standardizing colors). Thereafter, applying image augmentation (where I increase the dataset size by changing different things in the images - colors, blurring, rotating etc.).
This would (as far as I understand) for instance leave me with a dataset of 503number_of_augmentations images and 50x1000xnumber_of_augmentations.
Thereafter (and here is where I am starting to get a bit lost), I think I should be able to apply some image classification deep learning model that basically extracts “features” from the pictures - so where a picture gives me some kind of multidimensional vector of values. Instead of then passing these values to the layer that labels them into “cat”, “dog” or whatever the original model has been trained on, the last layer would be the output scores from the survey.
Could I also use these extracted features and use them as inputs to a traditional xgboost model, where the outcome are the survey answers? As far as I understand it, the last layer of the DN model would be a xgboost layer.
Is this sort of task feasible?
Could I obtain these feature vectors from the pictures using any image model (for instance ResNet)?
Some challenges I see:
The dataset is potentionally quite large (over 50k answers). However, there may be 50k respondents, who have all given a value to the pictures. But, If I were to look at it from the input perspective, I only have 150 unique images (although I will get more with data augmentation). Additionally, each image does not a single mutually exclusive class (like “dog” for a dog picture). Instead, the same image may receive several scores. If I downsized the dataset to take the mean of a picture, then I would only have 150 rows.
Have any of you dealt with a task like this, or is there anyone that has a link to a paper/article/blog or anything like this? I seem to only be able to find how to classify either:
What is happening in the image
The liking of the image (so classifiy a person as happy, sad, etc.)
I want to read or see examples of:
Persons (in my case survey respondents) responding to an image (in the form of survey answers). How should I search for this kind of problem on google?