1
$\begingroup$

Let's say I have images of cars. For each image in the dataset, I have let's say 3 pictures of the same car but in different angles.

1) The first image is the picture of the car from the front. 2) The second image is the picture of that car from either of the sides. 3) The third image is the picture of that car from the back.

If I have 1000 cars, I would have 3000 images as my dataset. Is there a way to find which angle of the car represents its label the most? In other terms, Which angle of the car image is most helpful in determining whether a particular image is a car or not?

My thoughts were to build a random guessing model to determine the lower bound of our loss and then iteratively use each of angle of images as train and predict on the same angle and come up with a loss score. Ex: First I train and predict with all images of angle 1 and come up with a loss score. So If I have 3 angles of images, I would end up with 3 loss values. So the angle with the lowest loss score is determined as the winner. But the problem is, I'm not sure if this is approach is right because if I have multiple classes in the dataset then the dominant class would have lower loss compared to any other non-dominant classes and moreover we would not be using the entire data at any point of time and so would possibly end up with wrong conclusions.

Thanks for replying in advance. This question has been lingering in my head for a long time now.

Improve this question
$\endgroup$
2
$\begingroup$

If you are training a classifier on only one angle, you will overfit to that particular position - it won't be able to recognise the same car from a different angle. It may learn something, I guess, and you could test generalisation by testing on the unseen images from the other two angles. So train on each set individually and test on all 3 sets individually with the usual caveats on training vs testing.

A better approach, however, would be to train on all your data. The network will automatically learn what is the most efficient and you can then look at the results on each of the 3 test sets to see which gives the best results. I am guessing that this is an example problem, so more generally it makes sense to do a statistical analysis on why the classifier fails. If you can identify which features make classification hard, you can work harder to create a training strategy that will improve the model. The main goal would be to make a model that generalises. I suspect that training using extra information from other angles will only improve the effectiveness for all cases.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks, This answers the first part of my question. If I had only cars in my dataset. the method you suggested should work fine. But if my data has multiple class of vehicle such as cars, motorcycles, trucks etc, Can I still train on the entire dataset of all these vehicles and test on 3 angles on each of those vehicles especially when the classes are imbalanced. Won't it classify the dominant class more accurately?And now I can get more images of the non-dominant class or tweak their weights using a multiplier something like {0:1, 1:4, 2:10} Right? $\endgroup$ – karthikeyan mg Aug 23 '19 at 17:59
  • 1
    $\begingroup$ I all likelihood, it might, but there is some variation. Some classes are simply easier to learn than others. It is certainly something to bear in mind. You can also weight your loss function based on imbalances in the data so that less frequent classes carry a greater penalty, but I think that is mainly used for large differences between classes. As with all these things, it's best to just try different approaches, look at the results ans see what to make of them. It's very case-dependent, but considering it is a sign of due diligence. $\endgroup$ – Chris Moorhead Aug 23 '19 at 22:50
1
$\begingroup$

From your question, I'm not sure what your motivation is, but if you just want to find out which visual features are the most representative for a given class label on a dataset of objects captured from different viewing angles, I would suggest the following approach:

  • For the training process, use pictures from all different angles, shuffle them, and train your CNN.
  • Then, take the test dataset and measure the accuracy as an overall benchmark.
  • In a second step, divide your data according to the viewing angle and measure the accuracy of the previously trained model on each viewing angle.
  • Now, take the angle on which you got the highest accuracy and choose a few images that have been classified correctly. Analyse those images with saliency maps.
  • Subsequently, take a few misclassified images from the angle with the lowest accuracy as well and analyse them in the same way.

From the saliency maps, you will get a deeper understanding which visual features cause your trained CNN to predict a certain label.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks. That's another interesting thing to do $\endgroup$ – karthikeyan mg Aug 24 '19 at 13:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.