Object Classification Dataset Creation

Question

There is a problem I've faced recently which I'm not sure my approach is proper or not. There is bunch of field videos which I run a semi-supervised detection model to extract crops to train my classifier. Until now everything seems fine. However the question/problem in here is that due to frame rate on detection model there is a lot of extracted crops that looks familiar (same or so tiny angle differences).

In my opinion it's not proper way to generate a dataset due to the lack of features on dataset so I remove some crops from dataset (lets say 10 crops with the same person and same/tiny different angle, I delete 8 of them)

Is that approach is proper or should I use all the data I collected regardless of whether the data is the same ?

Sincerely Alper

Answer 1

Yes you can remove similar frames from the video in order to create the dataset. In fact this is a widely used technique to prevent over-fitting or data leakage when training models.

This is because if there are similar frames with negligible differences and when you split the dataset into train and test, there is a very good chance that the similar frames present in the train set might end up in the test set which will lead to data leakage. Hence you will get an overly optimistic result from your model which will be false.

How many similar frames you remove depends on you. Decide on a threshold, train your model and see the results. If results are not satisfactory then repeat the process with a different threshold until you get proper results.

You can also apply this same processing technique when inferencing in order to increase the efficiency and speed of your deep learning model. This will also increase the frame rate your model can spit out and also it won't require high resources.

But again it depends on what kind of data your model is encountering. If most of the frames are similar then you can use this technique but if the objects in the video are constantly changing at a very high frequency then it is not advisable to use this technique!

Cheers!