I wish to train a model that detects the breed of a dog based on video input. I have a dataset containing 10 classes with 30 videos in each class. The problem is that for each of these videos, the dog is not present throughout the course of the video. The following are examples of 2 videos from the dataset:

Video 1: Video of backyard (first 5 seconds) --> Dog appears (15 seconds) --> Video of surrounding buildings (3 seconds)

Video 2: Video of grass (first 8 seconds) --> Dog appears (3 seconds) --> Video of nearby people (4 seconds)

I presume that my CNN would detect redundant features and hence give incorrect outputs if I trained my model on the videos as is. Hence, do I need to manually trim each of the 300 videos to show only the part where the dog appears or is there an easier way to approach this problem?


It depends on the scale.

I had several situations where doing a task manually would be a time-saver as it would not be scaled or repeated in the future.

I think you would spend at most five minutes for each video. That is three days of work time. If you need the data urgently, doing it manually and/or delegating would be faster than trying out different solutions, unless you have a lot of experience in the field.

If you are confident that you can implement and troubleshoot one of the proposed methods (two-stage net, frame-to-image) in less than a day, then I'd give automation a shot.
However, I think it's more likely that you will debug it for a week or longer.

Therefore, I suggest doing it manually if this task is a one-time thing and you need the data now.

  • $\begingroup$ Thanks for the answer! Do you know any software to quickly upload multiple video files and trim them individually. The software I have encountered only takes one input at a time and it's excruciatingly boring to have to upload, trim, save and compress each file individually. $\endgroup$ Aug 9 at 12:46
  • $\begingroup$ Unfortunately, I am not aware of such software. On a side note, if the process of saving and compressing is tedious, it might be possible to automate it with AutoHotkey or a similar program. $\endgroup$ Aug 9 at 14:40
  • $\begingroup$ I can't use AutoHotkey because I am working on a macOS. Nonetheless, thanks for the help! $\endgroup$ Aug 10 at 5:00
  • $\begingroup$ @learning123 If you made a spreadsheet of the timestamps of start and end points of each cut, you could script it with ffmpeg (which I think works on a Mac?). You might spend a couple of hours messing around with the script, so there is a trade-off there. A quick google found an example here: forum.videohelp.com/threads/365637-Batch-trim-using-FFMPEG Ooh, actually the answer here looks useful: video.stackexchange.com/q/22697/21168 $\endgroup$ Aug 10 at 7:57

One option is to create a hierarchical system. The first stage could be a model that detects the presence of "dog" / "not dog". The second sage could be if "dog" is presence then the specific breed.

Many Convolutional Neural Networks (CNNs) are designed for images. Thus, classification of the video would happen frame-by-frame.

Since this is a relative common task, find a pretrained CNN and see how well it performs on your specific data. If the performance is not acceptable, then label your data and fine-tune the model.

  • $\begingroup$ Hey, thanks for answering! The problem with a hierarchical system is that the model that detects the presence of "dog" would be prone to error and hence my accuracy would reduce. Furthermore, I will have to look for a new dataset and then connect the learnt features to a new model which may lead to some complications. In fact, out of the 10 classes, one of them is "not dog", so I intend to use that to indicate the presence of a dog at a specific moment. But first, I need a dataset that helps my model precisely learn the required features. Please correct me if I'm wrong. $\endgroup$ Aug 7 at 23:04

I would try to do this in several steps:

  1. transform the fideo to frames of image, then to grayscale, to csv.
  2. I suppose that in the majority of videos you have, the dog appears at some point in the video, then disappears (or not), so at the middle of the video, the dog almost always is there. so maybe cut out all but the frames in the middle of the video, and then also reduce their ammount (as it is quite a waste of resourses to run algorithms of frames with the difference in time of less than half a second (nothing changes).
  3. Now you can run normalization and the models you wish.

Hope this helps!

  • $\begingroup$ Hey, thanks for the answer! Is there a simple way to extract the middle portion of a large set of videos (300, in my case) without having to trim each video individually? $\endgroup$ Aug 7 at 23:06
  • $\begingroup$ of course there is such a way. I dont know exactly how to do it, but I think you might find your answer in OpenCV library. If not, then I have no clue (Dr Google will help a lot here:)). In any case I am almost certain that such a way exists. Good luck! $\endgroup$ Aug 8 at 7:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.