I'm working on a project where the task is to classify videos of birds and predict the species. I have tried to use machine learning models originally designed for deep action recognition and tweaked for my dataset.

The dataset consists of 9 different classes with around 3000 videos with labels. I have expanded the dataset to around 30000 videos using the following techniques:

 - adding rgb noise to video
 - random distortions

Here is a [video][1] of a sample dataset containing the different classes that my deep learning model should be able to predict:

[![https://www.youtube.com/watch?v=4iNpw7J5q9I]][2]][1]

Currently I have used this codebase for [temporal segment networks][3] to train my model and the resulting accuracy for my own dataset was about 87% (only with RGB and not using optical flow). Although depending on weather condition and lighting, the accuracy can be very different.

I have a few questions about doing video classification to predict bird species:

 - Are there better techniques for doing video classification compared to using temporal segment networks?
 - Are there any preprocessing steps that should be done to improve accuracy?
 - How to train the model to generalize better to be less sensitive to lighting and weather?
 - How can the model be trained to classify unknown species?

In short: What is the best deep learning model for video classification for bird species?

  [1]: https://www.youtube.com/watch?v=4iNpw7J5q9I
  [2]: https://i.sstatic.net/ExQEP.jpg
  [3]: https://github.com/yjxiong/temporal-segment-networks