I am given a selection of videos of users exploring simulated 3D enviroments (kind of looks like the Sims video game) and I am tasked with being able to classify each room using a tensorflow framework. E.g. if a user were to hover over a certain area in the environment, the model should be able to classify whether it's a security line
or baggage check etc.
I am fairly new to machine learning algorithms and techniques, however I have been given a working VGG16 CNN model that is able to characterize (given labeled input data) actors in a video clip, and I am expected to rework this into environment classification. Is such a thing possible given this framework? I am undecided on whether I should collect images from various angles of the room/environment, and labeling and training each image as a whole using VGG16/Resnet50, or to implement some sort of R-CNN/YOLO model using bounding boxes to classify the environment given known objects found on screen. I am not actually limited to the given tensorflow framework, so if there is a better way to approach this problem I can recreate a new code environment from scratch if needed.