after reading the state of the art about object detection using CNN (R-CNN Faster R-CNN ,YOLO, SSD...) I was wondering if there is a method that use RNN's or that combine the use of CNN's and RNN's for object detection ?? Thank you
Yes, there have been many attempts, but perhaps the most noteable one is the approach described in the paper of Andrej Karpathy and Li Fei-Fei where they connect a CNN and RNN in series (CNN over image region + bidirectional RNN + Multimodal RNN) and use this for labeling a scene with a whole sentence. Though, this one is more than just object detection as it leverages a data set of scenes and their descriptions to generate natural language descriptions of new unseen images.
Another example is Ming Liang and Xiaolin Hu's approche where they mix a CNN with an RNN and use this architecture for better object detection. As Ming and Xiaolin explained in their paper (linked above), the RNN is used to improve the CNN:
A prominent difference is that CNN is typically a feed-forward architecture while in the visual system recurrent connections are abundant. Inspired by this fact, we propose a recurrent CNN (RCNN) for object recognition by incorporating recurrent connections into each convolutional layer.
Recurrent Neural Networks (RNN) are the state of the art algorithm for sequential data and Long Short-Term Memory (LSTM) networks are an extension for RNN. This method can be used on object detection in case detect object in video or moving images, etc. You can try this https://github.com/tensorflow/models/tree/master/research/lstm_object_detection. It implementation from Tensorflow mobile video object detection implementation proposed in the following paper: Mobile Video Object Detection with Temporally-Aware Feature Maps (CVPR 2018). The link of paper: http://openaccess.thecvf.com/content_cvpr_2018/papers/Liu_Mobile_Video_Object_CVPR_2018_paper.pdf
Yes there is a lot of literature about object detection using RNNs and it often consists of object detection and tracking in videos or action detection.
- "Online Video Object Detection Using Association LSTM", 2018, Lu et al.
- "Re3 : Real-Time Recurrent Regression Networks for Visual Tracking of Generic Objects", 2017, Gordon et al.
- "Online Multi-Target Tracking Using Recurrent Neural Networks", 2016, Milan et al.
- "Action Recognition in Video Sequences using Deep Bi-Directional LSTM With CNN Features", 2018, Ullah et al.