4
$\begingroup$

after reading the state of the art about object detection using CNN (R-CNN Faster R-CNN ,YOLO, SSD...) I was wondering if there is a method that use RNN's or that combine the use of CNN's and RNN's for object detection ?? Thank you

$\endgroup$
7
$\begingroup$

Yes, there have been many attempts, but perhaps the most noteable one is the approach described in the paper of Andrej Karpathy and Li Fei-Fei where they connect a CNN and RNN in series (CNN over image region + bidirectional RNN + Multimodal RNN) and use this for labeling a scene with a whole sentence. Though, this one is more than just object detection as it leverages a data set of scenes and their descriptions to generate natural language descriptions of new unseen images.

Another example is Ming Liang and Xiaolin Hu's approche where they mix a CNN with an RNN and use this architecture for better object detection. As Ming and Xiaolin explained in their paper (linked above), the RNN is used to improve the CNN:

A prominent difference is that CNN is typically a feed-forward architecture while in the visual system recurrent connections are abundant. Inspired by this fact, we propose a recurrent CNN (RCNN) for object recognition by incorporating recurrent connections into each convolutional layer.

$\endgroup$
3
$\begingroup$

Actually, I do not think it should be a good way of using RNN only to do object detection work, because there is no "Receptive Field" conception in RNN compared with CNN, which I think should be a key point in doing vision related task.

$\endgroup$
2
$\begingroup$

Recurrent Neural Networks (RNN) are the state of the art algorithm for sequential data and Long Short-Term Memory (LSTM) networks are an extension for RNN. This method can be used on object detection in case detect object in video or moving images, etc. You can try this https://github.com/tensorflow/models/tree/master/research/lstm_object_detection. It implementation from Tensorflow mobile video object detection implementation proposed in the following paper: Mobile Video Object Detection with Temporally-Aware Feature Maps (CVPR 2018). The link of paper: http://openaccess.thecvf.com/content_cvpr_2018/papers/Liu_Mobile_Video_Object_CVPR_2018_paper.pdf

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.