What are some of the important CNN architectures one should know about? What CNN architectures did well on the ImageNet ILSVRC challenge? What CNN architectures are good candidates for transfer learning?

  • $\begingroup$ Although I did not downvote, I understand the concern that might have come up. Answers to this question can become outdated quickly! It might be best to search other websites for state-of-the-art neural network architectures and ask here when you stumble upon a specific concern about them. $\endgroup$ – E_net4 of the downvote brigade Aug 27 '17 at 22:48

Here is a list of several important CNN architectures:

  • LeNet-5 the first successful application of CNNs to digit recognition, developed by Yann LeCun in 1990. It consists of a sequence of Convolutional, ReLU, Max Pooling layers followed by a Fully Connected layer. Reference: Y. LeCun et. al., "Gradient-Based Learning Applied to Document Recognition", Proceedings of the IEEE, 1998
  • AlexNet popularized CNNs in computer vision, did really well on the ImageNet ILSVRC challenge in 2012 showing significant gains in performance. The network has similar architecture to LeNet but is deeper and bigger and features convolutional layers stacked on top of each other. Reference: A. Krizhevsky, et. al. "ImageNet Classification with Deep Convolutional Neural Networks", NIPS, 2012
  • VGG16 demonstrated the importance of depth as a critical component to good performance, it was a runner-up in ILSVRC 2014. The architecture consists of a stacked convolutional and max pooling layers with increasing depth and it uses a large number of parameters due to the final fully connected layers. Reference: K. Simonyan, "Very Deep Convolutional Networks for Large-Scale Image Recognition", ICLR, 2015
  • Inception original version called GoogLeNet from Google won the ILSVRC 2014 challenge the architecture consists of inception modules that dramatically reduce the number of parameters, it uses multi-scale 3x3, 5x5 convolutional filters including 1x1 convolutions for dimensionality reduction. Reference: C. Szegedy et al., "Going Deeper with Convolutions", CVPR, 2015
  • ResNet50 residual network was the winner of ILSVRC 2015, it introduces skip connections for easier training that enable very deep architectures and makes use of batch normalization. Reference: K. He et al., "Deep Residual Learning for Image Recognition", CoRR, 2015
| improve this answer | |

See my masters thesis. Chapter 2.2 and 2.3 for a high-level overview over the building blocks. Appendix D describes a couple of really well known CNN architectures in detail:

  • Lenet-5 (historic)
  • AlexNet (still in use, but not SotA)
  • VGG-16 D (Still in use, but not SotA)
  • Googlenet and the Inception-Nets (v2, v3, v4)

Not described in detail, but still important

  • Resnets
  • Densenets
| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.