Contemporary alternatives to SIFT for image feature extraction?

Question

I've been learning about SIFT and all the ways its descriptors can be used to do different tasks.

I am particularly interested in the way SIFT can be used for image classification. (e.g. A 2006 paper by Niester & Stewenius relies on SIFT descriptors to build a vocabulary tree).

However, as of 2017, 11 years later, Deep Learning has been replacing the classical approaches in many ways.

What are some alternatives to find feature descriptors for images that, as of 2017, have shown more promising results?

Answer 1

The Capsule Networks by Geoff Hinton et al. are based on a similar idea to SIFT: in fact, if you look at the 2011 paper, Transforming Auto-Encoders by Hinton et al. they explicitly cite SIFT as an inspiration for Capsules.

The main idea is that they construct a network with a new type of unit called a Capsule which outputs a vector (rather than neurons outputting scalars, as we know in traditional neural networks).

You can think of these vectors as an analogue to the SIFT keypoints and feature descriptors, but with the benefit that they can be learned via back-propagation.

There is a recent article, Dynamic Routing Between Capsules by Sabour, Frosst and Hinton elaborating on this which shows very promising results.

I have written a short summary of Capsule Networks on the AI Stack Exchange, see the question What's the main concept behind Capsule Networks?.

Answer 2

Depending on what you need, you should look at:

1. Fully convolutional networks (FCN) to obtain the mask of the input image

2. ResNets for very deep networks(101 layers)

3. MaskRCNN for instance segmentation