# What is representation learning?

I am reading the Chapter-1 of the Deep Learning book, where the following appears:

A wheel has a geometric shape, but its image may be complicated by shadows falling on the wheel, the sun glaring off the metal parts of the wheel, the fender of the car or an object in the foreground obscuring part of the wheel, and so on.

One solution to this problem is to use ML to discover not only the mapping from representation to output but also the representation to itself. This approach is called representation learning.

Here, I did not understand the exact definition of representation learning. I have referred to the wikipedia page and also Quora, but no one was explaining it clearly. The lack of explanation with a proper example is lacking too.

The authors cite the autoencoder as an example for a representation learning algorithm.

An autoencoder is the combination of an encoder function, which converts the input data into a different representation, and a decoder function, which converts the new representation back into the original format

Can someone explain in simpler terms what exactly representation learning is?

[Would be much better if explained with an example. The car wheel example would do too.]

• Interesting question.Why don't bring it to Artificial Intelligence Community. – quintumnia Aug 17 '17 at 16:32
• @quintumnia Honestly, I still don't have a clue what's on-topic and what's not, there. Trust me, I tried to understand that. Still no clue :D – Dawny33 Aug 17 '17 at 16:34
• ,This is on topic and what will matter there is tagging options.This is machine learning concept though I may I argued that a major distinction in machine learning is between predictive learning and representation learning. Now I’ll take a stab at summarizing what representation learning is about. Or, at least, what I think of as the first principal component of representation learning. – quintumnia Aug 17 '17 at 16:35

If you look at older machine learning algorithms, they rely on the input being a feature and learn a classifier, regressor, etc on top of that. Most of these features were hand crafted, meaning, they were designed by humans. Classical examples of features in computer vision include Harris, SIFT, LBP, etc .

The problem with these is that they were designed by humans based on heuristics. Images can be represented using these features and ML algorithms can be applied on top of that. However, they may not be the most optimal in terms of the objective function, i.e, it may be possible to design better features that can lead to lower objective function values. Instead of hand crafting these image representations, we can learn them. That is known as representation learning. We can have a neural network which takes the image as an input and outputs a vector, which is the feature representation of the image. This is the representation learner. This be followed by another neural network that acts as the classifier, regressor, etc .

So, in the wheel example, you can try to manually describe how a wheel should look like and how it can be represented. Say, it should be circular, be black in colour, have treads, etc. But these are all hand crafted features and may not generalize to all situations. For example, if you look at the wheel from a different angle, it might be oval in shape. Or the lighting may cause it to have lighter and darker patches. These kinds of variations are hard to account for manually. Instead, we can let the representation learning neural network learn them from data by giving it several positive and negative examples of a wheel and training it end to end.

Hope this makes sense. Let me know if I can clarify any point I made.

• So, representation learning algorithms act as inputs to algos. like regressors and/or classifiers? – Dawny33 Aug 22 '17 at 15:12
• Yes. Representations are just features, right? As for all ML algorithms, you have to input features. – ObisanKenobi Aug 23 '17 at 16:19

"One solution to this problem is to use ML to discover not only the mapping from representation to output but also the representation to itself. This approach is called representation learning."

1. Mapping the representation to output is called "Hetero association."

2. Mapping the representation to itself is called "Auto association."

Both approaches are about classification but in the first case you are associating a representation (or object, or item, or vector) to a label (for a category different from the original represented object, item, or vector). In the second case you are associating a representation (or object, or item, or vector) to a label (for the original represented object, item, or vector).

By representation, they mean an image, object, item, vector, etc..

So an auto associative encoder can take an image and output the category for the original image, while a hetero associative encoder can take an image an output a learned association for that image (for example, the word "dog").

How would autoassociative learning be useful? Spell checkers are a good example. Given a small lexicon of words ("dog, hog, hat, heat") each word forming its own category, we can create an auto encoder that when presented with each word, featurizes the word, and returns the original category / word. Let's featurize the words using trigrams. So we split each word into its feature set and let that feature set represent the orginal word as follows:

dog  --> {'  d', ' do', 'dog', 'og ', 'g  '} --> dog

hog  --> {'  h', ' ho', 'hog', 'og ', 'g  '} --> hog

hat  --> {'  h', ' ha', 'hat', 'at ', 't  '} --> hat

heat --> {'  h', ' he', 'hea', 'eat', 'at ', 't  '} --> heat


Now we present a new word, featurize it, and see where it falls:

hot --> {'  h', ' ho', 'hot', 'ot ', 't  '} --> ???

"hot" matches 0% of the features in "dog".

"hot" matches 40% of the features in "hog".

"hot" matches 40% of the features in "hat".

"hot" matches 33% of the features in "heat".


hot would be classified as either "hog" or "hat" if the match threshold were 40% or below. If we had a higher match threshold than 40% then there would be no match and hence no proper classification for the word hot.

The auto encoding of each of the lexicon entries (dog, hog, hat, heat) allowed us to take the original representation of each word and split it into features, which can be combined to point back to the original representation.

dog  --> {'  d', ' do', 'dog', 'og ', 'g  '} --> dog


A hetero encoding could take each of the lexicon entries and map it to a different category.

dog --> animal

hog --> animal

hat --> thing

heat --> thing


a simple hash table could suffice as the hetero encoder in this example.

Neural networks take an input vector of features and output the category either in an autoassociative or heterassociative manner.