The classification is the following:
- handcrafted are features that are manually engineered by the data scientist.
- learned features are ones that are automatically obtained from a machine learning algorithm
Let me give you an example:
Suppose you are doing an image classification task, where you wanted to classify cats from dogs. You want to build a classifier but you are faced with the dilemma, how do I input my data to the classifier?. You have two options:
- Use the raw pixel data. The issue with this is that you have a vast feature space, which makes it hard for models to generalize.
- Attempt to extract features from the image so that you can reduce your feature space.
Now if you choose the second you have two more options:
- Manually define a set of features and extract them. Some examples include edge detection, corner detection, histrograms, etc. The problem with this approach is that nothing guarantees that the number of corners is a good descriptor for classifying cat and dog images.
- The alternative is to train a ML model to identify and extract useful features for this specific classification task. This is exactly what a Convolutional Neural Network does. It searches for what features are best to classify the images?.
Traditionally the first approach was used extensively in Machine Learning. That changed however with the arrival of Deep Learning.
An example of what features CNNs identify when classifying different types of images:
Note that these features aren't general, low-level features like edges or corners; instead they are tailored for each class. That is the power of training a model to extract features.
Because learned features are extracted automatically to solve a specific task, they are extremely effective at it. In fact deep learning models that perform feature extraction and classification outperform models that classify manually extracted features by a large margin. This is one of the reasons why deep learning is so popular.
On the other hand, we have no control on what features the model will extract from the data. In many cases these features are only good for classifying the data and have no real-world interpretation. They are only good for the task that they were trained for.