I often see in a well-known datascience competition platform, that a lot of people apply some dimension reductions techniques, but instead of using it to reduce the number of features (complexity) of their models, they append the resulting features to their datasets. Is'nt that adding complexity to their model rather than simplifying it ?
$\begingroup$
$\endgroup$
1
-
1$\begingroup$ The goal of feature extraction is to do some of the legwork for the model, or provide it information it can not learn by itself. Simplifying the input is not the goal, learning the mapping between the output and input is, and if enriching the input helps achieve this goal, so be it. $\endgroup$– EmreJun 20, 2017 at 7:39
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
This is feature engineering. You just give the algorithm another look at the data, from another point of view. It often helps to understand better data when you have different point of views.
For example, let's assume you want to learn the US road directions to a simple ML model. You give him all examples from roads 1 to 100 mapping to 0 (if the road is east-west), or 1 (road is north-south), except for the number 47. Then you ask the model to predict 47 and it will answer 0 because it's between 46 and 48 who are labelled 0.
Now if you give the model another point of view (here, the parity of the road number), then it will obviously learn efficiently the road directions.
You can view the results of the PCA as the parity, just another view of the data to understand it better.
