The automatic construction of new features from raw data

Question

Some observations are far too voluminous in their raw state to be modeled by predictive modeling algorithms directly.

Common examples include image, audio, and textual data, but could just as easily include tabular data with millions of attributes.

Feature extraction is a process of automatically reducing the dimensionality of these types of observations into a much smaller set that can be modelled.

For tabular data, this might include projection methods like Principal Component Analysis and unsupervised clustering methods. For image data, this might include line or edge detection. Depending on the domain, image, video and audio observations lend themselves to many of the same types of DSP methods.

What about generating new features with higher predictive values from the raw data and concatenate them to the raw data?

For example I have data about student wealth, health, family status and I want somehow generate a new feature I can call Social status whic is generated from the raw data and with high predictive value? Is this possible? linear regression can be a good way I need to discover?

Answer 1

'high predictive value' is only defined if you a target which you are trying to predict upon. It seems that you don't, and your goal is to cluster data points according to some scale defined by a variety of factors. These can undoubtedly be used to cluster data points, and I'd advise you to look into the various methods available: some that may be interesting for you are Agglomerative and Hierarchical clustering.

Now to answer the question, you can surely generate new features from the ones present in your dataset that may or may not help you achieve your goal. You can:

• Bin your data: define some categories such as 'rich', 'average', 'poor' with specified ranges and create a new feature that maps a numerical value (wealth) to a bin
• One-hot-encode categorical variables

After these pre-processing steps are done, you could go ahead an apply the clustering methods I mentioned to group data together into their respective 'Social Status'. Of course, lots of tweaking and experimentation will be needed. As far as what I've encountered, there are not really automagic ways of generating new features, and mostly the available methods will depend greatly on the type of data and problem you are working on.