Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
Say I have a scatter plot like this:
Since I have many scatter plots like this I want to do feature transformation i.e. squash (x,y) in a single term to be input to a NN. Which transformation like x/y or (x/y)^2 or any other transformation will work best in this type of graphs i.e. increase the separation more while squashing it to a single term.
As @David Masip mentioned, Principal Component Analysis would be a good method to use here. Essentially PCA is a method by which a mapping is found between a high dimensional space to a lower dimenional space while keeping as much variation in the data as possible - perfect for dimensionality reduction of high dimensional data.
However, you mention that you want to use this reduced data to train a neural network model. It may be best to first train the neural net model and see how well it performs, as neural nets are usually very good at identifying interactions between features as well as other hidden structures in the data. If it doesn't perform well, then one approach to improve performance may be to use PCA - although this is highly dependent on your use case, content / type / amount of data, neural network architecture etc.
p.s PCA is also good to visualize high dimensional data (reduce the dimensionality to 2 or 3 dimensions, then plot it. This is better than plotting only 2 features at a time as you have done above).
Have a look at Linear Discriminant Analysis. This guide should give you an idea why it's more appropriate than PCA for your task.
I think what you are looking for is PCA (Principal Component Analysis). In your case, you have to take the first principal component. PCA allows to automate the process of deciding which is the linear combinations of the variables that explains the most of the data. In the picture you've showed, the first component will be roughly the vertical axis of your plot. If you don't know what PCA is, see this magnificient answer on cross-validated.
Taking your comment:
Well I don't want to use pca..It's just a 2 dimensional data..So I thought of some manual feature extraction
Something really simple you can do is just to use $y$ directly. It looks like $y \gtrapprox 23$ does a pretty good job separating the the red and green groups in the scatterplot.
One could generalize this approach to other pairings of variables $(i,j)$ by fitting decision trees with a depth of 1, yielding the best single variable split to separate the two groups for each pair. These rules can be used to select a single variable to use, $i$ or $j$, or a binary variable/flag to make, such as the $y \gtrapprox 23$ suggestion above.
