7
$\begingroup$

Say I have a scatter plot like this:

enter image description here

Since I have many scatter plots like this I want to do feature transformation i.e. squash (x,y) in a single term to be input to a NN. Which transformation like x/y or (x/y)^2 or any other transformation will work best in this type of graphs i.e. increase the separation more while squashing it to a single term.

$\endgroup$

4 Answers 4

5
$\begingroup$

As @David Masip mentioned, Principal Component Analysis would be a good method to use here. Essentially PCA is a method by which a mapping is found between a high dimensional space to a lower dimenional space while keeping as much variation in the data as possible - perfect for dimensionality reduction of high dimensional data.

However, you mention that you want to use this reduced data to train a neural network model. It may be best to first train the neural net model and see how well it performs, as neural nets are usually very good at identifying interactions between features as well as other hidden structures in the data. If it doesn't perform well, then one approach to improve performance may be to use PCA - although this is highly dependent on your use case, content / type / amount of data, neural network architecture etc.

p.s PCA is also good to visualize high dimensional data (reduce the dimensionality to 2 or 3 dimensions, then plot it. This is better than plotting only 2 features at a time as you have done above).

$\endgroup$
1
  • $\begingroup$ Well I don't want to use pca..It's just a 2 dimensional data..So I thought of some manual feature extraction $\endgroup$
    – DuttaA
    Jun 5, 2018 at 16:06
4
$\begingroup$

Have a look at Linear Discriminant Analysis. This guide should give you an idea why it's more appropriate than PCA for your task.

$\endgroup$
1
  • $\begingroup$ Answers having only links are discouraged... $\endgroup$
    – Aditya
    Jul 2, 2018 at 20:03
3
$\begingroup$

I think what you are looking for is PCA (Principal Component Analysis). In your case, you have to take the first principal component. PCA allows to automate the process of deciding which is the linear combinations of the variables that explains the most of the data. In the picture you've showed, the first component will be roughly the vertical axis of your plot. If you don't know what PCA is, see this magnificient answer on cross-validated.

$\endgroup$
3
$\begingroup$

Taking your comment:

Well I don't want to use pca..It's just a 2 dimensional data..So I thought of some manual feature extraction

Something really simple you can do is just to use $y$ directly. It looks like $y \gtrapprox 23$ does a pretty good job separating the the red and green groups in the scatterplot.

One could generalize this approach to other pairings of variables $(i,j)$ by fitting decision trees with a depth of 1, yielding the best single variable split to separate the two groups for each pair. These rules can be used to select a single variable to use, $i$ or $j$, or a binary variable/flag to make, such as the $y \gtrapprox 23$ suggestion above.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.