Is there a way to recognize which of these scatter plots is "better"?

Question

I doubt better is the correct adjective, apologies for that. What I mean is this: I have a set of files (1200~) each paired with a scatterplot image. I need to find a way to classify which data files will have scatter plots that a person would classify as "clearly separated data"(again, not the right words), and which ones as "not clearly separated". For example:

The first and second scatter plots are examples of data that would end classified as "good" or "clear", while the third one would be classified as "unclear". Are there metrics or distances that could be applied on the data to predict wheter the plots generated by that file would be of either the first kind or the second?

Thanks for taking the time to read this. I'm new to this, and english is not my first language.

Answer 1

You could:

1. Label manually few of them (say 100-150), then train a simple model to classify data. A small Random Forest could do the job well.

2. Train a super basic model on the each dataset used to produce each scatterplot. Something like a Linear Classifier. If the Classifier doesn't make mistakes, you have "clearly separated data", if it makes mistakes then it's likely to be the opposite.

Answer 2

Welcome to the community!

Very interesting question! I start with an introduction and the propose a few solutions:

If you did not have labels (seems like you have as you colored them) then there was not any justifiable argument for that. Evaluation of unsupervised tasks are theoretically impossible as you already mentioned (what is close? or what is clear?). There are ways to have some intuition about performance of an unsupervised tasks in practice (like devision of within cluster variation and between clusters difference which tells you to some extent how "well-separated" they are but for instance in this case a Gaussian distribution is assumed).

After this I jump directly to my suggestions:

Funny Way

Assuming that for each image you have the data points and their labels (orange and blue):

1. Use a linear classifier to classify them and in a cross validation pipeline, get the test score for them. We will use those scores later as an indicator of well-separation.
2. Use a nonlinear classifier and do the same as above. This time you get better scores. We will use them later as second stage of well-separation
3. Why linear vs non-linear? I think two classes which are linearly separable are more well-separated than two classes which are non-linearly separable. If your assumption differs from me then combine step (1) and (2).
4. At last take all linear scores and non-linear scores and combine them (you may simply add them: If classes are linearly separable, then both scores are high so the summ is highest, if nonlinearly separable then one is high one is lower so the summ is still high but lower than linear ones, if data is not separable i.e. like the third image then both scores are low so their sum is the lowest.). Now you can use these numbers as the indicator of well-separation.

A Bit More Serious

After reading the data (you have 2 features apparently) find the F-Value for those or use LDA to find the best axis of projection and then calculate F-Value there. Or use PCA to find the best axis of projection and calculate F-value on that feature. If you see so many non-linear classes, then use Kernel PCA.

Using Images

I assumed you have data files but if your project is to determine this well-separation from the images themselves i.e. doing Image Processing and Analysis, then comment under my answer so I update the answer.

Hope it helped. Good Luck!