Does f1 score evaluate only the model or does it also enable us to observe and evaluate the data?

Question

I have a dataset. This dataset consists of the data that the actual picture that needs to be drawn, that is, the 100-point graded paper, and the similarity between 100 and 0 points graded pictures that others are trying to draw on it. I obtain this similarity from ORB, SSIM, and VGG16 methods. Parameters in this dataset: ORB, SSIM, VGG16 and GRADED. Using KNN, I calculate the distances between the data in the test dataset and the 3D vector plane. I use k points weighted according to their distance to find the approximate grade. When I looked at using F1 score here, I had doubts about how I should find the TP, TN, FP, FN values. I think True values ​​indicate my data. I thought I should determine whether it is true or false depending on whether it falls within the margin of error I expect based on the orb ssim vgg16 values ​​in the data. I wonder if this is true. If this is the case, wouldn't F1 score give us an evaluation about the dataset?

UPDATE to be clear

this is a project to visually compare the plots drawn by students on an exam paper with the answer key and predict the grade based on the similarity score we get. My nodes are the grades given by the instructor. These nodes have 3 features. ORB, SSIM, VGG16. My dataset, which appears here as an example, is as follows: ORB SSIM VGG16 GRADED

                    0.7854, 0.71235, 0.682445, 75 .

I have a data list similar to these. Another parameter I use to calculate accuracy here is PREDICTED_GRADE: 71. I want to calculate an accuracy according to these values. I'm not exactly sure what my criteria should be for TP, TN, FP, FN values ​​to calculate with F1 score. What should I give as True and False values? One of the methods I have in mind is that I should set a +- range with the k% error rate that I promised before in this project. I think I can call those within this value range as Positive and those outside of this value range as Negative. So how should I determine the True and False values? Should I use some ORB SSIM VGG16 data that is inconsistent or consistent with grade?

Answer 1

So I'm really new to machine learning specifically but I have a fairly long background in data analysis and programming. And I have to admit I've only used KNN once, and I don't remember how to reconstruct the regions. But my understanding of TN, TP, FN, and FP is that some sort of hypothesis test is required.

For example, passing score equals PassScore

TP = number of test >= PassScore where train > PassScore

TN = number of test < PassScore where train < PassScore

FP = number of test >= PassScore where train < PassScore

FN = number of test < PassScore where train >= PassScore

the hypothesis doesn't necessarily have to be one-sided or set by the passing score, but generally speaking there has to be a division into binary categories associated with a null-hypothesis and a hypothesis to be tested to have a True/False, Positive/Negative division. Positive is associated with the test hypothesis and Negative is associated with the null-hypothesis. True is if the data agrees with the hypothesis and false is if it does not.

---

Edit:

What I mean is, if you've said

test agrees with train = positive

test disagrees with train = negative

Then you can't use

test agrees with train = true

test disagrees with train = false

because that doesn't give you enough constraints conceptually or mathematically

I'm not sure "test agrees with train" is a well-formed hypothesis.