0
$\begingroup$

Lets say I have 100 values in my dataset and split it 80% train 20% test. When predicting the last value, is the prediction based on previous 99 (80 test + 19 already predicted values) or only the original 80 train values?
For example: if kd-tree is used, is every data point inserted into the tree during the prediction?

Is it possible to use knn for the following scenario? I have 20 train values, when I add new observation I classify it and add it to the train dataset so there are 21 values, next time I add new value I classify it based on the 21 values in the dataset. I understand that this is probably not how it should be done, but imagine I am adding up to 50k values so the last one is classified based on the previous 49 999 values.

Another simplified example I came up with. n=2: on pictures 1,2,3 we see points as they were trained and one new green point which will be classified. then we take new observation, are the distances calculated to points as in 4a or as in 4b. link to visualization
Imagine its python sklearn module doing the classification. Up until the picture 1 we called .fit(X_train, y_train), where test dataset had 4 points. Then we called .predict(X_test) which had 2 points.

$\endgroup$

1 Answer 1

0
$\begingroup$

It depends which scenario you chose.

When you train any data science model, it won't move anymore. For example, if you train K-Means, you'll get at a result the cendroid of every cluster. If you train a random forest, you'll have as a result your trees.

Then, when you apply your model, it gives you an answer according to that. The answer will always be the same, if your entry is the same.

So if you trained your model in your 80 samples, after testing it on your 20 remaining, the model remains the same, trained on the 80 samples, and will give the exact same answer if you test it again on the same 20 samples.

However, it's possible to re-train your model : You make all your tests with your 80-20, and once you found the good parameters, you retrain a new model on your 100 samples, so it'll be more precise if it has to classify new samples.

What you want to do is avoid retraining your model, but include the result of new samples each time you encounter one : that's hard, not all models can do it, and that's not for a starter level at all. It also raises a lot of questions (I won't go into too much details)

What I'd suggest you is to manually set a threshold to re-train your model. Example : don't re-train your model until you have 1000 new samples to insert, or only re-train your model each month at a defined date, with the data you know at this time.

$\endgroup$
2
  • $\begingroup$ Hey, I certainly understand that retraining the model is supposedly bad, but the first question is about something that happens more under the hood of the knn(k-nearest neighbors) algorithm itself. Yes the results will be the same because distances between the data points did not change and are the same, but the question remains, when predicting new data point do you calculate distance to the points which were classified during prediction or not? I have edited my question and added new visualisation. $\endgroup$
    – montty
    Commented Jul 14, 2021 at 7:52
  • 1
    $\begingroup$ No, each prediction is independant, made only from what you used to train/fit your model $\endgroup$
    – Adept
    Commented Jul 15, 2021 at 6:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.