0
$\begingroup$

Can someone intuitively explain how interacting variables are being handled by KNN.according to the book "Introduction to Data Mining":

Nearest neighbor classifiers can handle the presence of interacting attributes, i.e., attributes that have more predictive power taken in combination then by themselves, by using appropriate proximity measures that can incorporate the effects of multiple attributes together.

is there an example to help to demonstrate it?

thank you

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ intuitively if one takes certain variables together, instead of singles, then they have more predictive power. So a kNN with an appropriate distance metric (which groups together these interacting variables in a neighborhood) should provide better results. $\endgroup$
    – Nikos M.
    Commented Jul 3, 2021 at 17:03

1 Answer 1

Reset to default
1
$\begingroup$

Interaction effect means the target is dependent on the interaction of two features i.e. X,Y but the data doesn't consider that Feature i.e. XY.

Simple model which tries to find a global pattern with the dataset will suffer from this issue e.g. LinearRegression.
Interaction Effect needs a full chapter for itself. Read here

Let's check this plot where the target is dependent on (X, Y, XY). For simplicity, I have kept the plot 2-D (ignoring Y).
We can easily observe that LinearRegression will have more challenges to predict the circled point.

While KNN will better approximate since it is not dependent on global pattern and will look on nearby data.

$\hspace{3cm}$enter image description here

In other words, the Interaction effect doesn't look prominent in a small portion of the space while it can have a large impact if we see the whole dataset together.
KNN works only with a small portion of space at a time(assuming small K).

Improve this answer
$\endgroup$
1
  • $\begingroup$ thanks @10xAI (nice name :) ), so I think intuitively is that I am calculating the distance between data points which invovle interacting features (e.g., x1, x2) anyway in the equation such as in euclidean distance but the only thing is that it is not getting multiplied (as usually interaction means multiplication impact) $\endgroup$
    – Abdulkarim Kanaan
    Commented Jul 7, 2021 at 8:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.