0
$\begingroup$

I have cleaned the data from nan values and infinite values, the only feature which has a large float is the column 8 (it's a sum)This is what the dataframe look like

I have no Idea how to fix this last error, I tried all previous solutions which are related to my question but nothing worked.

This is the result of : df.isnull().sum():

isnull

I am following this tutorial (because I'm just starting to learn about machine learning supervised algorithm KNN): Tutorial Link

Please help me out!

$\endgroup$
0
$\begingroup$

I'm almost certain that you have missing values, check that and fill them beforehand

Assuming you data frame is called df

df.info() will give you that info

Additionally df.describe() is a good method to validate the maxmimum value is not np.inf

$\endgroup$
  • $\begingroup$ I added the result of df.isnull().sum() $\endgroup$ – geekys Aug 24 '20 at 22:31
  • $\begingroup$ Does that sum 0 for all the columns? $\endgroup$ – Julio Jesus Aug 24 '20 at 22:32
  • $\begingroup$ Is this a classification task? If so... is you y matrix only binary-valued? $\endgroup$ – Julio Jesus Aug 24 '20 at 22:33
  • $\begingroup$ yes, I updated the question, you can see it on the image $\endgroup$ – geekys Aug 24 '20 at 22:33
  • $\begingroup$ The dataframe datatypes are: int and float only $\endgroup$ – geekys Aug 24 '20 at 22:34
0
$\begingroup$

You should scale your data before training your data. Try something simple:

df = (df - df.min()) / (df.max() - df.min())

If you're using the L2 distance in your knn model this means some values are squared (which is probably why you max out the float64). Btw float64 max is $2^{31} − 1$ so check the range of the columns just to be sure there might be an outlier.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.