0
$\begingroup$

I have cleaned the data from nan values and infinite values, the only feature which has a large float is the column 8 (it's a sum)This is what the dataframe look like

I have no Idea how to fix this last error, I tried all previous solutions which are related to my question but nothing worked.

This is the result of : df.isnull().sum():

isnull

I am following this tutorial (because I'm just starting to learn about machine learning supervised algorithm KNN): Tutorial Link

Please help me out!

$\endgroup$
1
$\begingroup$

You should scale your data before training your data. Try something simple:

df = (df - df.min()) / (df.max() - df.min())

If you're using the L2 distance in your knn model this means some values are squared (which is probably why you max out the float64). Btw float64 max is $2^{31} − 1$ so check the range of the columns just to be sure there might be an outlier.

$\endgroup$
0
$\begingroup$

I'm almost certain that you have missing values, check that and fill them beforehand

Assuming you data frame is called df

df.info() will give you that info

Additionally df.describe() is a good method to validate the maxmimum value is not np.inf

$\endgroup$
10
  • $\begingroup$ I added the result of df.isnull().sum() $\endgroup$
    – geekys
    Aug 24 '20 at 22:31
  • $\begingroup$ Does that sum 0 for all the columns? $\endgroup$ Aug 24 '20 at 22:32
  • $\begingroup$ Is this a classification task? If so... is you y matrix only binary-valued? $\endgroup$ Aug 24 '20 at 22:33
  • $\begingroup$ yes, I updated the question, you can see it on the image $\endgroup$
    – geekys
    Aug 24 '20 at 22:33
  • $\begingroup$ The dataframe datatypes are: int and float only $\endgroup$
    – geekys
    Aug 24 '20 at 22:34
0
$\begingroup$

Try converting your dataframe to integer values since its dtype('float64') is throwing the error. Assuming your dataframe is called df, use this:

 df["column_8"] = pd.to_numeric(df["column_8"])

This will convert all values in that column to integers.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.