I'm self-teaching myself some scikit-learn and tensorflow right now, and trying to get better at cleaning data since the rest seems reasonably straightforward.
I downloaded some Google Play Store data from here, and upon taking a peak, it appears to be primarily numerical, although the numbers stored are not of a recognized type.
<class 'pandas.core.frame.DataFrame'> RangeIndex: 10841 entries, 0 to 10840 Data columns (total 13 columns): App 10841 non-null object Category 10841 non-null object Rating 9367 non-null float64 Reviews 10841 non-null object Size 10841 non-null object Installs 10841 non-null object Type 10840 non-null object Price 10841 non-null object Content Rating 10840 non-null object Genres 10841 non-null object Last Updated 10841 non-null object Current Ver 10833 non-null object Android Ver 10838 non-null object dtypes: float64(1), object(12) memory usage: 1.1+ MB
A lot of these are obviously numerical anyway when checking with head(), but some entries have something like "3.0M" in place of 3000000, and so I can't have any fun with them yet (I'm trying to first make histograms of them to explore). I wrote this quick loop to try and address this by converting them to a string, taking off the "M", then making them floats, but it doesn't seem to be working because the output still isn't numeric.
k=0 for i in dat["Reviews"]: #make the entry a string: j=str(i) #remove the Ms- I'm assuming these mean million if j[-1]=='M': j=j[:-1] #cut off last letter j=float(j) j=j*10**6 dat[k,"Reviews"]=j k=k+1
There is likely a far easier way to do this that I'm missing, but the fact not even this works is strange. Does anyone have any ideas on how to deal with an unknown object type in a dataframe like this?