Is there a way in pandas to import NA fields as a string rather than NaN?

Question

I'm doing a Kaggle challenge, and a lot of entries in the data are NA. However, according to the data description, this doesn't actually mean "missing data", it means something like "Not applicable", in the sense of it just not having that quality (for example, not having a basement).

However, when I use pandas to import the data using read_csv(), and then use head() to look at it, it shows NaN for all those things that should be NA (comparing with the spreadsheet in LibreOffice).

I know how to just replace one value with another for a given column, but there's still a problem. NaN is usually what it should enter if there's just a missing value. So, if it's importing both "NA" and a blank cell as NaN, I won't know which it is.

It's probably not a huge issue, because I'll probably want to replace actual missing values with something common anyway, but it's good to know. I know I could also go into the csv with regular python and rename those actual "NA"s, but that's cumbersome. Is there a way I can just import certain columns as strings?

Answer 1

Use the pandas.read_csv() options:

d = pandas.read_csv('foo.csv', keep_default_na=False)

na_values : scalar, str, list-like, or dict, default None

Additional strings to recognize as NA/NaN. If dict passed, specific >per->column NA values. By default the following values are interpreted as >NaN: ‘’, ‘#N/A’, ‘#N/A N/A’, ‘#NA’, ‘-1.#IND’, ‘-1.#QNAN’, ‘-NaN’, ‘-nan’, >‘1.#IND’, ‘1.#QNAN’, ‘N/A’, ‘NA’, ‘NULL’, ‘NaN’, ‘n/a’, ‘nan’, ‘null’.

keep_default_na : bool, default True

If na_values are specified and keep_default_na is False the default NaN >values are overridden, otherwise they’re appended to.

from here

Answer 2

I think you can do something like this which might help in terms of specifying the data type.

d = pandas.read_csv('foo.csv', dtype={'ColumnName': 'S10'})

However, i think it is a wiser idea to "Find and Replace" all such values to something like blank cell or "Not Available" and then parse the file as this kind of data cleaning is better done before importing rather than after importing and processing