Do you use categorical data types?

Question

Personally I've never used the categorical data type in pandas, and leave eveything as objects. I've seen it has the capability to be saved as parquet files, saves data etc...

What are the pros and cons? Why would I not just transform every object type to categorical?

Answer 1

The main advantages of using categorical dtype are:

• Memory efficiency. The data is stored as integer codes, which are smaller in size than strings, the category type requires less memory to store the same amount of data compared to object type or int type data.
• Faster processing. Categorical data operations such as group by are generally faster than equivalent operations on object or int type data because they can be performed on the integer codes, which are more efficient to work with than strings.

The cons are:

• group by output: the output of the groupby is very messy. a lof of Nan are generated depending on your categories values.
• the same problem applies to the filtering.
• concatenation issue with category type: the category type is linked to a dictionary of values so when you concatenate or merge you will have trouble and the loss of the category dtype.

You could have more in-depth information from my article here.