0
$\begingroup$

Personally I've never used the categorical data type in pandas, and leave eveything as objects. I've seen it has the capability to be saved as parquet files, saves data etc...

What are the pros and cons? Why would I not just transform every object type to categorical?

$\endgroup$

1 Answer 1

0
$\begingroup$

The main advantages of using categorical dtype are:

  • Memory efficiency. The data is stored as integer codes, which are smaller in size than strings, the category type requires less memory to store the same amount of data compared to object type or int type data.
  • Faster processing. Categorical data operations such as group by are generally faster than equivalent operations on object or int type data because they can be performed on the integer codes, which are more efficient to work with than strings.

The cons are:

  • group by output: the output of the groupby is very messy. a lof of Nan are generated depending on your categories values.
  • the same problem applies to the filtering.
  • concatenation issue with category type: the category type is linked to a dictionary of values so when you concatenate or merge you will have trouble and the loss of the category dtype.

You could have more in-depth information from this article: https://medium.com/gitconnected/pandas-category-type-pros-and-cons-1bcac1bdea71

New contributor
Arli94 is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.