0
$\begingroup$

I have a dataset (42000, 10) which contains 7 categorical features and 3 numerical. I would like to separate both the numerical and categorical features into 2 different data frames i.e I would like 2 data frames where one contains only numerical data (42000, 3) and the other only categorical data (42000, 7), perform some pre-processing on both of them, and lastly concatenate them into one data frame.

So, my question is how do I separate my initial dataframe into 2 based on numerical and categorical data?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Simplest way is to use select_dtypes method in Pandas. This returns a subset of a dataframe based on the column dtypes:

      df_numerical_features = df.select_dtypes(include='number')
      df_categorical_features = df.select_dtypes(include='category')

Reference documentation of select_dtypes

This will also depend on the column datatypes of your dataframe. Considering you have categorical columns and few columns are either int64 or float you can go for:

  df_numerical_features = df.select_dtypes(exclude='object')
  df_categorical_features = df.select_dtypes(include='object')

Use the include/exclude option to choose based on the dtype. Other dtype information is as shown below:

  • To select all numeric types, use np.number or 'number'

  • To select strings you must use the object dtype, but note that this will return all object dtype columns

  • To select datetimes, use np.datetime64, 'datetime' or 'datetime64'

  • To select timedeltas, use np.timedelta64, 'timedelta' or 'timedelta64'

  • To select Pandas categorical dtypes, use 'category'

  • To select Pandas datetimetz dtypes, use 'datetimetz' (new in 0.20.0) or 'datetime64[ns, tz]'

Improve this answer
$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.