0
$\begingroup$

I am currently working a on project that requires me to convert all of the categorical variables to continuous (or binary) variables to build a regression model. The problem is that I have more than 100 categorical values and all of them have different measures. For example, let us say that users who have different types of cars and all of those car models have associated continuous variables such as the price of the cars, size of gas tank, avg price of the gas to fill the gas tank, etc. Here is an example of this dataset below:

user car_make car_model price gas_tank full_gas_tank_price
123_a Honda civic 17000 12 19
345_b Lexus RX 45000 16 35

One way to rebuild this feature set to only contain continuous variables is have car_make and car_models as columns that each could contain the price, gas_tank, and full_gas_tank_price values. For example:

user Honda_price honda_gas_tank honda_full_gas_tank_oprice Lexus_price Lexus_gas_tank Lexus_full_gas_tank_price
123_a 17000 12 19 0 0 0
345_b 0 0 35 45000 16 35

(I have not included the columns for civic model and RX model because the table is going to be wide). So, if I create the table like above, the table is going to include too many columns. Becuse for each car_make and car_model, there are 3 different features. So, I decided to create my featureset in a way that all of the car_make and car_models are binary variables and then have other continuous features as they are. Here is the example of the table:

user Honda Lexus civic RX price gas_tank full_gas_tank_price
123_a 1 0 1 0 17000 12 19
345_b 0 1 0 1 45000 16 35

This sounded reasonable to me because I would have less features, hence, better performance. However, I was a bit worried that it would affect the model's accuracy because this table does not specifically show the price of honda. Is just shows that the user has honda and then the price is on another column. What do you think?

(I understand that I can potentially remove features that do not correlated with the output, but let us assume that all of the existing features correlate with the output but there are just too many of them)

$\endgroup$
0
$\begingroup$

What you are doing is fine, you are one-hot encoding the categorical variables.

There are two ways to think of price: target or feature. If price is the target then the model will learn to weight the presence of Honda or Lexus as way to predict price. If price is a feature and you want to include the effect of Honda or Lexus, you'll have to add an interaction term to the regression model capture that relationship.

$\endgroup$
3
  • $\begingroup$ Thanks for your comment. Could you please elaborate more on the interaction term? $\endgroup$
    – Ryan
    Aug 11 '21 at 16:57
  • $\begingroup$ Sure - Added a hyperlink that goes into greater detail. $\endgroup$ Aug 11 '21 at 16:59
  • $\begingroup$ Got it! Thanks so much :) $\endgroup$
    – Ryan
    Aug 11 '21 at 17:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.