Creating a complex featureset for regression modeling

Question

I am currently working a on project that requires me to convert all of the categorical variables to continuous (or binary) variables to build a regression model. The problem is that I have more than 100 categorical values and all of them have different measures. For example, let us say that users who have different types of cars and all of those car models have associated continuous variables such as the price of the cars, size of gas tank, avg price of the gas to fill the gas tank, etc. Here is an example of this dataset below:

user	car_make	car_model	price	gas_tank	full_gas_tank_price
123_a	Honda	civic	17000	12	19
345_b	Lexus	RX	45000	16	35

One way to rebuild this feature set to only contain continuous variables is have car_make and car_models as columns that each could contain the price, gas_tank, and full_gas_tank_price values. For example:

user	Honda_price	honda_gas_tank	honda_full_gas_tank_oprice	Lexus_price	Lexus_gas_tank	Lexus_full_gas_tank_price
123_a	17000	12	19	0	0	0
345_b	0	0	35	45000	16	35

(I have not included the columns for civic model and RX model because the table is going to be wide). So, if I create the table like above, the table is going to include too many columns. Becuse for each car_make and car_model, there are 3 different features. So, I decided to create my featureset in a way that all of the car_make and car_models are binary variables and then have other continuous features as they are. Here is the example of the table:

user	Honda	Lexus	civic	RX	price	gas_tank	full_gas_tank_price
123_a	1	0	1	0	17000	12	19
345_b	0	1	0	1	45000	16	35

This sounded reasonable to me because I would have less features, hence, better performance. However, I was a bit worried that it would affect the model's accuracy because this table does not specifically show the price of honda. Is just shows that the user has honda and then the price is on another column. What do you think?

(I understand that I can potentially remove features that do not correlated with the output, but let us assume that all of the existing features correlate with the output but there are just too many of them)

Answer 1

What you are doing is fine, you are one-hot encoding the categorical variables.

There are two ways to think of price: target or feature. If price is the target then the model will learn to weight the presence of Honda or Lexus as way to predict price. If price is a feature and you want to include the effect of Honda or Lexus, you'll have to add an interaction term to the regression model capture that relationship.