How to measure the correlation between categorical variables and a continuous variable

Question

I have the following list of the names of the categorical variables in my dataset:

categorical_columns = ['MSSubClass', 'MSZoning', 'LotShape', 'LandContour', 'LotConfig', 'Neighborhood', 'Condition1',
                       'Condition2', 'BldgType', 'HouseStyle', 'RoofStyle', 'RoofMatl', 'Exterior1st', 'Exterior2nd',
                       'Foundation', 'Heating', 'Electrical', 'Functional', 'GarageType', 'PavedDrive', 'Fence',
                       'MiscFeature', 'SaleType', 'SaleCondition', 'Street', 'CentralAir']

in this function definition, I am one-hot encoding each column with the help of a for loop:

def feature_encoding(df, categorical_list):

    # One Hot Encoding the columns gathered in categorical_columns
    for col in categorical_list:

        # take one-hot encoding
        OHE_sdf = pd.get_dummies(df[categorical_list])

        # drop the old categorical column from original df
        df.drop(col, axis = 1, inplace = True)

        # attach one-hot encoded columns to original dataframe
        df = pd.concat([df, OHE_sdf], axis = 1, ignore_index = True)

I don't want to use all of these columns in my training so now I am at the dimensionality reduction phase of my work. I want to measure the correlation between each of these columns and my SalePrice variable (numerical) and weed out the columns with low correlation.

I've read that Chi-square test is generally used for measuring the correlation of categorical variables but I have not seen an implementation where it was a list of categorical variables vs a continuous variable.

Answer 1

The Chi-squared test measures the relationship between two categorical variables.

To measure the relationship between a categorical feature and a continuous feature, you can use an ANOVA test.

As an aside, you don't need to use a for loop for your encoding - get_dummies has the parameter columns which allows the user to specify which columns are encoded.