2
$\begingroup$

I'm trying to do feature selection in the bayesian framework with a Laplace prior with the following code in Python;

Code:

#nb_predictors = len(df.columns) - 1 # we remove the target variable
nb_predictors = 7
beta = list()

with pm.Model() as model:
    # Define priors
    intercept = pm.Normal('Intercept', mu=0, sd=1/25)

    for cpt in range(1, nb_predictors + 1):
        beta.append(pm.Laplace('beta_' + str(cpt), mu=0, b=np.sqrt(2)))

    # Define Likelihood    
    logit = intercept + beta[0] * df['satisfaction_level'] + beta[1] * df['last_evaluation'] \
                      + beta[2] * df['number_project'] + beta[3] * df['average_montly_hours'] \
                      + beta[4] * df['time_spend_company'] + beta[5] * df['Work_accident'] \
                      + beta[6] * df['promotion_last_5years']



    likelihood = pm.Bernoulli('left', pm.math.sigmoid(logit), observed=df['left'])

What I was wondering is what happen if I add a new categorical variable (by one hot encoding), like this sales variable. Can I still use a laplace prior and observe if the density is close to 0 (so it is probably not related with the target variable) or it makes no sense with categorical variable ? and it only works with continuous variable ?

# Define Likelihood    
logit = intercept + beta[0] * df['satisfaction_level'] + beta[1] * df['last_evaluation'] \
                  + beta[2] * df['number_project'] + beta[3] * df['average_montly_hours'] \
                  + beta[4] * df['time_spend_company'] + beta[5] * df['Work_accident'] \
                  + beta[6] * df['promotion_last_5years'] + beta[7] * df['sales_low'] + beta[8] * df['sales_high']
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.