6
$\begingroup$

Based on a training set we applied a simple Linear Regression on some attributes that all were numeric.

Now we have more attributes in terms of categories and of course we applied one-hot-encoding to transform the categories to binary attributes

Take for example this simple python code:

X_train, X_test, y_train, y_test = train_test_split(X, y, train_size = 0.8, test_size=0.2)

model = LinearRegression(normalize=True).fit(X_train, y_train)

printErrorMetrics(trueTargets=y_test, predictions=model.predict(X_test))

When the table X has only the original numeric attributes the scores from the printErrorMetrics function (RMSE, etc.) are all good enough

We were expecting better results after adding the one-hot-encoded categories but the results are so worse that the method does not seem to work.

Are we missing anything?

Do we need to preprocess the data after adding the one-hot-encoded columns/attributes?

$\endgroup$

1 Answer 1

7
$\begingroup$

One possible reason is that when you use one-hot-encoding for categorical data, you should set the intercept property in the function to be False:

model = LinearRegression(fit_intercept=False, normalize=True).fit(X_train, y_train)

This will avoid the dummy variable trap: http://www.algosome.com/articles/dummy-variable-trap-regression.html

You could also use dummy encoding to avoid this problem: http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.LabelEncoder.html

$\endgroup$
3
  • 2
    $\begingroup$ dropping a column for each category, as the link explaining the dummy-variable-trap-regression suggested, worked awesomely $\endgroup$ Oct 20, 2016 at 22:04
  • 1
    $\begingroup$ can also be done easily with pandas.get_dummies(data,drop_first=True) $\endgroup$
    – oW_
    Oct 21, 2016 at 2:09
  • $\begingroup$ @oW_, That's cool, always finding new things in pandas. $\endgroup$
    – Hobbes
    Oct 21, 2016 at 3:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.