6
$\begingroup$

Based on a training set we applied a simple Linear Regression on some attributes that all were numeric.

Now we have more attributes in terms of categories and of course we applied one-hot-encoding to transform the categories to binary attributes

Take for example this simple python code:

X_train, X_test, y_train, y_test = train_test_split(X, y, train_size = 0.8, test_size=0.2)

model = LinearRegression(normalize=True).fit(X_train, y_train)

printErrorMetrics(trueTargets=y_test, predictions=model.predict(X_test))

When the table X has only the original numeric attributes the scores from the printErrorMetrics function (RMSE, etc.) are all good enough

We were expecting better results after adding the one-hot-encoded categories but the results are so worse that the method does not seem to work.

Are we missing anything?

Do we need to preprocess the data after adding the one-hot-encoded columns/attributes?

$\endgroup$
7
$\begingroup$

One possible reason is that when you use one-hot-encoding for categorical data, you should set the intercept property in the function to be False:

model = LinearRegression(fit_intercept=False, normalize=True).fit(X_train, y_train)

This will avoid the dummy variable trap: http://www.algosome.com/articles/dummy-variable-trap-regression.html

You could also use dummy encoding to avoid this problem: http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.LabelEncoder.html

$\endgroup$
  • 2
    $\begingroup$ dropping a column for each category, as the link explaining the dummy-variable-trap-regression suggested, worked awesomely $\endgroup$ – Georgios Pligoropoulos Oct 20 '16 at 22:04
  • 1
    $\begingroup$ can also be done easily with pandas.get_dummies(data,drop_first=True) $\endgroup$ – oW_ Oct 21 '16 at 2:09
  • $\begingroup$ @oW_, That's cool, always finding new things in pandas. $\endgroup$ – Hobbes Oct 21 '16 at 3:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.