2
$\begingroup$

I have seen pages where they mention 5 methods of building models.

1) All-in
2) Backward Elimination
3) Forward Selection
4) Bidirectional Elimination
5) Score Comparision

I usually implement a linear regression or any algorithm using sklearn

from sklearn.linear_model import LinearRegression
lr = LinearRegression()
lr.fit(X, y)
y_pred = lr.predict(X_test)

How can one implement those 5 methods of building models?

Can anyone explain the importance of this and What is most commonly used one out of these?

$\endgroup$
2
$\begingroup$

Strictly speaking, those are not methods of "building" models, they are feature selection strategies.

Most of them are implemented in Scikit-learn.

$\endgroup$
2
$\begingroup$

As Qusai said in his answer, these are not methods to build models, these are selection mechanisms. There are others as well, like feature engineering (for instance dimensionality reduction can be seen as feature engineering).

Starting from your example, and assuming that X is your full data:

from sklearn.linear_model import LinearRegression
lr = LinearRegression()
lr.fit(X, y)
y_pred = lr.predict(X_test)

This is the all-in. From this, backward elimination is removing one of the columns and keeping the n-1 with the best result.

Then forward is starting from 0 and adding 1 feature, so just one column at a time, comparing all potential ones.

The bidirectional is to try adding or removing at each step (because non linear models may sometimes be better without one feature, but better with two of them).

Each time, compare the score, and more precisely, compare the cross-validated score. In a way, only step 5 is what you ALWAYS do.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.