8
$\begingroup$

I just fitted a logistic curve to some fake data. I made the data essentially a step function.

data = -------------++++++++++++++

But when I look at the fitted curve, the slope is very small. The function that best minimizes the cost function, assuming cross entropy, is the step function. Why does it not look like a step function? Is there some regularization, L1 or L2, done by default?

Logistic regression using scikit-learn

$\endgroup$
6
$\begingroup$

Please take a look at the documentation. The first line shows the default parameters, which include penalty='l2' and C=1.0.

You actually cannot disable regularization completely, you can only regularize less... try setting C=1e10 for example.

$\endgroup$
5
$\begingroup$

Yes, there is regularization by default. It appears to be L2 regularization with a constant of 1.

I played around with this and found out that L2 regularization with a constant of 1 gives me a fit that looks exactly like what sci-kit learn gives me without specifying regularization.

from sklearn.linear_model import LogisticRegression    
model = LogisticRegression()
model.fit(X, y)

is the same as

model = LogisticRegression(penalty="l2", C=1)
model.fit(X, y)

When I chose C=10000, I got something that looked a lot more like step function.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.