I just fitted a logistic curve to some fake data. I made the data essentially a step function.
data = -------------++++++++++++++
But when I look at the fitted curve, the slope is very small. The function that best minimizes the cost function, assuming cross entropy, is the step function. Why does it not look like a step function? Is there some regularization, L1 or L2, done by default?
Please take a look at the documentation. The first line shows the default parameters, which include penalty='l2' and C=1.0.
You actually cannot disable regularization completely, you can only regularize less... try setting C=1e10 for example.
penalty='none'. scikit-learn.org/stable/whats_new.html#id15
$\endgroup$
Yes, there is regularization by default. It appears to be L2 regularization with a constant of 1.
I played around with this and found out that L2 regularization with a constant of 1 gives me a fit that looks exactly like what sci-kit learn gives me without specifying regularization.
from sklearn.linear_model import LogisticRegression
model = LogisticRegression()
model.fit(X, y)
is the same as
model = LogisticRegression(penalty="l2", C=1)
model.fit(X, y)
When I chose C=10000, I got something that looked a lot more like step function.