New answers tagged

8

There is some important information missing in your question, i.e. what the standard parameters are and what kind of logistic regression you use. When you use sklearn.linear_model.LogisticRegression, you will see in the docs that the first hyperparameter is the penalty which defaults to l2. This means that by default "shrinkage" of parameters is ...


0

Regularization tries to reduce the chance of overfitting by reducing the sensitivity to small changes in the input data. This is not as much of an issue for the intercept term (relative to the coefficients) so it is often not included.


1

A sigmoid function is a function with specific properties, most notably it maps values to the interval $[0,1]$. Often "sigmoid" is used to describe "some" s-shaped function which maps values to the interval $[0,1]$. If you take the logistic function $$ f(x) = \frac{L}{ 1 + e^{-k(x-x_0)} },$$ with $k=1, x_0=0, L=1$ ($x_0$ midpoint, $L$ ...


0

Maximum Likelihood Maximum likelihood estimation involves defining a likelihood function for calculating the conditional probability of observing the data sample given probability distribution and distribution parameters. This approach can be used to search a space of possible distributions and parameters. The logistic model uses the sigmoid function (...


0

In short, Maximum Likelihood estimation is used to find parameters given target values y and x. The Maximum likelhood estimation finds the parameters maximises the probability of y given x. It has been proved that MLE estimation problem caan be solved by finding the parametrs which gives least cross entropy in case of binary classification. Gradients descent ...


0

When we have such a imbalanced dataset, it is always a good practice to do hyperparameter tuning of some randomly sampled data. Once you get the best parameters apply it complete datasets For dealing with imbalanced I think craig has already pointed out links.


2

For understanding this first we will have to look at the maths of logistic regression. The equation of linear regression is given by : P(y|x;w) = Sigmoid(wTx + b) Now if we take log on both sides and folow the match in the image below, it clearly show why log of odds linearly related to the predictor variables After step 6, shown in above image if you take ...


1

The Log of Odds is used for interpretation purposes if we want to compare Logisitic Regression to Linear Regression. Unlike linear regression, $\beta_0 + \beta_1X$ does not directly give you the estimated value of your response variable. It gives the estimated log of odds, here's a short derivation that you already may have seen: $$p = \frac{e^{\beta_0+\...


0

MLE (Maximum Likelihood Estimation) sets up the optimization problem and gradient descent is a method for finding a specific solution to the optimization problem. MLE defines the optimization problem as finding the values of the model parameters that maximize the likelihood function over the parameter space, selecting the parameter values that make the ...


0

Gradient descent is a numerical method used by a computer to calculate the minimum of a loss function. If that loss function is related to the likelihood function (such as negative log likelihood in logistic regression or a neural network), then the gradient descent is finding a maximum likelihood estimator of a parameter (the regression coefficients). For ...


1

You get the output of the logistic regression $\sigma$, which is between $0$ and $1$. Default option (is spit out out from most packages): In order to get class labels you simply map all values $\sigma \leq 0.5$ to $0$ and all values $\sigma >0.5$ to $1$. The $\sigma =0.5$ belonging to class $0$ can be different in different implementations (practically ...


0

(Throughout this, I will assume the classes are balanced. If that is not the case, $0.5$ is likely to be a poor threshold. (As the links in my comments describe, thresholds are even overrated.)) The good news is that this situation would be unusual, so it is unlikely to matter. If it does come up, you have a few options. First, if the model is so uncertain ...


1

Despite the interesting comments on setting appropriate threshold values, I think the main question was about wat the threshold value actually means for the prediction. There are different ways to implement a thresholding function. Your proposed way says that for a predicted probability p: if p > threshold, it is predicted to be 1 AND if p < threshold ...


0

This can be accomplished by weighting the samples in the loss function calculation. In sklearn, that's done using the sample_weights argument of the fit method. You'd set that parameter as an array, whatever function of your f1 that's most appropriate.


Top 50 recent answers are included