Skip to main content
OverflowAI is here! AI power for your Stack Overflow for Teams knowledge community. Learn more
24 votes

What does it mean to "share parameters between features and classes"

I will try to answer this question through logistic regression, one of the simplest linear classifiers. The simplest case of logistic regression is if we have a binary classification task ($y \in\{0,...
Djib2011's user avatar
  • 7,998
17 votes
Accepted

Python implementation of cost function in logistic regression: why dot multiplication in one expression but element-wise multiplication in another

In this case, the two math formulae show you the correct type of multiplication: $y_i$ and $\text{log}(a_i)$ in the cost function are scalar values. Composing the scalar values into a given sum over ...
Neil Slater's user avatar
16 votes
Accepted

Linear regression with non-symmetric cost function?

If I understand you correctly, you want to err on the side of overestimating. If so, you need an appropriate, asymmetric cost function. One simple candidate is to tweak the squared loss: $\mathcal L: ...
Emre's user avatar
  • 10.5k
15 votes

How to get p-value and confident interval in LogisticRegression with sklearn?

The short answer is that sklearn LogisticRegression does not have a built in method to calculate p-values. Here are a few other posts that discuss solutions to this, however. https://stackoverflow....
Hobbes's user avatar
  • 1,449
15 votes
Accepted

What is the difference between SGD classifier and the Logisitc regression?

Welcome to SE:Data Science. SGD is a optimization method, while Logistic Regression (LR) is a machine learning algorithm/model. You can think of that a machine learning model defines a loss function, ...
user12075's user avatar
  • 2,274
15 votes
Accepted

Why continuous features are more important than categorical features in decision tree models?

It could be the way that you encode categorical variables. If you do One Hot Encoding (dummy) each encoded feature will only have two possible values [0,1]. Binary variables normally have less ...
Carlos Mougan's user avatar
14 votes

How to get p-value and confident interval in LogisticRegression with sklearn?

One way to get confidence intervals is to bootstrap your data, say, $B$ times and fit logistic regression models $m_i$ to the dataset $B_i$ for $i = 1, 2, ..., B$. This gives you a distribution for ...
darXider's user avatar
  • 613
12 votes

Does scikit-learn use regularization by default?

Please take a look at the documentation. The first line shows the default parameters, which include penalty='l2' and C=1.0. You ...
stmax's user avatar
  • 1,637
12 votes
Accepted

How to plot logistic regression decision boundary?

Regarding the code You should plot the decision boundary after training is finished, not inside the training loop, parameters are constantly changing there; unless you are tracking the change of ...
Esmailian's user avatar
  • 9,322
12 votes

How to plot logistic regression decision boundary?

Your decision boundary is a surface in 3D as your points are in 2D. With Wolfram Language Create the data sets. ...
Edmund's user avatar
  • 705
11 votes
Accepted

How do I implement the sigmoid function in Octave?

This will compute the sigmoid of a scalar, vector or matrix. ...
gingermander's user avatar
11 votes
Accepted

The differences between SVM and Logistic Regression

If you use logistic regression and the cross-entropy cost function, it's shape is convex and there will be a single minimum. But during optimization, you may find ...
Green Falcon's user avatar
  • 14.1k
11 votes

My data is highly overlapping, but when I apply logistic regression, it is giving an impressive accuracy of 79%. Why?

Decision Tree, KNN, & Random Forest (Methods that are suitable for overlapping data) This statement is false. All those methods are good when the decision surface (separating surface) has a ...
rapaio's user avatar
  • 4,763
9 votes
Accepted

Logistic regression on biased data

Background I'll start with some background to help you research the solution yourself and then will add some specifics. What you refer to as "biased data" is more commonly known as ...
AN6U5's user avatar
  • 6,818
8 votes
Accepted

How does binary cross entropy work?

When doing logistic regression you start calculating a bunch of probabilities $p_i$ and your target is maximize the product of those probabilities (as they're considered independent events). The ...
Alberto's user avatar
  • 251
8 votes

Are linear models better when dealing with too many features? If so, why?

There is some important information missing in your question, i.e. what the standard parameters are and what kind of logistic regression you use. When you use ...
Peter's user avatar
  • 7,516
7 votes

Does scikit-learn use regularization by default?

Yes, there is regularization by default. It appears to be L2 regularization with a constant of 1. I played around with this and found out that L2 regularization with a constant of 1 gives me a fit ...
sebastianspiegel's user avatar
7 votes
Accepted

Preprocessing text before use RNN

Welcome to the Data Science forum. Yes, data preprocessing is an important aspect of sentiment analysis for better results. What sort of preprocessing to be done largely depends on the quality of ...
Manohar Swamynathan's user avatar
7 votes
Accepted

Do logistic regression and softmax regression do the same thing?

There is a key difference: Softmax regression provides class probabilities for mutually exclusive classes. Logistic regression treats class membership for each class separately. Classes do not need ...
Neil Slater's user avatar
7 votes
Accepted

Bad classification performance of logistic regression on imbalanced data in testing as compared to training

I suspect the reason is that the class balance in your test set is different from the class balance in your training set. That will throw everything off. The fundamental assumption made by ...
D.W.'s user avatar
  • 3,371
7 votes

Is this a good practice of feature engineering?

1) Yes, it makes sense. Trying to create features manually will help the learners (i.e. models) to graspe more information from the raw data because the raw data is not always in a form that is ...
Fansly's user avatar
  • 71
7 votes
Accepted

What is the difference between SVM and logistic regression?

Both logistic regression and SVM are linear models under the hood, and both implement a linear classification rule: $$f_{\mathbf{w},b}(\mathbf{x}) = \mathrm{sign}(\mathbf{w}^T \mathbf{x} + b)$$ Note ...
KT.'s user avatar
  • 2,121
7 votes
Accepted

Risk prediction vs classification model

I will try to answer your question as shortly as possible. Yes, if you define probability as a risk, then the probabilities are risk scores. But, there's a catch in these scenarios, you will have to ...
Danny's user avatar
  • 1,158
6 votes
Accepted

Should I use regularization every time?

Normally you use regularization. The exception is if you know the data generating process and can model it exactly. Then you merely estimate the model parameters. In general you will not know the ...
Emre's user avatar
  • 10.5k
6 votes

difference between logistic regression and binary logistic regression

binary:logistic is used for binary classification where the target variable takes binary output [0, 1] reg:logistic is used for ...
Tu N.'s user avatar
  • 509
6 votes

What cost function and penalty are suitable for imbalanced datasets?

So you ask how does class imbalance affect classifier performance under different losses? You can make a numeric experiment. I do binary classification by logistic regression. However, the ...
David Dale's user avatar
  • 1,551
6 votes

How to get p-value and confident interval in LogisticRegression with sklearn?

This is still not implemented and not planned as it seems out of scope of sklearn, as per Github discussion #6773 and #13048. However, the documentation on linear models now mention that (P-value ...
Lucas Morin's user avatar
  • 2,224
6 votes

Learning rate in logistic regression with sklearn

sklearn.linear_model.LogisticRegression doesn't use SGD, so there's no learning rate. I think sklearn.linear_model.SGDClassifier...
Icyblade's user avatar
  • 4,336
6 votes
Accepted

AUC and classification report in Logistic regression in python

In order to calculate the AUC, you need to have probabilities. Therefore you should use the following function: ...
Archie's user avatar
  • 863
6 votes

How to perform Logistic Regression with a large number of features?

In order to reduce your model down to 7 variables there are a few approaches you could take: PCA (unsupervised): this creates "new" linear combinations of your data where each proceding component ...
bnorm's user avatar
  • 533

Only top scored, non community-wiki answers of a minimum length are eligible