Skip to main content

Questions tagged [logistic-regression]

Refers generally to statistical procedures that utilize the logistic function, most commonly various forms of logistic regression

Filter by
Sorted by
Tagged with
30 votes
3 answers
138k views

How to get p-value and confident interval in LogisticRegression with sklearn?

I am building a multinomial logistic regression with sklearn (LogisticRegression). But after it finishes, how can I get a p-value and confident interval of my model? It only appears that sklearn only ...
hminle's user avatar
  • 401
28 votes
4 answers
46k views

Scikit-learn: Getting SGDClassifier to predict as well as a Logistic Regression

A way to train a Logistic Regression is by using stochastic gradient descent, which scikit-learn offers an interface to. What I would like to do is take a scikit-learn's SGDClassifier and have it ...
hlin117's user avatar
  • 685
26 votes
2 answers
18k views

Text categorization: combining different kind of features

The problem I am tackling is categorizing short texts into multiple classes. My current approach is to use tf-idf weighted term frequencies and learn a simple linear classifier (logistic regression). ...
elmille's user avatar
  • 361
24 votes
3 answers
40k views

Python implementation of cost function in logistic regression: why dot multiplication in one expression but element-wise multiplication in another

I have a very basic question which relates to Python, numpy and multiplication of matrices in the setting of logistic regression. First, let me apologise for not using math notation. I am confused ...
GhostRider's user avatar
23 votes
4 answers
13k views

Is logistic regression actually a regression algorithm?

The usual definition of regression (as far as I am aware) is predicting a continuous output variable from a given set of input variables. Logistic regression is a binary classification algorithm, so ...
joews's user avatar
  • 526
22 votes
3 answers
10k views

Linear regression with non-symmetric cost function?

I want to predict some value $Y(x)$ and I am trying to get some prediction $\hat Y(x)$ that optimizes between being as low as possible, but still being larger than $Y(x)$. In other words: $$\text{cost}...
asPlankBridge's user avatar
21 votes
1 answer
4k views

What does it mean to "share parameters between features and classes"

When reading this paper there is a line which says "linear classifiers do not share parameters among features and classes." What is the meaning of this statement? Does it mean that linear ...
joydeep bhattacharjee's user avatar
20 votes
5 answers
28k views

Choose binary classification algorithm

I have a binary classification problem: Approximately 1000 samples in training set 10 attributes, including binary, numeric and categorical Which algorithm is the best choice for this type of ...
IgorS's user avatar
  • 5,474
17 votes
5 answers
51k views

Should I use a decision tree or logistic regression for classification?

I am working on a classification problem. I have a dataset containing equal numbers of categorical variables and continuous variables. How do I decide which technique to use, between a decision tree ...
Arun's user avatar
  • 717
16 votes
2 answers
6k views

Binary classification model for unbalanced data

I have a dataset with the following specifications: Training dataset with 193,176 samples with 2,821 positives Test Dataset with 82,887 samples with 673 positives There are 10 features. I want to ...
tejaskhot's user avatar
  • 4,085
15 votes
2 answers
22k views

Does scikit-learn use regularization by default?

I just fitted a logistic curve to some fake data. I made the data essentially a step function. data = -------------++++++++++++++ But when I look at the fitted ...
sebastianspiegel's user avatar
13 votes
1 answer
9k views

What is the difference in xgboost binary:logistic and reg:logistic

What is the difference in R in xgboost between binary:logistic and reg:logistic? Is it only in evaluation metric? If yes, how does RMSE on binary classification compare to error rate? Is the ...
user2530062's user avatar
12 votes
3 answers
33k views

How to plot logistic regression decision boundary?

I am running logistic regression on a small dataset which looks like this: After implementing gradient descent and the cost function, I am getting a 100% accuracy in the prediction stage, However I ...
rrz0's user avatar
  • 311
12 votes
2 answers
4k views

The differences between SVM and Logistic Regression

I am reading about SVM and I've faced to the point that non-kernelized SVMs are nothing more than linear separators. Therefore, ...
David Masip's user avatar
  • 6,101
12 votes
2 answers
14k views

How to perform Logistic Regression with a large number of features?

I have a dataset with 330 samples and 27 features for each sample, with a binary class problem for Logistic Regression. According to the "rule if ten" I need at least 10 events for each feature to be ...
LucasRamos's user avatar
12 votes
1 answer
20k views

How do I implement the sigmoid function in Octave? [closed]

so given that the sigmoid function is defined as hθ(x) = g(θ^(T)x), how can I implement this funcion in Octave given that g = zeros(size(z)) ?
Shuryu Kisuke's user avatar
12 votes
1 answer
4k views

What's the relationship between an SVM and hinge loss?

My colleague and I are trying to wrap our heads around the difference between logistic regression and an SVM. Clearly they are optimizing different objective functions. Is an SVM as simple as saying ...
Simon's user avatar
  • 1,026
11 votes
4 answers
2k views

Learning ordinal regression in R?

I'm working on a project and need resources to get me up to speed. The dataset is around 35000 observations on 30 or so variables. About half the variables are categorical with some having many ...
Matt Hall's user avatar
  • 111
10 votes
4 answers
2k views

Is this a good practice of feature engineering?

I have a practical question about feature engineering... say I want to predict house prices by using logistic regression and used a bunch of features including zip code. Then by checking the feature ...
user3768495's user avatar
10 votes
1 answer
21k views

What is the difference between SGD classifier and the Logisitc regression?

To my understanding, the SGD classifier, and Logistic regression seems similar. An SGD classifier with loss = 'log' implements Logistic regression and loss = 'hinge' implements Linear SVM. I also ...
Akash Dubey's user avatar
10 votes
3 answers
9k views

What cost function and penalty are suitable for imbalanced datasets?

For an imbalanced data set, is it better to choose an L1 or L2 regularization? Is there a cost function more suitable for imbalanced datasets to improve the model score (...
red_GNS's user avatar
  • 101
9 votes
2 answers
2k views

Why continuous features are more important than categorical features in decision tree models?

I have both categorical and continuous features in my prediction model and want to select (and rank) most important features. I have converted all categorical variables into dummy variables using one ...
Shahab Kazemi's user avatar
9 votes
4 answers
4k views

Loss Function for Probability Regression

I am trying to predict a probability with a neural network, but having trouble figuring out which loss function is best. Cross entropy was my first thought, but other resources always talk about it in ...
ahbutfore's user avatar
  • 191
8 votes
1 answer
212 views

Examples where simple classifier systems out-perform deep learning

I have been working on a problem where published results using deep learning are substantially worse than results I have obtained on the same task (using the same experimental protocol) using simple ...
Dikran Marsupial's user avatar
8 votes
1 answer
5k views

sklearn: SGDClassifier yields lower accuracy than LogisticRegression

I'm participating in the kaggle Iceberg Classifier Challenge, where the idea is to classify whether an object present in a radar image is an iceberg or a ship. I am currently trying to implement ...
PrestonH's user avatar
  • 201
7 votes
2 answers
19k views

Trying to understand Logistic Regression Implementation

I'm currently using the following code as a starting point to deepen my understanding of regularized logistic regression. As a first pass I'm just trying to do a binary classification on part of the ...
razalfuhl's user avatar
  • 173
7 votes
4 answers
10k views

Bad classification performance of logistic regression on imbalanced data in testing as compared to training

I am trying to fit a logistic regression model to an imbalanced dataset (0.5/99.5) with high dimensionality(about 15k). I used random forest to select top 200 important features. Observations are ...
Alice's user avatar
  • 131
7 votes
2 answers
411 views

Understanding regularization

I'm currently trying to understand regularization for logistic regression. So far, I'm not quite sure whether I really got it. Basically, the problem is that when we add an additional features to a ...
Golo Roden's user avatar
  • 1,323
7 votes
1 answer
118 views

Tensorflow - logistic regression -oneHot Encoder - Transformed array of different size for both train and test

...
user80034's user avatar
7 votes
1 answer
4k views

Why does logistic regression in Spark and R return different models for the same data?

I've compared the logistic regression models on R (glm) and on Spark (LogisticRegressionWithLBFGS) on a dataset of 390 obs. of ...
SparkUser's user avatar
  • 113
7 votes
2 answers
774 views

Coursera ML - Does the choice of optimization algorithm affect the accuracy of multiclass logistic regression?

I recently completed exercise 3 of Andrew Ng's Machine Learning on Coursera using Python. When initially completing parts 1.4 to 1.4.1 of the exercise, I ran into difficulties ensuring that my ...
AKKA's user avatar
  • 123
7 votes
2 answers
7k views

Regression model to predict probability of rare event

I have a dataset with around 900.000 records, around 1000 of which are marked as positive (the studied event occurred). The probability of the event occurring is always low (i.e. < 0.1), and I ...
MDG's user avatar
  • 71
7 votes
2 answers
655 views

Concatenating embedding and hand-designed features for logistic regression

An interviewer told me that we cannot concatenate an embedding from a neural network (such as a pre-trained image representation) and hand designed features (such as image metadata) for use in a ...
plutothepup's user avatar
7 votes
3 answers
277 views

Understanding Classifier performance on text data

I am working on a multi-label text classification problem(Total target labels 90). The data distribution has a long tail and class imbalance and around 1900k records. Currently, I am working on a ...
joel's user avatar
  • 129
6 votes
2 answers
3k views

Logistic regression on biased data

I am currently working on a dataset to predict customer attrition based on past data and transactions of the customers. There are 2,40,000 customers in total out of which around 1,77,000 customers ...
Anonymint's user avatar
  • 155
6 votes
3 answers
14k views

When to use Random Forest

I understand Random Forest models can be used both for classification and regression situations. Is there a more specific criteria to determine where a random forest model would perform better than ...
Pedro Alves's user avatar
6 votes
1 answer
5k views

How does binary cross entropy work?

Let's say I'm trying to classify some data with logistic regression. Before passing the summed data to the logistic function (normalized in range $[0,1]$), weights must be optimized for desirable ...
ShellRox's user avatar
  • 409
6 votes
2 answers
3k views

Voting combined results from different classifiers gave bad accuracy

I used following classifiers along with their accuracies: Random forest - 85 % SVM - 78 % Adaboost - 82% Logistic regression - 80% When I used voting from above classifiers for final classification, ...
mach's user avatar
  • 367
6 votes
1 answer
262 views

Logistic Regression Modeling & Interpretation [closed]

I'm building a logistic regression model to predict the credit risk of lending company customers. I'm using dataset from kaggle : https://www.kaggle.com/datasets/ranadeep/credit-risk-dataset/code ...
Jovian Aditya's user avatar
6 votes
1 answer
13k views

Dealing with NaN (missing) values for Logistic Regression- Best practices?

I am working with a data-set of patient information and trying to calculate the Propensity Score from the data using MATLAB. After removing features with many missing values, I am still left with ...
stats_nerd's user avatar
6 votes
1 answer
11k views

How to determine threshold in Sigmoid function

Context: I picked up data-set from here and tried to run Logistic Regression on it. Since I am not very much aware of MATLAB, I converted "Strings" to "Numbers" with my own using "NUMBERS" software. ...
Gaurav Gupta's user avatar
6 votes
1 answer
2k views

Should I standardize first or generate polynomials first?

Recently I am dealing a classification problem with some algorithms, say logistic regression. When I preprocess my data, I standardize all my features and then generate polynomial features based on ...
Kulbear's user avatar
  • 199
6 votes
1 answer
3k views

Modeling uncertainty from Logistic Regression

Logistic regression is a part in a simulation pipeline that I use for some scenario analysis. The dataset that this is based on is not small but relatively noisy, and only one explanatory variable/...
Jan van der Vegt's user avatar
5 votes
2 answers
19k views

Learning rate in logistic regression with sklearn

In sklearn, for logistic regression, you can define the penalty, the regularization rate and other variables. Is there a way to set the learning rate?
user's user avatar
  • 2,003
5 votes
1 answer
9k views

Best or recommended R package for logit and probit regression

Could somebody please recommend a good R package for doing logit and probit regression? I have tried to find an answer by searching on Google but all the links I find go into lengthy explanations ...
Jerome Smith's user avatar
5 votes
1 answer
717 views

Risk prediction vs classification model

I am working on a binary classification model. Currently, when I use scikit logistic regression, it outputs binary values like 0s and 1s. However, I understand, from online reading, that it outputs ...
The Great's user avatar
  • 2,575
5 votes
2 answers
1k views

Logistic regression with high cardinality categorical variable

I have a logistic regression model where I care about predictive power solely over comprehensibility. I'm interested in predicting win rates in a video game. There are 133 characters. Each team picks ...
Lee Sin's user avatar
  • 153
5 votes
2 answers
19k views

Logistic regression does cannot converge without poor model performance

I have a multi-class classification logistic regression model. Using a very basic sklearn pipeline I am taking in cleansed text descriptions of an object and classifying said object into a category. <...
jasper's user avatar
  • 73
5 votes
3 answers
5k views

Stochastic gradient descent in logistic regression

I am very new to machine learning and in my first project have stumbled across a lot of issues which I really want to get through. I'm using logistic regression with R's ...
aasthetic's user avatar
5 votes
3 answers
484 views

Which machine (or deep) learning methods could suit my text classification problem?

I am long-time engineer with almost zero machine learning experience, who is trying to determine a good starting point to solve my problem (hopefully using machine learning). The problem (I'll keep ...
ConorD55's user avatar

1
2 3 4 5
15