Questions tagged [logistic-regression]

Refers generally to statistical procedures that utilize the logistic function, most commonly various forms of logistic regression

Filter by
Sorted by
Tagged with
1
vote
1answer
109 views

Relationship between log-odds and weighted sums in Logistic Regression

I've read several articles/tutorials on Logistic Regression and I've come across this idea of log-odds being equal to the weighted sum of features. i.e. if $p$ is the probability of a sample ...
0
votes
1answer
204 views

Is there a machine learning model suited well for longitudinal data?

I have a fairly large (>100K rows) dataset with multiple (daily) measurements per individual, for a few thousand individuals. The number of measurements per individual vary, and there are many null ...
1
vote
1answer
38 views

Handling Numerical Categorical Column in ML models in Python

When I was exploring the titanic dataset to estimate the probability of a person of surviving using the Logistic Model, I realized there are two ways of handling numerical categorical variables : Use ...
0
votes
1answer
30 views

should we include or exclude a variable in a logistic regression based on the description below? [duplicate]

should we include or exclude a variable in a logit regr. model which will only obtain values if a certain event takes place otherwise will show N/A? this variable tells whether or not a product will ...
7
votes
3answers
188 views

Understanding Classifier performance on text data

I am working on a multi-label text classification problem(Total target labels 90). The data distribution has a long tail and class imbalance and around 1900k records. Currently, I am working on a ...
0
votes
1answer
37 views

Topic Similarity Measure | Multi-class Text Classification Model

I am trying to build a multi class text-classifier that classifies whether the tweet belongs to one of the categories ( Advise or Science or others ) let the input be any tweet like this , Input : <...
1
vote
0answers
15 views

Gradient calculation for proportional odds (logistic) model

I am trying to calculate a gradient for a proportional odds model. http://fa.bianp.net/blog/2013/logistic-ordinal-regression/ What steps are required to take the derivative with respect to w? $$\...
3
votes
2answers
125 views

On simple 1D dataset, LogisticRegressionCV selects terrible hyperparameters, resulting scores are nonsensical

I am trying to use LogisticRegressionCV to fit a logistic regression model to a simple 1D dataset. Very oddly, when given a choice, it seems to select a tiny C value, which forces my model to select a ...
1
vote
2answers
44 views

XGBoost Objective is Changed

I am trying to use XGBoost in python for logistic regression. I am calling it as follows ...
1
vote
0answers
41 views

Linear and logistic regression output with same neural network

it seems like this should be a very common task, but I have not found anything useful in my research: How can I do linear and logistic regression with the same neural network? By example, what I mean ...
1
vote
0answers
25 views

What to do when feature engineering and parameter tuning don't add to the base model performance

I've been working on using LogisticRegression from scikit to try the Titanic Kaggle comp. I've found something interesting, and that is that no amount of feature engineering and paramater tuning is ...
1
vote
1answer
30 views

Two questions on hyper-parameter tuning

Question 1: In the example of logistic regression, I often see the regularization constant and penalty methods being tuned by a grid search. However, it seems like there are a lot more options for ...
1
vote
2answers
68 views

I intend to do classification modelling, but my target variable has only one value

Currently I have a dataset and I am trying to predict whether someone will default on their bank loan. The dataset is quite tricky. It covers those who have defaulted in the past, but is also ...
1
vote
1answer
71 views

improving accuracy of logistic model

I am trying to reproduce results from one paper, where authors minimized the following loss function \begin{align} \min_{w \in R^d} \frac{1}{n} \sum_{i \in [n]} log(1 + exp(-y_ix_i^Tw))+\frac{\lambda}{...
1
vote
1answer
637 views

Recursive Feature Elimination (RFE) with Logistic Regression and little correlation between the features and the target (SKLearn)

I have little experience in the field of Machine Learning, so I apologise in advance for my ignorance and lack of intuitions. I'm trying to build a classifying algorithm to identify if a subject is ...
2
votes
1answer
103 views

Kohen Kappa Coefficient of Naive Bayes with 62% overall accuracy is better than Logistic Regression with 98% accuracy?

I have been trying to evaluate my models used on fire systems dataset with a huge imbalance in the dataset. Most models failed to predict any true positives correctly however naive Bayes managed to do ...
1
vote
2answers
47 views

Why ROC value area under curve of two models is different whereas accuracy, precision, recall, f1-score and confusion matrix is same

I am applying logistic regression and support vector machines on the extacly same dataset with 70% data for training and 30% for test. Both perform exactly the same have the same precision, recall, f1-...
0
votes
1answer
193 views

How to compute Hessian matrix for log-likelihood function for Logistic Regression

I am currently studying the Elements of Statistical Learning book. The following equation is in page 120. It calculates the Hessian matrix for the log-likelihood function as follows \begin{equation} ...
1
vote
0answers
20 views

scaling credit risk scorecard

I need to build a credit risk scorecard using logistic and linear regression. The variables using to predict are all dummies, where each dummy is a bin of some variable. Let's say the variable age, I ...
4
votes
2answers
242 views

What is the best way to find minima in Logistic regression?

In the Andrew NG's tutorial on Machine Learning, he takes the first derivative of the error function then takes small steps in direction of the derivative to find minima. (Gradient Descent basically) ...
0
votes
1answer
38 views

Use all available data to build Logistic Regression model [duplicate]

Using K-Fold, I chose to use Logistic Regression for a project of mine. I made it learn on my X_train (80% of data), and tested it on my X_test, with good results. My question is : now that I need ...
-3
votes
1answer
47 views

LogisticRegression NotImplementedError on fit function

im a newbie in data science or machine learning. i try to implement code from here, but i got error when try to call fit function here is the code: ...
0
votes
1answer
23 views

When can you reorder log operations?

For example, you can reorder a softmax + nl (negative likelihood) to log_softmax + nll (negative log-likelihood) Essentially changing log(softmax(x)) to softmax(log(x)) However, what are the ...
3
votes
2answers
171 views

Logistic Regression vs SVM

Following Andrew Ng's machine learning course, he explains how we can modify logistic regression to obtain SVM algorithm. First he replaces (sort of approximating) cross entropy loss with hinge loss ...
1
vote
0answers
17 views

KS-Score Methods

I came across 2 methods to calculate KS-Score and select best probability threshold. Decible Method TPR - FPR Is there any specific scenario on which it depends which method to select. or we cna ...
3
votes
1answer
37 views

I have 3 graphs of a binary Logistic Regression that I want to understand better what is happening and learn of a strategy to make the model better

My problem is the following: I have a binary Logistic Regression model with a very imbalanced dataset that outputs the percentage of the prediction. As can be seen in the images, as the threshold is ...
1
vote
1answer
37 views

How does inverse of logistic function produces “linear relationship”, (so we can use least-squares)

I am reading about time-series analysis in "A First Course on Time Series Analysis". The book reviews the logistic function ($f_{log}(t)$). Part 1.6 (PDF page 16; book page 8; screenshot below) ...
0
votes
0answers
48 views

Reclassification machine learning measuring fatigue

I am very new to data science/ML field, so I apologize in advance if my logic may seem confusing. I would like to know if it's possible to perform reclassification on a dataset? So the process I ...
1
vote
1answer
75 views

Error calculation in Logistic Regression

Suppose we have a linear regression model that predicts an item’s price. If the item’s prediction is 8 USD and the actual value is 10 USD, then it is clear that the error is pow(10-8, 2)=4. But how is ...
2
votes
1answer
32 views

Explanation for Why Logistic Regression can be so Accurate in Sentiment Classification?

My question is about how a logistic regression model performs so accurately. In some exploratory experimentation, I compared a logistic regression model against a long short term memory recurrent ...
4
votes
3answers
109 views

Is my model over fitting or not?

I have 50000 observations with 70% positive and 30% negative target variable. I'm getting accuracy of around 96-99% which seems unreal of course and I'm worried that my model is over-fitting which I ...
1
vote
1answer
46 views

Is this scheme correct for logistic regression with stochastic gradient descent

I am implementing logistic regression with stochastic gradient descent, but it is not working as expected. I've tried many epochs and different learning rates $\alpha$ but the probability of belonging ...
2
votes
0answers
15 views

Connection between prob output LogisticReg/SVM and ROC

I have the following ROC generated using LPOCV and Logistic regression or SVM (l2 norm). Now, let's say I have a test set containing 10 patients and I get that the probabilities of those patients to ...
0
votes
2answers
175 views

Cannot make a single prediction: Is this behavior normal?

I am running a hate speech classifier published by Davidson et al. The principle is simple, the classifier takes as an input an annotated ('hateful', 'offensive', 'neither') dataset of tweets. It ...
0
votes
0answers
36 views

How can I test my trained model on a completely new dataset? [duplicate]

Preface I have an annotated text dataset on hate speech. Simply put, the dataset consists of a column called text which includes a piece of text, and a column ...
0
votes
1answer
139 views

Logistic Regression outperforms MLPClassifier

I am new in ML and I am trying to train classifier. I have a tiny dataset, just 90 examples, I divided it 70/30 train/test set and started to train. As I know MLP must outperform Logistic Regression, ...
1
vote
2answers
42 views

If a categorical feature only occurs a few times in a data set, should I drop it?

I have a data set of mostly categorical variables. When I one-hot encoded them some of the features occur less than 3% of the time. For instance the Tech-support feature only occurs 928 times in a ...
0
votes
1answer
55 views

How to select one record from multiple record for a subject during analysis?

I have a dataset where I am working on a binary classification. I have two classes of subjects. One is Outpatients and Other is Inpatients. (66:33 is the class proportion) My objective is to identify ...
1
vote
1answer
30 views

Require Guidance - Identifying risk factors

I have a dataset of patients who have been inpatients (admitted to hospital) and not admitted (but visited as outpatients). Class proportion is 66:34. I have collected a list of features for all ...
2
votes
1answer
229 views

Which metric to use for classify a binary logistic regression

I still novice in machine learning. My problem is as follow : I use two binary logistic regression models for making comparaison. Additionally to LogLoss and accuracy, I would like to use for metric :...
1
vote
0answers
12 views

What could be the impacts on the intercept and slops of each predictor in Logistic Regression by rare event cases?

I am wondering how data set with rare event cases would impact on my Logistic Regression model in terms of the intercept and slopes of each predictor?
3
votes
1answer
165 views

Error while plotting Logistic Regression Classification

I was trying to plot by using the following code ...
2
votes
1answer
35 views

Logistic Regression Maximum Likelihood

Is it true that we assume our P(y|x;theta) to follow Bernoulli's distribution given y has binary output in Logistic Regression? Is there any specific reason why we consider Bernoulli's distribution? ...
2
votes
0answers
846 views

Some of the p-values are NaN - logistic regression

I am trying to do logisitc regression, but have this issue - some of the p values are NaN ...
3
votes
1answer
186 views

Risk prediction vs classification model

I am working on a binary classification model. Currently, when I use scikit logistic regression, it outputs binary values like 0s and 1s. However, I understand, from online reading, that it outputs ...
1
vote
1answer
44 views

Optimum values of all predictors in logistic regression [closed]

I am trying to figure out in logistic regression, with the help of coefficients of $x$ variables, we could figure out that on unit change in any $x$ variable what will be the change in probability of ...
9
votes
2answers
640 views

Why continuous features are more important than categorical features in decision tree models?

I have both categorical and continuous features in my prediction model and want to select (and rank) most important features. I have converted all categorical variables into dummy variables using one ...
2
votes
1answer
73 views

batched CrossEntropyLoss in pytorch

I'm wondering how to implement this with pytorch built-ins. I've got a 3 dimensional input of uints called policy. Most of the entries are zero, and if I were to L1 normalize this I would have a (...
2
votes
0answers
69 views

Probability of the recipient to open the email

I am trying to build a model using logistic regression, where my dependent variable is y=1 if the mail was opened, y=0 if it was not. I have data approximately 10 records (10 rows) for every ...
1
vote
1answer
72 views

Why results of statsmodel logreg is different from scikit-learn logreg?

I am trying to do a binary classification. I have only 6 input variables and one output variables. Label 1 is 1554 records and Label 0 is 3558 records. As you can see below, the metrics that I get ...

1 2
3
4 5
11