0
$\begingroup$

I created bag of words (in which number of columns were around 15000 i.e greater than 1) and corresponding output data (0 or 1). After that I used LogisticRegression() (I didn't passed any parameters) from sklearn and used it for training and testing. It didn't gave any error or warning and worked completely fine.

However isn't logistic regression only for those dataset which have maximum 1 independent variables? How sigmoid function would work when number of independent variables are greater than 1? Sigmoid function is used in LogisticRegression and wherever I read it is described in 2 dimension space i.e with only one independent variable.

Improve this question
$\endgroup$
5
  • $\begingroup$ The “independent” variable is $x$. Only the $y$ needs to be binary in logistic regression (and there’s even a fairly natural extension to $y$ with three or more categories). $\endgroup$
    – Dave
    Nov 8, 2021 at 3:54
  • $\begingroup$ @Dave Apology from my side. I have done correction in my post. $\endgroup$
    – TheReal__Mike
    Nov 8, 2021 at 5:32
  • $\begingroup$ It still looks like you’ve mixed up $x$ and $y$. In particular, why shouldn’t multiple predictor variables be allowed? $\endgroup$
    – Dave
    Nov 8, 2021 at 11:03
  • $\begingroup$ @Dave Because I can't visualize in my mind how sigmoid function of logistic regression work or look when multiple predictor variables are present. $\endgroup$
    – TheReal__Mike
    Nov 8, 2021 at 14:55
  • $\begingroup$ Look at the answer I posted. The predicted probability is a function of the $\hat z$, which is just a number. That $\hat p = \dfrac{1}{1 + exp(-\hat z)}$ exists in the plane and looks like the usual sigmoid function. $\endgroup$
    – Dave
    Nov 8, 2021 at 14:57

1 Answer 1

Reset to default
0
$\begingroup$

Multiple predictor (independent) variables are allowed.

$$ \hat p =\dfrac{1}{1+\exp(-\hat z)}\\ \hat z = X\hat\beta = \hat\beta_0+\hat\beta_1x_1+\hat\beta_2x_2++\hat\beta_3x_3+\cdots $$

You can simulate this in R. Let’s use five predictor variables.

set.seed(2021)
N <- 100
p <- 5
X <- matrix(runif(N * p), N, p)
z <- X %*% c(1, 2, -3, 4, -5)
p <- 1/(1+exp(-z))
y <- rbinom(N, 1, p)
L <- glm(y ~ X, family = binomial)

EDIT

Let's look at the sigmoid function in the plane for my example.

z_hat <- predict(L)
p_hat <- 1/(1 + exp(-z_hat))
plot(z_hat, p_hat)

It's just a sigmoid curve in the plane, even though there are multiple predictor variables.

enter image description here

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ I am able to understand what you are trying to say. I have edited your answer and uploaded the picture after running your code. $\endgroup$
    – TheReal__Mike
    Nov 9, 2021 at 7:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.