sample data

I am trying to run a logistic regression on a data set where my dependent variable is a proportion of a binary variable, rather than the binary variable itself. I have seen a ton of documentation that says this is possible, but I am having trouble finding an example of how to actually do it. I am open to using scikit learn, statsmodels, or any other library that will do it.

I have added a photo showing a simplified version of my data.

successes here is just a count of a binary (1/0) outcome. instead of having the individual observations, I only have them rolled up, but my understanding is that it is still a logistic regression problem. I want to predict the dependent variable 'proportion' based on the features. I understand this conceptually, but am just trying to find an example of this in python. all of the examples I have seen assume a binary dependent variable.

your help is appreciated!

  • $\begingroup$ I'm voting to close this question as off-topic because this is just binomial regression - see stats.stackexchange.com/questions/208213/… Note the inputs there are not just the proportion but the two columns of attempts and successes, because the more attempts there are, the lower the variance on the estimate of the proportion. $\endgroup$ – Spacedman Dec 5 '16 at 23:12
  • $\begingroup$ I think this could be on topic here or on Cross Validated. It's in the overlap. $\endgroup$ – Sean Owen Dec 6 '16 at 4:33

As the link by @Spacedman shows, binomial regression works well. If you don't have the attempts and successes but, instead, have just the proportion, then you'll want to use beta-regression. After a little digging, it doesn't seem like this is available in Python. Here's a blog post demoing it in R.

| improve this answer | |
  • $\begingroup$ I do have the attempts and successes, as shown in my data pic. I'd really like to do the logistic regression, not the binomial regression. And I def can't do it in R. I have talked to two data scientists who were convinced this was possible in Python, but they didn't know where to find a specific example. $\endgroup$ – tarastar42 Dec 6 '16 at 1:00
  • $\begingroup$ Then what @Spacedman linked you to is the correct path. Alternatively, you could predict whether or not a single observation was a success or not, which is where you could use Logistic Regression. To do that, you'd have to unfold the data from group 1, ..., to Observation 1 in Group 1, Observation 2 in Group 1, etc. $\endgroup$ – franciscojavierarceo Dec 6 '16 at 1:53
  • $\begingroup$ @tarastar42 perhaps you should have talked to two statisticians instead of data scientists. There's no point doing this as a logistic regression unless you have individual level features (ie they were possibly different for each trial). Its not a logistic regression, its a binomial regression, in the same way that modelling count data (eg number of cars past the bus stop every minute) is a Poisson regression. These are all just types of Generalised Linear Model. $\endgroup$ – Spacedman Dec 6 '16 at 8:20
  • $\begingroup$ @Spacedman hits the nail on the head, you really shouldn't model it as a Logistic regression because it just increases the computational cost and understates the uncertainty of your predictions (since it'll artificially increase your sample size). You certainly can do it but it's like using a wrench to hammer in a nail; it works but isn't the best tool for the job. $\endgroup$ – franciscojavierarceo Dec 6 '16 at 8:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.