5
$\begingroup$

Logistic regression is a part in a simulation pipeline that I use for some scenario analysis. The dataset that this is based on is not small but relatively noisy, and only one explanatory variable/feature. Of course I can say something about this uncertainty using frequentist or Bayesian methods but I would like to use this in the sequential simulation step as well, to get a fairer final estimate. What I'm planning on doing should work but is somewhat computationally expensive and I would like to know if someone has a better idea.

  • Approximate distribution of the coefficients of logistic regression with MCMC

Now simulate this n times:

  • Sample one of the two coefficients from this distribution
  • Fix this coefficient and use MCMC again to get approximated conditional distribution over second coefficient (could I get this in a better way? This seems highly correlated with the first coefficient)
  • Do the rest of the simulation run using this sampled model

I feel like there should be a better way but I'm uncertain.

$\endgroup$
4
  • $\begingroup$ This is a standard approach for robustly determining uncertainty, but if the two coefficients are highly correlated, that points to problems with the model. $\endgroup$
    – Paul
    Oct 25 '16 at 15:25
  • $\begingroup$ I don't know if they are highly correlated but that was my assumption, for example in a regular linear model if the intercept is samples very low I assume the slope would on average get higher to make up for that, right? $\endgroup$ Oct 25 '16 at 15:27
  • 2
    $\begingroup$ I'm not sure I understand the rationale for fixing one of the coefficients and then running MCMC again to get the other. Can't you just get the joint distribution of the two coefficients from running MCMC once, and then sample the pairs of coefficients from that? $\endgroup$ Oct 27 '16 at 8:31
  • $\begingroup$ I think you are right, I'm very new to MCMC, I failed to realize every step of the walk is a combination of the parameters, so you just sample one tuple of parameters from the trace. If you turn it into an answer I will accept it :) $\endgroup$ Oct 27 '16 at 8:54
1
$\begingroup$

I see many facets of your question and in what follows will present my top 2 :-)

Facet 1 -- assessing the uncertainty in estimated coefficients

In logistic regression, assessing the uncertainty in the estimated coefficients is virtually the same as for least-squares regression .

In both logistic regression and least-squares regression, the regression coefficient table will include a column for the regression coefficients followed by a column of standard errors, then by a column of test statistics, and finally a column of p-values. The table below shows the example coefficient table output for the some regression problem (where the probability of buying some magazine for kids is estimated). example problem coefficients table

Note that the test statistics are labeled “z value” and the p-values are labeled “P(>|t|)” in the table above. The standard errors can be used to construct confidence intervals for the regression coefficients. Roughly speaking, going plus or minus 2 times the standard error from the regression coefficient gives approximately a 95% confidence interval for the coefficient.

For example, for Residence Length, the regression coefficient is 0.024680. The next column gives the standard error of the regression coefficient which is 0.013800. Thus an approximate 95% confidence interval for the Residence Length regression coefficient is:

$$ 0.024680 \pm 2 \times 0.013800 = 0.024680 \pm 0.0276\ $$

This means that the regression coefficient for Residence Length could be anywhere from -0.00292 to 0.05228 (with 95% confidence).

As is well known, we often use the odds-ratio, which is the exponential of the regression coefficient (i.e., $\exp(\beta)$ ), to help to interpret the meaning of the regression coefficient. The odds-ratio for the Residence Length coefficient, as shown in the coefficient table, is 1.0250. This means that there is a 2.5% increase in the odds of buying the magazine associated with each additional year of residence.

We can also compute the odds-ratios corresponding to the ends of the confidence interval. These odds-ratios will give us an equivalent confidence intervals for the odds. So continuing the example using Residence Length, the odds ratios corresponding to the ends of the confidence interval are

$$ \exp(-0.00292)=0.99708 $$ and $$ \exp(0.05228)=1.05367. $$ Thus, the interval [0.99708,1.05367] is an approximate 95% confidence interval for the odds ratio. This means that there could be anywhere from a 0.292% decrease to a 5.367% increase in the odds of buying the magazine associated with each additional year of residence.

Facet 2 -- modeling data with measurements uncertainty

Here again many options are known. Take a look at this to get started: https://pdfs.semanticscholar.org/0a7a/a5e3d407b24e2f5c24287bfdda20e573bd05.pdf

I will just cite the source:

in some datasets, the outcome of interest is measured with imperfect sensitivity and specificity. It is well known that the misclassification induced by such an imperfect diagnostic test will lead to biased estimates of the odds ratios and their variances.

In this paper, the authors show that when the sensitivity and specificity of a diagnostic test are known, it is straightforward to incorporate this information into the fitting of logistic regression models.

An EM algorithm that produces unbiased estimates of the odds ratios and their variances is described. The resulting odds ratio estimates tend to be farther from the null but have greater variance than estimates found by ignoring the imperfections of the test. The method can be extended to the situation where the sensitivity and specificity differ for different study subjects, i.e., nondifferential misclassification. The method is useful even when the sensitivity and specificity are not known, as a way to see the degree to which various assumptions about sensitivity and specificity affect one's estimates.

Hope this helps!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.