10
$\begingroup$

I'm working on a project and need resources to get me up to speed.

The dataset is around 35000 observations on 30 or so variables. About half the variables are categorical with some having many different possible values, i.e. if you split the categorical variables into dummy variables you would have a lot more than 30 variables. But still probably on the order of a couple of hundred max. (n>p).

The response we want to predict is ordinal with 5 levels (1,2,3,4,5). Predictors are a mix of continuous and categorical, about half of each. These are my thoughts/plans so far: 1. Treat the response as continuous and run vanilla linear regression. 2. Run nominal and ordinal logistic and probit regression 3. Use MARS and/or another flavor of non-linear regression

I'm familiar with linear regression. MARS is well enough described by Hastie and Tibshirani. But I'm at a loss when it comes to ordinal logit/probit, especially with so many variables and a big data set.

The r package glmnetcr seems to be my best bet so far, but the documentation hardly suffices to get me where I need to be.

Where can I go to learn more?

$\endgroup$
  • $\begingroup$ I suggest you add the R tag as well. $\endgroup$ – Christopher Louden Jun 19 '14 at 13:28
  • 1
    $\begingroup$ Given that this is a question about the statistical model, you may want to go to CrossValidated website, but keep in mind that it is a terrible practice to cross-post the questions: you would either want to formulate it to highlight the methodological issues you are facing, or migrate the whole question. $\endgroup$ – StasK Jun 27 '14 at 13:23
  • $\begingroup$ Without really explaining why, ISL notes (on p 137) that discriminant analysis (like LDA, QDA) is more often used than multiple class extensions of logistic regression. Packages like penalizedLDA may therefore be worth examining. $\endgroup$ – MattBagg Jun 27 '14 at 21:03
6
$\begingroup$

I suggest this tutorial on ordered logit: http://www.ats.ucla.edu/stat/r/dae/ologit.htm

It showcases the use of polr in the MASS package, and also explains the assumptions and how to interpret the results.

$\endgroup$
6
$\begingroup$

One fairly powerful R package for regression with an ordinal categorical response is VGAM, on the CRAN. The vignette contains some examples of ordinal regression, but admittedly I have never tried it on such a large dataset, so I cannot estimate how long it may take. You may find some additional material about VGAM on the author's page. Alternatively you could take a look at Laura Thompson's companion to Agresti's book "Categorical Data Analysis". Chapter 7 of Thompson's book describes cumulative logit models, which are frequently used with ordinal responses.

Hope this helps!

$\endgroup$
3
$\begingroup$

If you are totally unfamiliar with ordinal regression, I would try to read the Tabachnick / Fidell (http://www.pearsonhighered.com/educator/product/Using-Multivariate-Statistics-6E/0205849571.page) chapter on the topic first - while not written for R, the book is very good at conveying the general logic and the "do's" and "do nots".

As a question: What are your response catgeories exactly? If they are some sort of scale, like "good - bad" it would be ok to use a linear regression (market research does it all the time...), but if the items are more disjunct, an ordinal regression might be better. I dimly remember that some books about structural equatiotion modelling mentioned that linear regression was superior for good scales than probit - bit I cannot recall the book at the moment, sorry!

The most serious problem might be the number of dummy variables - a couple of hundred dummy variables will make the analysis slow, hard to interpret and probably unstable - are there enough cases for each dummy / dummy-combination?

$\endgroup$
3
$\begingroup$

One standard reference written from social science perspective is J Scott Long's Limited Dependent Variables book. It goes much deeper than say Tabachnik suggested in another answer: Tabachnik is a cookbook at best, with little to no explanations of the "why", and it seems like you would benefit from figuring this out in more detail that can be found in Long's book. Ordinal regression should be covered in most introductory econometrics courses (Wooldridge's Cross-Section and Panel Data is a great graduate-level book), as well as quantitative social science courses (sociology, psychology), although I would imagine that the latter will loop back to Long's book.

Given that your number of variables is wa-a-ay lower than the sample size, the R package you should be looking is probably ordinal rather than glmnetcr. Another answer mentioned that you can find this functionality in a more mainstream MASS package.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.