7
$\begingroup$

I've compared the logistic regression models on R (glm) and on Spark (LogisticRegressionWithLBFGS) on a dataset of 390 obs. of 14 variables.

The results are completely different in the intercept and the weights. How to explain this?

Here is the results of Spark (LogisticRegressionWithLBFGS) :

model.intercept  : 
 1.119830027739959
model.weights :
 GEST    0.30798496002530473
 DILATE  0.28121771009716895
 EFFACE  0.01780105068588628
 CONSIS -0.22782058111362183
 CONTR  -0.8094592237248102
 MEMBRAN-1.788173534959893
 AGE    -0.05285751197750732
 STRAT  -1.6650305527536942
 GRAVID  0.38324952943210994
 PARIT  -0.9463956993328745
 DIAB   0.18151162744507293
 TRANSF -0.7413500749909346
 GEMEL  1.5953124037323745

Here is the result of R :

             Estimate Std. Error z value Pr(>|z|)   
(Intercept)  3.0682091  3.3944407   0.904 0.366052    
GEST         0.0086545  0.1494487   0.058 0.953821    
DILATE       0.4898586  0.2049361   2.390 0.016835 *  
EFFACE       0.0131834  0.0059331   2.222 0.026283 *  
CONSIS       0.1598426  0.2332670   0.685 0.493196    
CONTR        0.0008504  0.5788959   0.001 0.998828    
MEMBRAN     -1.5497870  0.4215416  -3.676 0.000236 ***   
AGE         -0.0420145  0.0326184  -1.288 0.197725    
STRAT       -0.3781365  0.5860476  -0.645 0.518777    
GRAVID       0.1866430  0.1522925   1.226 0.220366    
PARIT       -0.6493312  0.2357530  -2.754 0.005882 **  
DIAB         0.0335458  0.2163165   0.155 0.876760    
TRANSF      -0.6239330  0.3396592  -1.837 0.066219 .  
GEMEL        2.2767331  1.0995245   2.071 0.038391 *  
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ I wouldn't call Limited-memory BFGS a "simple logistic regression". Except, I guess, in the sense that modern libraries make complex techniques very accessible :) $\endgroup$
    – nealmcb
    Commented May 7, 2015 at 17:08
  • $\begingroup$ You are right. Do you know a way to implement Limited-memory BFGS with R ? $\endgroup$
    – SparkUser
    Commented May 11, 2015 at 8:15
  • $\begingroup$ Also, suspect if it has something to do with the loss function csie.ntu.edu.tw/~cjlin/papers/spark-liblinear/… (Sec - IIIA) @SparkUser - is it possible to run LogisticRegressionWithLBFGS without the defaults and then compare the coefficients with the default R - glm. $\endgroup$
    – user45409
    Commented Apr 23, 2016 at 17:47

1 Answer 1

Reset to default
6
$\begingroup$

A quick glance at the docs for LogisticRegressionWithLBFGS indicates that it uses feature scaling and L2-Regularization by default. I suspect that R's glm is returning a maximum likelihood estimate of the model while Spark's LogisticRegressionWithLBFGS is returning a regularized model estimate. Note how the estimated model weights of the Spark model are all smaller in magnitude than those in the R model.

I'm not sure whether or not glm in R is implementing feature scaling, but this would also contribute to different model values.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I'm not sure it is possible to use feature scaling in R .. I can not find it.. Do you know how to perform logistic regression with LBFGS using R to compare the 2 models? $\endgroup$
    – SparkUser
    Commented May 11, 2015 at 8:13
  • $\begingroup$ Here, check the documentation of function glmnet from glmnet package and look and parameter standardize http://cran.r-project.org/web/packages/glmnet/glmnet.pdf - there also is possibleto use regularization $\endgroup$
    – Marcin
    Commented Jun 1, 2015 at 10:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.