7
$\begingroup$

I've compared the logistic regression models on R (glm) and on Spark (LogisticRegressionWithLBFGS) on a dataset of 390 obs. of 14 variables.

The results are completely different in the intercept and the weights. How to explain this?

Here is the results of Spark (LogisticRegressionWithLBFGS) :

model.intercept  : 
 1.119830027739959
model.weights :
 GEST    0.30798496002530473
 DILATE  0.28121771009716895
 EFFACE  0.01780105068588628
 CONSIS -0.22782058111362183
 CONTR  -0.8094592237248102
 MEMBRAN-1.788173534959893
 AGE    -0.05285751197750732
 STRAT  -1.6650305527536942
 GRAVID  0.38324952943210994
 PARIT  -0.9463956993328745
 DIAB   0.18151162744507293
 TRANSF -0.7413500749909346
 GEMEL  1.5953124037323745

Here is the result of R :

             Estimate Std. Error z value Pr(>|z|)   
(Intercept)  3.0682091  3.3944407   0.904 0.366052    
GEST         0.0086545  0.1494487   0.058 0.953821    
DILATE       0.4898586  0.2049361   2.390 0.016835 *  
EFFACE       0.0131834  0.0059331   2.222 0.026283 *  
CONSIS       0.1598426  0.2332670   0.685 0.493196    
CONTR        0.0008504  0.5788959   0.001 0.998828    
MEMBRAN     -1.5497870  0.4215416  -3.676 0.000236 ***   
AGE         -0.0420145  0.0326184  -1.288 0.197725    
STRAT       -0.3781365  0.5860476  -0.645 0.518777    
GRAVID       0.1866430  0.1522925   1.226 0.220366    
PARIT       -0.6493312  0.2357530  -2.754 0.005882 **  
DIAB         0.0335458  0.2163165   0.155 0.876760    
TRANSF      -0.6239330  0.3396592  -1.837 0.066219 .  
GEMEL        2.2767331  1.0995245   2.071 0.038391 *  
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
$\endgroup$
  • 1
    $\begingroup$ I wouldn't call Limited-memory BFGS a "simple logistic regression". Except, I guess, in the sense that modern libraries make complex techniques very accessible :) $\endgroup$ – nealmcb May 7 '15 at 17:08
  • $\begingroup$ You are right. Do you know a way to implement Limited-memory BFGS with R ? $\endgroup$ – SparkUser May 11 '15 at 8:15
  • $\begingroup$ Also, suspect if it has something to do with the loss function csie.ntu.edu.tw/~cjlin/papers/spark-liblinear/… (Sec - IIIA) @SparkUser - is it possible to run LogisticRegressionWithLBFGS without the defaults and then compare the coefficients with the default R - glm. $\endgroup$ – user45409 Apr 23 '16 at 17:47
6
$\begingroup$

A quick glance at the docs for LogisticRegressionWithLBFGS indicates that it uses feature scaling and L2-Regularization by default. I suspect that R's glm is returning a maximum likelihood estimate of the model while Spark's LogisticRegressionWithLBFGS is returning a regularized model estimate. Note how the estimated model weights of the Spark model are all smaller in magnitude than those in the R model.

I'm not sure whether or not glm in R is implementing feature scaling, but this would also contribute to different model values.

$\endgroup$
  • $\begingroup$ I'm not sure it is possible to use feature scaling in R .. I can not find it.. Do you know how to perform logistic regression with LBFGS using R to compare the 2 models? $\endgroup$ – SparkUser May 11 '15 at 8:13
  • $\begingroup$ Here, check the documentation of function glmnet from glmnet package and look and parameter standardize http://cran.r-project.org/web/packages/glmnet/glmnet.pdf - there also is possibleto use regularization $\endgroup$ – Marcin Kosiński Jun 1 '15 at 10:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.