1
$\begingroup$

I decided to apply the logistic regression method to my categorical and quantitative data. So, I followed these steps:

  • Eliminating the bad and inconsistent data.
  • Preparing the target variable (categorical variable).
  • Testing the dependencies between categorical variables and the target variable using the Ki-2 test for selecting the variables that are well linked with the target variable.
  • Testing the correlation between quantitative variables to avoid the choice of two correlated variables at a time.
  • Crossing some variables to improve their significance.

After all this I do not find any significant variables in my logistic regression model knowing that the base is well coherent and well cleaned.

I work with the R language and I used the glm function:

glm (formula, family = familytype (link = linkfunction), data =)

You can see my data base. Note that Achat_client is target variable:

    > M=data.frame(Type_peau,PEAU_CORPS,SENSIBILITE,IMPERFECTIONS,BRILLANCE ,GRAIN_PEAU,RIDES_VISAGE,ALLERGIES,MAINS,PEAU_CORPS,
+              INTERET_ALIM_NATURELLE,INTERET_ORIGINE_GEO,Crois_Prior1_Milieu,Crois_Profil_Prio,Crois_ALL_AGE,
+              INTERET_VACANCES,INTERET_COMPOSITION, PEAU_CORPS,Nbre_gift,w,Achat_client)
> str(M)
'data.frame':   836 obs. of  21 variables:
 $ Type_peau             : Factor w/ 5 levels "","Grasse","Mixte",..: 3 4 5 3 4 3 3 3 2 3 ...
 $ PEAU_CORPS            : Factor w/ 4 levels "","Normale","Sèche",..: 2 3 3 2 2 2 3 2 3 2 ...
 $ SENSIBILITE           : Factor w/ 4 levels "","Aucune","Fréquente",..: 4 4 4 2 4 3 4 2 4 4 ...
 $ IMPERFECTIONS         : Factor w/ 4 levels "","Fréquente",..: 3 4 3 4 3 2 3 4 3 3 ...
 $ BRILLANCE             : Factor w/ 4 levels "","Aucune","Partout",..: 4 2 2 4 4 4 4 4 3 4 ...
 $ GRAIN_PEAU            : Factor w/ 4 levels "","Dilaté","Fin",..: 4 4 4 2 4 2 4 4 2 4 ...
 $ RIDES_VISAGE          : Factor w/ 4 levels "","Aucune","Très visibles",..: 2 2 2 4 4 2 4 2 4 2 ...
 $ ALLERGIES             : Factor w/ 4 levels "","Non","Oui",..: 2 2 2 2 2 2 2 2 2 2 ...
 $ MAINS                 : Factor w/ 4 levels "","Moites","Normales",..: 3 4 4 3 3 3 3 4 4 4 ...
 $ PEAU_CORPS.1          : Factor w/ 4 levels "","Normale","Sèche",..: 2 3 3 2 2 2 3 2 3 2 ...
 $ INTERET_ALIM_NATURELLE: Factor w/ 4 levels "","Beaucoup",..: 2 4 4 4 2 2 2 4 4 2 ...
 $ INTERET_ORIGINE_GEO   : Factor w/ 5 levels "","Beaucoup",..: 2 4 2 5 2 2 2 2 2 2 ...
 $ Crois_Prior1_Milieu   : Factor w/ 14 levels "Per_nature_éclatante",..: 11 13 8 12 8 6 8 9 13 11 ...
 $ Crois_Profil_Prio     : Factor w/ 294 levels "Eclatante / Hydratée_éclatante",..: 141 227 221 74 56 184 13 86 227 68 ...
 $ Crois_ALL_AGE         : Factor w/ 6 levels "jeune_Avec_ALL",..: 2 4 2 4 2 2 4 4 6 2 ...
 $ INTERET_VACANCES      : Factor w/ 6 levels "","À la mer",..: 3 4 2 2 3 2 3 2 3 2 ...
 $ INTERET_COMPOSITION   : Factor w/ 4 levels "","Beaucoup",..: 2 2 2 4 2 2 2 2 4 2 ...
 $ PEAU_CORPS.2          : Factor w/ 4 levels "","Normale","Sèche",..: 2 3 3 2 2 2 3 2 3 2 ...
 $ Nbre_gift             : int  1 4 1 1 2 1 1 1 1 1 ...
 $ w                     : num  0.25 0.25 0.5 0.25 0.5 0 0 0 0 0.75 ...
 $ Achat_client          : num  0 0 0 0 0 0 1 0 0 0 ...

Then I split my data base into tow parts: 70% of data for modeling and 30% for testing then the model:

split = sample.split(M$Achat_client, SplitRatio = 0.70)


final.train = subset(M, split == TRUE)
final.test = subset(M, split == FALSE)

Then for application of logistic regression:

final.log.model <- glm(formula=Achat_client ~ .-1,family=binomial(link="logit"),data = final.train)

The result is:

> summary(final.log.model)

Call:
glm(formula = Achat_client ~ . - 1, family = binomial(link = "logit"), 
    data = final.train)

Deviance Residuals: 
     Min        1Q    Median        3Q       Max  
-1.60306  -0.26636  -0.04545  -0.00003   2.83714  

Coefficients: (5 not defined because of singularities)
                                                  Estimate Std. Error z value Pr(>|z|)    
Type_peauGrasse                                 -7.530e+00  2.811e+00  -2.679 0.007383 ** 
Type_peauMixte                                  -8.278e+00  2.535e+00  -3.265 0.001095 ** 
Type_peauNormale                                -7.902e+00  2.568e+00  -3.077 0.002087 ** 
Type_peauSèche                                  -9.710e+00  2.583e+00  -3.759 0.000171 ***
PEAU_CORPSSèche                                  8.639e-01  5.966e-01   1.448 0.147612    
PEAU_CORPSTrès sèche                             1.660e+00  8.959e-01   1.853 0.063843 .  
SENSIBILITEFréquente                            -1.978e-02  9.583e-01  -0.021 0.983531    
SENSIBILITEOccasionnelle                        -1.123e-01  8.905e-01  -0.126 0.899640    
IMPERFECTIONSOccasionnelle                      -2.825e-01  5.859e-01  -0.482 0.629712    
IMPERFECTIONSRares                              -1.234e+00  9.116e-01  -1.353 0.175956    
BRILLANCEPartout                                 1.720e+00  1.351e+00   1.273 0.202993    
BRILLANCEZone T                                  2.942e-01  7.643e-01   0.385 0.700299    
GRAIN_PEAUFin                                    1.901e+00  9.640e-01   1.972 0.048609 *  
GRAIN_PEAUMoyen                                  6.999e-01  6.976e-01   1.003 0.315696    
RIDES_VISAGETrès visibles                        2.350e+00  1.101e+00   2.134 0.032843 *  
RIDES_VISAGEVisibles                            -1.224e-02  5.696e-01  -0.021 0.982857    
ALLERGIESOui                                    -1.770e+01  4.714e+03  -0.004 0.997004    
MAINSNormales                                    1.767e+00  1.055e+00   1.675 0.093963 .  
MAINSSèches                                      1.679e+00  1.073e+00   1.565 0.117595    
PEAU_CORPS.1Sèche                                       NA         NA      NA       NA    
PEAU_CORPS.1Très sèche                                  NA         NA      NA       NA    
INTERET_ALIM_NATURELLEPas du tout               -6.929e-01  1.490e+00  -0.465 0.641833    
INTERET_ALIM_NATURELLEUn peu                    -1.801e+00  7.381e-01  -2.441 0.014666 *  
INTERET_ORIGINE_GEOPas du tout                   1.025e+00  1.293e+00   0.793 0.427673    
INTERET_ORIGINE_GEOUn peu                       -4.041e-01  6.494e-01  -0.622 0.533728    
Crois_Prior1_MilieuPer_nature_hydratée          -7.592e-01  2.015e+00  -0.377 0.706322    
Crois_Prior1_MilieuPer_nature_lisse             -3.454e-01  1.756e+00  -0.197 0.844073    
Crois_Prior1_MilieuPer_nature_matifiée          -1.509e-01  2.495e+00  -0.060 0.951761    
Crois_Prior1_MilieuPer_nature_nourrie           -1.860e+01  6.608e+03  -0.003 0.997754    
Crois_Prior1_MilieuPer_nature_purifiée          -3.315e-01  1.611e+00  -0.206 0.837001    
Crois_Prior1_MilieuPer_nature_reposée            1.284e+00  1.883e+00   0.682 0.495542    
Crois_Prior1_MilieuPer_urbain_éclatante         -1.819e-02  1.534e+00  -0.012 0.990538    
Crois_Prior1_MilieuPer_urbain_hydratée           3.449e-01  1.356e+00   0.254 0.799243    
Crois_Prior1_MilieuPer_urbain_lisse              1.080e+00  1.334e+00   0.810 0.418107    
Crois_Prior1_MilieuPer_urbain_matifiée           1.051e+00  1.398e+00   0.752 0.452290    
Crois_Prior1_MilieuPer_urbain_nourrie            3.070e+00  1.627e+00   1.887 0.059226 .  
Crois_Prior1_MilieuPer_urbain_purifiée          -1.456e+00  1.430e+00  -1.018 0.308601    
Crois_Prior1_MilieuPer_urbain_reposée            2.520e+00  1.493e+00   1.688 0.091498 .  
Crois_Profil_PrioLisse / Eclatante_éclatante     2.472e+00  1.486e+00   1.663 0.096222 .  
Crois_Profil_PrioMatifiée / Eclatante_éclatante  2.845e+00  1.631e+00   1.744 0.081110 .  
Crois_Profil_PrioNourrie / Eclatante_éclatante  -1.685e+01  7.795e+03  -0.002 0.998276    
Crois_Profil_PrioPurifiée / Eclatante_éclatante  1.182e+00  1.523e+00   0.776 0.437707    
Crois_Profil_PrioReposée / Eclatante_éclatante  -1.895e+01  5.224e+03  -0.004 0.997106    
Crois_Profil_PrioEclatante / Hydratée_hydratée  -1.763e+01  2.457e+03  -0.007 0.994276    
Crois_Profil_PrioLisse / Hydratée_hydratée      -1.409e-02  1.625e+00  -0.009 0.993081    
Crois_Profil_PrioMatifiée / Hydratée_hydratée   -1.623e+01  4.667e+03  -0.003 0.997225    
Crois_Profil_PrioNourrie / Hydratée_hydratée     1.917e+00  1.854e+00   1.034 0.301226    
Crois_Profil_PrioPurifiée / Hydratée_hydratée    1.841e+00  1.417e+00   1.299 0.193915    
Crois_Profil_PrioReposée / Hydratée_hydratée    -1.581e+01  7.573e+03  -0.002 0.998334    
Crois_Profil_PrioEclatante / Lisse_lisse         1.464e+00  1.427e+00   1.026 0.305064    
Crois_Profil_PrioHydratée / Lisse_lisse         -1.757e+01  3.171e+03  -0.006 0.995578    
Crois_Profil_PrioMatifiée / Lisse_lisse          3.308e+00  1.879e+00   1.760 0.078343 .  
Crois_Profil_PrioNourrie / Lisse_lisse           1.318e+00  1.829e+00   0.721 0.471155    
Crois_Profil_PrioPurifiée / Lisse_lisse         -1.844e+01  3.417e+03  -0.005 0.995695    
Crois_Profil_PrioReposée / Lisse_lisse          -1.627e+01  6.889e+03  -0.002 0.998115    
Crois_Profil_PrioEclatante / Matifiée_matifiée  -1.651e+01  4.405e+03  -0.004 0.997010    
Crois_Profil_PrioHydratée / Matifiée_matifiée    3.058e+00  1.669e+00   1.832 0.066970 .  
Crois_Profil_PrioLisse / Matifiée_matifiée       2.699e+00  1.761e+00   1.533 0.125319    
Crois_Profil_PrioNourrie / Matifiée_matifiée     3.910e+00  2.316e+00   1.688 0.091412 .  
Crois_Profil_PrioPurifiée / Matifiée_matifiée   -1.614e+01  3.076e+03  -0.005 0.995814    
Crois_Profil_PrioReposée / Matifiée_matifiée    -1.751e+01  1.235e+04  -0.001 0.998868    
Crois_Profil_PrioEclatante / Nourrie_nourrie     2.304e+00  1.620e+00   1.422 0.154909    
Crois_Profil_PrioHydratée / Nourrie_nourrie     -1.622e+01  3.909e+03  -0.004 0.996688    
Crois_Profil_PrioLisse / Nourrie_nourrie         4.018e+00  2.082e+00   1.931 0.053542 .  
Crois_Profil_PrioMatifiée / Nourrie_nourrie     -1.683e+01  9.424e+03  -0.002 0.998575    
Crois_Profil_PrioPurifiée / Nourrie_nourrie      2.889e+00  1.891e+00   1.528 0.126629    
Crois_Profil_PrioEclatante / Purifiée_purifiée   1.983e-01  1.773e+00   0.112 0.910932    
Crois_Profil_PrioHydratée / Purifiée_purifiée   -1.700e+00  2.025e+00  -0.840 0.401098    
Crois_Profil_PrioLisse / Purifiée_purifiée      -1.741e+01  5.381e+03  -0.003 0.997419    
Crois_Profil_PrioMatifiée / Purifiée_purifiée   -1.840e+01  4.515e+03  -0.004 0.996749    
Crois_Profil_PrioReposée / Purifiée_purifiée    -1.771e+01  1.039e+04  -0.002 0.998640    
Crois_Profil_PrioEclatante / Reposée_reposée     2.290e+00  1.796e+00   1.275 0.202363    
Crois_Profil_PrioHydratée / Reposée_reposée     -1.609e+01  3.890e+03  -0.004 0.996699    
Crois_Profil_PrioLisse / Reposée_reposée         2.916e+00  1.552e+00   1.879 0.060212 .  
Crois_Profil_PrioMatifiée / Reposée_reposée     -1.625e+01  9.461e+03  -0.002 0.998629    
Crois_Profil_PrioNourrie / Reposée_reposée      -1.724e+01  8.232e+03  -0.002 0.998329    
Crois_Profil_PrioPurifiée / Reposée_reposée      6.497e-01  1.724e+00   0.377 0.706274    
Crois_ALL_AGEjeune_Sans_ALL                     -7.512e-01  7.326e-01  -1.025 0.305170    
Crois_ALL_AGEsenior_Avec_ALL                     1.664e+01  4.714e+03   0.004 0.997183    
Crois_ALL_AGEsenior_Sans_ALL                     3.611e-01  6.555e-01   0.551 0.581703    
Crois_ALL_AGEvieux_Avec_ALL                      1.930e+01  4.714e+03   0.004 0.996733    
Crois_ALL_AGEvieux_Sans_ALL                             NA         NA      NA       NA    
INTERET_VACANCESÀ la montagne                    1.208e+00  5.537e-01   2.182 0.029090 *  
INTERET_VACANCESEn ville                         2.452e+00  9.143e-01   2.682 0.007317 ** 
INTERET_COMPOSITIONPas du tout                  -1.763e+01  4.579e+03  -0.004 0.996929    
INTERET_COMPOSITIONUn peu                        4.350e-01  8.082e-01   0.538 0.590461    
PEAU_CORPS.2Sèche                                       NA         NA      NA       NA    
PEAU_CORPS.2Très sèche                                  NA         NA      NA       NA    
Nbre_gift                                        2.144e-01  1.520e-01   1.411 0.158265    
w                                                1.796e+00  1.027e+00   1.749 0.080317 .  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 812.37  on 586  degrees of freedom
Residual deviance: 170.68  on 501  degrees of freedom
AIC: 340.68

Number of Fisher Scoring iterations: 19

Besides, I tested all types of family:

*Family Default Link Function
binomial (link = "logit")
gaussian (link = "identity")
Gamma (link = "inverse")
inverse.gaussian (link = "1 / mu ^ 2")
fish (link = "log")
quasi (link = "identity", variance = "constant")
quasibinomial (link = "logit")
quasipoisson (link = "log")*

Why then I can not find any significant variable in my model ???

Thanks,

Improve this question
$\endgroup$
7
  • $\begingroup$ Is your target variable Binary(1/0; yes/no)?, give some sample data of your categorical variables, why did you choose Logistic Regression? there are many intermediary steps before applying Logistic Regression, a bit more explanation would help us in giving you better suggestions $\endgroup$
    – Toros91
    Commented Nov 29, 2017 at 9:11
  • $\begingroup$ yes, It is a Binary.The target variable is Achat_client, Achat_client=sapply(ID_Final,function(x) if(sum(ID_Achat==x)<=4){0} else {1}) > class(Achat_client) [1] "numeric" > head(Achat_client) 10626 10855 11224 1349 14714 15180 0 0 1 0 1 1 $\endgroup$
    – Imed
    Commented Nov 29, 2017 at 9:18
  • $\begingroup$ couldn't understand, can you append that in your question? what about the other variables? $\endgroup$
    – Toros91
    Commented Nov 29, 2017 at 9:20
  • $\begingroup$ @Toros91, Done. $\endgroup$
    – Imed
    Commented Nov 29, 2017 at 9:37
  • $\begingroup$ what is the outcome of summary(final.log.model)?, you need some method to important variables and do prediction right? $\endgroup$
    – Toros91
    Commented Nov 29, 2017 at 9:40

1 Answer 1

Reset to default
0
$\begingroup$

Welcome to the site!

So far what you have done is good.

Before going to modelling you need to take care of couple of things(Exploratory Analysis), Like:

  • Removing all the unnecessary variables based on business knowledge
  • Imputing Missing values
  • Removing Outliers
  • removing unimportant variables
  • Correlation analysis between variables

once these are achieved then look into Modelling(you have done most of the above steps).

Now coming to your problem:

When you are trying to apply Logistic Regression on data to predict then you need to take care of couple of things:

  1. Remove all the factors with single level
  2. Then you need to make sure that data which is partitioned(Spliting between Test and Train) have Same Levels before you predict. For now you din't see the issue but once you predict then you will get this error.

Important thing to remember is, Logistic regression works by converting Categorical variable to dummies before applying model, because of this it is unable to return you anything.Since you have more Categorical variables in your data I would suggest you not use Logistic Regression.

To get Predictor Importance(Important Independent Variables), you can use a Package in R called Boruta, this link has the implementation in R.

Once you get important variables then you can apply Random Forest, Decision Trees(Rpart) and see how they perform.

Let me know if you have any issues.

Improve this answer
$\endgroup$
24
  • $\begingroup$ ,Thanks,But I don't well Inderstand the issue of Level?_Same Levels_ and factors with single level ?? You mean level=modality/option of a variable?? Then, in logistic regression, we need to split data into tow parts.So I think that it's normaly to get not the same levels between Train and test! $\endgroup$
    – Imed
    Commented Nov 29, 2017 at 10:21
  • $\begingroup$ Yes if the levels are not same then it will throw error, but for ur case it converts each and every level of a factorial variable to every category present on it. It’s very hard for a logistic model to tell you variable explains the most. $\endgroup$
    – Toros91
    Commented Nov 29, 2017 at 10:37
  • $\begingroup$ ,perphas, the Bourta methods has select just 3 important variables from 20!! what i can conclude So?? $\endgroup$
    – Imed
    Commented Nov 29, 2017 at 13:07
  • $\begingroup$ The other variables are not important or not significant enough. Only 3 variables are important. $\endgroup$
    – Toros91
    Commented Nov 29, 2017 at 13:09
  • $\begingroup$ I didn't deal with prediction using Boruta package. How can I do that??? $\endgroup$
    – Imed
    Commented Dec 18, 2017 at 14:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.