1
$\begingroup$

In the context of discrete choice models, what difference does it make in segmenting my sample based on a particular "criteria" and study the effects of explanatory variables on each segment VERSUS Just adding the "criteria" as another explanatory variable in the unsegmented population

Say, I want to build a commuter mode choice model using MNL, what difference does it make when I split my population via gender and study the effects of various explanatory variables VERSUS adding gender as another explanatory variable in the entire unsegmented population.

$\endgroup$
0
$\begingroup$

If you choose to build a model where one of the categorical features such as a gender plays a big role in the outcome, then the overall model results will give a large weight to the gender, and other features will have a smaller weight. While your model will be applicable for the unsegmented population, the predictive power might not be very good.

Now, suppose that you wanted to identify if the weights of the other features differed for males vs. females and if the segmented models had better predictive power, then by segmenting the population and creating models for each gender, you could explore changes in the magnitude or direction of the weights of each feature & evaluate the goodness of fit of your models to see if there is an improvement in the segmented population models vs. the overall model.

If performing logistic regression, you could scale the coefficients or Z-values returned in the model parameters for each model and check if the relative magnitude & direction of the overall vs. the segmented models gave you similar or different results. You could also look at the confusion matrix and use various accuracy measures to assess if the segmented models performed better, worse or the same as the overall model. In this way, you could determine whether for the outcome independent gender-based models are more appropriate for your question than a single overall model.

But, this leads to the question of if I have "n" features, should I create multiple models by segmenting the "x" categories in each of the "n" features? This really depends on the data question being asked. In some cases, it is valuable to run multiple models where each feature is dropped one-by-one to assess feature importance. In some cases, just evaluating the weight & relevance of each feature from the overall model is sufficient. Going beyond, one could look at optimizing models by choosing which of all "n" features should be segmented to yield the model with the best predictive power. This paper from FICO describes one methodology. This article details an approach and comparison of various models.

$\endgroup$
2
  • $\begingroup$ Thanks for the response. I have another question related to this. What are the statistical tests to check whether segmenting the model is superior to the unsegmented one. One such test is Chi square test for segmentation usually used for discrete choice models. $\endgroup$ Aug 7 '17 at 9:55
  • $\begingroup$ You could look at ROC and Precision-Recall curves to compare the separate model results against a test set. You could also evaluate the confusion matrix results and check for sensitivity & specificity of the model (depending on what is most useful for your problem). I couldn't find any existing questions about this on Stack Exchange, so if you propose a new question, i think you'll get better responses & it would be easier for someone to search in the future. $\endgroup$
    – gchaks
    Aug 8 '17 at 18:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.