0
$\begingroup$

I'm using Logistic Regression from sklearn for a multi-class classification problem. I have 5 classes, and 10 predictors. These predictors are semi-continuous (ordinal variables but are averaged so that values are in the range 1 - 5), and the target variables are classes 1, 2, 3, 4, and 5.

When I read the output, the model shows 55 features, whereas my input data has 10 columns and hence 10 features. Where in the pipeline does this split occur, and how can I debug the algorithm?

At first I thought sub-categories from the predictor variables were being formed, as is seen with hot encoded data, but the output features change with the sample size, not the format of the predictor variables. This is strange as sample size should not be correlated with the number of features that you end up with? Or am I wrong...

$\endgroup$
  • $\begingroup$ Can you share a snippet of your features please? $\endgroup$ – Saandeep Sreerambatla Feb 7 at 8:08
  • $\begingroup$ Unfortunately I cannot because of some legal constraints $\endgroup$ – Ali Faruque Feb 7 at 11:28
  • $\begingroup$ (1) If the categorical features have more levels in the larger dataset, there will be more dummy variables. (2) What do you mean by averaging ordinal variables? (3) Code snippets would help, as would a simplified and nonproprietary data example. $\endgroup$ – Ben Reiniger Feb 7 at 13:03
  • $\begingroup$ What exactly do you mean by "When I read the output, the model shows 55 features"? $\endgroup$ – Ben Reiniger Mar 8 at 16:30
0
$\begingroup$

It looks like, you have converted data in a One-Hot-Encoding and thats the reason each feature value converted into 5 sub-values.

One-Hot-Encoding used for mutually exclusive/categorical values like Male/Female.

If your data is having continuous value, then I dont think you need One-Hot Encoding.

| improve this answer | |
$\endgroup$
  • $\begingroup$ You are correct, but I do some further processing to avoid this happening. My predictors are averaged (after hot encoded) to reduce the feature space from 60 to 10. These final 10 predictors are then fed into the model. However, even if this was the case, 10*5 = 50 and I have 55 variables. $\endgroup$ – Ali Faruque Feb 7 at 11:34
  • 1
    $\begingroup$ share a data sample along with problematic piece of code. $\endgroup$ – vipin bansal Feb 7 at 11:47
  • $\begingroup$ I cant share the data unfortunately because of some legal constraints, and there is no part of the code that does not work. The problem is that logreg is producing coefficients that are proportional to the training sample size, which is odd because coefficients should not change with sample size. $\endgroup$ – Ali Faruque Feb 7 at 11:54
  • $\begingroup$ A bit difficult for me to map your problem...please do share other relevant information and will try my best. $\endgroup$ – vipin bansal Feb 10 at 7:19
0
$\begingroup$

Perhaps your settings (multi_class, solver) have been set to use one-vs-rest, so there are actually 5 separate models, each with 10 features plus a constant term, hence 55 coefficients.

I'm not sure how sample size would have an effect, unless you don't actually have all 5 classes in some run.

| improve this answer | |
$\endgroup$
  • $\begingroup$ That's exactly what happened, thank you for your comment. $\endgroup$ – Ali Faruque Jun 17 at 9:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.