I am new to machine learning and I got this task in my university. I have a dataset with over 100 columns and two target variables: $target1$ is categorical i.e. $0$ or $1$ and $target2$ is continuous i.e. values in range $0 \space to \space 100$.

How can I predict this type of problem?

I tried using multi-output classification from sklearn using the Random forest as an ensembler and it is predicting nicely for continuous target variable but not for categorical target variable.

  • $\begingroup$ I am also trying the same thing. I am trying to predict age and gender. For age, I am using regression algorithm and for gender, I am using classification algorithm and trying to Neural Network. Can you please share your solution?? $\endgroup$ – DataP Feb 3 '19 at 13:18

You have one classification task and one regression task, but sklearn's multioutput meta-estimators only support two tasks of the same type.

The best solution here is to train two models:

  1. A binary classifier to predict $target1$
  2. A regressor to predict $target2$

For example:

from sklearn.cross_validation import train_test_split
from sklearn.ensemble import RandomForestClassifier, RandomForestRegressor

# X, y = load training data

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

clf = RandomForestClassifier()
clf.fit(X_train, y_train[:,0])
print("classifier accuracy:", clf.score(X_test, y_test[:,0]))

regr = RandomForestRegressor()
regr.fit(X_train, y_train[:,1)
print("regressor R^2:", regr.score(X_test, y_test[:,1]))
| improve this answer | |

You should break this down into two models. I would solve this in the following manner:

  1. The first model would predict if its either Target 1 or Target 2 by looking at 100 columns
  2. The second model then would look at the 100 columns and additionally the output of model 1 and then predict 0 or 1 in case of target 1 or 0-100 in case of target 2.

I do not think you can achieve the result with just one single model. If you need more information, I could elaborate on it. But this should give you a starting point.

Here is an explanation of the 2 above points:

  1. Train a classifier, with all the data points you have with labels as Target 1/ Target 2. For this you could use any family of classifier. But you need to be very careful in the evaluation. If this models performs poorly, you will have a problem, as your classification would affect your next model. You also need to check if the distribution between target 1 and target 2 are appropriate before using a model to classify them.

  2. Once the classifier is done, you can then use regression with all the input features + class of the entry ( target 1 or 2 ).

| improve this answer | |
  • $\begingroup$ do you mean using classifier chain? Can you please elaborate? Thanks! $\endgroup$ – Rajeev Motwani Feb 12 '18 at 12:56
  • $\begingroup$ I am editing my answer to elaborate. $\endgroup$ – Nischal Hp Feb 12 '18 at 15:14

This depends on the meaning of your 2 response variables.

If your continues variable is actually 100 (integer) class as oppose to meaningful increasing magnitude (hence more of a regression), hence, what you can have a try is combining your 2 categorical response into one by concatenating. This new variable will be your new response with 200 class.

Otherwise, Nischal's answer would be the way to go.

| improve this answer | |

There may be benefit, i.e. improved performance, in solving for both problems together instead of as separate problems via separate models if the two targets have some sort of relationship. Look up multitask learning and structured prediction. The output layer in neural networks can be of any dimension, including 2 which is your case.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.