4
$\begingroup$

The classic classification problem is like finding the function $F:\mathbb{R}^n\mapsto \{0,1\}$. The label set will be [Apple,Banana,Banana,...,Apple].

What if I want to train a function $F:\mathbb{R}\mapsto[0,1]$? My samples could be something like "this sample has 80% probability to be Apple and 20% to be Banana".

It seems a multi-output neural network works, as we can apply the softmax loss with cross entropy loss. What about random forest or other algorithms? I have tried some common algorithms in scikit-learn without any luck.

For example, this code:

import numpy as np
from sklearn.ensemble import RandomForestClassifier

N_FEATURES = 10
N_SAMPLES = 1000
N_CLASSES = 2

train_x = np.random.rand(N_SAMPLES, N_FEATURES)
train_y = np.random.rand(N_SAMPLES, N_CLASSES)
train_y = np.apply_along_axis(lambda x: x/x.sum(), 1, train_y)

model = RandomForestClassifier(n_estimators=10).fit(train_x, train_y)

Yields a ValueError: Unknown label type: 'continuous-multioutput'.

$\endgroup$
7
  • 1
    $\begingroup$ Use predict_proba in sklearn instead of predict. $\endgroup$ Feb 22, 2019 at 3:10
  • $\begingroup$ @MatthewDrury I mean the training procedure, not test/evaluate/predict/inference. $\endgroup$
    – Icyblade
    Feb 22, 2019 at 12:19
  • $\begingroup$ Same training procedure. Almost all modern ML models (logistic regression, random forests, gradient boosting, neural networks) are designed to train probabilistic predictions. Which specific ones are troublesome? $\endgroup$ Feb 22, 2019 at 20:22
  • $\begingroup$ For example random forest in scikit-learn. The question has been updated. $\endgroup$
    – Icyblade
    Feb 25, 2019 at 11:24
  • $\begingroup$ Ahhh, I think I understand now, your target is a distribution across the labels? $\endgroup$ Feb 25, 2019 at 20:30

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.