2
$\begingroup$

I have an SFrame and a model:

train_data,test_data = products.random_split(.8, seed=0)
selected_words_model = graphlab.logistic_classifier.create(train_data,
                                                     target='sentiment',
                                                     features=selected_words,
                                                     validation_set=test_data)

After computing the accuracy of the model with `selected_words_model.evaluate(test_data) I'm asked "What is the accuracy majority class classifier on this task?" Yet I don't even know what this "means accuracy majority class classifier", shouldn't it be "accuracy of the majority class classifier" ?

Here is my attempt.

All these materials come from this coursera ML fundations course exercise.

$\endgroup$

2 Answers 2

2
$\begingroup$

I suspect you are right that there is a missing "of the," and that the "majority class classifier" is the classifier that predicts the majority class for every input. Such a classifier is useful as a baseline model, and is particularly important when using accuracy as your metric. This matches what your notebook comments in the next bullet, so that's likely what was intended.

$\endgroup$
1
  • $\begingroup$ Thank you ! That seems the most simple classification method ;) I still have one doube : how should I compare different learned models with the baseline approach where we are just predicting the majority class ? I tried a selected_words_model.show(view='Evaluation') but it still might depend on the threshold ? $\endgroup$ Sep 5, 2019 at 14:33
0
$\begingroup$

Naive classifiers, such as picking at random or always picking the majority class, serve as a braindead baseline to compare your model against. Hopefully your model can out-perform a naive model that simply randomly picks the response from the input rows, or doesn't look at the predictors at all!

It is also a warning about using metrics like accuracy for unbalanced datasets, ex. a dataset with response 90% 0s will give a majority classifier a 90% accuracy.

The majority classifier is in fact the optimal Bayes classifier assuming a prior based on the training class proportion, without looking at the predictor variables (so $Y$ and $X$ are independent):

$$ \hat y = \operatorname{argmax}_y P(y \mid \mathbf x) = \operatorname{argmax}_y P(y) $$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.