1
$\begingroup$

I was wondering if someone could direct me to a dataset for a classification task with the following conditions:

  • Multinomial logistic regression alone does not learn a good classifier
  • A series of fully connected layers is able to learn a good classifier
  • The task is not MNIST

Thank you

$\endgroup$
2
  • 1
    $\begingroup$ There are lots of datasets you can try but I would simply suggest XOR problem which logistic regression can not perform well on but fully connected layers can achieve a good performance. $\endgroup$
    – pythinker
    Apr 11, 2019 at 19:11
  • $\begingroup$ Multinomial logistic can learn almost everything. The point is that you yourself have to find the correct polynomials which is not easy in most of the cases. $\endgroup$ Apr 11, 2019 at 21:49

1 Answer 1

1
$\begingroup$

Here's a one-dimensional problem that should be impossible for logistic regression:

  • Generate x uniformly on [0, 10]
  • Let y = sin(2 * pi * x)

If you want a classification problem define y_disc as:

  • 0 if y > 1/sqrt(2)
  • 1 if 1/sqrt(2) > y > -1/sqrt(2)
  • 2 if y < -1/sqrt(2).

This is non-linear, so logistic regression should do poorly. Further, if you decide to put higher powers of x into your logistic regression classifier, I think you'll need a lot of powers to accurately represent the Taylor series near 10.

If fact, you could try adding a small noise term to y before discretizing - I suspect that will result in your logistic regressor overfitting long before it's able to accurately approximate the behavior near x=10.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.