4
$\begingroup$

I was given a target function to design neural network and train: (y = (x1 ∧ x2) ∨ (x3 ∧ x4))

The number of input and number of output seems obvious (4 and 1). And the training data can use truth table.

However, in order to train as a multilayer artificial neural network, I need to choose number of hidden units. May I know where can I find some general guideline for this?

Thank you!

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

You problem is linearly separable so you can use a single layer perceptron. Hidden units are necessary only for non linear problems (xor is a classic example).

As for general guidelines, I don't think anyone has ever needed more than one hidden layer. Also the number of neurons in the hidden layer doesn't need to exceed the number of inputs.

Improve this answer
$\endgroup$
11
  • $\begingroup$ On StackExchange, such short answers are usually (with rare exceptions) preferred in a form of a comment. Posting answer implies a more detailed/comprehensive treatment of a topic. Just FYI. $\endgroup$
    – Aleksandr Blekh
    Commented Mar 5, 2015 at 3:06
  • $\begingroup$ @Alex S Kinman multi hidden layer is common in deep learning architecture. $\endgroup$
    – Xin
    Commented Mar 5, 2015 at 13:40
  • $\begingroup$ @Aleksandr Blekh what if a question doesn't need a long answer at all? do you have a reference where your comment based on, meta, faq etc? thx $\endgroup$
    – Xin
    Commented Mar 5, 2015 at 13:43
  • $\begingroup$ @Xin , I don't know enough about deep neural networks. But for traditional neural networks, the universal function approximation theorem states that a multilayer network with a single hidden layer can approximate any continuous decision function. I am curious, are there any results proving that DNN have more representational power than regular ANN? $\endgroup$
    – Alex S Kinman
    Commented Mar 5, 2015 at 14:03
  • 1
    $\begingroup$ @AlexSKinman I deleted my previous comment because the 3-D case I gave was a bad example (that one is indeed linearly separable). I tried training a 4x1 network on the y = (x1 ∧ x2) ∨ (x3 ∧ x4) problem numerous times and could not get it to learn the function (the best it would do was accuracy of 0.9375 (i.e., one input was always misclassified). Using a 4x2x1 network, the function was easily learned. If you believe you can construct a separating hyperplane, perhaps you could provide its coefficients (i.e., the a coefficients for f(x) = a1*x1 + a2*x2 + a3*x3 + a4*x4 - a0). $\endgroup$
    – bogatron
    Commented Mar 6, 2015 at 6:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.