1
$\begingroup$

I'm new to machine learning, and was working on creating an ANN which would classify each observation to a certain value. I have worked primarily with the sigmoid function up to this point to get the probability of an observation (true/false or binary output).

In this instance, I want each observation to be classified to one of 5 values: 0, 1, 2, 3, 4. I am using the sklearn library's StandardScalar function to scale the input data. What would be recommended in terms of:

  1. Best way to scale the output data, and
  2. Appropriate activation function to use for the output layer.

Thanks!

$\endgroup$
1
$\begingroup$

You don't need to scale the output data. For classification with a ANN the best activation is the softmax function:

f(x) = e^x_i / sum e^x_j

Which normalizes an input vector by applying the exponential function element wise, then dividing by the sum. This produces a discrete probability distribution.

Then your output layer should have 5 output neurons, apply softmax, and you will get a discrete probability distribution over the set [0, 1, 2, 3, 4]. Then you don't need to normalize the data, just do a one-hot encoding. Note that this might differ depending on the framework used.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.