I'm doing research where a part of the collected data is of Ordinal type. I will implement ANN with Logistic Regression function in the Activation function. What I have learnt from documents of other websites as well as an answer in https://datascience.stackexchange.com/, the target value is of ordinal type while the independent variables are of ratio or interval type. But my independent data is of Ordinal type and the target data will be label (say Like or Unlike). How should I build a function for ANN if I'm not wrong in my understanding?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
5
I think that what is influenced by the type of your independent variables is not the structure of your neural network but the encoding of your data. If your independent variables are ordinal, then you can map them to integer numbers or other ordered sets (and maybe normalize them to make them between 0 and 1).
If your target variable is a label, it is a classification problem. Therefore, the neurons in the last layer in your neural network should have an activation function optimized for classification problems, like sigmoid.
You also need a specific loss function, like binary crossentropy. There was a case study, however, that showed that for ordinal classification (when your target labels are ordinal), mean squared error works well.
You can take any activation functions for all hidden layers in your network as long as they are non-linear. The choice of a particular activation function will influence the rate of convergence of your learning algorithm but I don't see how a particular activation function in hidden layers may be preferable for a certain type of input variables, like if they are ordinal or not.
answered Jun 3, 2018 at 7:15
-
$\begingroup$ Thanks for your answer. Did you mean "Ordinal" in place of "ordinary"? $\endgroup$– PS NayakCommented Jun 3, 2018 at 13:18
-
$\begingroup$ Ooops. yes, ordinal. Corrected $\endgroup$ Commented Jun 4, 2018 at 1:22
-
$\begingroup$ I agree with you. Now, if prediction comes into picture and in the output, there are more than five levels (say very negative, somewhat negative, neutral, somewhat positive, or very positive), then theoretically we can go for Ordinal Logistic Regression that computes the probabilities of the output set. What will be the activation function? $\endgroup$– PS NayakCommented Jun 4, 2018 at 11:23
-
$\begingroup$ I edited my answer. Removed about softmax. I am not sure if it can be used here. I think you need to use sigmoid activation, and the number of output neurons = number of classes. See this link they also report that squared error loss function works fine. $\endgroup$ Commented Jun 4, 2018 at 12:17
-
$\begingroup$ Check softmax, though. In one of my cases, it looks like it works. $\endgroup$ Commented Jun 4, 2018 at 12:54