I am working on a 4-class classification task using an Artificial Neural Network. My aim is to visualize how well those 4 classes can be separated, moreover how consistent the data coming from each class is and how far they are "away" from one another.
My approach is to use 2 neurons in the last hidden layer and plot the output of the neurons in a scatterplot. However my current output usually looks like this where the different colors stand for the actual classes the data is coming from:
Obviously this is not very useful. First of all I was hoping that the points would not all be all along the same line and second of all I am not sure if this can even be interpreted in the way that I want.
I am using the neuralnet package in R to train the network using linear output. I am guessing that if this approach can work the choice of activation function would be crucial.
I am new to Neural Networks and I am not sure whether or not something like this can be done or not and I am hoping to get some comments.