I'm building a neural net for classifying characters in pictures. The input can be any character a-z, lowercase and uppercase. I only care about classifying the characters, and not the case, so the neural net has an output vector of length 26; one for each character. It makes sense, intuitively, to have a hidden layer of size 26*2 just upstream of the output layer. It also makes intuitive sense for this layer not to be fully connected to the output layer, but instead having two and two hidden nodes connect to each output node.
I have some questions:
a) Does this make sense? I'm getting about 75 % success rate on a pretty hard data set with just one hidden layer, but I'm not certain on how to improve from there.
b) If so, What activation function should I use from the hidden layer with 26*2 nodes to the output layer? Maybe I should use an OR function for this, since both the lowercase and the uppercase version of a character should output for a single character.
c) Would it be wiser to have 26*2 output nodes instead, and just combine lowercase and uppercase outputs after the neural net?