i have created some logistic regression model (different preprocessing) with softmax function. and i mix all model with an ensemble with a hierarchical method. so the output of all model (base) will be used as input for the final model (logistic regression too).
the default base model used a softmax function. i think transform a confident value into a probability will lose much information. so i have a plan to change the softmax into activation function.
what i learn in my CNN class, Relu is the best default activation function for an image. but my case is multi-class classification email.
which activation function should i choose? Relu, sigmoid or other?