The neural network I am trying to evolve uses the tanh as an activation function in each neuron and has a topology of 1-5-1, so I need at least 5 weights. The solution of the GA is a real-number vector of length 5, which represents the weights of the network and each weight should take values between -5 to 5. I wrote an R function to use as a fitness function which returns the mean squared error (MSE) of the output data in comparison to the desired output. I want it to learn the cubic function. The input data I am using is
input<-seq(-1,1,0.02)
and the output data is its cubic function
des_out<-input^3
The evaluation function is the following
evalnn<-function(x){
mse<-0
for(i in 1:length(input)){
nn.out <- tanh(x[1]*input[i]) + tanh(x[2]*input[i]) + tanh(x[3]*input[i]) + tanh(x[4]*input[i]) + tanh(x[5]*input[i])
mse <- (des.out[i] - nn.out)^2 + mse
}
return(-(mse)/length(input))
}
I have set the return value to negative, because I want the smallest value to be thought as the best fit.
gann<-ga(type="real-valued", fitness=evalnn,min=c(-5,-5,-5,-5,-5),max=c(5,5,5,5,5),popSize=100,maxiter=150,pmutation=0.01,pcrossover=0.8)
What I always get back from the GA are weights that make up a linear function although I have been experimenting with the ga's parameters quite a lot, i.e. I have tried all crossover methods and mutations. https://cran.r-project.org/web/packages/GA/GA.pdf.
The cubic function plotted together with my output function.
I have used a linear function as training data before the cubic with this implentation and worked. It has trouble with the non-linear.
If anybody could figure out why do I get this, is there something I have missed. Thank you
