# Using the GA R package to optimize the weights of a MLP neural network

The neural network I am trying to evolve uses the tanh as an activation function in each neuron and has a topology of 1-5-1, so I need at least 5 weights. The solution of the GA is a real-number vector of length 5, which represents the weights of the network and each weight should take values between -5 to 5. I wrote an R function to use as a fitness function which returns the mean squared error (MSE) of the output data in comparison to the desired output. I want it to learn the cubic function. The input data I am using is

input<-seq(-1,1,0.02)


and the output data is its cubic function

des_out<-input^3


The evaluation function is the following

evalnn<-function(x){

mse<-0
for(i in 1:length(input)){

nn.out <- tanh(x[1]*input[i]) + tanh(x[2]*input[i]) + tanh(x[3]*input[i]) + tanh(x[4]*input[i]) + tanh(x[5]*input[i])
mse <- (des.out[i] - nn.out)^2 + mse

}
return(-(mse)/length(input))
}


I have set the return value to negative, because I want the smallest value to be thought as the best fit.

gann<-ga(type="real-valued", fitness=evalnn,min=c(-5,-5,-5,-5,-5),max=c(5,5,5,5,5),popSize=100,maxiter=150,pmutation=0.01,pcrossover=0.8)


What I always get back from the GA are weights that make up a linear function although I have been experimenting with the ga's parameters quite a lot, i.e. I have tried all crossover methods and mutations. https://cran.r-project.org/web/packages/GA/GA.pdf.
The cubic function plotted together with my output function.

I have used a linear function as training data before the cubic with this implentation and worked. It has trouble with the non-linear.

If anybody could figure out why do I get this, is there something I have missed. Thank you

• You use the tanh function in the hidden layer, shouldn't it be logical to use sigmoid instead, because your output-values are all in the range [0-1] and tanh is in a range of [-1 1] And maybe also a sigmoid function in the output layer? Do you have still our final code and can you share this here? – user32991 Jun 3 '17 at 10:04
• Could you share your code when you include the topology of the neural network? At this moment, I am trying to develop a forecasting model using neural network. the problem is I don't know how to optimise the architecture of my network. I hope your method could help me with it. Thank you. – ISKANDAR Dec 12 '17 at 5:24
• Can you please provide the rationale for the requirement that the weights should lie between-5 and 5? – aivanov Dec 12 '17 at 12:53