2
$\begingroup$

I am currently writing a program that would be able to play snake on an 25*25 grid. It works by optimizing a set of weights of 300 different solutions (each solution would be a different neural network) with the aid of an evolutionary strategy, thus by random mutation and parent selection. I have decided not to apply crossover on the parents pool due to the black box problem concerning MLPs and other neural networks. My population size is 300 and 10 parents are selected every generations (200 gen. total). Every other solution which is not a parent gets deleted to make place for 290 new mutated solutions based on the parents.

My network structure (MLP) is the following : 6 input nodes, (10,10) hidden nodes, 3 output nodes.

With my current version of the programme, I am able to reach a steady score of 60 apples (or points) eaten (around 15/200 have a score above 60). And this only after 50 generations! But now, it seems like i have bumped into a local minima that I am not able to overcome. Does anyone have any idea if further progress is still possible or that i might have hit the ceiling? The quickly elevating performance might be a sign that my exploration/eploitation balance might not be ideal, but with each solution containing 190 weights, more exploration would seem too computationally expensive and would take forever on my i7 10gen laptop :/.

Can i call this a premature convergence and is there a way to deal with this, or have i hit the limits of my idea?

Improve this question
$\endgroup$
9
  • $\begingroup$ I fail to understand how the MLPs are trained in your program. Are they trained using back-propagation as usualy done or they are evolutionary evolved via evolutionary programming? $\endgroup$
    – Nikos M.
    Apr 20, 2021 at 16:17
  • 1
    $\begingroup$ i extract the weights as a 190 dimensional vector (190 weights for 6 input nodes, (10,10) hidden nodes and 3 output nodes). I select the best performing networks and their weight sets accordingly. Now, with a pop. of 300 and 10 parents. each parent will be copied 29 times and inserted into the population (the parents themselves get reinserted too). But while inserting them (=> the copies), i make random changes in the weights sets and then update the weights. Redoing this over different generations. I like to refer to this as a form of random local search mixed with an evolutionary strategy. $\endgroup$
    – Nick Stevens
    Apr 20, 2021 at 16:54
  • $\begingroup$ Sorry for the undeterministic description, I have never had the correct theory behind it but i think this is what u refer to as evolutionary programming. $\endgroup$
    – Nick Stevens
    Apr 20, 2021 at 16:55
  • 1
    $\begingroup$ I think unless MLP architecture changes (eg add more hidden layers, or special layers like dropouts, ..) then most probably you have hit a ceiling $\endgroup$
    – Nikos M.
    Apr 20, 2021 at 17:13
  • $\begingroup$ What would be preferred? The biggest challenge is that I add a lot more complexity to training the MLP if more hidden layers are used. I might be able to speed up some processes with code optimization, but this will cost exponentionally more time. $\endgroup$
    – Nick Stevens
    Apr 20, 2021 at 17:20

1 Answer 1

Reset to default
1
$\begingroup$

You can take a look at dropout, it will get you out of the local minima you are stuck in.

Improve this answer
$\endgroup$
5
  • $\begingroup$ what dropout rate would u suggest? $\endgroup$
    – Nick Stevens
    Apr 20, 2021 at 12:40
  • $\begingroup$ 0.1 to 0.3 is what is usually used. $\endgroup$
    – Abhishek Verma
    Apr 20, 2021 at 13:26
  • $\begingroup$ @NickStevens, you can even evolve the dropout rate as you do with other parameters $\endgroup$
    – Nikos M.
    Apr 20, 2021 at 17:37
  • $\begingroup$ yes, that was my plan. Would u perhaps know if keras dropout layers offer a dynamic change of the dropout rate? $\endgroup$
    – Nick Stevens
    Apr 20, 2021 at 18:23
  • $\begingroup$ @NickStevens, i dont think this can be done in keras unless you dive into the implementation code $\endgroup$
    – Nikos M.
    Apr 20, 2021 at 18:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.