After coming across this article about evolution strategies http://blog.otoro.net/2017/10/29/visual-evolution-strategies/, it seems clear that the Covariance-Matrix Adaptation Evolution Strategy (CMA-ES) has the potential to be incredibly effective.

I'd like to try to create a similar algorithm to be used with a Tensorflow Keras Sequential model. I've only seen simple examples, but here's how I think it could be done with a Sequential model:

  1. Build the model as if backpropagation were being used and split the data into training and test sets
  2. For some number of items representing the search size of the CMA-ES model, initialize weights randomly for each item
  3. Use Tensorflow's model.predict method to evaluate the loss for each instance of the model on the training set
  4. Select the best instances (let's say the top 20%) based on the calculated loss
  5. Calculate the mean and standard deviation in accordance with the CMA-ES methodology for each weight among the weights of the selected models
  6. Generate a new list of random weights based on the new weight means and standard deviations
  7. Repeat steps 3-6 for some number of iterations or until convergence

The model I'm working on has almost 27k weights. So if I use 100 search instances and iterate through 50 times, that's 135M individual weights. With that said, before I go down this rabbit hole, I'd be curious if anyone has general feedback or experience with this method, but I'm specifically wondering:

  1. What is the best way to initiate weights for each instance of the model and set the initial standard deviations for each weight? How is Tensorflow doing this under the hood?
  2. Will the weights converge with high probability within a reasonable time like the image below (taken from the article) or could it get stuck between multiple low points?

enter image description here

I'd be interested to hear from anyone who has explored this topic.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.