What type of optimization problem is this?

Question

We have a deformable mirror controlled by 40 actuators that have input voltages from -1V to 1V. Prior to hitting the mirrors, a pulse of light enters a diffraction grating and then the light is spread over the mirror by wavelength. Each actuator applies a difference phase shift to each wavelength. In the end, the light gets reconverged and read by a CCD.

Our goal is to maximize the integral of the spectrum read by the CCD with respect to the 40 variable voltages. This optimization will be done in matlab. I mostly just need to know what type of problem this is...

The program will do something like:

1. Measure a spectrum and integrate
2. Input the new spectrum to the optimization toolbox.
3. Output a new set of voltages to try
4. Go back to 1

There is likely multiple local maxima and the positions will change from day to day as our setup is always changing. Any help on where to look for more information would be much appreciated!

Answer 1

Some flavor of evolutionary algorithm may suit your problem nicely, since:

• The gradient of the objective function is unavailable or cannot be computed.
• The objective function itself is somewhat expensive to compute.
• The objective function is may have a large number of local maxima.

In short, evolutionary algorithms do optimization by generating candidate solutions and iteratively:

1. Checking their "fitness" (the value of the objective function)
2. Selecting the best candidate solutions
3. Generating new candidate solutions, somehow based on the best candidates from the previous iteration

Naturally, the devil is in the details of how the best candidate solutions are selected and combined to generate new candidate solutions. This gives rise to a large number of evolutionary algorithm variants. Two popular variants are genetic algorithms and evolution strategies.

Genetic algorithms

Standard genetic algorithm implementations require you to encode your input variables in a binary fashion. This may be fine for categorical inputs, but your input variables take real values in the interval $[-1,1]$. It would be possible to implement a genetic algorithm for your problem, but it wouldn't be ideal. For your problem, I'd take a look at...

Evolution strategies

On the other hand, evolution strategies handle real-valued inputs by design and, after getting used to the jargon, are fairly easy to implement. You mentioned that you plan to use MATLAB for optimization. I found this toolbox on the FEX which implements evolution strategies, but I haven't taken it for a test drive myself. Whatever the case, I think an evolution strategy would be a good option to try, given the dynamic nature of your problem.

Answer 2

It sounds like a reinforcement learning problem to me:

• It's a high-dimensional input space with non-linear interactions
• The only way to get information from the system is to use it
• Its behavior is time-dependent given all the intricacies of laser/optics setups
• There is interplay between short-term and long-term rewards (multiple local maxima vs. a global max.)

You can search "reinforcement learning matlab" and "markov decision process" for some available projects and tutorials.