Is it possible to train a machine learning trainer?

Question

That's just a weird thought experiment I recently came up with. I am new to this field actually. Anyway, here is the idea:

Suppose we have data D, and a machine learning algorithm A1. Then we use A1 to train D to get the hypothesis H. I denote this process as follows:

A1(D) = H

Then, suppose we have a machine learning algorithm trainer A2, which we can use to train a better machine learning algorithm, A1*, by inputing all the row data, D, the hypothesises trained by the machine learning algorithms, H, and the algorithms itself A1:

A2(D,H,A1) = A1*(actually there should be a lot of H and A1)

And this process can go on and on. Hopefully you can get my idea. Sorry in advance if that doesn't make sense to you since English is not my native language. So do you guys think it is possible to train, let's say, a algorithm A3, which can be used to train a machine learning algorithm trainer?

Answer 1

I would try using a Genetic Algorithm to estimate optimal parameters for training. You would need to figure out the objective function of parameters of training algorithm $A_2$ which minimalised fitting $A_1(D)$. Those parameters could be regularization coefficients or numer of layers/neurons, or any other parameters describing your training algorithm.

There is also algorithm called bayesian regularization. It is very similiar to Levenberg - Marquardt training, but it is computing optimal regularization parameters $\alpha$ and $\beta$ by computing effective numbers of network's parameters.

I hope I understood your question well.
Regards, Max