Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
When creating a multi-objective optimisation/MCDM algorithm such as NSGA-ii, does it make sense to use a deep neural network trained on a supervised tabular regression prediction task, in place of a simple equation for the objective function?
Is possible or advantageous to replace a nonlinear equation with model.predict() function in Keras to be able to model more complex objective functions?
I am using pymoo with nsga-ii
This nonlinear equation will again be approximated with the net. There is no point in introducing this much computation complexity, if it is not learned by than than it wont be learned. Ocam rasor
There seems to be very little information on this subject using internet searches. Almost all searches link to information about optimizing the actual DNN training process.
My problem is similar to the one described above. I am using a DNN to create an objective function that I am then externally optimizing. I have 'real' input and output data HOWEVER my model is applied to the input data to create transformed input data. This model evolves the transformation over time.
So, my setup goes something like this. The model has 6 parameters plus one constraint. My external 'old school' NLO picks a set of 6 parameters, and a 7th that is 'close', then creates a unique vector of input data. I then train the DNN. I tweak this 7th parameter and re-train until the constraint is met. This now represents a 'valid' evaluation/prediction function F(x). I can then calculate my objective function using the predictions of the DNN.
I proceed by adjusting these 6 model parameters and re-training the DNN to create an F(x) for each evaluation of the objective. The best scheme for me is a hybrid simulated annealing with nearest neighbour for alternate evaluations. So, I don't have to worry about taking derivatives, though in principle I could take numerical derivatives. I eventually end up with a set of 6 parameters that minimize my objective function.
I haven't found papers/information about doing this type of thing. I have just followed my nose. Any links to similar approaches would be most appreciated.
