I have a huge huge model in SQL that nobody knows what it is doing. This model spits out some numbers and those numbers should be optimised to match another batch of 'correct' numbers as much as possible. So I get one set of numbers form my black box model that change according to how different parameters change inside that model - there are around 400 parameters that can be changed to get different numbers out of it (no, I don't want to even touch it, less trying to understand how it works) and I have another 'correct' set of numbers that I get for those parameters from a person in charge of it. What would be the best approach to 'optimise' my black box numbers to match the correct ones? The black box numbers will relatively match the correct numbers depending on how parameters are set up but I need to make them more correct. I was thinking maybe Bayesian optimisation in Python or GridSearch but not sure if that is the best approach. Any ideas would be appreciated.
For this, you can treat output provided by "person in charge" as ground truth. I assume that you have historical records of :
- Numbers generated by this person
- Parameters that were fed into the model to generate numbers
treat these 400 parameters are train_x and numbers generated by the person as train_y.
Once you have this data in CSV, you can explore various Algo to find a model that can match train_y with minimal error.
Since no details are available on "how" either of these outputs are produced, you need to pick a tool that enables quick experimentation.
I will suggest Weka since this is a UI tool and does not require any coding. Once you identify an Algo, you can implement it in Python.
Some tutorials :