0
$\begingroup$

I have a huge huge model in SQL that nobody knows what it is doing. This model spits out some numbers and those numbers should be optimised to match another batch of 'correct' numbers as much as possible. So I get one set of numbers form my black box model that change according to how different parameters change inside that model - there are around 400 parameters that can be changed to get different numbers out of it (no, I don't want to even touch it, less trying to understand how it works) and I have another 'correct' set of numbers that I get for those parameters from a person in charge of it. What would be the best approach to 'optimise' my black box numbers to match the correct ones? The black box numbers will relatively match the correct numbers depending on how parameters are set up but I need to make them more correct. I was thinking maybe Bayesian optimisation in Python or GridSearch but not sure if that is the best approach. Any ideas would be appreciated.

$\endgroup$
  • $\begingroup$ "person in charge of it.", does this person provide correct output of model, given an input ? $\endgroup$ – Shamit Verma Feb 22 '19 at 13:08
  • $\begingroup$ No, this person just has his own completely different way of doing it and our numbers should match his as much as possible but we can't have anything to do with his model. We just get his numbers. $\endgroup$ – BlueIvy Feb 22 '19 at 13:11
  • $\begingroup$ by "his numbers", do you mean output generated by this person's model ? $\endgroup$ – Shamit Verma Feb 22 '19 at 13:19
  • $\begingroup$ Yes, that is what I mean. $\endgroup$ – BlueIvy Feb 22 '19 at 13:20
  • $\begingroup$ What inputs does he use ? Are the same inputs fed into SQL model to generate numbers from model ? $\endgroup$ – Shamit Verma Feb 22 '19 at 13:25
0
$\begingroup$

For this, you can treat output provided by "person in charge" as ground truth. I assume that you have historical records of :

  1. Numbers generated by this person
  2. Parameters that were fed into the model to generate numbers

treat these 400 parameters are train_x and numbers generated by the person as train_y.

Once you have this data in CSV, you can explore various Algo to find a model that can match train_y with minimal error.

Since no details are available on "how" either of these outputs are produced, you need to pick a tool that enables quick experimentation.

I will suggest Weka since this is a UI tool and does not require any coding. Once you identify an Algo, you can implement it in Python.

Some tutorials :

https://www.youtube.com/watch?v=6tDnNyNZDF0

https://www.youtube.com/watch?v=kFKxUQDnLV8

https://www.youtube.com/watch?v=COpXNK0O8As

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.