1
$\begingroup$

I have the following vector my_vector=[0.059; 0.223;0.197;0.176; 0.173; 0.171; 0.0421; 0.209; 0.556;0.252;0.198; 0.255; 0.130; 0.176; 0.110; 0.0845; 0.270; 0.192; 0.199; 0.348]

I want to change the vector in such a way that the output derived vector will have the highest possible distribution change. This means the distribution of the two vectors should be highly different (say have large KLD values) but their distribution should be still from the same family (say the F-test or two-sample K–S test will not reject the hypothesis that they are from the same distribution).

How could I generate the new vector from the base vector while I meet the constraint (having "highest" "possible" change in distribution)? I need a machine learning (data-driven) method to achieve such a goal.

Thanks in advance for any help and recommendation.

$\endgroup$
2
  • $\begingroup$ Well, I think you would not think to start from your vector then. Just create a new vector with length similar to yours randomly sampled from the distribution of your interest and compute KLD, and repeat the process till KLD is maximized! To me this seams as a plausible method to start. I do not know if there is any off-the-shelf ML model to do this for you. Still the method would data-driven so to say. For the implementation, you can borrow most of materials from this post: towardsdatascience.com/… $\endgroup$ Apr 3, 2020 at 15:02
  • 1
    $\begingroup$ Thank you very much, TwinPenguins. I appreciate your comment. So, based on your proposed solution, I have to iterate it and every time calculate KLD of the new vector and also check the K-S test between two samples. However, I need a more systematic method since I will do it over and over. I mean from the newly generated vector, I should generate another one which will have the same mentioned situation. $\endgroup$
    – Arkan
    Apr 3, 2020 at 16:46

1 Answer 1

2
$\begingroup$

I think the answer(comment) of TwinPenguins provides you a good start. You could improve this by using a Metropolis Monte Carlo approach instead of just trying again and again generating new vectors. What you could do is:

  1. create a new vector with length similar to yours randomly sampled from the distribution of your interest and compute KLD$_{old}$ (cf. TwinPenguins)
  2. modify this vector and compute KLD$_{new}$.
  3. if (KLD$_{new}$ is better than KLD$_{old}$), the new vector becomes your old vector
  4. else discard the changes
  5. go back to 2, unless you reach a given target value of KLD

(Note that there may be issues with getting trapped in local minima, but search the Monte-Carlo literature and you'll find a solution...you can always run multiple times, and then take the best result.)

$\endgroup$
5
  • $\begingroup$ Thank you very much DannyVanpoucke. I appreciate it. As I have mentioned in my post, the problem is that there is no target value of KLD. Just should be the highest possible value ( possible means still be from the same family distribution). As I TwinPenguins in a comment, I will need to do this generation over and over. Does Metropolis Monte Carlo approach address this situation? Thanks. $\endgroup$
    – Arkan
    Apr 3, 2020 at 16:46
  • 2
    $\begingroup$ Step 5 can be whatever rule you decide. It could be that you limit it to for example 1000 updates, or maybe that you track the evolution of your KLD and decide that the improvement slows down sufficiently (or even stops). The main difference is that you will generate a vector once and modify it. however, will still need to do your tests, but you can do them smartly (cheapest first, and maybe simplify, as only a small part of your vector is changed.) $\endgroup$ Apr 3, 2020 at 16:52
  • $\begingroup$ Thanks @DannyVanpoucke for complementing. Great answer. I almost forgot that the update could be done in smarter way. $\endgroup$ Apr 3, 2020 at 20:14
  • 1
    $\begingroup$ @Arkan I think you have a good lead on this this answer. I am not sure what do you mean 'systematic method'? Although you say there is no target, but you need maximum KLD and define a finite epochs! Please note that though, KL is a non-convex function, and there is no truly one maximal, and you easily end up getting a different result (vector) here at every new run. I gather from you that this is not a concern, as long you have a new vector meeting your criteria. $\endgroup$ Apr 3, 2020 at 20:15
  • $\begingroup$ Thanks, @TwinPenguins. I don't mean we have a threshold for KLD. I mean the KLD will be maximal if we have a higher value than it, which will lead to having a completely different distribution from the base one. I mean the K-S test (or F-test) will tell us that two vectors (the base one and the derived one) are from two different distributions. $\endgroup$
    – Arkan
    Apr 3, 2020 at 20:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.