How to train and validate a model continously which affects its own future data?

Question

We are working with a online marketplace. Our problem is to predict whether certain products are profitable or not for our marketplace in near future(next one month horizon).

For example: Consider 2 products

1. Toy Giraffe (Predicted not profitable)
2. Sketch Pen (Predicted profitable)

For the 2nd product which was predicted profitable we take no action and as such we can get the profitability data of next month and check if our prediction was correct.

For 1st product which was predicted non-profitable we take actions which are supposed to help make it profitable. Now 2 scenarios arise: 1. It becomes profitable next month. 2. It doesn't. In both scenarios how do I know whether our initial prediction was correct or not. In scenario 1. It could have gotten profitable since my prediction was wrong or because of the corrective action. In 2. scenario also whether my prediction was wrong or the action taken was bad.

So how do I use the 1st products result in training my model next month?

Simple Tree diagram.

                                        start
                                          |
                                         / \
                                        /   \
                                       /     \
                                      /       \
                                     /         \
                                    /           \
                          Toy Giraffe          Sketch Pen
                            Predicted:         Predicted:
                            Non Profitable.    Profitable.
                                  |                |
                                  |                |
                                  |                |
                            Action Taken       No Action
                                  |
                            Wait 1 Month
                                  |
                                 / \
                                /   \
                               /     \
                              /       \
                             /         \
                            /           \
                           /             \
                          /               \
                     Case 1.             Case 2.
              Is Profitable.           Is not Profitable despite action. 
                   |                                  |
                   |                                  |  
                  / \                                / \
                 /   \                              /   \
                /     \                            /     \
               /       \                          /       \
              /         \                        /         \
             /           \                      /           \
            /             \                    /             \
      Prediction      Prediction           Prediction      Prediction
     was correct.   was not correct.      was correct.     was not correct.   
          |              |                    |                |
  Profitable due      Profitable          Action Failed.    Got non-profitable due to Action.
   to Action.         since our 
                   Prediction was wrong.

Under case 1. and case 2. for Toy giraffe nodes how do I identify whether my prediction was wrong/right or the taken action impacted the results. Who among prediction and action was responsible for the end result? The 4 leaf nodes are the possible events that can happen and I need to analyze each case.

References:

1. How to deal with a machine learning model which affects future ground truth data?

2. https://stats.stackexchange.com/questions/13533/what-is-reject-inferencing-and-how-can-it-be-used-to-increase-the-accuracy-of

Not sure if second link will be useful technique or not.

Also is Reinforcement learning something to look at for these kind of problems where the reward can be said to be delayed?

Answer 1

Reinforcement learning (RL) is a useful way to frame problems when actions that are taken change the environment.

The best way to assign credit for outcomes is through experimentation. By carefully designing an experiment, you can figure out if the intervention has a causal impact on the result. Often it takes a series of experiments to understand the cause-effect relationship.