I need help determining the best way to go about creating the target variable for a machine learning model that is trading a financial instrument (stocks, foreign currencies, crpyto, etc).
Below is some sample data to help with answering the question.
Time: When the data was recorded Price: Is the current price of the instrument and is the price I would be buying at. price_good: Price level that I need to sell at to make a profit price_bad: Price level that I need to sell at to minimize losses good_id: Returns the ID of the first row where price >= price_good bad_id: Returns the ID of the first row where price <= price_bad target: If good_id < bad_id, target = 1. Else target = 0
My ideal scenario is when price_good occurs before price_bad. The issue I'm running into is how to correctly set the target field. I have 2 ways that I can solve this -
Option #1 - This option uses all of the data and has no restrictions on creating the target field but has a slight mismatch on how it would work in production.
id time price price_good price_bad good_id bad_id target 1 01-01-19 100 110 90 4 nan 1 2 01-02-19 105 115 95 4 nan 1 3 01-03-19 109 120 99 4 nan 1 4 01-04-19 121 131 111 nan 5 0 5 01-05-19 110 120 97 nan nan nan
Option #2 - This option doesn't use all of the data and doesn't allow a target to be set until the initial row reaches it's target which is how it would work in production.
id time price price_good price_bad good_id bad_id target 1 01-01-19 100 110 90 4 nan 1 2 01-02-19 105 115 95 nan nan nan 3 01-03-19 109 120 99 nan nan nan 4 01-04-19 121 131 111 nan 5 0 5 01-05-19 110 120 97 nan nan nan
"In production" means how it would work after the model is built. For example, If my model predicted that id:1 would hit target_good, I wouldn't be able to make any other trades until that trade reaches price_good or price_bad which aligns with option #2. However, I'm losing the information about all of the other trades that I could of made from option #1 if I had more capital to invest.
If I go with option #1, the model will sometimes "overfit" because it has data that is closer together in time and uses mostly the same information.
If I go with option #2, it is a more accurate representation of how it would work in production but I lose a ton of data points. For example, what if I started at id:2 instead of id:1. I would end up with different data points that the model is built on. I could test starting at multiple places to find the most optimal starting place but that would take a ton of iterations and resources to test.
Any tips on how to model this correctly? Thanks!