I'm just looking for some high level recommendations on libraries, design patterns, or algorithms here. I obviously don't expect you to build a model for me.
I would like to predict the monetary impact that an outage will have on the electric grid.
I have data of historical outages that I could train a model on. This basically consists of the equipment-ID that was on outage, with a start and end timestamp.
An outage may cause zero or more constraints. A constraint is where a line becomes loaded to the extent that power must be flowed an alternate way. I already have data on which constraint-IDs occurred at what start and end timestamp, as well as the economic impact of that particular occurrence of that constraint. Note that a constraint may exist without any outages, so there is a bit of a loose relationship. Also, outages may occur without any constraints occurring, or it may take a combination of outages to cause a particular constraint. Generally speaking though, there is a moderate to strong relationship between outages and constraints.
I presume that there should be an algorithm of sorts, that can deduce which outages generally correlate to what constraints after they have been seen to overlap throughout history.
To recap, an outage may cause a constraint. I would like to feed the model outage data and constraint data, so that if I have planned outage data, it can predict if zero or more constraints are X% likely to occur.
I prefer to use Python, but am open to R.
How would you go about modeling a problem like this? In simplified terms, how do you correlate a row in one table to zero or more rows in another where there is no direct mapping? Most solutions I have seen correlate at the column level as opposed to the row.
After some thinking about this, I've come up with what I think is a solution. Make each row with a timestamp for that particular hour of the day. One hot encode all outages for that hour. Also, create a column for each potential constraint, with the monetary impact as the value (Scaled to a 0-1 scale).
The thing that seems crazy to me, is that this data set is going to end up being thousands of columns wide if I do it this way. Are super-wide data sets like this common in ML? - or, is there a better approach?