3
$\begingroup$

My data consists of entries when an event is True, namely, when a train crossing is down. So I will have entries within a day like so (just examples):

start,end,interval-length

10:43:02,10:46:02,180, 10:49:06,10:51:06,120

The state is of course True for these entries, and the times not logged throughout a day are when the state is False, aka traversable. (however I have a strong feeling I should explicitly pad these times in, so that I have full 24-hour cycles logged)

The data presented is a time series, based on which I wish to make predictions for coming days. I want to be able to ask "What are the chances that the crossing is down at time $t$, of day $d$?" (the second argument is important because I will have 7 different models, one for each day of the week due to the differing train traffic schedules in a week), taking into consideration the previously observed times.

This of course needs to take into account the fact that when the events happen, they have a certain length (as shown by my data). This is not inherently a classification problem because I am also interested in a confidence of the prediction, which is not as simple as taking a built model's performance parameters. Assuming though that this could be a promising way to tackle the problem, how can I build a model so that the times later on in a day don't wrongfully bias it?

Consider the simple example of some dataset, where days are represented as indexing numbers 1-7. An unsupervised classification algorithm would be biased to guess more towards saturday and sunday, because they are higher numbers (and in this case the solution is to have 7 different columns, each with boolean values instead - but I can't do anything similar to this in my case). I hope this example clears up my question a bit.

$\endgroup$
1
$\begingroup$

Not knowing your data in detail, my intuition is that you could go for dummy (one hot) encoding. You could split each day in (say) 10 min. intervals $x$ (144 columns) and attach labels (up/down) $y$. Each time interval $x$ would be one dummy encoded column (true/false).

The model would be a binary classification (logistic) like:

$$ y = \beta_0 + \beta_1 x_1 + ... + \beta_n x_n + u .$$

In case you have enough days, this could be a reasonable predictive model. It is likely, that some of the dummy encoded $x$ are irrelevant. So it could be worth to check a Lasso model, where some (not so helpful $x$) are shrunken or automatically excluded from the model.

https://web.stanford.edu/~hastie/glmnet/glmnet_alpha.html

Based on the same logic, you could also check boosting, e.g. lightGBM to make the classification.

| improve this answer | |
$\endgroup$
  • $\begingroup$ I had the same intuition of splitting the day into intervals of 1 minute (aka, padding in the dataset that I have), attaching labels for each point and collating the days, then throwing all of this into a classification algorithm. Does this sound legible? & I will look into boosting, thank you! $\endgroup$ – peterxz Jan 22 at 14:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.