The objective is to predict router fail/no fail (1/0) in a future time window with all the data collected over the last hour (i.e. binary target)

The data is received at two different levels:

  1. Router metrics: Memory, Temperature, CPU Usage, Idle time, etc..
  2. Connected devices metrics: Data collected from N different connected devices - received rates, signals, etc..

Every row in the training set should be a snapshot of data representing the hub + an aggregation of the N-different devices state (note that N may be different for every row).

Min, max and percentiles of the distribution of the connected device features could be added as new features. Are there other smarter techniques to preserve all the information in the connected clients?

  • $\begingroup$ You have to be more precise on your question : What's your problem ? Getting those variables ? Adding them to the model since you have a non-fixed number of individual devices ? etc $\endgroup$
    – Adept
    Jul 26 at 13:08
  • $\begingroup$ Clarified the purpose in the question $\endgroup$
    – molse
    Jul 26 at 13:15

Ok, your methodology looks good, but you're on a typical problem showing why Data Scientist are true specialists, and not just "running some copied code" : You have to create your variables yourself using your knowledge about the problem.

I'd say you first have to try to list the things you can measure about those devices, and that are worth giving a chance to : it can be from really simple (number of devices connected to, measure of 'data flow' from one device to another, ratio between 2 metrics etc) to more complicated features (example : difference between the number of devices you're connected to, and the number of devices you were connected to last hour, in your previous check. This is a way harder variable to handle, I won't go into details unless you want me to).

As far as I know, there's no general process to create variables (you can automate it by making each possible ratios from metrics you have, and then use feature selection, but I wouldn't bet on it). To me, what guides you on manually creating your variables if what you know about your problem.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.