8
$\begingroup$

Experts in my field are capable of predicting the likelyhood an event (binary spike in yellow) 30 minutes before it occurs. Frequency here is 1 sec, this view represents a few hours worth of data, i have circled in black where "malicious" pattern should be. Interactions between the dimensions exist, therefore dimensions cannot be studied individually (or can they?)

enter image description here

I'm trying to build a supervised ML model using Scikit Learn which learns a normal rythm, and detects when symptoms might lead to a spike. I am lost for which direction to take. I have tried Anomaly detection, but it only works for on the spot detection, not prior.

How could I detect "malicious" patterns prior to those events (taking them as target variables) ?

I welcome any advice on which algorithms or data processing pipeline might help, thank you :)

$\endgroup$
  • $\begingroup$ Are these brain waves? $\endgroup$ – JahKnows Apr 20 '17 at 13:49
7
$\begingroup$

This is a fun problem. This is a time series and from this time series you want to identify the trigger of a certain event. So it is a binary classification problem. Based on the information from the specified window will a spike occur? Yes or No.

The first step is to set up your database. What you will have is a set of instances (which can have some overlap but to avoid bias it is best for them to be independently drawn) and then for each instance a human needs to label if there was a spike or if there was not a spike.

Then you need to identify the time window you want to use for your time series analysis. You have done this and decided 30 minutes is a good start.

Now, you have 6 waveforms in a 30 minute window from which you can extract data to get information about your classification. You can use the raw data samples as your features, but this is WAY TOO many features and will lead to poor results. Thus you need some feature extraction, dimensionality reduction, techniques.

There are a million ways you can extract data from these waveforms. First, ask yourself, as a human what are the telltale signs that these other waveforms should have which would mean a spike would arise. For example, in seismic data, if you see agitation in a waveform from a neighboring town then you should expect to see agitation in your town soon.

In general, I like to extract all the basic statistics from my waveforms. Get the mean, standard deviation, fluctuation index, etc. Get whatever you think might help. Check how these statistics correlate with your labels. The more correlation the better they might be. Then there are some very good techniques for extracting time and frequency information from your time-series. Look into envelope mode decomposition and empirical mode decomposition. I have used empirical mode decomposition successfully on some time series data and obtained far better results than I expected.

Now even though you have your reduced feature space you can do better! You can apply some dimensionality reduction techniques such as PCA or LDA to get a lower dimensional space which may better represent your data. This might help, no guarantees.

Now you have a small dataset with instances that are a Frankenstein concoction which represents your 6 waveforms across the 30 minute window. Now you are all set to select your classifier. You will want a binary classification algorithm, luckily that is the most common. There are many to choose from. How to choose?


How many instances do you have?

$\# instances > 100* \#features$?

Then you are all set to use a deep learning technique such as neural networks, 1D convolutional neural networks, stacked autoencodders, etc...

Less than that!!!! The you should stick with shallow methods. Check out kernel support vector machines, random forests, k-nearest neighbors etc..


Common misconception: A shallow method CAN and WILL perform better than a deep learning technique if you have properly selected your features. feature extraction is the most important aspect of a machine learning architecture.


I want to use anomaly detection!

This would also work and there are some good techniques that would do this. However, the nature of anomaly detection is to learn the distribution of the nominal case. So you would feed your algorithm all the instances in your dataset that did not result in a spike. Then from this your algorithm would be able to identify when a novel instance is significantly different from this nominal distribution and it will flag it as a n anomaly. This would mean that a spike will occur in your context.

Check out:

Learning Minimum Volume Sets http://www.stat.rice.edu/~cscott/pubs/minvol06jmlr.pdf

Anomaly Detection with Score functions based on Nearest Neighbor Graphs https://arxiv.org/abs/0910.5461

New statistic in P-value estimation for anomaly detection http://ieeexplore.ieee.org/document/6319713/

You can also use more rudimentary anomaly detection techniques such as a generalized likelihood ratio test. But, this is kind of old-school.

$\endgroup$
  • $\begingroup$ Great Walkthrough, thank you for your level of detail. The timeseries shown here are already a selection of the original features, and made 'visualisable' for the human eye using rolling mean for example. The "raw" data feed consists of around 50 features, and a million or so instances. I guess that leads me to deep learning techniques ! I am diving into feature extraction, and will definately post the results of my journey here :) $\endgroup$ – William D Apr 21 '17 at 8:14
  • $\begingroup$ Yeah that would be great keep us posted. If you have 50 features and 1 million instances. Go for the neural network first, its the easiest to code up since so many packages already exist (ex: Keras in python). Also try PCA and LDA to transform your feature space to a more representative one. $\endgroup$ – JahKnows Apr 21 '17 at 14:47
0
$\begingroup$

You need to do feature extraction or feature engineering to create variables in your training data that "catch" those patterns you boxed in and then have a target variable saying "malice found" or "malice found not found"

Take a really simple example: predicting if it is going to rain. You could come up with a reasonably good predictor of rain in the next 30 minutes that checked every 30 minutes if 1. it suddenly got cloudy and 2. barometric pressure dropped.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.