# Tag Info

8

I am actively maintaining an efficient implementation of both PrefixSpan and BIDE in Python 3, supporting mining both frequent and top-k (closed) sequential patterns.

6

The only Python package I've found is on Github. They have an implementation of BIDE there, but it's not maintained code.

2

Have you considered to write it by yourself? Because there is probably no up-to-date maintained library right now. Check this out, its the basic - PrefixSpan and Closed/Maximal patterns are actually not that hard to implement.

2

There is unlikely to be any useful pattern analysis for this problem. I cannot prove it, but I think it highly likely that the raindrops are being generated using a pseudo-random process. Even if they are not, you have been given more than enough information in what your controller can "sense" in this simulation, so that predicting future raindrops is not ...

2

If it is really close to random (background noise, bits from random number generators or hash codes), I don't expect there is much you can do, except create a large collector. (Or are you only allowed one collector?) You may want to throw in a few random checks. Those won't prove anything but may give you some insight into the pattern. If you want to ...

1

This basically asks for a recurrent network, like the LSTM. But if you only have 2 properties that are dynamic, I don't think you will have as much luck because they might be affected from external factors as your boss said. However, this will happen regardless of the model you're using. You should not throw away static properties, unless they are the same ...

1

I suggest to start with "outlier detection", "anomaly detection", filtering methods. Its pretty wide topic to cover but you need to start from somewhere.

1

Try generating a dictionary of patterns you want to identify. You can then use convolutions/ cross-correlations to identify where these patterns appear in your data. https://en.wikipedia.org/wiki/Convolution https://en.wikipedia.org/wiki/Cross-correlation http://paulbourke.net/miscellaneous/correlate/ This method is also called 'matched filter'.

1

The visNetwork R package is the best I've worked with. It renders with the vis.js Javascript package right in your RStudio window. Nodes and edge visualization is fully customizable and may be data-driven. They implement click-functionality on your graphs, so you can rearrange them or highlight a node and its neighbors. I've found people really enjoy ...

1

If your signal follow simple square patterns like you've displayed, why not using a simpler solution? Some smoothing to reducte noise (ex: kalman filter) + a derivative function (ex: diff function in numpy) to detect ups and downs should be enough to detect the signal patterns, including their durations.

1

Seq2Pat: Sequence-to-Pattern Generation Library might be relevant to your case. The library is written in Cython to take advantage of a fast C++ backend with a high-level Python interface. It supports constraint-based frequent sequential pattern mining. Here is an example that shows how to mine a sequence database while respecting an average constraint for ...

1

The fastest way to train a model to predict each item's label is using Conditional Random Fields (CRT) like in this example. h/t @erwin

1

It looks to me like what you propose makes sense, but there has been some research done around these questions of time representation already. I'd suggest you check the state of the art in this domain, if only not to reinvent the wheel or miss important cases. I'm not very knowledgeable about it but I can at least point you to TimeML and the related ...

1

I'm not entirely sure but it looks like sequence labeling might be what you need: Sequences of varying length Supervised: you would need to train a model with a sample of sequences annotated with a label at every step (not sure that this is your use case?) Can handle any number of features Conditional Random Fields is the state of the art method, there ...

1

Your problem is commonly called discrete sequence anomaly detection. One way to begin is to generate an anomaly score for the current item. If the anomaly score is above a threshold, label it as "anomalous". An example of this approach is "A Sequence Anomaly Detection Approach Based on Isolation Forest Algorithm for Time-Series" by Weng ...

1

While an RNN using one-hot encoded moves is possible, I would suggest that your model needs to understand chess (or similar complex games) at a deeper level to be able to associate comments to positions. I would encode the position itself (eg a layered representation like in Alpha-Zero paper), and pass those through a conv-RNN to model the temporal ...

1

You can rank the paths by frequency using the database (SQL), constraining the start-, end- points as necessary once you fix the length of the window. If you let the path length be variable then you will not be able to do it all in SQL. In that case you can learn the transition probabilities between states then solve a weighted shortest path problem, where ...

1

You could reframe the problem as a regression - prediction of a single real-valued dependent variable based on several independent variables. If you choose to model with regression, then it technically a "calibration" problem. A calibration problem takes a known observation of the dependent variables is used to predict corresponding explanatory variables.

1

SPMF sounds like a useful library for pattern mining.

1

I've used fim's fpgrowth function in the past and it worked well. It's kind of a pain to install on Windows machines however. It seems to be an academic website so I'm not sure if they're doing many updates to the code over time...

1

I think you might want to check Andrej Karpathy's work around charRNNs some pretty cool work is being done. The github link has all the relevant code too. If you are looking for a more applied way - you can check my blog on deep learning gender from name which effectively uses character level LSTM RNNs to learn patterns : medium.com/@prdeepak.babu/deep-...

1

Welcome to DS stack exchange! Some of your questions/statements are not too clear, I'll try to answer to the best of my abilities but try to ask more precise questions in the future to get better answers (and avoid being down-voted). To your points 1. and 2.: I would say it's highly unlikely that you'd be able to attribute a small fluctuation like 8% to ...

1

The established procedure is called symbolic aggregate approximation or SAX in which you first first do piecewise aggregate approximation (PAA), namely send non-overlapping mean (boxcar) windows of width $w$ over the data, and then transform the obtained values into symbols by equal-frequency-splitting the resulting data distribution into $k$ letters of the ...

Only top voted, non community-wiki answers of a minimum length are eligible