I have dataset of traders' transaction data: trade id, date, stock id, sector of stock id, buy-or-sell, volume $

The goal is to identify anomalies in transactions data of traders. For example to find suspicious transactions.

As I dont have information how real outliers look like so it is unsupervised algorithms. As usual unsupervised algorithms I tried k-mean, z-score, lot, one class svm and different things with moving average etc.

Ive got results, but I was able to use only volume and date for every trader.

Is there any algorithm that can help me to include sector data/stock data/buy-sell data to do my model better? Can I include this information in above unsupervised algorithms or Im missing something?

How would you tackle this task, are there any other solutions maybe?

Thanks in advance.

  • 1
    $\begingroup$ If you have prior labelled data, where labelled are based upon inlier vs outlier, in such case you develop anomaly detection models. Like outliers or novelty detection. Scikit have good explanation for these. $\endgroup$ Jul 2 '19 at 1:56
  • 1
    $\begingroup$ When using z score, it should be done within its group (same product.. Seller?) also.. If your data is presented in time dimension, you should consider residual (changes on time step to avoid growing - collapsing behavior) $\endgroup$
    – krayyem
    Jul 2 '19 at 2:36
  • $\begingroup$ Once I played around Autoencoders on Normal Bank Transactions, and it worked quite OK. But it was on a benchmark dataset, not a real world dataset. Still you can try Autoencoders or Variational Autoencoders. The problem is, if you literally have no data or idea what anomalies in transactions data of traders look like, it is hard to evaluate any unsupervised algorithms. Maybe it is best to think about labeling small part of your data for testing, if not available for training! $\endgroup$ Jul 2 '19 at 5:34
  • $\begingroup$ @vipin bansal I tied, here is good link for it scikit-learn.org/stable/modules/outlier_detection.html. But those models only use volume and datetime data. How can I add sector/stock/buy-sell data? Or you meant something else? $\endgroup$
    – John Doe
    Jul 2 '19 at 7:00
  • $\begingroup$ @krayyem could you please send a link with a good explatation about it? $\endgroup$
    – John Doe
    Jul 2 '19 at 7:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.