2
$\begingroup$

I have dataset of traders' transaction data: trade id, date, stock id, sector of stock id, buy-or-sell, volume $

The goal is to identify anomalies in transactions data of traders. For example to find suspicious transactions.

As I dont have information how real outliers look like so it is unsupervised algorithms. As usual unsupervised algorithms I tried k-mean, z-score, lot, one class svm and different things with moving average etc.

Ive got results, but I was able to use only volume and date for every trader.

Is there any algorithm that can help me to include sector data/stock data/buy-sell data to do my model better? Can I include this information in above unsupervised algorithms or Im missing something?

How would you tackle this task, are there any other solutions maybe?

Thanks in advance.

Improve this question
$\endgroup$
11
  • 1
    $\begingroup$ If you have prior labelled data, where labelled are based upon inlier vs outlier, in such case you develop anomaly detection models. Like outliers or novelty detection. Scikit have good explanation for these. $\endgroup$
    – vipin bansal
    Jul 2, 2019 at 1:56
  • 1
    $\begingroup$ When using z score, it should be done within its group (same product.. Seller?) also.. If your data is presented in time dimension, you should consider residual (changes on time step to avoid growing - collapsing behavior) $\endgroup$
    – krayyem
    Jul 2, 2019 at 2:36
  • $\begingroup$ Once I played around Autoencoders on Normal Bank Transactions, and it worked quite OK. But it was on a benchmark dataset, not a real world dataset. Still you can try Autoencoders or Variational Autoencoders. The problem is, if you literally have no data or idea what anomalies in transactions data of traders look like, it is hard to evaluate any unsupervised algorithms. Maybe it is best to think about labeling small part of your data for testing, if not available for training! $\endgroup$
    – TwinPenguins
    Jul 2, 2019 at 5:34
  • $\begingroup$ @vipin bansal I tied, here is good link for it scikit-learn.org/stable/modules/outlier_detection.html. But those models only use volume and datetime data. How can I add sector/stock/buy-sell data? Or you meant something else? $\endgroup$
    – John Doe
    Jul 2, 2019 at 7:00
  • $\begingroup$ @krayyem could you please send a link with a good explatation about it? $\endgroup$
    – John Doe
    Jul 2, 2019 at 7:00

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.