Questions tagged [anomaly-detection]

Anomaly detection refers to the problem of finding patterns in data that do not conform to expected behaviour. This is also known as outlier detection.

Filter by
Sorted by
Tagged with
1
vote
1answer
182 views

Anomoly detection method selection

I need to decide between SVM (One-Class Support Vector Machine) and PCA (PCA-Based Anomaly Detection) as anomaly detection methods. Azure ML is used and provides SVM and PCA as methods - hence the ...
7
votes
3answers
1k views

Which outlier detection can detect these outliers?

I have a vector and want to detect outliers in it. The following figure shows the distribution of the vector. Red points are outliers. Blue points are normal points. Yellow points are also normal. ...
6
votes
1answer
2k views

Netflow anomaly detection python packages

Is anyone aware of any open source / python packages for Netflow Anomaly detection ? I found some on github but anyone who has more experience with it. please advise.
5
votes
1answer
668 views

Time Series pattern recognition and classification problem

I have some labeled sensor data. Now, I would like to know how to extract features from time series using DFT, DWT, and HAAR transforms. I know that the transformations above transform a signal to ...
1
vote
0answers
107 views

In the context of anomaly detection, which is a better language to use, python or R? [closed]

In the context of anomaly detection, which is a better language to use, python or R ?
1
vote
0answers
236 views

Network Anomaly detection [closed]

I am working on a problem to identify anomaly in Network. I am stuck at how to handle the following issues 1. VPN Land Based Violation (Login from Multiple locations within unrealistic situation 2. ...
3
votes
2answers
830 views

Which Outlier Detection Method? Why?

For detecting an outlier in a vector I have tested different well known outlier detection methods. Finally, I used combination of different methods and an agreement between those methods. Now, a ...
5
votes
1answer
3k views

Anomaly detection for transaction data

I have transaction details for credit data (bank transfers, peer to peer transfers, etc). Currently, I have one year worth of data which I cannot properly classify. I'm looking for input and ...
0
votes
0answers
521 views

HMM - Matlab for data set to detect anomaly

I have a dataset of oil temperatures. The time series consist of 100 hours of measurement at every second. There is an anomaly in the data that I would like to detect using Hidden Markov Models (HMM). ...
0
votes
1answer
898 views

Anomaly detection on a time-series data in a CSV format using python

I have a time versus current data for a days work which is as follows: ...
0
votes
1answer
59 views

What could be possible features of a textual word?

I wish to construct feature vectors of words in a document and then calculate their linkage distance to detect anomalies. My question is how can I model these features ? If possible please give an ...
6
votes
2answers
4k views

What methods can be used to detect anomalies in temporal texual data?

I've been looking for methods that can help figure out anomalies in textual data stored in databases. Major goal is to use a unsupervised learning method to detect the anomalies. Further how can I ...
2
votes
0answers
1k views

How does Elastic's Prelert (formerly Splunk Anomaly Detective App) work?

Background: In recent months, Elastic has purchased Prelert and will actively incorporate it into the Elastic stack (and also discontinue the Splunk Anomaly Detective App!). I am trying to understand ...
2
votes
1answer
199 views

statistics or robust statistics for identifying multivariate outliers

For the single variate data sets, we can use some straightforward methods, such as box plot or [5%, 95%] quantile to identify outliers. For multivariate data sets, are there any statistics that can be ...
3
votes
1answer
4k views

Anomaly Detection In Univariate Time Series Data Using ARIMA In Python With Updating

I have trained an ARIMA model on some 15 minute incremented time series data by using the statsmodels library. I would like to determine how anomalous the next 15 minute increment's data I observe is. ...
1
vote
0answers
263 views

Outlier Detection

I have a dataset which has two class. It has 13 features. They are values which are sent from 13 sensors. Label is True or False. When I use mad outlier detection, when the label is false(really there ...
1
vote
0answers
29 views

how to learn from unlabeled samples but labeled group of samples?

I'm trying to perform anomaly detection on the open data from citibike. They are giving bikeshare trips for the past 30+ months, as well as monthly reports. In those reports they say how many bikes ...
3
votes
0answers
162 views

Clustering with Replicator Neural Network

I'm trying to cluster an unknown set of data with a replicator neural network. The number of clusters is determined by the number of neuron units in the middle layer, multiplied by the number of steps ...
8
votes
1answer
7k views

Difference: Replicator Neural Network vs. Autoencoder

I'm currently studying papers about outlier detection using RNN's (Replicator Neural Networks) and wonder what is the particular difference to Autoencoders? RNN's seem to be treaded for many as the ...
2
votes
2answers
244 views

How do I approach grouped anomaly detection?

I currently have a large stream of data, with points such as HTTP request/response codes (200, 404, 500, etc.). Essentially, I want to perform anomaly detection for when too many signals that are NOT ...
2
votes
1answer
473 views

Find outliers in time-series data

I want to find outliers in power consumption in real-time, at hourly rate, i.e., at the end of the hour, I should say whether power consumption in current hour was outlier/anomalous or not. Approach: ...
4
votes
1answer
483 views

If a time series has random time events, how to detect patterns?

My app receives messages with a random number of bits at a random time. But two weeks ago I started to notice some almost regular patterns on the metrics of my app. I suspect they are some bots ...
2
votes
2answers
512 views

Industry Standard Process for Fraud/Outlier/Anomaly Detection

I want to write my thesis about Fraud Detection in ERP Databases. I'm looking for a Industry Standard Processs such as CRISP-DM for Data Mining Projects, in order to justify my approach in solving the ...
3
votes
2answers
3k views

Outlier detection for unbalanced classes

I have to make a predictive model for predicting a boolean Won/Lost variable based on some other numeric data; and further find out the features of observations that have 'Won'. However, the number ...
1
vote
1answer
2k views

Anomaly detection in Time Series Data - Help Required [closed]

I am looking for algorithms on Anomaly detection for time series data. It is uni-variate analysis, considering single parameter (inlet pressure) of air compressor sensor data. The objective is to ...
0
votes
1answer
880 views

GE Predix's machine learning (anomaly detection) capabilities

Background: I am investigating time-series anomaly detection for industrial machine data, and have stumbled upon GE Predix. It seems like a promising tool, however, I am not familiar with their ...
5
votes
3answers
4k views

change detection

I have a question related to change detection. Application domain is robotics/planning. Background/setting: There is a sensor detecting distance from obstacle (ultrasonic / sonar sensor) at a ...
2
votes
1answer
82 views

Accept any suggestion to create training data from correlation matrix to find odd one out to identify difference in variation

I have N time varying feature vectors obtained by recording different parameters over time.This results in N*N similarity matrix which contains one to one correlations value for each feature. We need ...
11
votes
2answers
6k views

Tools for automatic anomaly detection on a SQL table?

I have a large SQL table that is essentially a log. The data is pretty complex and I'm trying to find some way to identify anomalies without me understanding all the data. I've found lots of tools for ...
2
votes
1answer
1k views

anomaly detection alert system

I need to design and implement my 3rd year project which is a real-time alert system for parents to keep their kids safe with their smartphones. My data is time-based and unsupervised, which means I ...
33
votes
5answers
46k views

Is it necessary to standardize your data before clustering?

Is it necessary to standardize your data before cluster? In the example from scikit learn about DBSCAN, here they do this in the line: ...
72
votes
7answers
81k views

Open source Anomaly Detection in Python

Problem Background: I am working on a project that involves log files similar to those found in the IT monitoring space (to my best understanding of IT space). These log files are time-series data, ...
3
votes
4answers
174 views

Definition of "inside" in K-means?

After conducting a cluster analysis using K-means, I have new data coming online that I need to detect anomalies with. Anomalies are assumed to not be within the clusters. So, how is one to define "...
3
votes
2answers
1k views

What are some good sources to learn fraud/anomaly detection in normal/time-series data?

I would like to know more on fraud/anomaly detection. I am looking for good source or survey article/book etc out there which will give me some preliminary idea of the area. Any suggestion is ...
7
votes
5answers
2k views

What would be a good way to use clustering for outlier detection?

For simplicity let's assume the feature space is the XY plane.
4
votes
2answers
794 views

Anomaly detection in multiple parameters

I am a newbie to data science with a typical problem. I have a data set with metric1, metric2 and metric3. All these metrics are interdependent on each other. I want to detect anomalies in metric3. ...

1 2 3 4 5
6