I want to use Unsupervised Leaning to detect Anomalies within a huge Csv file (consisting of headers that are named and thousands of rows belonging to them)

Here on this link I have read about the learning process of AI and Machine Learning.

I have some uncertainties hoping to clear them up from advice in here. Start of the project, I ran a Dataset (csv) into the Machine Learning Algorithm.

Does this create a model file that future dataset can be tested on? if so, what format is the file?

I have this question because a year ago I did a self-driving RC car, where first I would drive it with the remote control, collect data, train it and then I would get an output file. I would just run the output file and then the car would drive autonomously upon a path which was given.

The whole idea is the create a Anomaly Detection for Data, that is flexible and I don’t have to go and hardcode the rules.

  • $\begingroup$ The answer depends on which specific method and library you use, but in theory yes: unsupervised learning learns a model which can later be applied to fresh instances. Whether this model is obtainable as a file depends on your specific library. $\endgroup$ – Erwan Jan 20 at 19:10
  • $\begingroup$ @Erwan, I am planning to try some multivariant mathods (because my data might depend on multiple variables) . Which one would you recommend ? and Which methods have obtainable models as a file. Would really appreciate if you would give an answer around this. thank you $\endgroup$ – be1995 Jan 20 at 20:37
  • $\begingroup$ Sorry I don't really know any anomaly detection library, but it should be easy to find out if a particular library does what you need in its documentation. I would expect any reasonably good library can do that, normally it's a fairly standard feature with unsupervised methods. $\endgroup$ – Erwan Jan 20 at 23:11
  • $\begingroup$ What exactly are the kind of anomalies you wish to detect? It is not clear to me why you consider this to be ananomaly detection problem $\endgroup$ – jonnor Feb 13 at 9:34
  • $\begingroup$ @jonnor, first start with detecting anomalies in integers so one identifier. Later start with mutlivariant $\endgroup$ – be1995 Feb 13 at 10:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.