I am trying to make a malicious software detector using machine learning.I have obtained a dataset from internet which containes md5 hash and other features of pe files.I fed this databse to weka to find out the most important features,which came out to be only one(static_prio).I tried running the K-NN algorithm on the data using only one feature(static_prio) and I got 99.82% accuracy but I am a little confused about my approach here.I am naive with machine learning so please help by putting me on the right track.Thank you.
It is not clear in your post whether or not you are using training/test sets so I will answer in this way:
The first thing to do if you want to validate your results is to cut your set into a training set and a validation set. This way you train the K-NN method on your training set, and you use the trained classifier on the validation set. Then you can monitor the validation error, and the training error.
The aim is to obtain a model that generalizes well, which means being able to accurately predict the class a sample belongs to, when it has never seen the sample before.
When trying to classify fraud/malware/etc,
don't score your model based only on Accuracy ... and always check for imbalanced data.
Check out this Github code - it's very much suitable for begginers...
The problem with your model is that it's almost always classifying the software as "not malware", and due to the very small number of "malware" it gets high accuracy.
Good Luck :)