1
$\begingroup$

I am trying to build a classifier that detects if I have a malaware by predicting the provenance compiler. To do so I have a dataset composed of assembly code in json format :

enter image description here

In particular, I want to select as features the instructions, so the push, mov, jmp,..etc and create a feature vector that contains the number of times a feature appears. So, I want to apply the bag of words. To do so my code is the following:

#libraries
import numpy as np
import pandas as pd
import json as j
import re
import nltk
from nltk.tokenize import word_tokenize

from sklearn.model_selection import train_test_split
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.naive_bayes import *
from sklearn.metrics import confusion_matrix, classification_report
from sklearn import svm

#for visualizing data
import matplotlib.pyplot as plt
import seaborn as sns; sns.set(font_scale=1.2)

%matplotlib inline

json_data = None;
with open('training_dataset.jsonl') as data_file:
    lines = data_file.readlines()
    joined_lines = "[" + ",".join(lines)+"]"

    json_data = j.loads(joined_lines)  

data = pd.DataFrame(json_data)
data.head()



vect = CountVectorizer()

data['instructions'] = data['instructions'].apply(lambda x: ' '.join(x))

vect.fit_transform(data['instructions'])

a =vect.vocabulary_
a

and from this I obtain a dictionary:

enter image description here

with more and more key, value pairs. At this point what I tried to do is to eliminate the registers so my idea to do this is to iterate the dictionary and eliminate the keys that have numbers, and so I have written the following:

def hasNumbers(inputString):
    return any(char.isdigit() for char in inputString)

for k in list(a.keys()):
    if hasNumbers(k):
        del a[k]

but actually it wasn't a really good idea because I have also registers without numbers, for example rdi.

At this point, I don't know how to move, and I am not sure I am going in the correct way. Can somebody please help me? Thank's in advance.

[EDIT 2] Now I am trying to take the json file, not opening it with pandas, but open it as a dictionary, so:

enter image description here

and my intenction is try to split, as suggested, each key in such a way to eliminate the registers. So, my code for this is :

for key in json_data:
    splitted = key.split[0]

but I recieve the following error message:

AttributeError: 'dict' object has no attribute 'split'

I am getting really confused about how to operate now.

[EDIT 3] or I tried to do the following:

for v in json_data.values():
    splitted = v.split[0]

but it gives me the following error message:

AttributeError: 'list' object has no attribute 'values'

a problem I think is the fact that the various push r12,.. etc are in a list, so I don't know how to do.

[EDIT 4] I actually tried to solve the above problem this way:

for v in json_data[0].values():
    splitted = v.split[0]

but again I get the error message:

AttributeError: 'list' object has no attribute 'split'

[EDIT 5] Now, I am trying to do this using the file opened with pandas, so:

data['features'] = [i.split[0] for i in data['instructions']]

but still I get the following error:

AttributeError: 'list' object has no attribute 'split'

or I tried also:

feat = [i.split[0] for i in json_data[0].values()]

but get the same error as above.

$\endgroup$
0
$\begingroup$

In particular, I want to select as features the instructions, so the push, mov, jmp,..etc and create a feature vector that contains the number of times a feature appears.

Instead of reading each instruction entirely for a sample program, you can read the first word of each instruction to get the opcode and then create a feature vector out of these opcodes.

Suppose,

sample_program[i] =  ['push r12', 'push rbp', 'xor edx edx', ...]

i.e. a list of instructions, then define opcodes[i] as

opcodes[i] = [instruction.split[0] for instruction in sample_program[i]] 
#['push', 'push', 'xor']

Now you can create feature vectors out of collections of such opcodes.

$\endgroup$
  • $\begingroup$ thank's for your answer. I tried to do what you suggested, but it gives me the error message : 'list' object has no attribute 'split' . I don' t understand why I have a list instead of strings in my json file when I open it. $\endgroup$ – J.D. Oct 26 at 6:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.