0
$\begingroup$

I found a dataset about DDOS attacks id like to use but the tool used to extract features from pcap files seems to have indented the columns a little weird as seen below there are indents in unusual places. like at the start of some feature names in particular im interested in the label or in this case Label (where there is a space in front). Syn is the name of the dataframe.

Syn.columns

Index(['Unnamed: 0', 'Flow ID', ' Source IP', ' Source Port', ' Destination IP', ' Destination Port', ' Protocol', ' Timestamp', ' Flow Duration', ' Total Fwd Packets', ' Total Backward Packets', 'Total Length of Fwd Packets', ' Total Length of Bwd Packets', ' Fwd Packet Length Max', ' Fwd Packet Length Min', ' Fwd Packet Length Mean', ' Fwd Packet Length Std', 'Bwd Packet Length Max', ' Bwd Packet Length Min', ' Bwd Packet Length Mean', ' Bwd Packet Length Std', 'Flow Bytes/s', ' Flow Packets/s', ' Flow IAT Mean', ' Flow IAT Std', ' Flow IAT Max', ' Flow IAT Min', 'Fwd IAT Total', ' Fwd IAT Mean', ' Fwd IAT Std', ' Fwd IAT Max', ' Fwd IAT Min', 'Bwd IAT Total', ' Bwd IAT Mean', ' Bwd IAT Std', ' Bwd IAT Max', ' Bwd IAT Min', 'Fwd PSH Flags', ' Bwd PSH Flags', ' Fwd URG Flags', ' Bwd URG Flags', ' Fwd Header Length', ' Bwd Header Length', 'Fwd Packets/s', ' Bwd Packets/s', ' Min Packet Length', ' Max Packet Length', ' Packet Length Mean', ' Packet Length Std', ' Packet Length Variance', 'FIN Flag Count', ' SYN Flag Count', ' RST Flag Count', ' PSH Flag Count', ' ACK Flag Count', ' URG Flag Count', ' CWE Flag Count', ' ECE Flag Count', ' Down/Up Ratio', ' Average Packet Size', ' Avg Fwd Segment Size', ' Avg Bwd Segment Size', ' Fwd Header Length.1', 'Fwd Avg Bytes/Bulk', ' Fwd Avg Packets/Bulk', ' Fwd Avg Bulk Rate', ' Bwd Avg Bytes/Bulk', ' Bwd Avg Packets/Bulk', 'Bwd Avg Bulk Rate', 'Subflow Fwd Packets', ' Subflow Fwd Bytes', ' Subflow Bwd Packets', ' Subflow Bwd Bytes', 'Init_Win_bytes_forward', ' Init_Win_bytes_backward', ' act_data_pkt_fwd', ' min_seg_size_forward', 'Active Mean', ' Active Std', ' Active Max', ' Active Min', 'Idle Mean', ' Idle Std', ' Idle Max', ' Idle Min', 'SimillarHTTP', ' Inbound', ' Label'], dtype='object')

The data in the label is

print("Label")
print(Syn[' Label'].unique())

Label ['Syn' 'BENIGN']

I want to change it to 1 and 0 so i can use it in logistic regression and other machine learring algorithms and deep learning. i tried the following two aproaches but cant seem to get it to work as i get errors on both methods as seen below

def change_label(cols):
    Syn[' Label'] = cols[0]

    if  Syn[' Label']== 'Syn':
        return 1
    elif Syn[' Label'] == 'BENIGN':
        return 0

Syn[' Label'] = Syn[[' Label']].apply(change_label,axis=1)

and the error i get is ValueError Traceback (most recent call last) in ----> 1 Syn[' Label'] = Syn[[' Label']].apply(change_label,axis=1)

~/anaconda3/lib/python3.7/site-packages/pandas/core/frame.py in apply(self, func, axis, 
broadcast, raw, reduce, result_type, args, **kwds)
   6485                          args=args,
   6486                          kwds=kwds)
-> 6487         return op.get_result()
   6488 
   6489     def applymap(self, func):

~/anaconda3/lib/python3.7/site-packages/pandas/core/apply.py in get_result(self)
    149             return self.apply_raw()
    150 
--> 151         return self.apply_standard()
    152 
    153     def apply_empty_result(self):

 ~/anaconda3/lib/python3.7/site-packages/pandas/core/apply.py in apply_standard(self)
    255 
    256         # compute the result using the series generator
--> 257         self.apply_series_generator()
    258 
    259         # wrap results

~/anaconda3/lib/python3.7/site-packages/pandas/core/apply.py in apply_series_generator(self)
    284             try:
    285                 for i, v in enumerate(series_gen):
--> 286                     results[i] = self.f(v)
    287                     keys.append(v.name)
    288             except Exception as e:

<ipython-input-104-14e7797e3b86> in change_label(cols)
      2     Syn[' Label'] = cols[0]
      3 
----> 4     if  Syn[' Label']== 'Syn':
      5         return 1
      6     elif Syn[' Label'] == 'BENIGN':

~/anaconda3/lib/python3.7/site-packages/pandas/core/generic.py in __nonzero__(self)
   1476         raise ValueError("The truth value of a {0} is ambiguous. "
   1477                          "Use a.empty, a.bool(), a.item(), a.any() or a.all()."
-> 1478                          .format(self.__class__.__name__))
   1479 
   1480     __bool__ = __nonzero__

ValueError: ('The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), 
a.any() or a.all().', 'occurred at index 0')

The second way i tried was to map the desired values but got a different error

my_dict = {'Syn':'1','BENIGN':'0'}
Syn[' Label'] = Syn[[' Label']].map(my_dict)

and i got the following

AttributeError                            Traceback (most recent call last)
<ipython-input-107-01d9e9f42923> in <module>
      1 my_dict = {'Syn':'1','BENIGN':'0'}
----> 2 Syn[' Label'] = Syn[[' Label']].map(my_dict)
      3 
      4 #temp_data['Gender'] = temp_data[['Gender']].apply(impute_gender,axis=1)

 ~/anaconda3/lib/python3.7/site-packages/pandas/core/generic.py in __getattr__(self, name)
   5065             if self._info_axis._can_hold_identifiers_and_holds_name(name):
   5066                 return self[name]
-> 5067             return object.__getattribute__(self, name)
   5068 
   5069     def __setattr__(self, name, value):

AttributeError: 'DataFrame' object has no attribute 'map'

any solution to this problem would be greatly appreciated.

After applying the solution suggested by @user47 the following occurred:

columns = Syn.columns.tolist()
columns = ['_'.join(item.split()) for item in columns]
Syn.columns = columns  

ran perfectly with the new edit but the next part gave an error

import pandas as pd
# above code here
pd.get_dummies(Syn,prefix=['Label'], drop_first=True)

error was

ValueError                                Traceback (most recent call 
last)
<ipython-input-84-ca05afe070ad> in <module>
      1 import pandas as pd
      2 # above code here
----> 3 pd.get_dummies(Syn,prefix=['Label'], drop_first=True)

 ~/anaconda3/lib/python3.7/site- 
packages/pandas/core/reshape/reshape.py in get_dummies(data, prefix, 
prefix_sep, dummy_na, columns, sparse, drop_first, dtype)
    823                     raise ValueError(len_msg)
    824 
--> 825         check_len(prefix, 'prefix')
    826         check_len(prefix_sep, 'prefix_sep')
    827 

~/anaconda3/lib/python3.7/site-packages/pandas/core/reshape/reshape.py 
    in check_len(item, name)
    821                     len_msg = len_msg.format(name=name, 
len_item=len(item),
    822                                              
len_enc=data_to_encode.shape[1])
--> 823                     raise ValueError(len_msg)
    824 
    825         check_len(prefix, 'prefix')

ValueError: Length of 'prefix' (1) did not match the length of the 
columns being encoded (8).

I tried using the following but couldnt see it in the column list when i used Syn.columns. I expected to see Target in the list which would be the encoded version of Label

Target = pd.get_dummies(Syn['Label'],drop_first=True)
$\endgroup$
1
$\begingroup$

First thing first, you should remove all the space from the columns, this would create problems when you have written enough code and one mistake in spacing would stop the program from running.

So since you're columns are in the Syn dataframe, maybe use this to remove the spaces and fill the spaces between words with '_' :

columns = Syn.columns.tolist()
columns = ['_'.join(item.split()) for item in columns]
Syn.columns = columns

Your dataframe's columns would look something like this now :

'Unnamed:_0', 'Flow_ID', 'Source_IP', 'Source_Port', 'Destination_IP', 'Destination_Port'

Next since you need to change the 'Label' attribute, do the following :

import pandas as pd
# above code here
pd.get_dummies(Syn,prefix=['Label'], drop_first=True)

You can also use :

Syn['Label] = [1 if item == 'Syn' else 0 for item in Syn('Label)]

I would recommend the former approach if you have a dataset with multiple classes.

| improve this answer | |
$\endgroup$
  • $\begingroup$ I encountered a problem with the third line of code as suggested, I added the new error to the end of the original question up above, I would be grateful for an further assistance $\endgroup$ – wild card Mar 29 at 14:16
  • $\begingroup$ I apologize, there was a small bug there, I corrected the code, try the code now $\endgroup$ – user47 Mar 29 at 22:54
  • $\begingroup$ that fixed that problem but the encoding part still doesnt work i added the change plus error to the end of the original question plus a variation of your code i made that ran but i dont think it did what i wanted $\endgroup$ – wild card Mar 31 at 20:44
  • $\begingroup$ I could look at the notebook and the data, But since that is not an option, Try this : stackoverflow.com/questions/37292872/… $\endgroup$ – user47 Mar 31 at 22:49
  • $\begingroup$ Thank you i will look trough this if your curious the data came from the Syn.csv dataset at unb.ca/cic/datasets/ddos-2019.html though i wouldnt recommend downloading it as the zip file is 2.2 gigabytes approximately $\endgroup$ – wild card Apr 1 at 18:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.