User identification using machine learning

Question

I am quite new to machine learning, and the task I was given was to find attributes that will allow me to identify users.

I have a set of data for training that contains roughly 48,000 instances, where every instance is labeled with user id. The data are TCP sessions with data on destination, source, ttl, start time, endtime, destination IP addresses, and more.

My task is to find attributes so that in given a difference TCP flows in the same format form the same group of users, I will be able to labeled them with the appropriate user Id.

There are 10 users in the data set.

My question is what the best way to do it, I tried using weka to run number of machine learning algorithm, but I couldn't configure them right, for example I tried to use K-means but the results didn't fit with the labeled user id.

Answer 1

From your question I'm interpreting that you've extracted data from a log which has TCP session related data, some of the fields you've mentioned are: source/dest IP address, TTL, start time, end time. You're trying to segregate this network traffic to isolate the traffic belonging to 10 unique client machines.

There are several ways to approach this client traffic segregation problem. But first you need to identify which variables are going to be most meaningful under different circumstances, since the network setup is not described, I'll list the different possible scenarios:

1. Clients reaching the server without intervening firewall/proxy: In this case each client will have a unique source ip-address which should help segregate the traffic without further analysis.

2. Clients behind a proxy which filters traffic and changes the ip-address to set a unique ip-address of the gateway for all the traffic coming to the server. In this case, you could use a combination of the following:

   a) Segregate clients by TCP/IP packet attributes: Attributes such as TTL, SYN packet size, TCP window size, etc, can identify client OS. For example - Linux and Mac-OSX have default starting TTL values of 64, whereas windows has a default of 128, others like Cisco/Solaris 2.x have 255. So you can roughly segregate the traffic by OS type.

   b) Since the TTL is decremented by 1 for each router placed between the client and the server, so even if you have 2 windows machines, but placed in a network segment separated by one more router, then the TTL number will be one lesser than the other windows clients.

   c) Further identify unique sessions by the sequence number and acknowledgement number in the TCP headers. Each acknowledgement number of the destination server can be correlated by the seq number of subsequent packets from the client for that session. This will let you segregate users connecting to the same server at the same time e.g. 10 different sessions to stackexchange.com.

   d) In case you can get further web-server log details, then attributes such as user-agent strings and HTTP cookies can help you identify a client.

   e) In some cases, the random source port no. used by clients for the TCP session might overlap for different clients accessing different dest. ip-addresses at the same time, you can then safely assume these are traffic form two different clients.

This still leaves you with the problem of segregating traffic for different servers at different points of time. For this you could analyze the data in the following manner:

1. Identify a client using their ISN: There is a certain logic in generating random initial sequence numbers by each OS, and if you have a large sample of ISN nos available, then you could identify the attributes of the random number generator to pinpoint the client. This ISN generation process has been exploited for DoS spoofing attacks, and many OS's have strengthened it since then, but this may still provide some clues to identify machines uniquely.
2. Analyse SSL handshakes to identify clients via their attributes such as supported cipher suites information exchanged during the SSL setup handshake, etc. See this paper for details about this technique.
3. The source port nos. are usually assigned in increasing order, and it is unlikely that every machine started at the same time and had the same TCP traffic volume; this helps identify clients uniquely.
4. TCP/IP timestamps: Most OS's will increment the timestamps in TCP headers and this keeps increasing as long as the system is up. Since systems may be switched on at different times, it helps segregate clients even if they have exactly the same OS and browser versions. See this RFC for mode details.
5. Some proxies my use extra HTTP headers (Via, X-Forwarded-For) to indicate the exact client hostname or address requesting the data. This provides a straightforward way to identify the client, but only if you can access the HTTP header information from the packet.

After segregating traffic using all these means, then you may use network traffic time-series analysis to identify distinct users from the periodicity in their usage. For example, the repetitive pattern of a person who logs into work everyday at 8:15 AM and checks on some specific websites for 5 minutes can be identified and that client uniquely segregated from the rest.

More hypothesis and directions to take can only be identified after exploring the dataset in detail.

Once you're satisfied with this preparatory work, you could then calculate and add all these features along with the other direct indicators to train a multinomial classifier for predicting these 10 user ids. You could use a random forest, LogitBoost or Neural Networks as starters. If a variable is not identified as important for the classification, then discard it to reduce the total number of features.

For new traffic, you would need to generate all these features and then use the final trained model to classify the traffic by user.