Tagging Unix/Non-Unix logs using NLP

Question

I have a set of unstructured data consisting command output logs for different operating systems like Unix, Windows, etc.

For example:

Releasing version 0.0.1 for Stackoverflow, on 01/01/2019. The coverage is 99% and build is passed.

(This is just an example, and not related to actual use case). This output is different for different operating systems. I want to perform tagging on this data

For the case above, the output should be:

Releasing version VERSION_NUMBER for PRODUCT_NAME, on releaseDate. The TEST_TYPE is TEST_VALUE and TEST_TYPE is TEST_VALUE

Here, some words are replaced with their corresponding sample tags.

I have studied techniques like POS tagging, NER, LSTM, but I don't know which one is suitable for this particular problem. How can I gather data from raw output and how to apply those techniques here.

Thanks to everyone who is willing to help me with this.

Answer 1

PoS tagging works for natural language only and identifies grammatical parts of the sentence, nothing more. LSTM is an algorithm that can be used to predict series. Named Entity Recognition (NER) and Terminology Extraction could work if you have already data to engage in Information Extraction (IE). However, In order to use a these techniques you need to have a trained model and in order to train one you need data. In your case that would involve identifying and tagging parts of the sentence by hand and later training a model with that data.

The best approach, in my opinion, is just use regex to identify parts of the sentence as one of the approaches of Information Extraction and use hard coded rules to best identify what you are trying to replace later.

For instance, if you want to search the version in the example:

import re

s = 'Releasing version. 0.0.1 for Stackoverflow, on 01/01/2019. The coverage is 99% and build is passed.'

re.search(r'Releasing version(.*?)for', s).group(1)

' 0.0.1 '

Check out these resources that will help you parse a log in Python using Regex.

https://pythonicways.wordpress.com/2016/12/20/log-file-parsing-in-python/

https://medium.com/devops-challenge/apache-log-parser-using-python-8080fbc41dda

But if you prefer to use Named Entity Recognition or Terminology Extraction techniques you could hack a NER model and train it yourself with your data. Keep in mind though, that according to Poibeau, Thierry; Kosseim, Leila (2001). "Proper Name Extraction from Non-Journalistic Texts". Language and Computers. 37 (1): 144–157.:

Research indicates that even state-of-the-art NER systems are brittle, meaning that NER systems developed for one domain do not typically perform well on other domains.

And according to Wikipedia: Considerable effort is involved in tuning NER systems to perform well in a new domain; this is true for both rule-based and trainable statistical systems.

So even if you pull that off, accuracy will be less than if you just extracted data using regex. An approximate of a reduction of 97% to 93% only for named entities (companies, names, etc.) Accuracy reduction will be much less in you case.

Check this link for more information about Information Extraction: https://web.stanford.edu/~jurafsky/slp3/17.pdf