1
$\begingroup$

I am new to data science and trying to find possibilities of using datascience in tasks. I have a set of logs which I want to convert to json. The logs are more or less of same format and I can write a script which parse them or aggregate them but instead of doing it manually I want to use help of machine learning. This thought is also inspired by the fact that at sometime some new log line may come. I guess pre-trained LLM models can identify the context and information from logs. But I am not sure how exactly can ML be used for for this purpose? The question may sound stupid but please pardon.

$\endgroup$

1 Answer 1

1
$\begingroup$

this question is not stupid, it's great that you want to extract knowledge from logs. However, if you are at the very beginning of this journey, I wouldn't recommend not pondering over LLM but starting from simpler stuff. If I were you, I would start by parsing these logs and organizing an infrastructure for doing that regularly (in a real-time best). Having done that you can consider your first NLP analysis, but for me, the key would be to do something hands-on with this data like visualizing on the Grafana dashboard or creating an alerting. When you trudge along this data, you will see what has potential.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.