I'd like to train a model to predict the constant and variable parts in log messages.
For example, considering the log message:
Example log 1, the trained model would be able to identify:
1as the variable
loglabeled as the constants.
To train the model, I'm thinking of leveraging a training dataset that would have all tokens in all of the log entries annotated. For example, for a particular log entry in the dataset, we would have a number of 8 tokens, of which 6 would be constants and 2 would be variables. However, from what I've seen so far, most NER tasks only annotate part of the textual entries, rather than annotating all tokens in the training data. Thus, is this the right way to tackle this problem? Should I formulate the problem differently, namely not as a NER task, maybe?
OBS: To clarify the difference between constants and variables, these refer to the parts that constitute an original code logging statement. In such a statement, the constants are the textual parts written by developers which remain the same during the execution of the system, whereas the variables is information that is generated during runtime.