Is it recommended to train a NER model using a dataset that has all tokens annotated?

Question

I'd like to train a model to predict the constant and variable parts in log messages.
For example, considering the log message: Example log 1, the trained model would be able to identify:

• 1 as the variable
• Example, log labeled as the constants.

To train the model, I'm thinking of leveraging a training dataset that would have all tokens in all of the log entries annotated. For example, for a particular log entry in the dataset, we would have a number of 8 tokens, of which 6 would be constants and 2 would be variables. However, from what I've seen so far, most NER tasks only annotate part of the textual entries, rather than annotating all tokens in the training data. Thus, is this the right way to tackle this problem? Should I formulate the problem differently, namely not as a NER task, maybe?

OBS: To clarify the difference between constants and variables, these refer to the parts that constitute an original code logging statement. In such a statement, the constants are the textual parts written by developers which remain the same during the execution of the system, whereas the variables is information that is generated during runtime.

Answer 1

There's no problem with this.

Technically a sequence labeling model (this is the general name of the problem of which NER is a particular example) actually always annotates all the tokens. For example, POS tagging is another sequence labeling task in which all the tokens must be receive a label. In the case of the NER task, one is only interested in extracting the entities from the text, this is why any other token is assigned a default label (the "Outside" label in the BIO format, for Begin/Inside/Outside).

What matters for the model is whether enough information is provided in the context to recognize the class. The features are usually designed through patterns which describe conditions about the current word or any word in the context. For example a feature could be used to represent whether the current token is a word or made of digits, or whether the previous token belongs to a predefined set of words, etc.