3
$\begingroup$

Suppose I am processing 3rd party vendor error log files that I am unable to change the schema from the source. And I am trying to predict a label.

Once logs are collected and I have all I need, in varying log formats, with each missing random information compared to the others.

-------------------------------------------------------
| vendor | variable_one | variable_two |variable_three|
-------------------------------------------------------
|    1   |       -      |"some string" |     NULL     |
-------------------------------------------------------
|    3   |      GET     |"some string" |     1270     |
-------------------------------------------------------
|  ...   |     POST     |    NULL      |      760     |
-------------------------------------------------------  

Even after all the preprocessing and data transformations to change labels to Binary, and Categorical values and such. After cleaning, feature selection and training. Having a model trained, evaluated, and tested. (label not depicted)

-------------------------------------------------------
| vendor | variable_one | variable_two |variable_three|
-------------------------------------------------------
|    1   |      GET     |"some string" |        0     |
-------------------------------------------------------
|    3   |      GET     |"some string" |     1270     |
-------------------------------------------------------
|  ...   |     POST     |     ""       |      760     |
-------------------------------------------------------  

The questions are:

When I go back to send my vendors logs to through the model, I am still sending it through with, potentially, missing values from what was altered, or selected as a feature for the model.

  1. Is the model then useless?
  2. Do the transformations made in data cleansing need to be applied to each log entry that comes through, somewhere upstream?
  3. Assuming the model only knows "nice pretty" cleaned data, and I send log entries with missing values. Is the model making assumptions or less accurate prediction?
$\endgroup$
1
$\begingroup$

This question has no general answer; it depends on the specifics of both the statistical method in question and the implementation of that method in whatever language you used.

For example if you used a logistic GLM, then a missing value is an error since data records must be complete. Presumably you would have included a default value, and you will have to manually apply that if you want those records to be scored.

Alternatively, if you were using gradient boosted trees (gbm), then they can cope with missing values, but do so in an implementation-defined way. The answers would be different in the gbm and xgboost R packages, for example.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.