# Putting dirty data back into a model

Suppose I am processing 3rd party vendor error log files that I am unable to change the schema from the source. And I am trying to predict a label.

Once logs are collected and I have all I need, in varying log formats, with each missing random information compared to the others.

-------------------------------------------------------
| vendor | variable_one | variable_two |variable_three|
-------------------------------------------------------
|    1   |       -      |"some string" |     NULL     |
-------------------------------------------------------
|    3   |      GET     |"some string" |     1270     |
-------------------------------------------------------
|  ...   |     POST     |    NULL      |      760     |
-------------------------------------------------------


Even after all the preprocessing and data transformations to change labels to Binary, and Categorical values and such. After cleaning, feature selection and training. Having a model trained, evaluated, and tested. (label not depicted)

-------------------------------------------------------
| vendor | variable_one | variable_two |variable_three|
-------------------------------------------------------
|    1   |      GET     |"some string" |        0     |
-------------------------------------------------------
|    3   |      GET     |"some string" |     1270     |
-------------------------------------------------------
|  ...   |     POST     |     ""       |      760     |
-------------------------------------------------------


# The questions are:

When I go back to send my vendors logs to through the model, I am still sending it through with, potentially, missing values from what was altered, or selected as a feature for the model.

1. Is the model then useless?
2. Do the transformations made in data cleansing need to be applied to each log entry that comes through, somewhere upstream?
3. Assuming the model only knows "nice pretty" cleaned data, and I send log entries with missing values. Is the model making assumptions or less accurate prediction?

Alternatively, if you were using gradient boosted trees (gbm), then they can cope with missing values, but do so in an implementation-defined way. The answers would be different in the gbm and xgboost R packages, for example.